In What’s Missing From Your Wi-Fi 6 Router? OFDMA back in January, I said that OFDMA was beginning to look at a lot like MU-MIMO. Since then, I’ve continued to work on developing OFDMA test methods, trying different approaches, as you’ll see shortly.
The bottom line is I still can’t consistently find evidence of the promised benefits of improved airtime efficiency, higher total bandwidth use with multiple devices, and lower latency.
In the previous article, the testbed STAs consisted of an octoScope Box18 with four Samsung S10e smartphones inside. Four octoScope high-gain dual-band antennas capture RF and a four-channel 90 dB range programmable attenuator (quadAtten) attenuates signal level.
The quadAtten feeds into a four-channel two-way RF splitter/combiner, whose second port connects to octoScope Pal-24 and Pal-5 partner devices that have their own attenuator. This provides the option of adding 2.4 GHz b/g/n and 5 GHz a/n/ac STAs into the test mix. The splitter connected to four octoScope high-gain dual-band antennas in a 38-inch octoScope box containing the device under test (DUT).
OFDMA testbed – first attempt
The test method measured the total UDP throughput for the four phones with OFDMA enabled and disabled.
In the end, this approach, UDP traffic with small packet sizes, which was suggested by Broadcom, didn’t produce significant or repeatable throughput gains.
This time, I decided to focus on measuring latency, to try to verify the claim that OFDMA significantly reduces it. The new testbed uses four octoScope STApals. These are teenie ASUS pico-ITX format computers, hosting an Intel AX200 Wi-Fi 6 station card. The STApal comes in two versions, one based on Ubuntu Linux, the other running Windows 10 Pro. The Windows STApal version was chosen because its AX200 driver at the time of this test has better up and downlink throughput performance than the Linux driver. (octoScope is working to correct this.)
OFDMA testbed – version 2
The testbed also has an octoScope Pal6. It can be used to add traffic load, but I use it to monitor channel congestion. More on that later.
The test is run by a Python script using the octoScope octoBox server platform to run the test and report results. octoScope’s multiperf traffic generator (using iperf3 at its core) provides traffic generation.
The basic test approach was inspired by the Make Wi-Fi Fast Project’s RRUL (Realtime Response Under Load) bufferbloat test tools. Specifically, I used Flent’s rtt_fair_var test ported to the octoScope test platform. All initial explorations, however, were performed using Flent natively, with a lot of help from Flent’s creator, Toke Høiland-Jørgensen. (Thanks, Toke!)
As an additional thanks to Toke, here’s a shameless plug and link to his PhD thesis “Bufferbloat and BeyondRemoving Performance Barriers in Real-World Networks”. It goes into more detail on Flent and Toke’s work on “ending the wifi anomaly”
Flent uses netperf, not iperf/iperf3 as its primary traffic generator/benchmarking tool. Netperf is (relatively) easily installed on Linux systems (you need to compile it), but not on Windows. Long story short, I ended up having Windows binaries of netperf and netserver created (download a zip of both files).
The rtt_fair_var test is a variant of the Realtime Response Under Load (RRUL) test that forms the heart of Flent’s test suite. For each test STA, the test simultaneously runs a ping at 200ms interval to measure round trip time (latency) and unlimited bandwidth TCP/IP traffic using netperf to flood the connection with traffic.
Shown below are some typical results from early tests using Flent. The DUT is an ASUS GT-AX11000, which at the time supported OFDMA for 5 GHz DL (download) traffic only. The first plot is the result with OFDMA off…
Flent RTT Fair test – CDF latency plot – BE priority – OFDMA OFF
…and the second plot is OFDMA on. A CDF (cumulative distribution function) plot shows the probability that a quantity (in this case ping or RTT) will be less than or equal to a value. 80% (0.8 on the graphs) is typically used as a benchmark, so we see that with OFDMA off average ping time for all three traffic pairs 80% of the time will be 30 ms or lower, increasing to ~ 120 ms or lower with OFDMA on. Not the result we’re looking for!