FYI/A: Bufferbloat 101

X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
Waveform.com website actually mentions that they do not recommend to use wireless for testing.
https://www.waveform.com/tools/bufferbloat

2. Make sure your connection to your router is at least as fast as your Internet connection itself

For example, if you have a gigabit fiber internet connection, make sure youā€™re connected over a gigabit ethernet cable, not over WiFi.

Our speed test wonā€™t be able to ā€œsaturateā€ your Internet connection to test for bufferbloat if youā€™re limited by your WiFi connection.
 
hwzlite

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,066
Reaction score
3,205
xiaofan said:
Waveform.com website actually mentions that they do not recommend to use wireless for testing.
https://www.waveform.com/tools/bufferbloat

2. Make sure your connection to your router is at least as fast as your Internet connection itself

For example, if you have a gigabit fiber internet connection, make sure youā€™re connected over a gigabit ethernet cable, not over WiFi.

Our speed test wonā€™t be able to ā€œsaturateā€ your Internet connection to test for bufferbloat if youā€™re limited by your WiFi connection.
Click to expand...






From Waveform FAQ:

Should I test bufferbloat over ethernet or WiFi?
Ethernet is generally a better choice.
Here's why:
Bufferbloat happens any time data packets traveling to and from the Internet reach a buffer. Buffers are used to control the flow of data whenever there is a change in the bandwidth of the conenction, or "the size of the pipe." So, for example, a buffer is needed when a fast router is connected to a slow cable modem.
If you're connecting over WiFi, the "size of the pipe" actually changes three times. So there are actually three buffers. Looking at the download path, there is:
  1. A buffer between your cable, DSL, LTE, satellite or fiber modem and your router;
  2. A buffer between your router and the WiFi access point (which may both be in the same physical box);
  3. A buffer between your phone or laptop's WiFi chipset and the actual computer.
By connecting with an ethernet cable, you're eliminating the second and third buffers, allowing you to specifically isolate your "modem to router bufferbloat." This is generally the most important buffer since it impacts every device on the network.

It's also worth testing WiFi bufferbloat on different devices. If you notice high bufferbloat, it may be worth upgrading either your access points and/or your devices.
Click to expand...

Once you properly SQM tuned the internet-ethernet link, the next course of action is to "saturate" the bufferbloat buried deep in multiple layers of queues in the wireless networking stack and tame it.
 
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
It is interesting cake-autorate does help to improve the Waveform.com bufferbloat testing score even though it is said to be not the right solution for my use case (to improve bufferbloat test score for wireless).

But waveform.com upload speed seems to be very low now -- so we may need to take the score improvement with a pinch of salt.

It does not seem to negatively impact the close-range OOkla SpeedTest much. That is the good thing.

Setup: OpenWRT virtual router (Intel N100 running Proxmox PVE 8.1) + ZTE BE7200 Pro+ AP
Client: Windows 11 laptop with Intel AX201 WiFi 6 adapter

Near range: A--> A+
https://www.waveform.com/tools/bufferbloat?test-id=6adf05a1-dca7-4043-ab5e-7ed499fdf166
Bash: 
PS C:\work\speedtest\ookla-speedtest-1.2.0-win64> .\speedtest.exe -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.96 ms   (jitter: 0.44ms, low: 2.64ms, high: 3.30ms)
    Download:   720.76 Mbps (data used: 759.3 MB)
                  8.10 ms   (jitter: 2.97ms, low: 3.62ms, high: 28.51ms)
      Upload:   441.02 Mbps (data used: 755.9 MB)
                  7.59 ms   (jitter: 3.63ms, low: 2.53ms, high: 49.13ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/54b28b15-a8f8-4524-8f74-e1eb73419e0a

After a thick wall: F ->C
https://www.waveform.com/tools/bufferbloat?test-id=2ab07a92-b270-4222-9e0d-4006812023ce

After one more wall: F-->C
https://www.waveform.com/tools/bufferbloat?test-id=7511580d-9e20-4d87-9d5c-aa28f23b9aa0

Earlier test results: A->A+; F-> B/C; F --> C/F.
https://github.com/lynxthecat/cake-autorate/issues/274#issuecomment-2065634714
 
Last edited:
hwzlite

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,066
Reaction score
3,205
Finally, i have succumbed to FOMO, upgraded M1 500Mbps to 1Gbps. :giggle:

Here are the stats -

Wired:
OpenWRT Soft Router -
Have to fallback resort to using fq_codel with simplest_tbf SQM , cos CAKE SQM cpu overload one of the n100 cores. And also speed limited both upload/down tuned to 0.9Gbps to keep bufferbloat at bay.




Wireless:
Xiaomi AX3000T(Openwrt) built-in TXQS/ATF/AQL features , configured as Dumb AP operate at DFS channel 64 @80Mhz, tweaked with "aql_txq_limit = 1500" setting and test with Xiaomi Poco X5 Pro (client).


Heat Map generated by WiFi Analyzer and Surveyor :
Screenshot-2024-04-22-145326.png

X = DumpAP


LR (-33dBm) -

BR3 (-46dBm) -

MB (-52dBm) -

Store (-53dBm) -

Kitchen (-57dBm) -

BR2 (-62dBm) -

Bath1 (-66dBm) -

Bath2 (-76dBm) -


Agaration: anything ā‰¤ -55dBM signal strength is the sweet spot for "A/A+" rating. :grin:
 
Last edited:
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
hwzlite said:
Finally, i have succumbed to FOMO, upgraded M1 500Mbps to 1Gbps. :giggle:

Here are the stats -

Wired:
OpenWRT Soft Router -
Have to fallback resort to using fq_codel with simplest_tbf SQM , cos CAKE SQM cpu overload one of the n100 cores. And also speed limited both upload/down tuned to 0.9Gbps to keep bufferbloat at bay.
Click to expand...

Hmm, that is interesting, by right Intel N100 should have no issues with CAKE SQM.

But probably that is because you run more stuff on your Intel N100 PVE envioronment.
 
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
hwzlite said:
Wireless:
Xiaomi AX3000T(Openwrt) built-in TXQS/ATF/AQL features , configured as Dumb AP operate at DFS channel 64 @80Mhz, tweaked with "aql_txq_limit = 1500" setting and test with Xiaomi Poco X5 Pro (client).

Heat Map generated by WiFi Analyzer and Surveyor :
Screenshot-2024-04-22-145326.png

X = DumpAP


LR (-33dBm) -

BR3 (-46dBm) -

MB (-52dBm) -

Store (-53dBm) -

Kitchen (-57dBm) -

BR2 (-62dBm) -

Bath1 (-66dBm) -

Bath2 (-76dBm) -

Agaration: anything ā‰¤ -55dBM signal strength is the sweet spot for "A/A+" rating. :grin:
Click to expand...

Nice results and good correlations of Bufferbloat score vs signal strength.

-76dBm is quite weak signal already in Singapore context. So I think "F" results are kind of expected. "B" for -62dBm and "C" for -66dBM are quite respectable as well.

I expect results using other wireless routers which do not support TXQS/ATF/AQL will be worse.

Reference:
++++++++++++++
https://dongknows.com/wi-fi-signal-strength-dbm-explained/
Generally, meaningful dBm values on the receiving end range from -10 (strongest signals) to -90 (weakest). Here are the general dBm readouts:

North of -30 dBm: Too good to be true, or signal saturation (not good).
-30 dBm: The best possible.
-50 dBm: Excellent signals.
-60 dBm: Very good signals.
-65 dBm: Good, reliable signals. Up to now, youā€™ve always had full bars.
-70 dBm: This is the threshold where you might have lost a signal bar and are about to lose another, if not already. But the connection is still solid.
-75 dBm: This is where things start getting problematic, but the connection might still be useable.
-80 dBm: Borderline useless ā€” you barely have just one bar.
-90 dBm: The signal is really weak, (almost) impossible to connect to.
South of -90 dBm: Forget about it.
+++++++++++++
 
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
@hwzlite

I am not so sure if M1 has better routing to waveform.com test servers or not.

The reason why I ask is that you seem to have decent wired and wireless speed from waveform.com bufferbloat test.

Could you share the mtr results to waveform.com? (but the test server may be different).
Bash: 
                                                 My traceroute  [v0.95]
OpenWrt (219.xx.xxx.xx) -> waveform.com (104.26.1.157)                                         2024-04-22T19:04:07+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                               Packets               Pings
 Host                                                                        Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. bb219-xx-xxx-254.singnet.com.sg                                           0.0%    43    1.4   2.2   1.2  14.9   2.0
 2. 165.21.193.22                                                             0.0%    42    2.1   2.3   2.0   4.1   0.3
 3. 165.21.193.21                                                             0.0%    42    1.9   3.1   1.4  27.5   4.7
 4. 165.21.138.245                                                            0.0%    42    2.2   2.8   1.6  19.1   2.8
 5. SN-SINQT1-BO403-ae1.singnet.com.sg                                        0.0%    42    2.1   1.9   1.6   2.6   0.2
 6. ip-202-147-32-136.asianetcom.net                                          0.0%    42   43.2  49.1  42.8 127.8  17.1
 7. unknown.telstraglobal.net                                                70.7%    42   48.9  50.8  48.5  61.5   4.1
 8. 172.70.146.3                                                              9.5%    42   48.3  50.2  47.8  70.0   5.6
 9. 104.26.1.157                                                              0.0%    42   43.0  43.0  42.7  43.2   0.1

Wireless speedtest of Asus RT-AX86U and Acer Windows 11 laptop (Asus Adaptive QoS ON, automatic bandwidth setting, default Learn from Home profile).

OOKla SpeedTest
Bash: 
PS C:\work\speedtest\ookla-speedtest-1.2.0-win64> .\speedtest.exe -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     3.45 ms   (jitter: 0.09ms, low: 3.31ms, high: 3.55ms)
    Download:   650.03 Mbps (data used: 1.1 GB)
                 12.81 ms   (jitter: 14.19ms, low: 3.96ms, high: 309.94ms)
      Upload:   450.86 Mbps (data used: 633.1 MB)
                 12.64 ms   (jitter: 11.36ms, low: 4.37ms, high: 233.55ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/9a875163-eb76-4fb1-952d-7921395d7c93

Waveform.com wireless test -- speed is quite low for upload, even though the scores are "A".
https://www.waveform.com/tools/bufferbloat?test-id=e7393a06-fe22-4730-9a05-3a20b2ffeb9e
https://www.waveform.com/tools/bufferbloat?test-id=4f89ed98-7b16-4d2c-9a52-b694adc027e7
 
Last edited:
Mach3.2

Mach3.2

Great Supremacy Member
HWZ Insider
Joined
Apr 8, 2011
Messages
72,500
Reaction score
2,478
Bash: 
xxxxxx (10.80.30.11) -> waveform.com (172.67.74.141)                                2024-04-22T20:22:43+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                    Packets               Pings
 Host                                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS???    _gateway                                              0.0%   166    0.1   0.1   0.1   0.4   0.1
 2. AS4773   1.192.199.112.unknown.m1.com.sg                       1.2%   166    2.7   3.0   1.7   9.0   1.1
 3. AS4773   204.246.65.202.unknown.m1.com.sg                      0.6%   166    2.8   3.0   1.7  14.6   1.7
 4. AS4773   205.246.65.202.unknown.m1.com.sg                      0.6%   165    3.8   3.7   1.8  40.9   4.3
 5. AS4773   37.246.65.202.unknown.m1.com.sg                       1.2%   165    5.1   4.9   2.5  19.4   3.0
 6. AS4773   221.246.65.202.unknown.m1.com.sg                      1.2%   165    3.8   5.3   2.9  32.2   3.8
 7. AS13335  162.158.160.220                                       0.0%   165    3.4  12.3   3.3  52.6  10.5
 8. AS13335  162.158.104.3                                        90.9%   165    7.2  11.4   3.6  38.6  10.6
 9. AS13335  172.67.74.141                                         1.8%   165    4.1   4.4   3.3   9.0   1.0


xxxxxx (2401:7400:c80b:xxxx::xx:xx) -> 2606:4700::6810:3d08 (2606:4700::6810:3d08)  2024-04-22T20:24:43+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                    Packets               Pings
 Host                                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   101    0.1   0.1   0.1   0.4   0.1
 2. (waiting for reply)
 3. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   100    3.0   2.5   1.8   4.4   0.4
 4. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   100    3.0   3.4   2.3  35.1   3.2
 5. AS4773   2401:7400:0:1::                                       0.0%   100    4.5   3.5   2.4  12.8   1.4
 6. AS4773   2401:7400:0:6::1                                      0.0%   100   18.0   9.6   3.1  23.7   5.6
 7. AS13335  2400:cb00:35:200::194                                 0.0%   100   25.0  11.0   3.7  48.7  10.0
 8. AS13335  2400:cb00:410:3::                                     0.0%   100    4.6  10.8   3.8  97.6  17.3
 9. AS13335  2606:4700::6810:3d08                                  0.0%   100    4.0   3.9   3.0   4.3   0.3


xxxxxx (2401:7400:c80b:xxxx::xx:xx) -> 2606:4700:3036::ac43:bbea (2606:4700:3036::ac43:bbea)  2024-04-22T20:25:40+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                              Packets               Pings
 Host                                                                       Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   101    0.1   0.1   0.1   0.4   0.1
 2. (waiting for reply)
 3. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   101    3.0   2.7   2.0  13.3   1.1
 4. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   100    3.5   4.3   2.3  84.9   8.5
 5. AS4773   2401:7400:0:1::                                                 0.0%   100    2.8   4.5   2.4  31.7   4.3
 6. AS4773   2401:7400:0:6::1                                                0.0%   100    3.8   4.0   3.1  15.2   1.4
 7. AS13335  2400:cb00:35:200::194                                           0.0%   100    4.4   9.2   3.6  45.9   8.4
 8. AS13335  2400:cb00:35:3::                                                2.0%   100    4.6   6.2   3.9  27.7   4.4
 9. AS13335  2606:4700:3036::ac43:bbea                                       0.0%   100    3.5   4.2   3.1  36.9   3.3

I pulled 2 IPv6 addresses while running the bufferbloat test, also ran MTR against those addresses.

M1 peers directly with Cloudflare so connection is pretty decent; Singtel doesn't, and for reasons only known to SingTel, your traffic trombones at HK via Telstra IP transit. :)


xiaofan said:
speed is quite low for upload, even though the scores are "A".
Click to expand...
I think this one is pretty self explanatory; not much bufferbloat becasue your link is not saturated.
 
Last edited:
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
Mach3.2 said:
I think this one is pretty self explanatory; not much bufferbloat becasue your link is not saturated.
Click to expand...

That is exactly the problem I am facing -- I can not trust Waveform.com bufferbloat test results since Singtel has bad route to Waveform.com.

And then there are no better public test servers to carry out similar tests...
 
hwzlite

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,066
Reaction score
3,205
Mach3.2 said:
Bash: 
xxxxxx (10.80.30.11) -> waveform.com (172.67.74.141)                                2024-04-22T20:22:43+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                    Packets               Pings
 Host                                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS???    _gateway                                              0.0%   166    0.1   0.1   0.1   0.4   0.1
 2. AS4773   1.192.199.112.unknown.m1.com.sg                       1.2%   166    2.7   3.0   1.7   9.0   1.1
 3. AS4773   204.246.65.202.unknown.m1.com.sg                      0.6%   166    2.8   3.0   1.7  14.6   1.7
 4. AS4773   205.246.65.202.unknown.m1.com.sg                      0.6%   165    3.8   3.7   1.8  40.9   4.3
 5. AS4773   37.246.65.202.unknown.m1.com.sg                       1.2%   165    5.1   4.9   2.5  19.4   3.0
 6. AS4773   221.246.65.202.unknown.m1.com.sg                      1.2%   165    3.8   5.3   2.9  32.2   3.8
 7. AS13335  162.158.160.220                                       0.0%   165    3.4  12.3   3.3  52.6  10.5
 8. AS13335  162.158.104.3                                        90.9%   165    7.2  11.4   3.6  38.6  10.6
 9. AS13335  172.67.74.141                                         1.8%   165    4.1   4.4   3.3   9.0   1.0


xxxxxx (2401:7400:c80b:xxxx::xx:xx) -> 2606:4700::6810:3d08 (2606:4700::6810:3d08)  2024-04-22T20:24:43+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                    Packets               Pings
 Host                                                             Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   101    0.1   0.1   0.1   0.4   0.1
 2. (waiting for reply)
 3. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   100    3.0   2.5   1.8   4.4   0.4
 4. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                              0.0%   100    3.0   3.4   2.3  35.1   3.2
 5. AS4773   2401:7400:0:1::                                       0.0%   100    4.5   3.5   2.4  12.8   1.4
 6. AS4773   2401:7400:0:6::1                                      0.0%   100   18.0   9.6   3.1  23.7   5.6
 7. AS13335  2400:cb00:35:200::194                                 0.0%   100   25.0  11.0   3.7  48.7  10.0
 8. AS13335  2400:cb00:410:3::                                     0.0%   100    4.6  10.8   3.8  97.6  17.3
 9. AS13335  2606:4700::6810:3d08                                  0.0%   100    4.0   3.9   3.0   4.3   0.3


xxxxxx (2401:7400:c80b:xxxx::xx:xx) -> 2606:4700:3036::ac43:bbea (2606:4700:3036::ac43:bbea)  2024-04-22T20:25:40+0800
Keys:  Help   Display mode   Restart statistics   Order of fields   quit
                                                                              Packets               Pings
 Host                                                                       Loss%   Snt   Last   Avg  Best  Wrst StDev
 1. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   101    0.1   0.1   0.1   0.4   0.1
 2. (waiting for reply)
 3. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   101    3.0   2.7   2.0  13.3   1.1
 4. AS4773   0.0.4.7.1.0.4.2.ip6.arpa                                        0.0%   100    3.5   4.3   2.3  84.9   8.5
 5. AS4773   2401:7400:0:1::                                                 0.0%   100    2.8   4.5   2.4  31.7   4.3
 6. AS4773   2401:7400:0:6::1                                                0.0%   100    3.8   4.0   3.1  15.2   1.4
 7. AS13335  2400:cb00:35:200::194                                           0.0%   100    4.4   9.2   3.6  45.9   8.4
 8. AS13335  2400:cb00:35:3::                                                2.0%   100    4.6   6.2   3.9  27.7   4.4
 9. AS13335  2606:4700:3036::ac43:bbea                                       0.0%   100    3.5   4.2   3.1  36.9   3.3

I pulled 2 IPv6 addresses while running the bufferbloat test, also ran MTR against those addresses.

M1 peers directly with Cloudflare so connection is pretty decent; Singtel doesn't, and for reasons only known to SingTel, your traffic trombones at HK via Telstra IP transit. :)



I think this one is pretty self explanatory; not much bufferbloat becasue your link is not saturated.
Click to expand...


Thanks for the head-out :cool:

More DPI details from ntopng, tested from client which IPv6 taking precedence over IPv4 :

Screenshot-2024-04-22-225159.png
 
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
Mach3.2 said:
the lazy way would be to set the bandwidth limiter to ~930Mbps for both upload and download.

Wireless is really best effort liao
Click to expand...

Looks like that is the right way to go. I will not use the plain bandwidth limiter though but rather SQM CAKE. Due to Singtel ZTE ONT issue, it is better to limit the upload speed to be below 800Mbps, I choose to go with 700Mbps.

The following works well for me with wired connection. And as you mentioned, wireless is just best effort. I may still want to get an OpenWRT capable WiFi 6 AP later though.
Bash: 
root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option enabled '1'
        option interface 'eth0'
        option download '900000'
        option upload '700000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'ethernet'
        option debug_logging '0'
        option verbosity '5'
        option overhead '44'

But my interest now is not really to sort out the bufferbloat issue as I do not really have bufferbloat issue anyway for my use cases. Rather I am interested in how to carry out mearningful bufferbloat tests using tools like crusader or OOkla Speedtest now that I will just give up on Waveform.com.

I am also not giving up on cake-autorate itself. It seems to be an interesting project even though it is not suitable for my use case.

My involvement of Open Source projects are often like that -- I become more interested in the tool or the development library itself than the original goal (to sort out an issue). :LOL:
 
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
hwzlite said:
Yeah, CAKE SQM is a single-threaded service that overloads single core, tried with futile attempts on irqbalance/smp_affinity.... :confused:
Click to expand...

hwzlite said:
Have to fallback resort to using fq_codel with simplest_tbf SQM , cos CAKE SQM cpu overload one of the n100 cores. And also speed limited both upload/down tuned to 0.9Gbps to keep bufferbloat at bay.
Click to expand...

Ineed you are right, I just checked and indeed CAKE SQM saturated one of the two virtual core assigned to OpenWRT (using htop) during download stage of OOkla SpeedTest. Then I also checked PVE 8.1 host and indeed one of the 4 physical cores reached 100% for a short time and then other core also reached 100% for a short time.

Bash: 
root@OpenWrt:~# ./speedtest_ookla/speedtest

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.25 ms   (jitter: 0.05ms, low: 1.21ms, high: 1.31ms)
    Download:   842.53 Mbps (data used: 381.4 MB)
                  1.36 ms   (jitter: 0.32ms, low: 0.93ms, high: 2.29ms)
      Upload:   656.42 Mbps (data used: 753.0 MB)
                  1.25 ms   (jitter: 0.19ms, low: 1.04ms, high: 11.24ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/58d15017-cf0d-43f1-86f0-c770defc8c02
root@OpenWrt:~# cat /etc/config/sqm

config queue 'eth1'
        option enabled '1'
        option interface 'eth0'
        option download '900000'
        option upload '700000'
        option qdisc 'cake'
        option script 'piece_of_cake.qos'
        option linklayer 'ethernet'
        option debug_logging '0'
        option verbosity '5'
        option overhead '44'
 
Last edited:
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
Just for fun. CAKE SQM at 2Gbps on virtual OpenWRT router with two virtual CPU core on the Intel N00 PVE 8.1 host.

Initially I tried 8Gbps download/upload and it does not work properly. I think the two virtual CPUs may be able to handle 2.5Gbps up/down or even 3Gbps to 4Gbps.

iperf3 server -- LxC container 192.168.50.15
OpenWRT virtual router WAN: 192.168.50.168
OpenWRT virtual router LAN: 192.168.48.1
iperf3 client -- LxC container 192.168.48.6

Bash: 
root@OpenWrt:~# service sqm restart
SQM: Stopping SQM on eth0
SQM: Starting SQM script: piece_of_cake.qos on eth0, in: 2000000 Kbps, out: 2000000 Kbps
SQM: piece_of_cake.qos was started on eth0 successfully

root@OpenWrt:~# exit
Connection to 192.168.48.1 closed.

root@ubuntu2204ct21:~# iperf3 -c 192.168.50.15 -R
Connecting to host 192.168.50.15, port 5201
Reverse mode, remote host 192.168.50.15 is sending
[  5] local 192.168.48.6 port 52128 connected to 192.168.50.15 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   219 MBytes  1.84 Gbits/sec               
[  5]   1.00-2.00   sec   192 MBytes  1.61 Gbits/sec               
[  5]   2.00-3.00   sec   204 MBytes  1.71 Gbits/sec               
[  5]   3.00-4.00   sec   203 MBytes  1.71 Gbits/sec               
[  5]   4.00-5.00   sec   211 MBytes  1.77 Gbits/sec               
[  5]   5.00-6.00   sec   220 MBytes  1.85 Gbits/sec               
[  5]   6.00-7.00   sec   206 MBytes  1.73 Gbits/sec               
[  5]   7.00-8.00   sec   192 MBytes  1.61 Gbits/sec               
[  5]   8.00-9.00   sec   200 MBytes  1.67 Gbits/sec               
[  5]   9.00-10.00  sec   209 MBytes  1.75 Gbits/sec               
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.04  sec  2.01 GBytes  1.72 Gbits/sec   26             sender
[  5]   0.00-10.00  sec  2.01 GBytes  1.73 Gbits/sec                  receiver

iperf Done.
root@ubuntu2204ct21:~# iperf3 -c 192.168.50.15 
Connecting to host 192.168.50.15, port 5201
[  5] local 192.168.48.6 port 43732 connected to 192.168.50.15 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   209 MBytes  1.75 Gbits/sec    1    969 KBytes     
[  5]   1.00-2.00   sec   214 MBytes  1.79 Gbits/sec    0   1.10 MBytes     
[  5]   2.00-3.00   sec   194 MBytes  1.63 Gbits/sec    0   1.22 MBytes     
[  5]   3.00-4.00   sec   196 MBytes  1.65 Gbits/sec    0   1.34 MBytes     
[  5]   4.00-5.00   sec   196 MBytes  1.65 Gbits/sec    1   1.01 MBytes     
[  5]   5.00-6.00   sec   199 MBytes  1.67 Gbits/sec    0   1.15 MBytes     
[  5]   6.00-7.00   sec   194 MBytes  1.63 Gbits/sec    0   1.26 MBytes     
[  5]   7.00-8.00   sec   195 MBytes  1.64 Gbits/sec    0   1.37 MBytes     
[  5]   8.00-9.00   sec   194 MBytes  1.63 Gbits/sec    0   1.47 MBytes     
[  5]   9.00-10.00  sec   198 MBytes  1.66 Gbits/sec    0   1.57 MBytes     
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.94 GBytes  1.67 Gbits/sec    2             sender
[  5]   0.00-10.05  sec  1.94 GBytes  1.66 Gbits/sec                  receiver

iperf Done.
 
hwzlite

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,066
Reaction score
3,205
hwzlite said:
Finally, i have succumbed to FOMO, upgraded M1 500Mbps to 1Gbps. :giggle:

Here are the stats -

Wired:
OpenWRT Soft Router -
Have to fallback resort to using fq_codel with simplest_tbf SQM , cos CAKE SQM cpu overload one of the n100 cores. And also speed limited both upload/down tuned to 0.9Gbps to keep bufferbloat at bay.




Wireless:
Xiaomi AX3000T(Openwrt) built-in TXQS/ATF/AQL features , configured as Dumb AP operate at DFS channel 64 @80Mhz, tweaked with "aql_txq_limit = 1500" setting and test with Xiaomi Poco X5 Pro (client).


Heat Map generated by WiFi Analyzer and Surveyor :
Screenshot-2024-04-22-145326.png

X = DumpAP


LR (-33dBm) -

BR3 (-46dBm) -

MB (-52dBm) -

Store (-53dBm) -

Kitchen (-57dBm) -

BR2 (-62dBm) -

Bath1 (-66dBm) -

Bath2 (-76dBm) -


Agaration: anything ā‰¤ -55dBM signal strength is the sweet spot for "A/A+" rating. :grin:
Click to expand...

Ideally for best results, both AP and client should have TXQS/ATF/AQL supported mac80211 linux drivers working in both ways.

Here are the results from the TXQS/ATF/AQL capable wireless client - FENVI AX1800 External USB (Mediatek MT7921au) connected to ThinkPad E485 booted from Kali Linux Live Boot USB :

LR (-32dBm) -


BR3 (-45dBm) -


Store (-50dBm) -


MB (-52dBm) -


Kitchen (-53dBm) -


BR2 (-63dBm)


Bath1 (-60dBm) -


Bath2 (-73dBm) -

Note: signal level verified by iwconfig.

If desperate obsession with A+ rating, might be able to further tweak the client with "aql_txq_limit = 1500" setting :grin:
 
Last edited:
X

xiaofan

High Supremacy Member
HWZ Insider
Joined
Sep 16, 2018
Messages
34,937
Reaction score
11,634
hwzlite said:
Ideally for best results, both AP and client should have TXQS/ATF/AQL supported mac80211 linux drivers working in both ways.

Here are the results from the TXQS/ATF/AQL capable wireless client - FENVI AX1800 External USB (Mediatek MT7921au) connected to ThinkPad E485 booted from Kali Linux Live Boot USB :

LR (-32dBm) -

Bath2 (-73dBm) -

Note: signal level verified by iwconfig

If desperate obsession with A+ rating, might be able to further tweak the client with "aql_txq_limit = 1500" setting :grin:
Click to expand...

Very nice results.

Would you please also run OOkla SpeedTest CLI as a comparison, just for Living room and Bath 2. Thanks.

As mentioned earlier, unfortunately Singtel has not good connection with Waveform.com and I can not really use it for testing.
 
hwzlite

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,066
Reaction score
3,205
xiaofan said:
Very nice results.

Would you please also run OOkla SpeedTest CLI as a comparison, just for Living room and Bath 2. Thanks.

As mentioned earlier, unfortunately Singtel has not good connection with Waveform.com and I can not really use it for testing.
Click to expand...

Have to resort booting to Debian Live ISO (debian-live-12.5.0-amd64-xfce.iso) as speedtest-cli does not have package support for Kali Linux Live.

And it seems that there is some form of download speed capping/throttling at <270Mbps which I validated only both CTCSCI TECH LTD (selected for testing) & Singtel servers can reach higher speed.

Code: 
user@debian:~$ speedtest --selection-details

   Speedtest by Ookla

Selecting server:
       7311:   2.65 ms; M1 Limited - Singapore
      23467:   3.19 ms; PT. Indonesia Comnets Plus - Singapore
      13058:   2.65 ms; PT Indosat Tbk - Singapore
      62216:   3.18 ms; CTCSCI TECH LTD - Singapore
      62713:  39.09 ms; Neunix Media Data - Singapore
       4235:   2.42 ms; StarHub Ltd - Singapore
        367:   3.08 ms; NewMedia Express - Singapore
       5935:   5.94 ms; MyRepublic Singapore - Singapore
      13623:   4.15 ms; Singtel - Singapore
       7556:   2.00 ms; PT FirstMedia - Singapore
       2054:   3.21 ms; ViewQwest - Singapore

      Server: PT FirstMedia - Singapore (id: 7556)
         ISP: M1
Idle Latency:     2.64 ms   (jitter: 0.20ms, low: 2.44ms, high: 2.70ms)
    Download:   266.61 Mbps (data used: 342.7 MB)                                                 
                  5.02 ms   (jitter: 1.10ms, low: 2.95ms, high: 10.78ms)
      Upload:   698.62 Mbps (data used: 601.0 MB)                                                 
                 11.77 ms   (jitter: 3.49ms, low: 3.19ms, high: 27.97ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/28afb228-f861-4075-a53e-d933b43cba57
user@debian:~$


SPEEDTEST CLI :

LR (-29dBm)


Bath2 (-74dBm)



LOL test until prompted this limits : :s22:
user@debian:~$ speedtest -s 62216
[2024-04-29 00:17:46.397] [error] Limit reached:

Speedtest CLI. Too many requests received. To maintain a fair and stable environment, please review and adjust the frequency of your requests.
Click to expand...
 
Last edited:
You must log in or register to reply here.
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ Forums. Forum members and moderators are responsible for their own posts. Please refer to our Community Guidelines and Standards and Terms and Conditions for more information.
Top