FYI/A: Bufferbloat 101

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
Impact for bufferbloat on uploads are more of you noticing it taking longer to load websites coz the request to the server to send you data is delayed. If doing videoconferencing, other ppl may notice you are lagging since your vide stream to them is not real time but buffered. Bandwidth isnt always everything.

That said, the issue might be something that is tolerable to some and not to others, coz the feeling of lag is still somewhat subjective, although measurable.

I would think a badly implemented QoS/SQM solution is far more harmful than that any supposed harm done on underpowered home routers on gigabit lines, coz the vast majority of ppl dont saturate their connections all the time. Personally I would rather sacrifice some underutilized bandwidth for better latency.

Good point. Latency is more important than pure bandwidth which is usually only useful in showing off Ookla SpeedTest results. I admit I am also guilty of that. :s13:

I was attributing occasional slow website loading to bad ping on SingTel network. But that was mainly attributed to bad routing of SingTel, compared to M1/Viewquest/MyRepublic. Bufferbloat was seldom mentioned in the discussions.

So which one is more damaging, bad ISP routing, or ONT/ONR/Router bufferbloat? Or are they intertwined?
 

Darkshadows

Arch-Supremacy Member
Joined
Aug 1, 2008
Messages
14,811
Reaction score
4,602
Any of you use ASUS RT-AC86U router? Can share what settings you guys insert into this router or similar models? I'm using M1 500Mbps plan. The settings in the router are too cheem to me as I'm not IT savvy. Just want to key in a good setting and leave it. TIA!
 

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
Any of you use ASUS RT-AC86U router? Can share what settings you guys insert into this router or similar models? I'm using M1 500Mbps plan. The settings in the router are too cheem to me as I'm not IT savvy. Just want to key in a good setting and leave it. TIA!

Nothing much to set for normal users for M1 500Mbps plan. Just set up the wireless password and then use.

But if you encounter issues, you can ask in the dedicated RT-AC86U thread.
https://forums.hardwarezone.com.sg/...-clinic-4/asus-rt-ax86u-rt-ax82u-6315347.html
 
Last edited:

uncle_josh

Master Member
Joined
Jun 16, 2018
Messages
2,717
Reaction score
511
There isn't any settings for home routers stock firmware. But you can try QoS prioritizing by Application (Skype, Http) or Port Range (P2P)

If your router support DD-WRT, then you glad to know that DD-WRT firmware have this SQM feature.
bufferbloat-kill_dd-wrt_screenshot.png


For more details, refer to here : How turn on SQM on DD-WRT firmware


Any of you use ASUS RT-AC86U router? Can share what settings you guys insert into this router or similar models? I'm using M1 500Mbps plan. The settings in the router are too cheem to me as I'm not IT savvy. Just want to key in a good setting and leave it. TIA!
 

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
:D me getting obsessive :D, here was the result under optimized pure CAKE mode which by design works best without HTB/HFSC scheduler.
(For some strange reason, the developer of DD-WRT did not make a standalone Option for CAKE , so have to tweak it via CLI ) .

http://www.dslreports.com/speedtest/66656741

It seems to me you are using wired to test and your speed is on the low side for M1 500Mbps. Just wondering if this is because of using CAKE and this is the trade-off to make. The speed in your first post is even lower.

What will be the results of not using CAKE or other bufferbloat countermeasures, just normal settings for your DIR-868L?

I do understand latency can be more important than speed in many use cases. Just curious of the comparisons.
 
Last edited:

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,018
Reaction score
3,118
It seems to me you are using wired to test and your speed is on the low side for M1 500Mbps. Just wondering if this is because of using CAKE and this is the trade-off to make. The speed in your first post is even lower.

What will be the results of not using CAKE or other bufferbloat countermeasures, just normal settings for your DIR-868L?

I do understand latency can be more important than speed in many use cases. Just curious of the comparisons.

Yeah, sacrificing throughput for godsend low latency :)
Will update here again when I got the chance to update DD-WRT to latest firmware with factory default setting.
 

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
its the zte ont, not the router. i use mikrotik hapac2 which is different from xiaofan. this test is via the router tho. i cant test directly from ont at the moment since ppl need to WFH in my home

btw my this upload bufferbloat begins once upload speed exceeds around 400mbps for me. verified this by limiting upload bw manually on my pc. my mikrotik cant handle qos at such high speeds :)

https://mikrotik.com/product/hap_ac2
Just curious, the MikroTik hAP ac2 is more meant to be running as an AP. Are you using it as an router?

Then for direct ONT connection, how do you deal with Singtel VLAN, if you are using a Windows 10 laptop?
 

miloaisdino

Senior Member
Joined
Mar 25, 2016
Messages
843
Reaction score
26
https://mikrotik.com/product/hap_ac2
Just curious, the MikroTik hAP ac2 is more meant to be running as an AP. Are you using it as an router?

Then for direct ONT connection, how do you deal with Singtel VLAN, if you are using a Windows 10 laptop?

its smb/soho router. it handles gigabit well and i also run ipsec vpn through it, ipsec throughput about 170mbps which is considered good for a home router. (i did test without the router instead but bufferbloat was even worse, got "F" grade)

im running it at 900Mhz (quad core), which is already faster than routers eg asus ac2600 etc
 
Last edited:

bert64

Senior Member
Joined
Jan 20, 2020
Messages
1,026
Reaction score
536
Okay, let's say certain ONT on certain ISP, and some routers, may have issues with bufferbloat score of "C" or "D", is there a real problem in reality?

Take example of SingTel new ZTE ONT, the upload gets bufferbloat issues. Is there a real impact? In most use cases, download data are much more than the upload. And take note upload speed is still good in the tests.

From what I read, QoS and SQM are usually doing more damages to symmetrical 1Gbps and above Fibre internet services, as typical home routers can not handle QoS/SQM properly at 1Gbps, including Ubiquiti/MikroTik.

Yes, bufferbloat was a thing in the days of dialup.. You'd have a (typically) 64kb transmit buffer for a 33.6kbps (max) modem, so it would take nearly 20 seconds to empty the contents of the buffer. It was designed this way to improve throughput, because operating systems of the time (especially windows) could not respond quickly enough to send more data once the modem was ready to transmit it.

It happened on things like ADSL too, as many providers would give you an extremely weak uplink it may still take a couple of seconds to clear the buffers.

For a 1gbps connection, the buffers are tiny relative to the speed of the line - they should get cleared very quickly. Similarly, your internal devices probably aren't capable of throwing data at the router faster than the upstream line can accept it.

What's more likely a cause of latency, is the router itself not being able to handle the traffic rate. Adding extra complexity like QoS, NAT, firewall rules etc will exacerbate the situation and slow it down further. Remember these devices are built to a price point, with corners cut to make the price more attractive. You'll get better performance with a high end layer 3 switch, but most home users won't pay the price or tolerate the fan noise.
 

bert64

Senior Member
Joined
Jan 20, 2020
Messages
1,026
Reaction score
536
its smb/soho router. it handles gigabit well and i also run ipsec vpn through it, ipsec throughput about 170mbps which is considered good for a home router. (i did test without the router instead but bufferbloat was even worse, got "F" grade)

im running it at 900Mhz (quad core), which is already faster than routers eg asus ac2600 etc

What matters more than the core clockrate, is the quality of the network interfaces, wether there is any hardware offload and the complexity of your configuration (including if your config is causing the hardware offload to not be used).

The Mikrotik page you linked also has benchmarks which highlight some of the bottlenecks. Direct bridging is fastest, followed by direct unfiltered routing, and smaller packets eat up a lot more resources than large packets. As you add extra complexity such as filter rules, state tracking/NAT, QoS etc, performance will drop. You need to significantly overspec the router to handle worst case conditions, or a flood of small packets (interactive applications often generate smaller packets) could drag overall performance down.

Best performance will be had using direct routed IPv6, no nat or filtering.
 

miloaisdino

Senior Member
Joined
Mar 25, 2016
Messages
843
Reaction score
26
What matters more than the core clockrate, is the quality of the network interfaces, wether there is any hardware offload and the complexity of your configuration (including if your config is causing the hardware offload to not be used).

The Mikrotik page you linked also has benchmarks which highlight some of the bottlenecks. Direct bridging is fastest, followed by direct unfiltered routing, and smaller packets eat up a lot more resources than large packets. As you add extra complexity such as filter rules, state tracking/NAT, QoS etc, performance will drop. You need to significantly overspec the router to handle worst case conditions, or a flood of small packets (interactive applications often generate smaller packets) could drag overall performance down.

Best performance will be had using direct routed IPv6, no nat or filtering.

yes, but the fact remains that the oversaturated buffers are on the ont and not my router, since bufferbloat waa ridiculously bad (thousands of ms) with direct connection to ont (which i have since tested yesterday).

i am happy with the performance of my router as it does not slow down even with torrent and i set the routing rules to fasttrack connections not destined for my vpn

in fact the zte ont is less powerful than the mikrotik. just like huawei onrs, they probably do not run in "true layer 2 bridge" mode but instead run a layer 3 bridge with DHCP relay, zte underpowered processor can easily cause the bottleneck here.
 
Last edited:

hwzlite

Master Member
Joined
Jan 27, 2007
Messages
3,018
Reaction score
3,118
yes, but the fact remains that the oversaturated buffers are on the ont and not my router, since bufferbloat waa ridiculously bad (thousands of ms) with direct connection to ont (which i have since tested yesterday).

Can't agree more.

Examples: Results from those untamed top-tier 1GB plan, upload bufferload (lag) under load can reach as high ~900ms :s22:

 
Last edited:

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
its the zte ont, not the router. i use mikrotik hapac2 which is different from xiaofan. this test is via the router tho. i cant test directly from ont at the moment since ppl need to WFH in my home

btw my this upload bufferbloat begins once upload speed exceeds around 400mbps for me. verified this by limiting upload bw manually on my pc. my mikrotik cant handle qos at such high speeds :)

I just setup bandwidth limiter in the RT-AX82U router (QoS function) to limit my Mac Mini M1 upload speed to below 300Mbps, indeed bufferbloat gets "A" grade now.
http://www.dslreports.com/speedtest/66673766
66673766.png


Interesting, just enable QoS and bandwidth limitter, the grade gets "A" already.

http://www.dslreports.com/speedtest/66673827
66673827.png


Or probably today is a good day, without any QoS, I still get an "A".
http://www.dslreports.com/speedtest/66673848
66673848.png
 
Last edited:

miloaisdino

Senior Member
Joined
Mar 25, 2016
Messages
843
Reaction score
26
I just setup bandwidth limiter in the RT-AX82U router (QoS function) to limit my Mac Mini M1 upload speed to below 300Mbps, indeed bufferbloat gets "A" grade now.
http://www.dslreports.com/speedtest/66673766
66673766.png


Interesting, just enable QoS and bandwidth limitter, the grade gets "A" already.

http://www.dslreports.com/speedtest/66673827
66673827.png


Or probably today is a good day, without any QoS, I still get an "A".
http://www.dslreports.com/speedtest/66673848
66673848.png

yep. it follows my testing showing that if upload is kept below 400mbps, there is less bufferbloat (but still not A+). but its unfortunately unacceptable to me as my family member needs to upload large video files periodically, 400mbps is way too slow
 

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
yep. it follows my testing showing that if upload is kept below 400mbps, there is less bufferbloat (but still not A+). but its unfortunately unacceptable to me as my family member needs to upload large video files periodically, 400mbps is way too slow
Looks like it is a good day for my Singtel connection

Test using Mac Mini M1 with Singtel 1Gbps plan
ZTE ONT -- Singtel Mesh Router -- Asus AX82U (Double NAT) -- wireless -- Huawei AX3 Pro as repeater -- LAN cable to Mac Mini M1.

SpeedTest Results from Ookla.
261597184.png


DSL Reports test result
http://www.dslreports.com/speedtest/66674671
66674671.png
 

xiaofan

High Supremacy Member
Joined
Sep 16, 2018
Messages
30,596
Reaction score
8,478
Looks like it is a good day for my Singtel connection
Test using Mac Mini M1 with Singtel 1Gbps plan
ZTE ONT -- Singtel Mesh Router -- Asus AX82U (Double NAT) -- wireless -- Huawei AX3 Pro as repeater -- LAN cable to Mac Mini M1.
DSL Reports test result
http://www.dslreports.com/speedtest/66674671
66674671.png
Similar A grade with a Windows laptop
ZTE ONT -- Singtel Mesh Router -- Asus AX82U (Double NAT) -- wireless -- Huawei AX3 Pro as repeater -- Acer Swift 3 Windows 10 with Ugreen USB 3 Ethernet adapter

http://www.dslreports.com/speedtest/66674984

66674984.png


But then if I used wired links to either Asus or Singtel Mesh router then the BufferBloat grade gets lower to C again...

Laptop connected to Singtel Mesh router

http://www.dslreports.com/speedtest/66674838

66674838.png


Laptop connected to Asus RT-AX82U

http://www.dslreports.com/speedtest/66674864

66674864.png
 
Last edited:

ZrE0_Cha0s

Arch-Supremacy Member
Joined
Jun 21, 2011
Messages
16,704
Reaction score
26
but its still quite weird that singtel upload is higher than download :s22:
 
Important Forum Advisory Note
This forum is moderated by volunteer moderators who will react only to members' feedback on posts. Moderators are not employees or representatives of HWZ. Forum members and moderators are responsible for their own posts.

Please refer to our Community Guidelines and Standards, Terms of Service and Member T&Cs for more information.
Top