Openwrt Router Firmware

xiaofan · Oct 8, 2023

BradenHeat said:
just for reference for anyone wondering

just doesnt stick, thus wondering where am i doing it wrong

luci has the necessary files , kmod-tcp-bbr , SQM etc,

really lost on this one lmao

Totally no idea about this one. You may want to ask in the OpenWRT forum.

BBR was (maybe still is) popular among the Chinese netizens hosting Great Firewall bypassing solutions on overseas VPS providers.

I tend to think it is useless for Singapore users. All in all, I have quite different views as yours on the usefulness of QoS/SQM/etc with 1Gbps plan or higher. I tend to think they are all useless, as long as you have a decent router and clients. It is just my personal opinion though.

Still please help to update when you get the issue resolved and report how good this is. Thanks.

hwzlite · Oct 8, 2023

BradenHeat said:
im trying to switch on bbr tcp on the gl-inet,

Tips from resident expert @ https://forum.openwrt.org/t/can-i-use-tcp-bbr-with-cake/69771/3 :grin:

xiaofan · Oct 8, 2023

hwzlite said:
Tips from resident expert @ https://forum.openwrt.org/t/can-i-use-tcp-bbr-with-cake/69771/3

BTW, I have doubt whether the IPQ6000 CPU in GL-AXT1800 can handle Cake effectively at 1Gbps or not.
https://www.gl-inet.com/products/gl-axt1800/

I feel it is better using low power x86_64 CPU based mini PC to play with Cake SQM in the end.

From the other thread.

hwzlite said:
Actually for 1Gbps plan, consumer routers based on mvebu a9 SoCs can do 1Gbps.
Check out this old gem on Comparative Throughput Testing Including NAT, SQM, WireGuard, and OpenVPN

As per the following thread, we need to take the benchmart test results of the mvebu a9 SoCs with a grain of salt. In the end, x86 based mini PC still beats Linksys WRT3200ACM hands-down, and not with 1Gbps, just 330Mbps line using Cake SQM.

Reference:
https://forum.openwrt.org/t/what-is-currently-the-fastest-non-x64-router/134604?page=4

xiaofan · Oct 8, 2023

Another interesting read of ECN and he mentions that "BBR, which currently more or less ignores packet loss on its quest to own the link, and currently has no ecn response."
https://www.bufferbloat.net/projects/ecn-sane/wiki/dtaht_ecn_editorial/

And OpenWRT SQM page: the following note means you have to have beefy hardware to get A+. For my Linksys EA7500v2, SQM is basically useless since we have to use Software/Hardware offload to achieve 1Gbos line speed.
"6. To reach A+ score in Waveform's Blufferbloat test, it is required to disable Software and Hardware offload"
https://openwrt.org/docs/guide-user/network/traffic-shaping/sqm

hwzlite · Oct 8, 2023

xiaofan said:
BTW, I have doubt whether the IPQ6000 CPU in GL-AXT1800 can handle Cake effectively at 1Gbps or not.
https://www.gl-inet.com/products/gl-axt1800/

I feel it is better using low power x86_64 CPU based mini PC to play with Cake SQM in the end.

From the other thread.

As per the following thread, we need to take the benchmart test results of the mvebu a9 SoCs with a grain of salt. In the end, x86 based mini PC still beats Linksys WRT3200ACM hands-down, and not with 1Gbps, just 330Mbps line using Cake SQM.

Reference:
https://forum.openwrt.org/t/what-is-currently-the-fastest-non-x64-router/134604?page=4

Or another contender like those Rockchip RK3588S Soc based NanoPi R6S tested from https://wiki.stoplagging.com/books/technical-guides/page/sqm-for-1-gbps-lines-with-openwrt :

2023.08.15 Update - I highly recommend the R6S ($140+$30 for power adapter) instead if you have 1Gbps or higher. It's 3x more powerful than a R4S and has dual 2.5G ports. The R4SE ($80+$10 for power adapter) can only handle up to 700Mbps on cake and could be a good alternative if you're not aiming for Gigabit.

The R6S has been tested to handle up to 1300Mbps on Cake using fiber.google.com/speedtest to the San Francisco, Comcast server on my own Comcast connection (I'm on the 1.2 Gbps DL Plan). Others have confirmed 1500-2000Mbps range with Cak. But in the real world experience that I have 900 Mbps cap seems to have the best results.

xiaofan · Oct 8, 2023

hwzlite said:
Or another contender like those Rockchip RK3588S Soc based
NanoPi R6S
tested from
https://wiki.stoplagging.com/books/technical-guides/page/sqm-for-1-gbps-lines-with-openwrt
:

Ah nice, this makes sense. RK3588S based Linux SBCs are one of the fastest low cost ARM SBCs out there, even faster than the newly announced Raspberry Pi 5.

And of course it is much faster than the poor IPQ6000 CPU in the GL.iNet GL-AXT1800.

Take note he is not using Waveform.com bufferbloat test, but rather internal Speedtest using iperf3 and ping.

xiaofan · Oct 8, 2023

The good thing is that waveform.com now has no problem handling 1Gbps line speed.

1) Router setup
OpenWRT PVE VM + Huawei AX3 Quad Core local version as AP. No SQM rules enabled.
Singtel ZTE ONT -- TP-Link TL-SG105E smart switch -- OpenWRT PVE 7.4 VM (Intel J4105 mini PC) -- Huawei AX3 Quad Core local version as wireless access point, on VLAN 150.

2) Client setup
Acer Swift 3 2021 model running Windows 11 (Intel Core i5-1135G7 16GB RAM, 512GB SSD) + UGreen USB 3.0 to Gigabit Ethernet adapter

3) Wired Test results: A
https://www.waveform.com/tools/bufferbloat?test-id=84d6c38e-6c34-417b-af84-4508c8f044fc

4) Reference OOkla SpeedTest result

xiaofan · Oct 8, 2023

From waveform.com guide, it is not recommended to test bufferbloat using WiFi, which is actually what the potential problem happens.

So the question is what is the prefered settings for WiFi when there are quite some interferences (CH40, link speed: 735/600Mbps in this case).

++++++++++++++++++++++++++++++++++++++
https://www.waveform.com/tools/bufferbloat

Before You Start

1. Disable any bandwidth-heavy tasks on your network

It’s important that your Internet connection’s upload and download connections aren’t saturated already.

Make sure no devices are currently uploading or downloading large files from the Internet (e.g. backup/sync tasks by Google Drive, Dropbox, etc).

2. Make sure your connection to your router is at least as fast as your Internet connection itself

For example, if you have a gigabit fiber internet connection, make sure you’re connected over a gigabit ethernet cable, not over WiFi.

Our speed test won’t be able to “saturate” your Internet connection to test for bufferbloat if you’re limited by your WiFi connection.
+++++++++++++++++++++++++++++++++++++++

So far I do not see problems using wired Ethernet, but I can see "C" or "D" using WiFi with the paticular setup. Using SQM seems to help but only getting B.

1) Waveform.com test result: D, without SQM rules

2) Reference OOkla SpeedTest result

3) With simple SQM rules on VLAN 150 -- 100Mbps download, 100Mbps upload -- Cake -- piece_of_cake.qos -- Ethernet -- 44Bytes overhead: Result is B. Download is very good but not download

Code:

root@OpenWrt:~# tc qdisc | grep cake
qdisc cake 8097: dev ifb4br-lan.150 root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44

https://www.waveform.com/tools/bufferbloat?test-id=9e2a463b-3924-4b2a-87ea-2a58f9c5cb5e

Test result can be "C" as well, because of download.
https://www.waveform.com/tools/bufferbloat?test-id=f3bb20cc-b3f3-4ca2-a4dd-8176af8d1eca

xiaofan · Oct 8, 2023

The following script seems to be pretty comprehensive but a bit daunting as well so I dare not to try.
https://github.com/Last-times/CAKE-QoS-Script-OpenWrt

There are some troubleshooting tips which I think may be useful.

Code:

https://github.com/Last-times/CAKE-QoS-Script-OpenWrt

root@OpenWrt:/etc/config# cat sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option script 'piece_of_cake.qos'
        option qdisc 'cake'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'

root@OpenWrt:~# tc qdisc | grep cake
qdisc cake 8097: dev ifb4br-lan.150 root refcnt 2 bandwidth 100Mbit besteffort triple-isolate nonat wash no-ack-filter split-gso rtt 100ms noatm overhead 44

root@OpenWrt:~# ip link show | grep veth
152: veth0@veth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
153: veth1@veth0: <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 qdisc noqueue master br-lan state UP mode DEFAULT group default qlen 1000
root@OpenWrt:~# sysctl net.core.default_qdisc
net.core.default_qdisc = fq_codel
root@OpenWrt:~# sysctl net.ipv4.tcp_congestion_control
net.ipv4.tcp_congestion_control = cubic
root@OpenWrt:~# sysctl net.ipv4.tcp_ecn
net.ipv4.tcp_ecn = 2
root@OpenWrt:~# uci show irqbalance.irqbalance.enabled
irqbalance.irqbalance.enabled='0'
root@OpenWrt:~# uci show network.globals.packet_steering
uci: Entry not found

With the following simple SQM settings (using fq_codel and not Cake), I can get more consistent results of "B".

Code:

root@OpenWrt:~# cat /etc/config/sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'
        option qdisc 'fq_codel'
        option script 'simplest_tbf.qos'

Results:
https://www.waveform.com/tools/bufferbloat?test-id=37c5ffe8-af82-4ca0-9ff3-bc85f2fe4e6f
https://www.waveform.com/tools/bufferbloat?test-id=3f34b0d2-90b6-48ee-851f-e9dac2d4e9e2

xiaofan · Oct 8, 2023

@BradenHeat

Maybe you can post your configurations (eg: /etc/config/sqm files and other settings related to your BBR tcp conjestion control settings), I may be able to give it a try as well. Maybe it is useful for SQM for wireless.

xiaofan · Oct 8, 2023

@hwzlite

Do you have some experiences with Wireless LAN SQM settings? I guess that my setup is a bit more difficult as the wireless AP is not under OpenWRT control, rather I am controlling the LAN interface (at this case, VLAN150).

I am reading this topic in OpenWRT forum.
https://forum.openwrt.org/t/sqm-for-wlan-need-seperate-settings/52916/27

BradenHeat · Oct 8, 2023

xiaofan said:
@BradenHeat

Maybe you can post your configurations (eg: /etc/config/sqm files and other settings related to your BBR tcp conjestion control settings), I may be able to give it a try as well. Maybe it is useful for SQM for wireless.

lol if i can figure out how gl-inet saves the config, it doesn’t stick whenever i power it off, still stuck at that area.

but yea,

if fq_codel + bbr , on Mỹ Firewalla purple it goes A+

will update once i can

and yes,, i even double confirm in vi sudo and nano iterminal command lolz

BradenHeat · Oct 8, 2023

xiaofan said:
BTW, I have doubt whether the IPQ6000 CPU in GL-AXT1800 can handle Cake effectively at 1Gbps or not.
https://www.gl-inet.com/products/gl-axt1800/

I feel it is better using low power x86_64 CPU based mini PC to play with Cake SQM in the end.

From the other thread.

As per the following thread, we need to take the benchmart test results of the mvebu a9 SoCs with a grain of salt. In the end, x86 based mini PC still beats Linksys WRT3200ACM hands-down, and not with 1Gbps, just 330Mbps line using Cake SQM.

Reference:
https://forum.openwrt.org/t/what-is-currently-the-fastest-non-x64-router/134604?page=4

technically [ piece of cake was what i used, and gotten 700 up down , consistently, ] but im just drawn to trying flow control with BBR , but urrrggghhhhh its driving me nuts lols

xiaofan · Oct 8, 2023

BradenHeat said:
lol if i can figure out how gl-inet saves the config, it doesn’t stick whenever i power it off, still stuck at that area.

but yea,

if fq_codel + bbr , on Mỹ Firewalla purple it goes A+

will update once i can

and yes,, i even double confirm in vi sudo and nano iterminal command lolz

When you get A+, what is the WAN speed achieved with the Firewalla Purple. The CPU is mentioned as 6-core 64bit ARM which may be faster than IPQ6000. I am not sure.

Wireguard performance and OpenVPN performance of GL-AXT1800 are actually comparable with Firewall Purple.
https://www.gl-inet.com/products/gl-axt1800/ (OpenVPN 120Mbps, Wireguard 550Mbps)
https://firewalla.com/products/firewalla-purple (OpenVPN 120Mbps, Wireguard 500Mbps,same as Firewalla Gold which is supposed to be a bit faster using 64bit Intel CPU).

GL-MT6000 is faster with the MediaTek Filogic 830 CPU.
https://www.gl-inet.com/products/gl-mt6000/ (OpenVPN 190Mbps, Wireguard 900Mbps).

BradenHeat said:
technically [ piece of cake was what i used, and gotten 700 up down , consistently, ] but im just drawn to trying flow control with BBR , but urrrggghhhhh its driving me nuts lols

I think 700Mbps up/down is pretty decent for IPQ6000 CPU (Quad Core Arm Cortex A53 CPU, up to 1.2GHz).

For the BBR thingy, I think you may have to ask in the OpenWRT forum.

BTW, there is a popular discussion and new OpenWRT package called Qosify. Maybe you want to play with that.
https://forum.openwrt.org/t/qosify-new-package-for-dscp-marking-cake/111789/1275

BradenHeat · Oct 8, 2023

laptop, m1 old 13"mbp

wireless, just next to router,

cake+ bbr

b grade was from ipad, but it was with EVERYone in the house using youtube

https://www.waveform.com/tools/bufferbloat?test-id=421163d4-fef2-48e4-a98c-b329aea2ee0b

--> lan cable on the m1 macbook

URGGGGHhhh stupid GL-iNet ahhhhhaaaa

BradenHeat · Oct 8, 2023

xiaofan said:
When you get A+, what is the WAN speed achieved with the Firewalla Purple. The CPU is mentioned as 6-core 64bit ARM which may be faster than IPQ6000. I am not sure.

Wireguard performance and OpenVPN performance of GL-AXT1800 are actually comparable with Firewall Purple.
https://www.gl-inet.com/products/gl-axt1800/ (OpenVPN 120Mbps, Wireguard 550Mbps)
https://firewalla.com/products/firewalla-purple (OpenVPN 120Mbps, Wireguard 500Mbps,same as Firewalla Gold which is supposed to be a bit faster using 64bit Intel CPU).

GL-MT6000 is faster with the MediaTek Filogic 830 CPU.
https://www.gl-inet.com/products/gl-mt6000/ (OpenVPN 190Mbps, Wireguard 900Mbps).

I think 700Mbps up/down is pretty decent for IPQ6000 CPU (Quad Core Arm Cortex A53 CPU, up to 1.2GHz).

For the BBR thingy, I think you may have to ask in the OpenWRT forum.

BTW, there is a popular discussion and new OpenWRT package called Qosify. Maybe you want to play with that.
https://forum.openwrt.org/t/qosify-new-package-for-dscp-marking-cake/111789/1275

COolie , will try it on the purple,

but yup its 6 core

ive NO trouble installing and doing the /etc/sysctl.conf on FWPur, but dang this travel router is kicking back hard lolz

thanks for the qosify, will take a loooooong look during my long trip back to in-laws

hwzlite · Oct 9, 2023

xiaofan said:
@hwzlite

Do you have some experiences with Wireless LAN SQM settings? I guess that my setup is a bit more difficult as the wireless AP is not under OpenWRT control, rather I am controlling the LAN interface (at this case, VLAN150).

I am reading this topic in OpenWRT forum.
https://forum.openwrt.org/t/sqm-for-wlan-need-seperate-settings/52916/27

Interesting thread there

Maybe you can add Wireless LAN setting via CLI > SQM configuration /etc/config/sqm ?
Which you can try to identify the wireless or related vlan interface via "ls /sys/class/net or ip link" command.

For my DD-WRT case , I simply configured with 1 liner (tc qdisc add ...) on wan upload side only as my poor workhorse Dlink-DIR868L almost reach cpu limit liao.

xiaofan · Oct 9, 2023

hwzlite said:
Interesting thread there

Maybe you can add Wireless LAN setting via CLI > SQM configuration /etc/config/sqm ?
Which you can try to identify the wireless or related vlan interface via "ls /sys/class/net or ip link" command.

For my DD-WRT case , I simply configured with 1 liner (tc qdisc add ...) on wan upload side only as my poor workhorse Dlink-DIR868L almost reach cpu limit liao.

I see.

I have tried two configurations, as mentioned before, but I can only get "B".

Code:

root@OpenWrt:/etc/config# cat sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option script 'piece_of_cake.qos'
        option qdisc 'cake'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'

Or.

Code:

root@OpenWrt:~# cat /etc/config/sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'
        option qdisc 'fq_codel'
        option script 'simplest_tbf.qos'

BradenHeat · Oct 12, 2023

currently back to in-laws ,

and glad ive gotten the gl-inet ,

ive ripped out the hotel's router and replaced directly

yep, even if its 500mbps or less, the sheer number of devices required for attention will surely kill it,

p.s this is their router on average :

TL-WR820N

300 Mbps Multi-Mode Wi-Fi Router

xiaofan · Oct 13, 2023

xiaofan said:

I see.

I have tried two configurations, as mentioned before, but I can only get "B".

Code:

root@OpenWrt:/etc/config# cat sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option script 'piece_of_cake.qos'
        option qdisc 'cake'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'

Or.

Code:

root@OpenWrt:~# cat /etc/config/sqm

config queue
        option interface 'br-lan.150'
        option debug_logging '0'
        option verbosity '5'
        option linklayer 'ethernet'
        option overhead '44'
        option enabled '1'
        option upload '100000'
        option download '100000'
        option qdisc 'fq_codel'
        option script 'simplest_tbf.qos'

Wired will always get A+ with the above settings. But not very useful as I only use 10% of the 1Gbps bandwidth.

Openwrt Router Firmware

High Supremacy Member

Master Member

High Supremacy Member

High Supremacy Member

Master Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

Before You Start​

High Supremacy Member

High Supremacy Member

High Supremacy Member

Supremacy Member

Supremacy Member

High Supremacy Member

Supremacy Member

Supremacy Member

Master Member

High Supremacy Member

Supremacy Member

TL-WR820N​

300 Mbps Multi-Mode Wi-Fi Router​

High Supremacy Member

Before You Start

TL-WR820N

300 Mbps Multi-Mode Wi-Fi Router