Tools for troubleshooting the network performance

seowbin · Apr 24, 2024

Mach3.2 said:
i'll also run a test once I receive the bidi tranceiver and single mode fibre from taobao.

simi test?

being a supporter of bidi reciver for quite a whiel. it's just another transceiver but live on 1 strand of fiber :s34:

now can run 25g or even 100g over bidi liao

xiaofan · Apr 24, 2024

seowbin said:
simi test?

being a supporter of bidi reciver for quite a whiel. it's just another transceiver but live on 1 strand of fiber

now can run 25g or even 100g over bidi liao

Test iperf3 vs Microsoft ntttcp.
https://techcommunity.microsoft.com...should-not-use-iperf3-on-windows/ba-p/4117876
https://www.cnx-software.com/2024/04/22/testing-ntttcp-iperf3-alternative-windows-11-linux/

Mach3.2 · Apr 24, 2024

seowbin said:
simi test?

ntttcp vs iperf3 on windows.
completely arbitrary, but i feel ntttcp a bit more consistent vs iperf3. on windows, iperf3 can be slow depending on what core it is scheduled on.
No such issue on my ubuntu vm, iperf3 very consistent in ubuntu.

seowbin said:
being a supporter of bidi reciver for quite a whiel. it's just another transceiver but live on 1 strand of fiber

yeah I bought one of those translucent simplex single mode fibre. Just gonna use those transparent cable hooks to run the fibre along my ceiling into the DB closet

25G maybe in a few years. I think for now 10GbE enough for me liao. My weekly backups transfer peak at 5 to 6Gbps only.

seowbin · Apr 24, 2024

I got these transparent one.. originally they used for fttr. Didn't install yet.

Click to buy is one thing. Install is another thing

Mach3.2 · Apr 25, 2024

seowbin said:
Install is another thing

yep. hopefully it's not too big of a pain

xiaofan · Apr 25, 2024

OpenSpeedTest seems to have a bit of inflated number, from SGIX OpenSpeedTest server, wireless.
Asus RT-AX86U router, CH60, 160MHz channel bandwidth
Acer Windows 11 computer with Intel AX201 WiFi 6 adapter.
http://speedtest.sgix.sg/

Mach3.2 · Apr 26, 2024

Actually I realised the SGIX speedtest server is only uplinked at 1Gbps.

I ran the download test simultanuously on both fibre and mobile data, and the combined speed didn't exceed 1Gbps.
When I stop the test midway on mobile data, the download speed immediately went back up on the fibre connection.

xiaofan · Apr 26, 2024

For people who are new to iperf3, here is an introduction.
Make sure you download latest version for Windows from the links provided by the first post.

xiaofan · Apr 26, 2024

Usually we use iperf3 to test bandwidth, but you can use it to measure latency as well using udp test.

Example local wireless test.

Bash:

PS C:\work\speedtest\iperf-3.16-win64> .\iperf3.exe -c 192.168.50.16 -u -b 1000M -R
Connecting to host 192.168.50.16, port 5201
Reverse mode, remote host 192.168.50.16 is sending
[  5] local 192.168.50.175 port 56470 connected to 192.168.50.16 port 5201
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-1.00   sec  29.8 MBytes   249 Mbits/sec  0.038 ms  64541/85932 (75%)
[  5]   1.00-2.01   sec  33.2 MBytes   277 Mbits/sec  0.027 ms  59759/83613 (71%)
[  5]   2.01-3.00   sec  33.1 MBytes   279 Mbits/sec  0.027 ms  61153/84940 (72%)
[  5]   3.00-4.01   sec  40.0 MBytes   335 Mbits/sec  0.046 ms  59661/88386 (68%)
[  5]   4.01-5.01   sec  47.8 MBytes   398 Mbits/sec  0.031 ms  51857/86194 (60%)
[  5]   5.01-6.00   sec  44.8 MBytes   380 Mbits/sec  0.026 ms  52664/84828 (62%)
[  5]   6.00-7.01   sec  49.3 MBytes   410 Mbits/sec  0.060 ms  50955/86353 (59%)
[  5]   7.01-8.01   sec  48.5 MBytes   407 Mbits/sec  0.025 ms  50738/85551 (59%)
[  5]   8.01-9.01   sec  46.0 MBytes   385 Mbits/sec  0.042 ms  52705/85742 (61%)
[  5]   9.01-10.01  sec  48.0 MBytes   403 Mbits/sec  0.032 ms  51196/85659 (60%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.02  sec  1.17 GBytes  1000 Mbits/sec  0.000 ms  0/0 (0%)  sender
[  5]   0.00-10.01  sec   420 MBytes   352 Mbits/sec  0.032 ms  555229/857198 (65%)  receiver

iperf Done.

xiaofan · Apr 26, 2024

It is good that Singtel is now providing a public iperf3 server.
https://github.com/R0GGER/public-iperf3-servers

iperf3 -c 89.187.162.1 (not working)	-R	10	SG	Singapore
iperf3 -c lg-sg-sin.webhorizon.net -p 9201-9209	-R, -6	10	SG	Singapore
iperf3 -c speedtest.singnet.com.sg -p 5201-5210	-R		SG	Singapore
iperf3 -c speedtest.sin1.sg.leaseweb.net -p 5201-5210	-R, -6	10	SG	Singapore

Bash:

root@debian12ct01:~# iperf3 -c speedtest.singnet.com.sg -p 5208 -R
Connecting to host speedtest.singnet.com.sg, port 5208
Reverse mode, remote host speedtest.singnet.com.sg is sending
[  5] local 192.168.50.16 port 56328 connected to 165.21.70.1 port 5208
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   104 MBytes   875 Mbits/sec
[  5]   1.00-2.00   sec   111 MBytes   933 Mbits/sec
[  5]   2.00-3.00   sec   112 MBytes   939 Mbits/sec
[  5]   3.00-4.00   sec   112 MBytes   939 Mbits/sec
[  5]   4.00-5.00   sec   112 MBytes   939 Mbits/sec
[  5]   5.00-6.00   sec   112 MBytes   939 Mbits/sec
[  5]   6.00-7.00   sec   112 MBytes   937 Mbits/sec
[  5]   7.00-8.00   sec   112 MBytes   939 Mbits/sec
[  5]   8.00-9.00   sec   111 MBytes   935 Mbits/sec
[  5]   9.00-10.00  sec   111 MBytes   935 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec  1060             sender
[  5]   0.00-10.00  sec  1.08 GBytes   931 Mbits/sec                  receiver

iperf Done.

root@debian12ct01:~# iperf3 -c speedtest.singnet.com.sg -p 5208
Connecting to host speedtest.singnet.com.sg, port 5208
[  5] local 192.168.50.16 port 41074 connected to 165.21.70.1 port 5208
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  94.8 MBytes   795 Mbits/sec    0    587 KBytes
[  5]   1.00-2.00   sec   112 MBytes   938 Mbits/sec    0    587 KBytes
[  5]   2.00-3.00   sec   112 MBytes   941 Mbits/sec    0    587 KBytes
[  5]   3.00-4.00   sec   112 MBytes   938 Mbits/sec    0    587 KBytes
[  5]   4.00-5.00   sec   112 MBytes   939 Mbits/sec    0    587 KBytes
[  5]   5.00-6.00   sec   112 MBytes   936 Mbits/sec    0    587 KBytes
[  5]   6.00-7.00   sec   112 MBytes   937 Mbits/sec    0    587 KBytes
[  5]   7.00-8.00   sec   111 MBytes   935 Mbits/sec    0    587 KBytes
[  5]   8.00-9.00   sec   112 MBytes   940 Mbits/sec    0    587 KBytes
[  5]   9.00-10.00  sec   112 MBytes   941 Mbits/sec    0    587 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  1.08 GBytes   924 Mbits/sec    0             sender
[  5]   0.00-10.00  sec  1.07 GBytes   922 Mbits/sec                  receiver

iperf Done.

xiaofan · Apr 26, 2024

iperf3 under Windows is not as good as under Linux/macOS/BSDs. So you may get lower than expected speed.

One tip for wireles test is to use -P parameter. Usually I will use -P 4 for my Windows 11 laptop.

Bash:

PS C:\work\speedtest\iperf-3.16-win64> .\iperf3 -c speedtest.singnet.com.sg -p 5207 -R
Connecting to host speedtest.singnet.com.sg, port 5207
Reverse mode, remote host speedtest.singnet.com.sg is sending
[  5] local 192.168.50.175 port 10366 connected to 165.21.70.1 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  43.9 MBytes   365 Mbits/sec
[  5]   1.01-2.01   sec  47.2 MBytes   396 Mbits/sec
[  5]   2.01-3.00   sec  55.4 MBytes   469 Mbits/sec
[  5]   3.00-4.00   sec  60.0 MBytes   503 Mbits/sec
[  5]   4.00-5.00   sec  63.6 MBytes   533 Mbits/sec
[  5]   5.00-6.00   sec  59.9 MBytes   502 Mbits/sec
[  5]   6.00-7.01   sec  63.1 MBytes   529 Mbits/sec
[  5]   7.01-8.01   sec  61.5 MBytes   515 Mbits/sec
[  5]   8.01-9.01   sec  61.1 MBytes   512 Mbits/sec
[  5]   9.01-10.00  sec  59.2 MBytes   501 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   576 MBytes   483 Mbits/sec  6465             sender
[  5]   0.00-10.00  sec   575 MBytes   482 Mbits/sec                  receiver

iperf Done.
PS C:\work\speedtest\iperf-3.16-win64> .\iperf3 -c speedtest.singnet.com.sg -p 5207 -P4 -R
Connecting to host speedtest.singnet.com.sg, port 5207
Reverse mode, remote host speedtest.singnet.com.sg is sending
[  5] local 192.168.50.175 port 10370 connected to 165.21.70.1 port 5207
[  7] local 192.168.50.175 port 10371 connected to 165.21.70.1 port 5207
[  9] local 192.168.50.175 port 10372 connected to 165.21.70.1 port 5207
[ 11] local 192.168.50.175 port 10373 connected to 165.21.70.1 port 5207
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  23.0 MBytes   192 Mbits/sec
[  7]   0.00-1.01   sec  25.0 MBytes   208 Mbits/sec
[  9]   0.00-1.01   sec  24.1 MBytes   201 Mbits/sec
[ 11]   0.00-1.01   sec  22.6 MBytes   189 Mbits/sec
[SUM]   0.00-1.01   sec  94.8 MBytes   789 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   1.01-2.01   sec  22.1 MBytes   185 Mbits/sec
[  7]   1.01-2.01   sec  24.2 MBytes   203 Mbits/sec
[  9]   1.01-2.01   sec  23.2 MBytes   195 Mbits/sec
[ 11]   1.01-2.01   sec  23.6 MBytes   198 Mbits/sec
[SUM]   1.01-2.01   sec  93.2 MBytes   781 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   2.01-3.01   sec  20.8 MBytes   174 Mbits/sec
[  7]   2.01-3.01   sec  20.2 MBytes   170 Mbits/sec
[  9]   2.01-3.01   sec  22.6 MBytes   190 Mbits/sec
[ 11]   2.01-3.01   sec  23.9 MBytes   200 Mbits/sec
[SUM]   2.01-3.01   sec  87.5 MBytes   734 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   3.01-4.00   sec  23.8 MBytes   201 Mbits/sec
[  7]   3.01-4.00   sec  22.8 MBytes   192 Mbits/sec
[  9]   3.01-4.00   sec  22.8 MBytes   192 Mbits/sec
[ 11]   3.01-4.00   sec  21.9 MBytes   185 Mbits/sec
[SUM]   3.01-4.00   sec  91.1 MBytes   771 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   4.00-5.00   sec  23.4 MBytes   196 Mbits/sec
[  7]   4.00-5.00   sec  22.0 MBytes   184 Mbits/sec
[  9]   4.00-5.00   sec  22.0 MBytes   184 Mbits/sec
[ 11]   4.00-5.00   sec  22.5 MBytes   188 Mbits/sec
[SUM]   4.00-5.00   sec  89.9 MBytes   752 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   5.00-6.00   sec  24.1 MBytes   202 Mbits/sec
[  7]   5.00-6.00   sec  22.5 MBytes   189 Mbits/sec
[  9]   5.00-6.00   sec  21.9 MBytes   183 Mbits/sec
[ 11]   5.00-6.00   sec  23.9 MBytes   200 Mbits/sec
[SUM]   5.00-6.00   sec  92.4 MBytes   774 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   6.00-7.01   sec  24.2 MBytes   203 Mbits/sec
[  7]   6.00-7.01   sec  22.9 MBytes   191 Mbits/sec
[  9]   6.00-7.01   sec  24.5 MBytes   205 Mbits/sec
[ 11]   6.00-7.01   sec  22.8 MBytes   190 Mbits/sec
[SUM]   6.00-7.01   sec  94.4 MBytes   790 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   7.01-8.01   sec  23.0 MBytes   193 Mbits/sec
[  7]   7.01-8.01   sec  23.4 MBytes   196 Mbits/sec
[  9]   7.01-8.01   sec  23.2 MBytes   195 Mbits/sec
[ 11]   7.01-8.01   sec  25.4 MBytes   213 Mbits/sec
[SUM]   7.01-8.01   sec  95.0 MBytes   796 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   8.01-9.00   sec  22.2 MBytes   188 Mbits/sec
[  7]   8.01-9.00   sec  23.2 MBytes   196 Mbits/sec
[  9]   8.01-9.00   sec  21.6 MBytes   182 Mbits/sec
[ 11]   8.01-9.00   sec  24.4 MBytes   206 Mbits/sec
[SUM]   8.01-9.00   sec  91.5 MBytes   772 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   9.00-10.00  sec  24.6 MBytes   206 Mbits/sec
[  7]   9.00-10.00  sec  23.0 MBytes   193 Mbits/sec
[  9]   9.00-10.00  sec  21.4 MBytes   179 Mbits/sec
[ 11]   9.00-10.00  sec  24.9 MBytes   209 Mbits/sec
[SUM]   9.00-10.00  sec  93.9 MBytes   787 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec   232 MBytes   194 Mbits/sec  11185             sender
[  5]   0.00-10.00  sec   231 MBytes   194 Mbits/sec                  receiver
[  7]   0.00-10.00  sec   230 MBytes   193 Mbits/sec  10375             sender
[  7]   0.00-10.00  sec   229 MBytes   192 Mbits/sec                  receiver
[  9]   0.00-10.00  sec   228 MBytes   191 Mbits/sec  9634             sender
[  9]   0.00-10.00  sec   227 MBytes   191 Mbits/sec                  receiver
[ 11]   0.00-10.00  sec   236 MBytes   198 Mbits/sec  9945             sender
[ 11]   0.00-10.00  sec   236 MBytes   198 Mbits/sec                  receiver
[SUM]   0.00-10.00  sec   926 MBytes   776 Mbits/sec  41139             sender
[SUM]   0.00-10.00  sec   924 MBytes   775 Mbits/sec                  receiver

iperf Done.

xiaofan · Apr 27, 2024

Adding bgp.tools in Post #6.

https://forums.hardwarezone.com.sg/...ussion-part-2.5658375/page-587#post-152181678

Mach3.2 said:
Dunno, I don't use their services.
Just pull the IP while streaming, then run traceroute, see where it goes.

Generally...
If it goes directly from M1's router to their router, they are peering directly.
If there's an internet exchange hop in between (Equinix/SGIX/BBIX), then they are peering via an internet exchange.
If it goes through another provider(Arelion/PCCW/TATA/Starhub/Singtel/etc.), then traffic is most likely going via IP transit, which is the worse case loh, because buffering can be due to upstream IP transit congestion.

Usually you can sort of tell what each hop is base off the reverse DNS. I don't think BBIX Singapore have rDNS configured, so in those cases you just throw the IP address into sites like bgp.tools and they will bring you to the IX's page. Then ctrl + F the same IP and you'll find out that IP is assigned to who liao.

Mach3.2 · May 6, 2024

4 stream send

4 stream receive

4 stream send

4 stream receive

ntttcp slightly more efficient in the send test, and the difference becomes more obvious during the receive test.

tl:dr ntttcp more efficient on windows.

xiaofan · Sep 7, 2024

Good article for troubleshooting DNS related issues under Linux
https://community.bigbeartechworld.com/t/comprehensive-guide-to-diagnosing-dns-issues-on-linux/221

xiaofan · Sep 7, 2024

Docker Networking Trouble-shooting Swiss-Army Container
https://hub.docker.com/r/nicolaka/netshoot

More about containers to use for home networking
https://forums.hardwarezone.com.sg/threads/containers-for-home-network.6656260

xiaofan · Sep 7, 2024

Singtel public iperf3 server vs SIngel OOkla SpeedTest server -- slower in terms of download, similar in terms of upload.

Bash:

mcuee@miniroute10g:~/build/networking/speedtest$ ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.43 ms   (jitter: 0.33ms, low: 1.27ms, high: 1.75ms)
    Download:  5650.03 Mbps (data used: 7.2 GB)
                  1.04 ms   (jitter: 0.35ms, low: 0.74ms, high: 7.13ms)
      Upload:  4983.85 Mbps (data used: 4.8 GB)
                  5.73 ms   (jitter: 0.40ms, low: 1.55ms, high: 7.19ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/477f5711-2bcb-467d-af3c-ef53aeba6057

mcuee@miniroute10g:~$ iperf3 -c speedtest.singnet.com.sg -p 5201 -R
Connecting to host speedtest.singnet.com.sg, port 5201
Reverse mode, remote host speedtest.singnet.com.sg is sending
[  5] local 192.168.1.3 port 57102 connected to 165.21.70.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.00   sec   359 MBytes  3.01 Gbits/sec
[  5]   1.00-2.00   sec   483 MBytes  4.05 Gbits/sec
[  5]   2.00-3.00   sec   470 MBytes  3.94 Gbits/sec
[  5]   3.00-4.00   sec   483 MBytes  4.05 Gbits/sec
[  5]   4.00-5.00   sec   475 MBytes  3.99 Gbits/sec
[  5]   5.00-6.00   sec   481 MBytes  4.04 Gbits/sec
[  5]   6.00-7.00   sec   475 MBytes  3.99 Gbits/sec
[  5]   7.00-8.00   sec   471 MBytes  3.95 Gbits/sec
[  5]   8.00-9.00   sec   488 MBytes  4.10 Gbits/sec
[  5]   9.00-10.00  sec   473 MBytes  3.97 Gbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  4.55 GBytes  3.91 Gbits/sec  122             sender
[  5]   0.00-10.00  sec  4.55 GBytes  3.91 Gbits/sec                  receiver

iperf Done.
mcuee@miniroute10g:~$ iperf3 -c speedtest.singnet.com.sg -p 5201
Connecting to host speedtest.singnet.com.sg, port 5201
[  5] local 192.168.1.3 port 32858 connected to 165.21.70.1 port 5201
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   283 MBytes  2.37 Gbits/sec  413   1.08 MBytes
[  5]   1.00-2.00   sec   593 MBytes  4.98 Gbits/sec    0   1.08 MBytes
[  5]   2.00-3.00   sec   595 MBytes  4.99 Gbits/sec    0   1.08 MBytes
[  5]   3.00-4.00   sec   593 MBytes  4.98 Gbits/sec    0   1.08 MBytes
[  5]   4.00-5.00   sec   593 MBytes  4.98 Gbits/sec    0   1.08 MBytes
[  5]   5.00-6.00   sec   594 MBytes  4.98 Gbits/sec    0   1.08 MBytes
[  5]   6.00-7.00   sec   594 MBytes  4.99 Gbits/sec    0   1.08 MBytes
[  5]   7.00-8.00   sec   593 MBytes  4.97 Gbits/sec    0   1.08 MBytes
[  5]   8.00-9.00   sec   593 MBytes  4.97 Gbits/sec    0   1.08 MBytes
[  5]   9.00-10.00  sec   594 MBytes  4.98 Gbits/sec    0   1.08 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  5.50 GBytes  4.72 Gbits/sec  413             sender
[  5]   0.00-10.00  sec  5.49 GBytes  4.72 Gbits/sec                  receiver

iperf Done.

xiaofan · Sep 15, 2024

Alternative SpeedTest client using Go, still slower than official OOkla SpeedTest CLI.
https://github.com/showwin/speedtest-go

Scripts to carry out many tests using OOkla SpeeedTest
https://github.com/su-haris/simple-network-speedtest
https://nws.sh/

SpeedTest server list and test script
https://github.com/spiritLHLS/speedtest.net-CN-ID
https://github.com/spiritLHLS/speedtest.cn-CN-ID
https://github.com/spiritLHLS/ecsspeed

xiaofan · Sep 15, 2024

Speedtest-go server list for Singapore

Note: Singtel to M1 can be slow if using Singtel IPv6.

Bash:

$ ./speedtest-go.exe  -l

    speedtest-go v1.7.9 (git-dd78c65) @showwin

✓ ISP: 115.xx.xx.x (Singtel Fibre) [1.3254, 103.7433]
✓ Found 50 Public Servers
[ 4235]      6.91km 5ms         Singapore (Singapore) by StarHub Ltd
[  367]      6.91km 3ms         Singapore (Singapore) by NewMedia Express
[ 7311]      6.91km 4ms         Singapore (Singapore) by M1 Limited
[ 5935]      6.91km 4ms         Singapore (Singapore) by MyRepublic Singapore
[ 7556]      6.91km 5ms         Singapore (Singapore) by PT FirstMedia
[13623]      6.91km 3ms         Singapore (Singapore) by Singtel
[26654]      6.91km Timeout     Singapore (Singapore) by fdcservers.net
[ 2054]      6.91km Timeout     Singapore (Singapore) by ViewQwest
[14863]      6.91km 117ms       Singapore (Singapore) by SMA PASUNDAN 8 BANDUNG
[28921]      6.91km 5ms         Singapore (Singapore) by PhoenixNAP Global IT Services
[41358]      6.91km 224ms       Singapore (Singapore) by Contabo
[42688]      6.91km 50ms        Singapore (Singapore) by PT XL Axiata Tbk
[45345]      6.91km 38ms        Singapore (Singapore) by Melbikomas UAB
[46304]      6.91km 69ms        Singapore (Singapore) by IDCloudHost Singapore
[37744]      6.91km 77ms        Singapore (Singapore) by Telekomunikasi Indonesia International Pte Ltd (Telin Singapore)
[54359]      6.91km Timeout     Singapore (Singapore) by Matrix NAP Info
[44817]      6.91km 3ms         Singapore (Singapore) by SPTEL PTE. LTD.
[40508]      6.91km 5ms         Singapore (Singapore) by i3D.net
[50406]      6.91km 5ms         Singapore (Singapore) by Verizon
[41801]      6.91km 43ms        Singapore (Singapore) by IX Telecom
[56340]      6.91km 15ms        Singapore (Singapore) by PT Lintas Jaringan Nusantara
[56066]      6.91km 211ms       Singapore (Singapore) by PT Herza Digital Indonesia
[31180]      6.91km 171ms       Singapore (Singapore) by Campana
[59186]      6.91km 7ms         Singapore (Singapore) by DCS1 Pte Ltd
[59016]      6.91km 36ms        Singapore (Singapore) by CBN
[62020]      6.91km 4ms         Singapore (Singapore) by Axclusive Pte Ltd
[62530]      6.91km 36ms        Singapore (Singapore) by Symphony Communication PCL
[63375]      6.91km 195ms       Singapore (Singapore) by HSPNet
[31293]      7.97km 11ms        Singapore (Singapore) by Pacific Internet (S)
[23467]      9.02km 5ms         Singapore (Singapore) by PT. Indonesia Comnets Plus
[13058]      9.02km 70ms        Singapore (Singapore) by PT Indosat Tbk
[12687]      9.02km Timeout     Singapore (Singapore) by SIMBA Telecom
[47713]      9.02km 4ms         Singapore (Singapore) by PT. Mora Telematika Indonesia
[62216]      9.02km 36ms        Singapore (Singapore) by CTCSCI TECH LTD
[56684]     18.62km 9ms         Johor Bahru (Malaysia) by U Mobile Sdn Bhd
[18031]     18.62km 178ms       Johor Bahru (Malaysia) by Telekom Malaysia Berhad
[57659]     18.62km 81ms        Johor Bahru (Malaysia) by Celcom Axiata Bhd
[51518]     40.74km 35ms        Batam (Indonesia) by VNT NETWORKS - BTM ID
[30838]     41.90km 5ms         Batam (Indonesia) by PT. Aplikanusa Lintasarta
[48161]     41.90km 50ms        Batam (Indonesia) by PT Cipta Informatika Cemerlang
[52293]     41.90km 18ms        Batam (Indonesia) by PT Lintas Jaringan Nusantara
[18108]     41.90km 71ms        Batam (Indonesia) by PT. Indosat Tbk
[52307]     41.90km 4ms         Batam (Indonesia) by SKYNETWORK
[40196]     41.90km 37ms        Batam (Indonesia) by PT.MEDIANUSA PERMANA
[40272]     41.90km 5ms         Batam (Indonesia) by SDI
[59120]     41.90km 41ms        Batam (Indonesia) by PT. Telekomunikasi Indonesia
[61291]     41.90km 212ms       Batam (Indonesia) by GONET INDONESIA
[61693]     41.90km 6ms         Batam (Indonesia) by PalapaNet
[62770]     41.90km 4ms         Batam (Indonesia) by Iplus
[63397]     41.90km 5ms         Batam (Indonesia) by PT Solnet Indonesia

xiaofan · Nov 16, 2024

xiaofan said:
Now we have a very fast 4Gbps capable Crusader server for Singapore (hosted on Linode) thanks to the famous Dave Täht.

I have done my first wired tests with the server with my Intel N100 mini PC (Singtel 5Gbps plan, ZTE F8648P ONR) and the results are actually very good for both IPv4 and IPv6.
https://github.com/Zoxc/crusader/issues/9

xiaofan said:
For those who are interested in latency testing (wired and wireless), you can take a look at here. Contributions of testing results are also welcome.
https://github.com/Zoxc/crusader/discussions

Latest result from my Singtel 5Gbps plan, running from the main OpenWRT router (Singtel ZTE F8648P ONR 10G LAN port bridged).

Bash:

root@OpenWrt:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink
.taht.net
[2024-11-16 03:23:31] Client version 0.3.2 running
[2024-11-16 03:23:32] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-16 03:23:34] Idle latency to server 2.36 ms
[2024-11-16 03:23:36] Testing download...
[2024-11-16 03:25:06] Testing upload...
[2024-11-16 03:26:36] Testing both download and upload...

-- Download test --
          Throughput: 4041.24 Mbps
             Latency: 2.7 ms (1.1 ms down, 1.6 ms up)
         Packet loss: 0.03% down, 0% up

-- Upload test --
          Throughput: 4782.38 Mbps
             Latency: 10.1 ms (1.0 ms down, 9.1 ms up)
         Packet loss: 0.01% down, 0% up

-- Bidirectional test --
          Throughput: 8551.04 Mbps (3716.90 Mbps down, 4834.13 Mbps up)
             Latency: 10.1 ms (1.2 ms down, 8.9 ms up)
         Packet loss: 0.01% down, 0.01% up

[2024-11-16 03:28:10] Writing data...
[2024-11-16 03:28:10] Saved raw data as crusader-results/test 2024-11-16 03.28.10.crr
[2024-11-16 03:28:10] Saved plot as crusader-results/test 2024-11-16 03.28.10.png

xiaofan · Jan 17, 2025

Somehow for my wireless speed test results, iperf3 (local speedtest) is always slower than OOkla SpeedTest using Singtel server (Internet speedtest).

As I am using Singtel 5Gbps plan, WAN Speed is not a limitation for wireless speed test (usually <3.5Gbps for all my devices and wireless routers). My fastest wireless router is TP-Link HB710 triband BE18000.

One example test:
Using Intel N100 virtual OpenWRT router as the iperf3 server, crusader server for tests.
Wireless router --> Asus TUF-BE6500 dual band BE6500 (Wireless speed <=2Gbps in general)
Wireless client --> Ryzen 5600G Desktop PC with built-in RealTek 8852BE AX1800 rated PCI-E adapter (up to 1200Mbps PHY speed and < 840Mbps wireless speed in terms of real world wifi speed), 4m away from router.

Bash:

PS C:\work\networking\ookla-speedtest-1.2.0-win64> .\speedtest.exe -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.89 ms   (jitter: 0.35ms, low: 2.53ms, high: 3.27ms)
    Download:   797.85 Mbps (data used: 1.4 GB)
                 15.15 ms   (jitter: 7.26ms, low: 2.67ms, high: 36.46ms)
      Upload:   770.44 Mbps (data used: 788.3 MB)
                 31.62 ms   (jitter: 9.26ms, low: 5.75ms, high: 55.88ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/3e462515-9819-4fd6-9183-32423163d748

PS C:\work\networking\crusader_bin\v0.3.2> .\crusader.exe test 192.168.18.1
[2025-01-17 14:03:42] Client version 0.3.2 running
[2025-01-17 14:03:42] Connected to server 192.168.18.1:35481
[2025-01-17 14:03:44] Idle latency to server 1.41 ms
[2025-01-17 14:03:46] Testing download...
[2025-01-17 14:03:58] Testing upload...
[2025-01-17 14:04:15] Testing both download and upload...
[2025-01-17 14:04:30] Warning: Load termination timed out. There may be residual untracked traffic in the background.

-- Download test --
          Throughput: 570.02 Mbps
             Latency: 15.1 ms (7.5 ms down, 7.6 ms up)
         Packet loss: 0.50% down, 0% up

-- Upload test --
          Throughput: 627.06 Mbps
             Latency: 170.4 ms (13.4 ms down, 157.0 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 687.08 Mbps (147.98 Mbps down, 539.10 Mbps up)
             Latency: 178.8 ms (10.3 ms down, 168.5 ms up)
         Packet loss: 0%

[2025-01-17 14:04:30] Writing data...
[2025-01-17 14:04:30] Saved raw data as crusader-results\test 2025-01-17 14.04.30.crr
[2025-01-17 14:04:30] Saved plot as crusader-results\test 2025-01-17 14.04.30.png

PS C:\work\networking\iperf-3.17.1-win64> .\iperf3.exe -c 192.168.18.1 -R
Connecting to host 192.168.18.1, port 5201
Reverse mode, remote host 192.168.18.1 is sending
[  5] local 192.168.50.143 port 59570 connected to 192.168.18.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  75.6 MBytes   630 Mbits/sec
[  5]   1.01-2.00   sec  79.8 MBytes   672 Mbits/sec
[  5]   2.00-3.01   sec  84.2 MBytes   699 Mbits/sec
[  5]   3.01-4.01   sec  85.8 MBytes   721 Mbits/sec
[  5]   4.01-5.01   sec  82.4 MBytes   694 Mbits/sec
[  5]   5.01-6.00   sec  82.8 MBytes   697 Mbits/sec
[  5]   6.00-7.01   sec  82.2 MBytes   683 Mbits/sec
[  5]   7.01-8.01   sec  81.2 MBytes   685 Mbits/sec
[  5]   8.01-9.01   sec  83.9 MBytes   704 Mbits/sec
[  5]   9.01-10.01  sec  76.8 MBytes   644 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.01  sec   818 MBytes   685 Mbits/sec  129             sender
[  5]   0.00-10.01  sec   815 MBytes   683 Mbits/sec                  receiver

iperf Done.

PS C:\work\networking\iperf-3.17.1-win64> .\iperf3.exe -c 192.168.18.1
Connecting to host 192.168.18.1, port 5201
[  5] local 192.168.50.143 port 59572 connected to 192.168.18.1 port 5201
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-1.01   sec  49.2 MBytes   408 Mbits/sec
[  5]   1.01-2.01   sec  32.1 MBytes   270 Mbits/sec
[  5]   2.01-3.01   sec  48.0 MBytes   404 Mbits/sec
[  5]   3.01-4.00   sec  52.0 MBytes   437 Mbits/sec
[  5]   4.00-5.00   sec  59.4 MBytes   500 Mbits/sec
[  5]   5.00-6.01   sec  70.5 MBytes   585 Mbits/sec
[  5]   6.01-7.01   sec  72.8 MBytes   614 Mbits/sec
[  5]   7.01-8.00   sec  70.4 MBytes   593 Mbits/sec
[  5]   8.00-9.01   sec  68.4 MBytes   568 Mbits/sec
[  5]   9.01-10.01  sec  63.2 MBytes   533 Mbits/sec
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate
[  5]   0.00-10.01  sec   586 MBytes   491 Mbits/sec                  sender
[  5]   0.00-10.01  sec   586 MBytes   491 Mbits/sec                  receiver

iperf Done.

Tools for troubleshooting the network performance

Great Supremacy Member

High Supremacy Member

Great Supremacy Member

Great Supremacy Member

Great Supremacy Member

High Supremacy Member

Great Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

Great Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member