Tools for troubleshooting the network performance

[xiaofan]

Singtel public iperf3 server vs SIngel OOkla SpeedTest server -- slower in terms of download, similar in terms of upload.

Using -P4 will yield better results. The following is wired speedtest from my main virtual OpenWRT router.

Singtel 5Gbps plan --> 5.45Gbps download, 4.99Gbps upload, using Singtel public iperf3 server.

Singtel OOkla SpeedTest server is still a bit faster for download --> 5.66Gbps download and 4.978Gbps upload.

 

[xiaofan]

nperf is another alternative to OOkla SpeedTest.

Wireless router --> Asus TUF-BE6500 dual band BE6500 (Wireless speed <=2Gbps in general)
Wireless client --> Ryzen 5600G Desktop PC with built-in RealTek 8852BE AX1800 rated PCI-E adapter (up to 1200Mbps PHY speed and < 840Mbps wireless speed in terms of real world wifi speed), 4m away from router.

https://www.nperf.com/en/r/3557572234428890-bEPS7Gjb

 

[xiaofan]

Same computer, reboot to the dual boot Ubuntu 24.04 installation.

mcuee@PCRyzen5600G:~/build/networking/ookla$ ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.63 ms   (jitter: 0.23ms, low: 2.04ms, high: 2.82ms)
    Download:   859.24 Mbps (data used: 759.7 MB)                                                   
                 16.46 ms   (jitter: 11.20ms, low: 3.08ms, high: 249.93ms)
      Upload:   556.35 Mbps (data used: 265.9 MB)                                                   
                 28.39 ms   (jitter: 9.75ms, low: 4.39ms, high: 61.22ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/40088127-7f27-4e56-9a3c-4eba43b5c1de

https://www.nperf.com/en/r/3557573554344952-oj8EIase

 

[xiaofan]

Found a better ways to list OOkla SpeedTest servers.

Just change the last parameter for the following query to other country if you want to look at speedtest servers in the specific country.

Tick the Pretty-Print checkbox to have a nice output.

Singapore:
https://www.speedtest.net/api/js/servers?engine=js&https_functional=true&limit=100&search=Singapore

China:
https://www.speedtest.net/api/js/servers?engine=js&https_functional=true&limit=100&search=China

Example output:

  {
    "url": "https://m1speedtest1.m1net.com.sg:8080/speedtest/upload.php",
    "lat": "1.3000",
    "lon": "103.8000",
    "distance": 7,
    "name": "Singapore",
    "country": "Singapore",
    "cc": "SG",
    "sponsor": "M1 Limited",
    "id": "7311",
    "preferred": 0,
    "https_functional": 1,
    "host": "m1speedtest1.m1net.com.sg:8080"
  },
  {
    "url": "http://speedtest.myrepublic.com.sg:8080/speedtest/upload.php",
    "lat": "1.3000",
    "lon": "103.8000",
    "distance": 7,
    "name": "Singapore",
    "country": "Singapore",
    "cc": "SG",
    "sponsor": "MyRepublic Singapore",
    "id": "5935",
    "preferred": 0,
    "https_functional": 1,
    "host": "speedtest.myrepublic.com.sg.prod.hosts.ooklaserver.net:8080"
  },

 

[xiaofan]

SpeedTest results for my Singtel 5Gbps plan with different SpeedTest servers in Singapore, mainly ISP servers, but also include two fast non-ISP SpeedTest servers.

Singtel to M1 has a problem because of bad routing through IPv6 (M1 is using IPv6 for its SpeedTest server by default if the test client has working IPv6 address).

Test done on the main OpenWRT virtual router (WAN Speed).

root@OpenWrt:~/ookla# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.48 ms   (jitter: 0.21ms, low: 1.24ms, high: 1.65ms)
    Download:  5652.59 Mbps (data used: 2.7 GB)
                  1.92 ms   (jitter: 11.50ms, low: 1.41ms, high: 213.21ms)
      Upload:  4981.59 Mbps (data used: 2.2 GB)
                  8.25 ms   (jitter: 0.60ms, low: 1.66ms, high: 9.30ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/ff3444bf-fed2-4f5d-aa69-cb21fc18a41b

root@OpenWrt:~/ookla# ./speedtest -s 5935

   Speedtest by Ookla

      Server: MyRepublic Singapore - Singapore (id: 5935)
         ISP: Singtel Fibre
Idle Latency:     1.99 ms   (jitter: 0.18ms, low: 1.80ms, high: 2.19ms)
    Download:  5378.96 Mbps (data used: 5.1 GB)
                  1.70 ms   (jitter: 0.18ms, low: 1.33ms, high: 2.17ms)
      Upload:  4938.90 Mbps (data used: 2.3 GB)
                  7.84 ms   (jitter: 0.49ms, low: 2.03ms, high: 9.45ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/e2d6c204-2cae-4f99-9e60-f3359de06bfe

root@OpenWrt:~/ookla# ./speedtest -s 2054

   Speedtest by Ookla

      Server: ViewQwest - Singapore (id: 2054)
         ISP: Singtel Fibre
Idle Latency:     2.99 ms   (jitter: 0.08ms, low: 2.93ms, high: 3.11ms)
    Download:  5457.63 Mbps (data used: 6.4 GB)
                  2.91 ms   (jitter: 0.27ms, low: 2.56ms, high: 4.84ms)
      Upload:  4933.94 Mbps (data used: 2.5 GB)
                  9.04 ms   (jitter: 0.81ms, low: 3.12ms, high: 11.95ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/64aa5b71-b341-4dc8-8104-97c5572ce314

root@OpenWrt:~/ookla# ./speedtest -s 4235

   Speedtest by Ookla

      Server: StarHub Ltd - Singapore (id: 4235)
         ISP: Singtel Fibre
Idle Latency:     1.64 ms   (jitter: 0.14ms, low: 1.43ms, high: 1.89ms)
    Download:  4750.32 Mbps (data used: 7.3 GB)
                  1.57 ms   (jitter: 0.18ms, low: 1.21ms, high: 2.20ms)
      Upload:  4942.41 Mbps (data used: 4.7 GB)
                  7.30 ms   (jitter: 0.75ms, low: 1.40ms, high: 9.35ms)
 Packet Loss: Not available.
  Result URL: https://www.speedtest.net/result/c/dfc9acb4-d441-4e4c-87cc-dc92a8073353

root@OpenWrt:~/ookla# ./speedtest -s 7311

   Speedtest by Ookla

      Server: M1 Limited - Singapore (id: 7311)
         ISP: Singtel Fibre
Idle Latency:    83.32 ms   (jitter: 0.34ms, low: 83.00ms, high: 83.52ms)
    Download:  2222.69 Mbps (data used: 3.3 GB)
                107.15 ms   (jitter: 29.92ms, low: 71.43ms, high: 142.97ms)
      Upload:  1057.31 Mbps (data used: 1.3 GB)
                106.11 ms   (jitter: 28.67ms, low: 69.71ms, high: 141.47ms)
 Packet Loss: Not available.
  Result URL: https://www.speedtest.net/result/c/97a7086e-7bdb-4d25-bab7-afbb43e5bf76

root@OpenWrt:~/ookla# ./speedtest -s 62020

   Speedtest by Ookla

      Server: Axclusive Pte Ltd - Singapore (id: 62020)
         ISP: Singtel Fibre
Idle Latency:     2.55 ms   (jitter: 0.29ms, low: 2.15ms, high: 2.79ms)
    Download:  5399.80 Mbps (data used: 4.6 GB)
                  2.61 ms   (jitter: 0.25ms, low: 2.23ms, high: 3.82ms)
      Upload:  4936.36 Mbps (data used: 2.4 GB)
                  8.52 ms   (jitter: 0.47ms, low: 2.07ms, high: 10.08ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/eaa544fe-6c92-4859-bdcc-59ce18d2780a

root@OpenWrt:~/ookla# ./speedtest -s 62216

   Speedtest by Ookla

      Server: CTCSCI TECH LTD - Singapore (id: 62216)
         ISP: Singtel Fibre
Idle Latency:     1.47 ms   (jitter: 0.20ms, low: 1.28ms, high: 1.58ms)
    Download:  5573.76 Mbps (data used: 2.7 GB)
                  2.23 ms   (jitter: 11.80ms, low: 1.28ms, high: 217.82ms)
      Upload:  4938.13 Mbps (data used: 2.3 GB)
                  8.07 ms   (jitter: 0.70ms, low: 1.89ms, high: 9.32ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/bf2679b3-c852-41cd-b308-5c33688016cf

 

[RLWZ21]

just checking if anyone know of any free real time traffic monitoring apps?

 

[xiaofan]

just checking if anyone know of any free real time traffic monitoring apps?

Some routers have this function.

If you ralk about Windows laptop/desktop, you can use Task Manager --> Performance tab.

If you want to go deeper, then Wireshark is the tool often used by more adavnced users, not really a real time tool though.

 

[RLWZ21]

Some routers have this function.

If you ralk about Windows laptop/desktop, you can use Task Manager --> Performance tab.

If you want to go deeper, then Wireshark is the tool often used by more adavnced users, not really a real time tool though.

just checking which are the routers? as my tplink can only look into speed on app but on web login like cannot view anything real time.

well im looking at somewhat abit deeper as my google nest is kinda mess up after the new update on oct, as everything was okay just the the network isn't stable and the status are all green speed getting is as per my sign up speed but network stabilitiy kinda on and off even though connecting is good but this happens on wifi wired has no issues.

have run through some website fourm on the latest update on affecting user like me but so far i have seen is only a few not many and those commenting also not many. did those usual stuff like restart nework and reset and pair and the result still the same.

 

[xiaofan]

just checking which are the routers? as my tplink can only look into speed on app but on web login like cannot view anything real time.

well im looking at somewhat abit deeper as my google nest is kinda mess up after the new update on oct, as everything was okay just the the network isn't stable and the status are all green speed getting is as per my sign up speed but network stabilitiy kinda on and off even though connecting is good but this happens on wifi wired has no issues.

have run through some website fourm on the latest update on affecting user like me but so far i have seen is only a few not many and those commenting also not many. did those usual stuff like restart nework and reset and pair and the result still the same.

I see.

I am probably biased here but I will say just give up on Google's Nest WiFi (also the original Google WiFi), which I consider as one of the worst mesh solutions out there in the market (eg: no mesh function in AP mode).

As for routers, Asus has the traffic analyzers as well (but it may slow down the internet).
https://www.asus.com/sg/support/faq/114483/

OpenWRT can do that as well.

More advanced APs like Ubiquiti Unifi can do this as well. Some of the models can even do Spectrum Analysis real-time.

 

[xiaofan]

FreeBSD now includes OOkla Speedtest CLI in the 14.x version and 15-CURRENT version.

mcuee@freebsd142vmn100:~/build $ uname -r
14.2-RELEASE

mcuee@freebsd142vmn100:~ $ pkg search speedtest
ookla-speedtest-1.2.0          Internet connection measurement for developers
py311-speedtest-cli-2.1.3_1    Command line interface for testing internet bandwidth
speedtest-go-1.7.10_1          CLI and Go API to test internet speed using speedtest.net

mcuee@freebsd142vmn100:~/build $ which speedtest
/usr/local/bin/speedtest

mcuee@freebsd142vmn100:~/build $ speedtest -L
Closest servers:

    ID  Name                           Location             Country
==============================================================================
 13623  Singtel                        Singapore            Singapore
 63900  PT Cyber Network Indonesia     Singapore            Singapore
 31180  Campana                        Singapore            Singapore
 31293  Pacific Internet (S)           Singapore            Singapore
  4235  StarHub Ltd                    Singapore            Singapore
   367  NewMedia Express               Singapore            Singapore
  7311  M1 Limited                     Singapore            Singapore
  7556  PT FirstMedia                  Singapore            Singapore
  5935  MyRepublic Singapore           Singapore            Singapore
 50344  GSL Networks                   Singapore            Singapore
 26654  fdcservers.net                 Singapore            Singapore

mcuee@freebsd142vmn100:~/build $ speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.41 ms   (jitter: 0.23ms, low: 2.10ms, high: 2.59ms)
    Download:  2361.18 Mbps (data used: 1.7 GB)
                 50.92 ms   (jitter: 50.86ms, low: 1.71ms, high: 518.63ms)
      Upload:  2362.92 Mbps (data used: 1.1 GB)
                  3.12 ms   (jitter: 1.19ms, low: 1.38ms, high: 8.28ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/a158d09d-9dab-402b-9582-78b74423035c

 

[xiaofan]

Tools to check why 10Gbe NIC is slow.
https://forums.hardwarezone.com.sg/threads/5gbe-and-beyond-ethernet-adapters.7068236/page-24

I noticed after i installed asus xg c100c onto my desktop motherboard, my pc seems to slow down alot although my speedtest is able to hit 7000+ mbps.streaming YouTube, surfing net all these things . Not sure why. Thinking to changed my 16gb ddr4 ram to 32gb ram to see can improved or not .

Basic check with Task Manager is like a magic crystal ball
Adv. check with 20 Fixes and Solutions to Get Great 10GbE SPEEDS!

 

[xiaofan]

1. Cloudflare tool to check ISP routing information and Routing Anomalies
https://radar.cloudflare.com/routing/sg

fyi with multiple collateral Routing Anomalies events as well...

2. Cloudflare tool to check ISP Internet quality
https://radar.cloudflare.com/quality/sg

 

[xiaofan]

https://radar.cloudflare.com/routing/sg

From the following table we can probably see why Singtel is not really pushing for IPv6, even Starhub and M1 have quite a lot of IPv4 addresses.

Order byASN	Order byName	Order byCustomer cone size The number of downstream ASes relying on the given AS for transit	Order byIPv4 addresses	Order byIPv6 addresses
AS63949	Linode AS63949	1	1.4M	2.78e30
AS9506	Singapore Telecommunications Ltd, Magix Services	2	983K	3.17e29
AS55430	StarHub AS55430	1	726K	1.59e29
AS4657	StarHub AS4657	1574	715K	3.17e29
AS3758	SingNet	148	713K	2.38e29
AS4773	M1 Limited	2	402K	7.92e28
AS135354	NAVER BUSINESS PLATFORM ASIA PACIFIC	1	304K	0
AS150436	Byteplus Pte. Ltd.	1	275K	1.55e27
AS139341	ACE CDN	96	271K	2.71e26
AS17547	M1 Net Ltd	15	261K	1.35e30
AS138345	Starhub Ltd.	1	155K	2.39e25
AS45143	Singapore Telecom Mobile Pte Ltd	1	112K	7.92e28
AS4628	Pacific Internet Singapore	19	105K	7.92e28
AS17645	NTT DATA Enterprise Internet Services - SG	16	83K	7.92e28
AS931	Hyonix	322	76K	6.19e26
AS134963	Alibaba Cloud (Singapore) Private Limited	1	73K	9.67e24
AS4817	SIMBA Telecom	1	72K	9.90e28
AS963	N963	2	72K	7.92e28
AS139070	Google Cloud Korea	1	70K	1.98e28
AS140666	ANY DIGITAL PTE. LTD	1	70K	2.06e25

Order byASN	Order byName	Order byCustomer cone size The number of downstream ASes relying on the given AS for transit	Order byIPv4 addresses	Order byIPv6 addresses
AS140928	Starhub Ltd.	7	70K	1.21e24
AS197537	WISDOM CLOUD	5	68K	2.46e30
AS23856	SPTEL AS23856	31	68K	7.92e28
AS7472	National University of Singapore	1	66K	0
AS9419	Nanyang Technological University	1	66K	1.21e24
AS9649	Pacific Internet Pte Ltd	1	66K	1.21e24
AS9877	Ngee Ann Polytechnic	1	66K	0
AS45133	Singapore Polytechnic	1	66K	7.92e28
AS139190	Google Cloud Indonesia	1	66K	1.98e28
AS7473	Singapore Telecommunications Limited	2183	65K	7.92e28
AS10122	NETSTAR	17	62K	7.92e28
AS141995	Contabo Asia	1	60K	3.17e29
AS134078	Netpluz Asia Pte Ltd	1353	55K	7.92e28
AS4765	Pacific Internet Thailand	3	54K	7.92e28
AS18106	ViewQwest	31	53K	2.43e29
AS59253	Leaseweb Singapore	1	47K	7.92e28
AS135600	Whiz Communications Pte Ltd	1	46K	7.92e28
AS56300	MyRepublic Singapore	4	44K	9.28e26
AS139057	EdgeNext	168	43K	1.22e26
AS24319	Akamai Technologies, Inc.	1	39K	4.96e25