IPv6 discussions

bert64 · Nov 22, 2024

xiaofan said:
I will try to test with 2.5Gbe connection as well later: to see if Singtel 6rd IPv6 has higher penalty in this case compared with native IPv6 and IPv4.

I have not set up Singtel 6rd IPv6 on the 5Gbe connection yet.

[To be updated]

5% overhead is reasonable for a tunnel - encapsulation overhead plus extra latency, plus the state tracking which they do (most tunnels like he.net dont do state tracking).

xiaofan said:

Testing 6rd IPv6 performance with a 1Gbps Fibre internet connection with Singtel 6rd Internet.

Virtual OpenWRT with Singtel 6rd (1Gbps connection to upstream Singtel ONR) --> Ubuntu 22.04 LxC container, both running on Intel N100 CPU running Proxmox PVE 8.2, both assigned two virutal CPU core.

Using crusader to test both speed and latency.
https://github.com/Zoxc/crusader/

Crusader server: 4Gbps capable, on Linode Singapore, supports IPv4 and IPv6
https://github.com/Zoxc/crusader/issues/9
Last time I got similar results using IPv4 and IPv6, with native Singtel IPv6 connection on a Ubuntu 24.04 Linux physical machine with Intel N100 CPU. Okay you can say IPv6 result is slightly lower but no big difference.

Crusader v0.3.2 client: the above Ubuntu 22.04 LxC container.

From the below results, 6rd does have a bit of degradation, 900Mbps vs 940Mbps, about 5% performance loss with this 1Gbps connection, not too bad.

1) IPv6 test result

Bash:

root@ubuntu2204ct11:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink.taht.net
[2024-11-21 11:42:23] Client version 0.3.2 running
[2024-11-21 11:42:23] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-21 11:42:25] Idle latency to server 3.53 ms
[2024-11-21 11:42:27] Testing download...
[2024-11-21 11:43:57] Testing upload...
[2024-11-21 11:45:27] Testing both download and upload...

-- Download test --
          Throughput: 902.59 Mbps
             Latency: 42.5 ms (40.5 ms down, 2.0 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 902.50 Mbps
             Latency: 13.6 ms (1.1 ms down, 12.4 ms up)
         Packet loss: 0% down, 0.01% up

-- Bidirectional test --
          Throughput: 1497.84 Mbps (894.39 Mbps down, 603.45 Mbps up)
             Latency: 41.0 ms (38.9 ms down, 2.2 ms up)
         Packet loss: 0% down, 0.03% up

[2024-11-21 11:47:01] Writing data...
[2024-11-21 11:47:01] Saved raw data as crusader-results/test 2024-11-21 11.47.01.crr
[2024-11-21 11:47:01] Saved plot as crusader-results/test 2024-11-21 11.47.01.png

2) IPv4 results of the same LxC container with IPv6 disabled --> it can basically run at more or less the full bandwidth.

Bash:

root@ubuntu2204ct11:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink.taht.net
[2024-11-21 11:49:33] Client version 0.3.2 running
[2024-11-21 11:49:33] Connected to server 172.236.148.60:35481
[2024-11-21 11:49:35] Idle latency to server 2.49 ms
[2024-11-21 11:49:37] Testing download...
[2024-11-21 11:51:07] Testing upload...
[2024-11-21 11:52:37] Testing both download and upload...

-- Download test --
          Throughput: 941.28 Mbps
             Latency: 47.7 ms (46.2 ms down, 1.5 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 941.11 Mbps
             Latency: 181.6 ms (1.0 ms down, 180.5 ms up)
         Packet loss: 0% down, 0.03% up

-- Bidirectional test --
          Throughput: 1713.35 Mbps (816.72 Mbps down, 896.63 Mbps up)
             Latency: 72.1 ms (3.2 ms down, 68.9 ms up)
         Packet loss: 0.04% down, 0.02% up

[2024-11-21 11:54:11] Writing data...
[2024-11-21 11:54:11] Saved raw data as crusader-results/test 2024-11-21 11.54.11.crr
[2024-11-21 11:54:11] Saved plot as crusader-results/test 2024-11-21 11.54.11.png

3) OOkla SpeedTest (IPv4) as a reference. The speed is higher than Crusader which I attribute to difference in implementation.

Bash:

root@ubuntu2204ct11:~/ookla# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.45 ms   (jitter: 0.18ms, low: 1.09ms, high: 1.55ms)
    Download:   950.19 Mbps (data used: 428.8 MB)                                                  
                  7.65 ms   (jitter: 0.43ms, low: 1.50ms, high: 8.89ms)
      Upload:   948.67 Mbps (data used: 430.3 MB)                                                  
                 34.20 ms   (jitter: 2.16ms, low: 2.23ms, high: 36.83ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/5d3ee5f4-ebef-4a9a-a401-d2a5dbfad593

latency is much higher on legacy ip, that suggests your local link is close to saturation, given that 6rd cannot saturate the link it would seem you're hitting the limit of the tunnel server (maybe the tunnel server has 1gbps total and other users using it?)

xiaofan · Nov 22, 2024

bert64 said:
latency is much higher on legacy ip, that suggests your local link is close to saturation, given that 6rd cannot saturate the link it would seem you're hitting the limit of the tunnel server (maybe the tunnel server has 1gbps total and other users using it?)

1. More tests to be done to see the tunnel behaviour

In this paticular test, native IPv6 does not work on the router because of upstream WAN link limitation, so I can only test 6rd tunnel vs legacy IPv4.

OpenWRT virtual router has 1Gbps WAN link, with 6rd IPv6 running.

6rd IPv6 test is with the Linux LxC container LAN client with SLAAC IPv6 network configuration.

Legacy IPv4 test is with the same LxC container with different network configuration (IPv6 disabled). In this case, I think it should not use the 6rd tunnel, even though the OpenWRT virtual router has 6rd IPv6 running. But NAT is present in this case.

I will carry out more tests to see if this is consistent or not.

I will try other test setups to see if I can test native IPv6 vs 6rd tunnel vs IPv4.

2. No much difference in terms of latency and speed when I tested native IPv6 vs legacy IPv4 last time.

Test was done with the Intel N100 mini PC running Ubuntu 24.04, physical machine, not a VM
(Singtel 5Gbps plan, ZTE F8648P ONR) and the results are actually pretty good and similar for both IPv4 and IPv6.
https://github.com/Zoxc/crusader/issues/9

Main router --> Singtel ONR with native IPv6 enabled from Singtel backend (ONR not bridged).

1) native IPv6

Bash:

mcuee@miniroute10g:~/build/networking/crusader_bin/v0.3.1$ ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink.taht.net
[2024-10-03 12:48:51] Client version 0.3.1 running
[2024-10-03 12:48:51] Connected to server 172.236.148.60:35481
[2024-10-03 12:48:53] Idle latency to server 2.73 ms
[2024-10-03 12:48:55] Testing download...
[2024-10-03 12:50:25] Testing upload...
[2024-10-03 12:51:55] Testing both download and upload...
[2024-10-03 12:53:33] Warning: Load termination timed out. There may be residual untracked traffic in the background.

-- Download test --
          Throughput: 3927.62 Mbps
             Latency: 2.4 ms (0.8 ms down, 1.7 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 4935.37 Mbps
             Latency: 9.9 ms (0.9 ms down, 9.0 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 8427.17 Mbps (3579.31 Mbps down, 4847.85 Mbps up)
             Latency: 9.7 ms (1.0 ms down, 8.7 ms up)
         Packet loss: 0% down, 0.02% up

[2024-10-03 12:53:33] Writing data...
[2024-10-03 12:53:33] Saved raw data as crusader-results/crusader-results/test 2024-10-03 12.53.33.crr
[2024-10-03 12:53:33] Saved plot as crusader-results/test 2024-10-03 12.53.33.png

2) legacy IPv4 (NAT involved)

mcuee@miniroute10g:~/build/networking/crusader_bin/v0.3.1$ ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 2600:3c15::f03c:95ff:fe7e:75a2
[2024-10-03 12:57:07] Client version 0.3.1 running
[2024-10-03 12:57:07] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-10-03 12:57:09] Idle latency to server 2.82 ms
[2024-10-03 12:57:11] Testing download...
[2024-10-03 12:58:46] Testing upload...
[2024-10-03 13:00:16] Testing both download and upload...
[2024-10-03 13:01:50] Warning: Load termination timed out. There may be residual untracked traffic in the background.

-- Download test --
          Throughput: 4057.47 Mbps
             Latency: 2.8 ms (1.0 ms down, 1.8 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 4873.10 Mbps
             Latency: 10.3 ms (0.1 ms down, 10.2 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 8280.88 Mbps (3577.16 Mbps down, 4703.72 Mbps up)
             Latency: 10.2 ms (0.6 ms down, 9.6 ms up)
         Packet loss: 0%

[2024-10-03 13:01:50] Writing data...
[2024-10-03 13:01:50] Saved raw data as crusader-results/crusader-results/test 2024-10-03 13.01.50.crr
[2024-10-03 13:01:51] Saved plot as crusader-results/test 2024-10-03 13.01.50.png

xiaofan · Nov 22, 2024

bert64 said:
5% overhead is reasonable for a tunnel - encapsulation overhead plus extra latency, plus the state tracking which they do (most tunnels like he.net dont do state tracking).

Thanks for the insight.

I will try to test two more test cases: 2.5Gbps uplink and 5Gbps uplink.

For the 5Gbps uplink test case, I will need to test this on the main OpenWRT virtual router which can be disruptive to the home network. So it may not happen soon.

BTW, previously I could not carry out latency related test over the internet easily because of the lack of fast public test server (OOkla or OpenSpeedTest or libreSpeed servers, Waveform.com bufferbloat test servers, public iperf3 test servers, public flent test servers, or public crusader test servers). It is even more difficult to find suitable servers which support IPv6. I had to set up the test server by myself.

Now at least the crusader Singapore server hosted on linode by Dave Täht is pretty fast (up to 4Gbps).

astones153 · Nov 22, 2024

ok wow it seems a lot of websites really fucking hate any traffic that comes from the HE tunnel IPv6 subnet (2001:470::/32). I got locked out of reddit while browsing without an account, google constantly redirects me to google.com.hk, I see why people run into problems with this. Seems it can only be useful for self-hosting and testing stuff. At least I get to have VLANs and subnets in IPv6 now.

tomb · Nov 22, 2024

bert64 said:
Can you try using your phone on its own mobile data as a client? (is the phone also using M1?)

You have fully routable addressing with v6, so it should be able to connect from anywhere so long as the router doesn't have firewall rules to block it. If it's not working then most likely your rules are incorrect.

If you don't have anywhere outside to test from, there are various ipv6 portscan sites eg http://www.ipv6scanner.com, you will find that tcp scanning is more reliable than udp (which wireguard uses) but you can set up some tcp test service to verify that your rules are correct first, and then switch them to udp for wireguard.

As far as i know M1 uses CGNAT for legacy traffic - ie does your legacy wan address start 100.x or 10.x? do a whois lookup and see if it actually belongs to M1 or if its reserved address space. In this scenario you won't be able to use legacy ip for any kind of server hosted there, v6 is your only option.

Also you've not assigned any v6 address range for inside the tunnel.

I tried with a phone on Simba with public v6 address and it did not work.

http://www.ipv6scanner.com shows the port and all common ports as filtered. It seems that OpenWRT blocks inbound connections to v6 addresses. This seems to be the issue (I hope). I will try to work out how to solve this.

The address starts with 10.x and is detected as a private IP address by www.whatismyip.com.

For v6 address range, I will use fd00::1 for server and fd00::x for the clients. Is this correct?

Thanks for your advice so far.

bert64 · Nov 23, 2024

tomb said:
I tried with a phone on Simba with public v6 address and it did not work.

http://www.ipv6scanner.com shows the port and all common ports as filtered. It seems that OpenWRT blocks inbound connections to v6 addresses. This seems to be the issue (I hope). I will try to work out how to solve this.

The address starts with 10.x and is detected as a private IP address by www.whatismyip.com.

For v6 address range, I will use fd00::1 for server and fd00::x for the clients. Is this correct?

Thanks for your advice so far.

Yes the OpenWRT firewall will block inbound by default, you will need to add rules to allow the traffic. That shouldn't be too difficult for you to troubleshoot if you have external tools to check ports from.

Simba also blocks inbound traffic at the telco level, but M1 does not.

For internal v6 you are technically supposed to pick a random range within the ULA space to reduce the risk of conflicts (eg if you connect to the VPN from somewhere else that is also using fd00::/64 you will have problems). Using fd00::/64 will work subject to the above caveat.

Ideally you would use fully routable address space delegated by the ISP, but unfortunately M1 only give you a single /64. The standard is /56. With ULA space you will be able to access your own internal devices, but not route back out.

The 10.x legacy IP is behind NAT, so no chance of inbound with that.

bert64 · Nov 23, 2024

astones153 said:
ok wow it seems a lot of websites really fucking hate any traffic that comes from the HE tunnel IPv6 subnet (2001:470::/32). I got locked out of reddit while browsing without an account, google constantly redirects me to google.com.hk, I see why people run into problems with this. Seems it can only be useful for self-hosting and testing stuff. At least I get to have VLANs and subnets in IPv6 now.

That's weird because www.reddit.com does not support v6, you have to go to https://ipv6.reddit.com to get v6 support currently.

But yes the he.net tunnel being a free service gets heavily abused, and is widely blacklisted.

The behaviour of google is a pet hate of mine... It uses it what it believes to be your location based on geoip to set the language, when the HTTP standard has had the Accept-Language header since pre HTTP/1.0.
This ls exceptionally annoying when you're traveling, eg if you're transiting in dubai and it serves you up a page in arabic despite the fact that your browser is explicitly asking for english.

xiaofan · Nov 23, 2024

Retest today 23-Nov-2024:

6rd performance drop is about 5% in terms of speed, no obvious latency issues either, both with WAN side and LAN client.

Singtel ONR bridged --> Singtel 6rd IPv6 on the virtual OpenWRT setup (1Gbps capable WAN/LAN, no Double NAT), Intel N100 Mini PC running Proxmox PVE 8.2 --> Debain 12 LxC container on PVE 8.2 (SLAAC)

Bash:

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'dda5:edda:cf50::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth1'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.38.1'
        option netmask '255.255.255.0'
        option delegate '0'
        option ip6assign '64'
        list ip6class 'wan6'

config interface 'wan'
        option device 'eth0'
        option proto 'dhcp'

config interface 'wan6'
        option proto '6rd'
        option peeraddr '202.166.127.6'
        option ip6prefix '2400:d803::'
        option ip6prefixlen '32'

root@OpenWrt:~# cat /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option cachesize '1000'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'
        option ednspacket_max '1232'
        option filter_aaaa '0'
        option filter_a '0'
        list server '/mask.icloud.com/'
        list server '/mask-h2.icloud.com/'
        list server '/use-application-dns.net/'
        list server '127.0.0.1#5053'
        list server '127.0.0.1#5054'
        option doh_backup_noresolv '-1'
        option noresolv '1'
        list doh_backup_server '/mask.icloud.com/'
        list doh_backup_server '/mask-h2.icloud.com/'
        list doh_backup_server '/use-application-dns.net/'
        list doh_backup_server '127.0.0.1#5053'
        list doh_backup_server '127.0.0.1#5054'
        list doh_server '127.0.0.1#5053'
        list doh_server '127.0.0.1#5054'
        option serversfile '/var/run/adblock-fast/dnsmasq.servers'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv4 'server'
        option ra 'server'
        option ndp 'relay'
        option dhcpv6 'server'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

config dhcp 'wan6'
        option interface 'wan6'
        option ignore '1'
        option ra 'relay'
        option dhcpv6 'relay'
        option ndp 'relay'

OpenWRT WAN side Crusader IPv6 test results

Bash:

root@OpenWrt:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --str
eam-stagger 4 singapore.starlink.taht.net
[2024-11-23 09:05:06] Client version 0.3.2 running
[2024-11-23 09:05:06] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-23 09:05:08] Idle latency to server 3.48 ms
[2024-11-23 09:05:10] Testing download...
[2024-11-23 09:06:40] Testing upload...
[2024-11-23 09:08:10] Testing both download and upload...

-- Download test --
          Throughput: 902.54 Mbps
             Latency: 42.5 ms (40.7 ms down, 1.7 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 902.51 Mbps
             Latency: 13.6 ms (1.5 ms down, 12.1 ms up)
         Packet loss: 0% down, 0.08% up

-- Bidirectional test --
          Throughput: 1573.33 Mbps (893.69 Mbps down, 679.65 Mbps up)
             Latency: 41.0 ms (39.3 ms down, 1.7 ms up)
         Packet loss: 0% down, 0.01% up

[2024-11-23 09:09:44] Writing data...
[2024-11-23 09:09:45] Saved raw data as crusader-results/test 2024-11-23 09.09.44.crr
[2024-11-23 09:09:45] Saved plot as crusader-results/test 2024-11-23 09.09.44.png

OOkla SpeedTest for the WAN (OpenWRT router), using IPv4 (Singtel SpeedTest server is only with IPv4, M1 SpeedTest IPv6 test server has issues with Singtel because of bad routing)

Bash:

root@OpenWrt:~# ./speedtest 0s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.48 ms   (jitter: 0.29ms, low: 1.30ms, high: 1.88ms)
    Download:   949.61 Mbps (data used: 427.6 MB)                                                  
                  7.23 ms   (jitter: 0.39ms, low: 1.25ms, high: 8.55ms)
      Upload:   948.99 Mbps (data used: 428.8 MB)                                                  
                 33.72 ms   (jitter: 2.05ms, low: 1.45ms, high: 35.89ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/170f1987-bde9-4b5b-83cb-db02f302b748

OpenWRT LAN client Crusader IPv6 test results, Linux LxC container, SLAAC configuration.

Bash:

root@debian12ct11:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink.taht.net
[2024-11-23 09:33:34] Client version 0.3.2 running
[2024-11-23 09:33:34] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-23 09:33:36] Idle latency to server 3.71 ms
[2024-11-23 09:33:38] Testing download...
[2024-11-23 09:35:08] Testing upload...
[2024-11-23 09:36:38] Testing both download and upload...

-- Download test --
          Throughput: 902.45 Mbps
             Latency: 42.7 ms (40.8 ms down, 1.9 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 901.73 Mbps
             Latency: 13.7 ms (1.7 ms down, 12.1 ms up)
         Packet loss: 0% down, 0.02% up

-- Bidirectional test --
          Throughput: 1529.63 Mbps (894.70 Mbps down, 634.93 Mbps up)
             Latency: 41.4 ms (39.8 ms down, 1.6 ms up)
         Packet loss: 0% down, 0.03% up

[2024-11-23 09:38:12] Writing data...
[2024-11-23 09:38:12] Saved raw data as crusader-results/test 2024-11-23 09.38.12.crr
[2024-11-23 09:38:12] Saved plot as crusader-results/test 2024-11-23 09.38.12.png

OOkla SpeedTest for the LAN client, using IPv4 (Singtel SpeedTest server is only with IPv4, M1 SpeedTest IPv6 test server has issues with Singtel because of bad routing)

Bash:

root@debian12ct11:~# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.46 ms   (jitter: 0.10ms, low: 1.35ms, high: 1.53ms)
    Download:   950.44 Mbps (data used: 427.5 MB)                                                  
                  7.13 ms   (jitter: 0.52ms, low: 0.94ms, high: 9.24ms)
      Upload:   948.76 Mbps (data used: 428.0 MB)                                                  
                 34.27 ms   (jitter: 1.73ms, low: 1.43ms, high: 36.62ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/4bc9b0ac-5549-4b89-bb5f-be9ef40f4bfd

astones153 · Nov 23, 2024

Seems like APNIC themselves also don't care about proper prefix delegations, maybe this is why most other ISPs in SEA also give out /64 prefixes.

An LIR can assign a /64 to /48 to an end site customer network based on their requirements. The following guidelines may be useful:

/64 where it is known that only one subnet is required.

/56 for small sites where it is expected only a few subnets will be required within the next two years. Subscribers can receive a /56 when connecting through on-demand or always-on connections such as small office and home office enterprises.

/48 for larger sites, or if an end site is expected to grow into a large network.

Mach3.2 · Nov 23, 2024

astones153 said:
Seems like APNIC themselves also don't care about proper prefix delegations, maybe this is why most other ISPs in SEA also give out /64 prefixes.

/64 is disgusting, and unnecessarily stingy.

xiaofan · Nov 23, 2024

astones153 said:
Seems like APNIC themselves also don't care about proper prefix delegations, maybe this is why most other ISPs in SEA also give out /64 prefixes.

You may want to read the following blog --> the chief scientist of APNIC seems to indicate that IPv6 transition may not go much futher already because of the architecture changes of the Internet and IPv4/NATs perform the same function adequately as IPv6.

1) From the blog of APNIC chief scientist.
https://blog.apnic.net/2024/10/22/the-ipv6-transition/

Now, let’s return to the situation of the transition to IPv6. The responsibility falls on networks and network operators to invest in transitioning to a dual-stack platform initially, with the eventual goal of phasing out IPv4 support. But this change is not really visible, or even crucial, to the content or service world. If IPv4 and NATs perform the carriage function adequately, then there is no motivation for the content and service operators to pay a network a premium to have a dual-stack platform.
...
The implication of these observations is that the transition to IPv6 is progressing very slowly not because this industry is chronically short-sighted. There is something else going on here. IPv6 alone is not critical to a large set of end-user service delivery environments. We’ve been able to take a 1980s address-based architecture and scale it more than a billion-fold by altering the core reliance on distinguisher tokens from addresses to names. There was no real lasting benefit in trying to leap across to just another 1980s address-based architecture (with only a few annoying stupid differences, apart from longer addresses!).
...

2) The above is a very big difference from his 2022 post.
https://blog.apnic.net/2022/05/04/the-transition-to-ipv6-are-we-there-yet/

Are we there yet with IPv6?
Well, clearly no.
But we are closing in. If the endpoint is to provide an IPv6-only service to customers that meet their requirements, then ongoing efforts of IPv6 adoption in the content and service platform space are having some impact on when we might reach this point.

xiaofan · Nov 23, 2024

From another configuration and I can see 6rd has big performance penalty in terms of speed (more than 50%). Maybe this is because of multiple links. I will try a simpler setup to see how it goes.

Main virtual OpenWRT router with native IPv6 --> Asus TUF-BE6500 with IPv6 disabled (2.5G WAN/LAN)--> secondary virtual OpenWRT router using 6rd IPv6 (2.5G WAN/LAN).

Bash:

root@OpenWrt:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink
.taht.net
[2024-11-23 10:47:51] Client version 0.3.2 running
[2024-11-23 10:47:52] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-23 10:47:53] Idle latency to server 3.82 ms
[2024-11-23 10:47:55] Testing download...
[2024-11-23 10:49:25] Testing upload...
[2024-11-23 10:50:56] Testing both download and upload...

-- Download test --
          Throughput: 811.05 Mbps
             Latency: 24.9 ms (23.3 ms down, 1.7 ms up)
         Packet loss: 0%

-- Upload test --
          Throughput: 1060.71 Mbps
             Latency: 5.8 ms (1.8 ms down, 4.1 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 1137.54 Mbps (693.45 Mbps down, 444.09 Mbps up)
             Latency: 28.3 ms (26.2 ms down, 2.1 ms up)
         Packet loss: 0.01% down, 0% up

[2024-11-23 10:52:30] Writing data...
[2024-11-23 10:52:30] Saved raw data as crusader-results/test 2024-11-23 10.52.30.crr
[2024-11-23 10:52:30] Saved plot as crusader-results/test 2024-11-23 10.52.30.png

Running OOKla Speedtest from the seondary OpenWRT router

Bash:

root@OpenWrt:~# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.52 ms   (jitter: 0.22ms, low: 2.26ms, high: 2.67ms)
    Download:  2344.70 Mbps (data used: 1.9 GB)
                 36.73 ms   (jitter: 41.97ms, low: 2.11ms, high: 255.43ms)
      Upload:  2363.02 Mbps (data used: 1.2 GB)
                 11.89 ms   (jitter: 1.72ms, low: 1.62ms, high: 20.87ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/185a774d-c881-4d50-b6d4-c82b21c0c747

xiaofan · Nov 23, 2024

Crusader and OOkla Speedtest also work on the Asus TUF-BE6500 (Qualcomm IPQ5322 CPU, quad-core Arm Cortex A53 at 1.5GHz).

I can see that enable 6rd tunnel IPv6 will cause quite a bit of speed drop.
Download has 31% penalty and no idea why upload is super slow.

Main virtual OpenWRT router with native IPv6 --> Asus TUF-BE6500 with IPv6 disabled (2.5G WAN/LAN)

Running OOKla Speedtest from the TUF-BE6500 router, using IPv4

Bash:

xiaofan@TUF_6500-5020:/tmp/opt/root/ookla# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     2.19 ms   (jitter: 0.32ms, low: 1.83ms, high: 2.53ms)
    Download:  2367.49 Mbps (data used: 1.1 GB)
                 10.45 ms   (jitter: 30.93ms, low: 1.04ms, high: 218.40ms)
      Upload:  2369.90 Mbps (data used: 2.0 GB)
                  2.91 ms   (jitter: 0.69ms, low: 1.29ms, high: 5.05ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/2bb29787-8025-434b-ad10-4ea1fd2691b8

Running Crusader using IPv4 --> results are in line with OOkla SpeedTest.

Bash:

xiaofan@TUF_6500-5020:/tmp/opt/root/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger
 4 singapore.starlink.taht.net
[2024-11-23 11:34:22] Client version 0.3.2 running
[2024-11-23 11:34:22] Connected to server 172.236.148.60:35481
[2024-11-23 11:34:24] Idle latency to server 2.91 ms
[2024-11-23 11:34:26] Testing download...
[2024-11-23 11:35:56] Testing upload...
[2024-11-23 11:37:26] Testing both download and upload...

-- Download test --
          Throughput: 2324.82 Mbps
             Latency: 4.8 ms (3.5 ms down, 1.3 ms up)
         Packet loss: 0.12% down, 0% up

-- Upload test --
          Throughput: 2304.24 Mbps
             Latency: 5.8 ms (1.6 ms down, 4.2 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 4274.01 Mbps (2099.28 Mbps down, 2174.73 Mbps up)
             Latency: 8.6 ms (4.4 ms down, 4.2 ms up)
         Packet loss: 0.33% down, 0% up

[2024-11-23 11:39:01] Writing data...
[2024-11-23 11:39:01] Saved raw data as crusader-results/test 2024-11-23 11.39.00.crr
[2024-11-23 11:39:01] Saved plot as crusader-results/test 2024-11-23 11.39.00.png

Once I enable 6rd IPv6 on the Asus TUF-BE6500 router, I can clearly see the penalty of 6rd tunnel (download 31% penalty). No idea why upload is so slow.

Main virtual OpenWRT router with native IPv6 --> Asus TUF-BE6500 with 6rd IPv6 (2.5G WAN/LAN)

Bash:

xiaofan@TUF_6500-5020:/tmp/opt/root/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger
 4 singapore.starlink.taht.net
[2024-11-23 11:24:46] Client version 0.3.2 running
[2024-11-23 11:24:46] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-23 11:24:48] Idle latency to server 3.63 ms
[2024-11-23 11:24:50] Testing download...
[2024-11-23 11:26:20] Testing upload...
[2024-11-23 11:27:50] Testing both download and upload...

-- Download test --
          Throughput: 1598.18 Mbps
             Latency: 16.5 ms (14.6 ms down, 1.8 ms up)
         Packet loss: 0.09% down, 0% up

-- Upload test --
          Throughput: 18.03 Mbps
             Latency: 3.7 ms (1.9 ms down, 1.8 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 1472.84 Mbps (1466.32 Mbps down, 6.52 Mbps up)
             Latency: 18.4 ms (16.7 ms down, 1.6 ms up)
         Packet loss: 0.04% down, 0% up

[2024-11-23 11:29:25] Writing data...
[2024-11-23 11:29:25] Saved raw data as crusader-results/test 2024-11-23 11.29.25.crr
[2024-11-23 11:29:25] Saved plot as crusader-results/test 2024-11-23 11.29.25.png

tomb · Nov 25, 2024

bert64 said:
Yes the OpenWRT firewall will block inbound by default, you will need to add rules to allow the traffic. That shouldn't be too difficult for you to troubleshoot if you have external tools to check ports from.

Simba also blocks inbound traffic at the telco level, but M1 does not.

For internal v6 you are technically supposed to pick a random range within the ULA space to reduce the risk of conflicts (eg if you connect to the VPN from somewhere else that is also using fd00::/64 you will have problems). Using fd00::/64 will work subject to the above caveat.

Ideally you would use fully routable address space delegated by the ISP, but unfortunately M1 only give you a single /64. The standard is /56. With ULA space you will be able to access your own internal devices, but not route back out.

The 10.x legacy IP is behind NAT, so no chance of inbound with that.

I have disabled firewall on the openwrt but ports still seem to be closed. is it a telco issue?

bert64 · Nov 25, 2024

tomb said:
I have disabled firewall on the openwrt but ports still seem to be closed. is it a telco issue?

Have you verified that the ports are open/unfiltered on your target machine?

It never used to be blocked, can you try putting the simcard into a phone and see if you get the same results?

tomb · Nov 26, 2024

bert64 said:
Have you verified that the ports are open/unfiltered on your target machine?

It never used to be blocked, can you try putting the simcard into a phone and see if you get the same results?

tried on a phone and ports are detected as closed.

Update: Finally got wireguard to work with an IPv6 endpoint using another sim router (Telit) and a different M1 sim card. Did not have to do much config on this router. I will verify if it's a router or sim card issue.

bert64 · Nov 27, 2024

tomb said:
tried on a phone and ports are detected as closed.

Update: Finally got wireguard to work with an IPv6 endpoint using another sim router (Telit) and a different M1 sim card. Did not have to do much config on this router. I will verify if it's a router or sim card issue.

Closed implies that the traffic reaches the phone, and the phone is rejecting connections to the ports.... So looks like the connection is open, most likely it's firewall rules on your router blocking it.

roidred · Nov 27, 2024

Anyone using HP MSR954 router for IPV6. I need some help in configuration part. I want to propagate the received /64 prefix from starhub on the WAN port to LAN port. How to bridge it or other ways to do that.

bert64 · Nov 27, 2024

roidred said:
Anyone using HP MSR954 router for IPV6. I need some help in configuration part. I want to propagate the received /64 prefix from starhub on the WAN port to LAN port. How to bridge it or other ways to do that.

You don't bridge, you route...
You get a WAN address (one address in a /64 shared with other customers) and a prefix delegation that you can apply to LAN, eg:

WAN: 2001:db8:666:666::123/64
Delegated prefix: 2001:db8:111:111::/64

You need to configure your router as a DHCPv6 client, and to receive a prefix via prefix delegation, and then route that delegated prefix to LAN. Sorry i'm not familiar with the HP MSR954 so can't advise on how to actually configure it there.

xiaofan · Nov 27, 2024

Just switched my main OpenWRT router from Singtel native IPv6 to Singtel 6rd to see the penality of the 6rd tunnel, Intel N100 CPU based mini PC, two virutal core assigned to the OpenWRT VM, virutal Linux bridge as WAN/LAN.

Reference IPv4 WAN (Internet speed), Singtel 5Gbps, 6rd tunnel does not affect IPv4 speed.

Bash:

root@OpenWrt:~/ookla# ./speedtest -s 13623

   Speedtest by Ookla

      Server: Singtel - Singapore (id: 13623)
         ISP: Singtel Fibre
Idle Latency:     1.32 ms   (jitter: 0.22ms, low: 0.97ms, high: 1.45ms)
    Download:  5635.14 Mbps (data used: 2.5 GB)
                  1.81 ms   (jitter: 0.81ms, low: 1.11ms, high: 5.35ms)
      Upload:  4977.62 Mbps (data used: 2.4 GB)
                  6.63 ms   (jitter: 0.50ms, low: 1.25ms, high: 7.25ms)
 Packet Loss:     0.0%
  Result URL: https://www.speedtest.net/result/c/3616a3d2-707e-4118-92fa-d1e5c533b315

6rd tunnel does seem to have quite a bit of penality on the upload speed, and dual birection speed got greatly reduced. Download speed is not affected as the test server is limited to 4Gbps.

The good thing is that the tunnel is at least able to cope with 4Gbps.

Bash:

root@OpenWrt:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink
.taht.net
[2024-11-27 12:01:50] Client version 0.3.2 running
[2024-11-27 12:01:51] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-27 12:01:52] Idle latency to server 2.88 ms
[2024-11-27 12:01:54] Testing download...
[2024-11-27 12:03:24] Testing upload...
[2024-11-27 12:04:54] Testing both download and upload...

-- Download test --
          Throughput: 4034.61 Mbps
             Latency: 3.6 ms (2.2 ms down, 1.4 ms up)
         Packet loss: 0.18% down, 0% up

-- Upload test --
          Throughput: 3071.63 Mbps
             Latency: 7.1 ms (1.6 ms down, 5.4 ms up)
         Packet loss: 0.04% down, 0% up

-- Bidirectional test --
          Throughput: 5663.09 Mbps (2905.34 Mbps down, 2757.75 Mbps up)
             Latency: 8.8 ms (6.8 ms down, 2.0 ms up)
         Packet loss: 0.15% down, 0% up

[2024-11-27 12:06:29] Writing data...
[2024-11-27 12:06:29] Saved raw data as crusader-results/test 2024-11-27 12.06.29.crr
[2024-11-27 12:06:29] Saved plot as crusader-results/test 2024-11-27 12.06.29.png

Switching back to native IPv6 and carry out the test again. Somehow the download is a bit slower but I can see the test server is not limited on the upload speed.

Bash:

root@OpenWrt:~/crusader_bin/v0.3.2# ./crusader test --load-duration 60 --streams 8 --stream-stagger 4 singapore.starlink
.taht.net
[2024-11-27 12:22:40] Client version 0.3.2 running
[2024-11-27 12:22:41] Connected to server [2600:3c15::f03c:95ff:fe7e:75a2]:35481
[2024-11-27 12:22:42] Idle latency to server 2.31 ms
[2024-11-27 12:22:44] Testing download...
[2024-11-27 12:24:15] Testing upload...
[2024-11-27 12:25:45] Testing both download and upload...

-- Download test --
          Throughput: 3812.55 Mbps
             Latency: 2.1 ms (0.7 ms down, 1.4 ms up)
         Packet loss: 0.03% down, 0% up

-- Upload test --
          Throughput: 4873.02 Mbps
             Latency: 9.5 ms (0.8 ms down, 8.7 ms up)
         Packet loss: 0%

-- Bidirectional test --
          Throughput: 8022.09 Mbps (3174.82 Mbps down, 4847.27 Mbps up)
             Latency: 9.5 ms (0.9 ms down, 8.6 ms up)
         Packet loss: 0.02% down, 0% up

[2024-11-27 12:27:19] Writing data...
[2024-11-27 12:27:19] Saved raw data as crusader-results/test 2024-11-27 12.27.19.crr
[2024-11-27 12:27:19] Saved plot as crusader-results/test 2024-11-27 12.27.19.png

IPv6 discussions

Senior Member

High Supremacy Member

High Supremacy Member

Member

High Supremacy Member

Senior Member

Senior Member

High Supremacy Member

Member

Great Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

High Supremacy Member

Senior Member

High Supremacy Member

Senior Member

Member

Senior Member

High Supremacy Member