IPv6 discussions

[bert64]

Okay, Google seems to say that some routers respond to traceroute/mtr command with internal / LAN side IPv4/IPv6 address and some respond with external / WAN side IPv6 address. But I have only seen the first case (internal, LAN side IPv4/IPv6 address).

LLA fe80::xxxx:xxxx:xxxx:63ae (last 64 bits matching the last 64 bits of the first hop in the mtr record: 2400:d802:dd3:xxxx:xxxx:xxxx:xxxx:63ae, which makes sense)
GUA 2400:d802:d10::xxxx
DNS 2400:d800::1 and 2400:d800::2
IPv6 Gateway fe80::200:xxxx:xxxx:145 (LLA address of the WAN??)

Still have two questions (still got a lot to learn about IPv6).
1) How do I confirm the above IPv6 gateway address is the LLA of the WAN interface?

Okay, I can answer this one, from the Web UI I know the WAN MAC address and from there I can get the LLA address. So yes the above IPv6 gateway is the LLA of the WAN interface.

Lazy to do it manually, just used the online tool here.
https://nettools.club/mac2ipv6

2) How do I confirm that I have /56 or /64?

Maybe the answer to the second question is that it is not possible to know since I cannot SSH into the Singtel ONR, the price to pay now that I cannot bridge the Singtel XGS-PON ONR...

Your gateway will usually be the LL address, but what shows in a traceroute will usually be the LAN address of the router (which may be ::1, may be based on EUI-64 etc). When the router sends traffic (eg ICMPv6 responses to your traceroute) to your client it will source the traffic as per its routing table. While traffic sourced from an LL address will work to a directly connected device it will not cross routers, so sending from a GUA wherever possible is the most sane default.

Your router (any router in fact) does not technically need to have an address within your LAN GUA space, and since it can't send traffic with the source address set to its LL it will either show as a timeout on any traceroute, or source a different GUA (eg the WAN address) as the source of the traffic.
I have a situation the other way round here, the WAN side does not get allocated a GUA address at all but communicates with the upstream routers via LL, if you traceroute to a LAN device from the outside you see the LAN side address of the router before the final host.

For a /56, the last 2 bytes of the block will usually be 00, for instance:
2400:d802:d10:1100::/64
The router usually picks the first /64 but technically you have 00 - FF (256 subnets) available to you.

Lack of downstream PD basically wastes the /56, some ISP routers do support downstream PD (eg Comcast and AT&T in the US). AT&T at least use several of the /64 networks for their own purposes (voip, iptv etc)

If you use "passthrough" then it's basically bridging v6 traffic and natting legacy traffic, you might as well just use a dumb access point which will bridge everything.

 

[bert64]

Thanks. Then it is quite similar to Singtel.

Singtel 5G SA -- IPv6
Singtel 5G NAS -- inconsistent IPv6
Singtel 4G -- no IPv6

Singtel will enable v6 for you on 4G if you call and speak to the right people, and you enable it manually on your handset (easy on android, painful on iOS since you need to use apple mobile configurator to make a profile).

Since they've enabled it on 5GSA they might have updated the iOS carrier bundle so the latter may no longer be necessary.

 

[xiaofan]

Your gateway will usually be the LL address, but what shows in a traceroute will usually be the LAN address of the router (which may be ::1, may be based on EUI-64 etc). When the router sends traffic (eg ICMPv6 responses to your traceroute) to your client it will source the traffic as per its routing table. While traffic sourced from an LL address will work to a directly connected device it will not cross routers, so sending from a GUA wherever possible is the most sane default.

Your router (any router in fact) does not technically need to have an address within your LAN GUA space, and since it can't send traffic with the source address set to its LL it will either show as a timeout on any traceroute, or source a different GUA (eg the WAN address) as the source of the traffic.
I have a situation the other way round here, the WAN side does not get allocated a GUA address at all but communicates with the upstream routers via LL, if you traceroute to a LAN device from the outside you see the LAN side address of the router before the final host.

Good info and it mateches my findings. Thanks.

 

[xiaofan]

For a /56, the last 2 bytes of the block will usually be 00, for instance:
2400:d802:d10:1100::/64
The router usually picks the first /64 but technically you have 00 - FF (256 subnets) available to you.

Lack of downstream PD basically wastes the /56, some ISP routers do support downstream PD (eg Comcast and AT&T in the US). AT&T at least use several of the /64 networks for their own purposes (voip, iptv etc)

If you use "passthrough" then it's basically bridging v6 traffic and natting legacy traffic, you might as well just use a dumb access point which will bridge everything.

1. Indeed Singtel ZTE F8648P ONR's handling of IPv6 is not good. As you mentioned, /56 is wasted.

2. Luckily I am able to bridge the Singtel ZTE F8648P ONR now. So I can get /56 prefix delegation on my own router (OpenWRT virtual router in this case).

And indeed WAN GUA (/128) is not within the LAN GUA space (/56 prefix delegated).

3. As for IPv6 Passthrough, I like to play with different sub-routers (OpenWRT/pfSense/OPNsense/Asus/Xiaomi/etc) behind the main router (OpenWRT in this case but sometimes I will switch to pfSense). In that case, I have Double NAT on purpose.

When using unbridged Singtel ONR, IPv6 passthrough seems to be the only way to get client devices to work with IPv6 (not counting NAT66 from OpenWRT). Take note pfSense/OPNsense do not work with NAT66.

OpenWRT's IPv6 "relay" seems to work fine. Same for Xiaomi's IPv6 passthrough. Asus' IPv6 Passthrough does not seem to work reliably.

pfSense/OPNsense do not seem to work with IPv6 passthrough -- I may be wrong here though.

I do have a wireless AP (Asus RT-AX86U now) which is connected to the main OpenWRT router --> clients have no issues to get working IPv6 from the wireless AP.

4. Now that I can bridge the Singtel ONR and use my own router as the main router, I may not need IPv6 passthrough any more --> I will need to carry out more experiments (kind of repeat the learning in Nov/Dec 2023) to be able to at least delagate /64 prefix to downstream routers.

 

[bert64]

1. Indeed Singtel ZTE F8648P ONR's handling of IPv6 is not good. As you mentioned, /56 is wasted.

2. Luckily I am able to bridge the Singtel ZTE F8648P ONR now. So I can get /56 prefix delegation on my own router (OpenWRT virtual router in this case).

And indeed WAN GUA (/128) is not within the LAN GUA space (/56 prefix delegated).

3. As for IPv6 Passthrough, I like to play with different sub-routers (OpenWRT/pfSense/OPNsense/Asus/Xiaomi/etc) behind the main router (OpenWRT in this case but sometimes I will switch to pfSense). In that case, I have Double NAT on purpose.

When using unbridged Singtel ONR, IPv6 passthrough seems to be the only way to get client devices to work with IPv6 (not counting NAT66 from OpenWRT). Take note pfSense/OPNsense do not work with NAT66.

OpenWRT's IPv6 "relay" seems to work fine. Same for Xiaomi's IPv6 passthrough. Asus' IPv6 Passthrough does not seem to work reliably.

pfSense/OPNsense do not seem to work with IPv6 passthrough -- I may be wrong here though.

I do have a wireless AP (Asus RT-AX86U now) which is connected to the main OpenWRT router --> clients have no issues to get working IPv6 from the wireless AP.

4. Now that I can bridge the Singtel ONR and use my own router as the main router, I may not need IPv6 passthrough any more --> I will need to carry out more experiments (kind of repeat the learning in Nov/Dec 2023) to be able to at least delagate /64 prefix to downstream routers.

pfsense/opnsense can do NAT66 and NAT-PT, although these features are rarely used. NAT-PT will only work if you have a full /64 routed to the device, it cannot overlay on top of another active network.

They have no explicit passthrough support for v6 traffic, although they do support bridging. IPv6 passthrough is just bridging with conditional rules to only forward v6 traffic.

Does the ONR provide you an option to create static routes? You could route traffic to a downstream router that way, although it would break every time your prefix changed.

 

[xiaofan]

pfsense/opnsense can do NAT66 and NAT-PT, although these features are rarely used. NAT-PT will only work if you have a full /64 routed to the device, it cannot overlay on top of another active network.

They have no explicit passthrough support for v6 traffic, although they do support bridging. IPv6 passthrough is just bridging with conditional rules to only forward v6 traffic.

Does the ONR provide you an option to create static routes? You could route traffic to a downstream router that way, although it would break every time your prefix changed.

I do not want to play with the ONR any more --> the new change from Singtel side makes the unbridged ports can no longer access the Internet even though the ONR itself has a proper public IPv4 and legitimate IPv6 GUA address, no idea why.

So I just treat the bridged Singtel ONR as an ONT now. I will play with my own router (Intel N100 Mini PC with dual SFP+ ports running Proxmox PVR and then OpenWRT/pfSense VM on top of PVE) which is connected to the bridged 10G LAN port.

pfSense/OPNSense support NPt but not NAT 66, even if the underlying FreeBSD OS itself supports NAT 66.

OpenWRT NAT66 and IPv6 masquerading
https://openwrt.org/docs/guide-user/network/ipv6/ipv6.nat6

pfSense NPt:
https://docs.netgate.com/pfsense/en/latest/network/ipv6/nat.html
https://docs.netgate.com/pfsense/en/latest/nat/npt.html

pfSense NAT 66 discussion

 

[Mach3.2]

pfSense/OPNSense support NPt but not NAT 66

I'm doing NAT66 on my pfsense box

but ya it's disgusting and you won't get official support for it.

 

[xiaofan]

I'm doing NAT66 on my pfsense box

but ya it's disgusting and you won't get official support for it.

Hmm, just wondering why you need to use NAT66 -- I remember you were using NPt since M1 gives you /64 IPv6 prefix.

And I thought VQ did not support IPv6 so that you would not be able to use IPv6 with your VQ 10Gbps plan.

Or are you saying that VQ does not support IPv6 properly but you can still get a /128 IPv6 address for the WAN port, in that case, using NAT66 for your VQ 10Gbps plan makes sense.

 

[bert64]

Hmm, just wondering why you need to use NAT66 -- I remember you were using NPt since M1 gives you /64 IPv6 prefix.

And I thought VQ did not support IPv6 so that you would not be able to use IPv6 with your VQ 10Gbps plan.

Or are you saying that VQ does not support IPv6 properly but you can still get a /128 IPv6 address for the WAN port, in that case, using NAT66 for your VQ 10Gbps plan makes sense.

You would need to dedicate the entire /64 to NPT use, you can't have one network with native v6 and others with NPT overlaid on top. It's basically 1:1 translation, the only real use for it is multi homing where you're using the ISP's address space, or if you want to use a static ULA prefix and map it to a dynamic prefix supplied by the ISP.

It still causes problems because internal devices have no standard way to determine their external address, so they would announce the internal address to torrent trackers or SIP peers etc.

I've used NAT66 on pfsense for a portable lab and it worked as well as can be expected, i wouldn't use it for live traffic.

VQ seem to have some v6 capability, at least they have some probes on ripe atlas... Not sure how difficult it would be to get something as a customer tho.

 

[xiaofan]

Today's progress with regard to get downstream router to work with IPv6 through prefix delegation.
Not working yet.

Main OpenWRT router --> assign /60 for the LAN port and allow downstream prefix delegation. LAN clients have proper IPv6 address and can access Internet using IPv6.

OpenWRT subrouter --> get /61 using DHCP-PD, assigned /62 to LAN. LAN client devices get routable IPv6 address but can not access Internet using IPv6. DNS64 is okay. So looks like IPv6 gateway is the issue.

Asus TUF-BE6500 subrouter --> get /62 using DHCP-PD on the LAN side. LAN client devices get routable IPv6 address but can not access Internet using IPv6. DNS64 is okay. So looks like IPv6 gateway is the issue.

 

[bert64]

Today's progress with regard to get downstream router to work with IPv6 through prefix delegation.
Not working yet.

Main OpenWRT router --> assign /60 for the LAN port and allow downstream prefix delegation. LAN clients have proper IPv6 address and can access Internet using IPv6.

OpenWRT subrouter --> get /61 using DHCP-PD, assigned /62 to LAN. LAN client devices get routable IPv6 address but can not access Internet using IPv6. DNS64 is okay. So looks like IPv6 gateway is the issue.

Asus TUF-BE6500 subrouter --> get /62 using DHCP-PD on the LAN side. LAN client devices get routable IPv6 address but can not access Internet using IPv6. DNS64 is okay. So looks like IPv6 gateway is the issue.

I think you need to assign /64 for LAN, and separately assign it a pool for PD.

If you have shell access, have you verified that the upstream router is creating a route to the delegated block via the downstream router? And you don't have any firewall rules blocking the traffic from the delegated block?

 

[Mach3.2]

Hmm, just wondering why you need to use NAT66 -- I remember you were using NPt since M1 gives you /64 IPv6 prefix.

And I thought VQ did not support IPv6 so that you would not be able to use IPv6 with your VQ 10Gbps plan.

Or are you saying that VQ does not support IPv6 properly but you can still get a /128 IPv6 address for the WAN port, in that case, using NAT66 for your VQ 10Gbps plan makes sense.

I'm doing NAT66 to translate the GUA address from the Hurricane Electric /48 that I'm using for my LAN networks to the /128 from Cloudflare warp+ VPN. It's really only for one of my software atlas probes.

I'm using NPt to share the /64 i got from M1 between multiple VLANs. The way I'm using NPt is off label and unsupported, which is why I'm not going into details because frankly it's also pretty disgusting.

 

[Mach3.2]

Not sure how difficult it would be to get something as a customer tho.

Contacted their technical support, but was told ipv6 is not supported despite seeing probes with ipv6 address from VQ's allocation.

 

[xiaofan]

I think you need to assign /64 for LAN, and separately assign it a pool for PD.

If you have shell access, have you verified that the upstream router is creating a route to the delegated block via the downstream router? And you don't have any firewall rules blocking the traffic from the delegated block?

I will try the suggestion for the LAN IPv6 assignment --> that is possible for OpenWRT but not for the Asus router. The Asus router has limited IPv6 settings.

I have SSH access to both the OpenWRT sub-router and the Asus TUF-BE6500 sub-router.

The upstream OpenWRT router (main router) does not have Firewall rules to block IPv6 traffic.

 

[yjwong]

I am able to bridge Singtel ZTE F8648P ONR now. Good thing is that Singtel native IPv6 still works with my own router connected to the bridged port.

I am using Asus TUF-BE6500 (only capable of 2.5Gbps) as a temp router before I set up my Intel N100 CPU based Miniroute R1 mini PC (dual 10G SFP+ ports and dual 2.5G ports) with OpenWRT (or pfSense).

I can see that IPv6 prefix length is 56. Need to check if I can get the sub-router to use /64 prefix delegation.

Wait, bridging and native IPv6 works now? I will try later.

Previously, I had to choose between having the ability to bridge vs native IPv6. If they enabled native IPv6, they would lock me out of the "support" account, removing my ability to bridge. But recently, I discovered that they configured the ONR to use 6rd rather than native v6, leading to poor performance.

 

[xiaofan]

Wait, bridging and native IPv6 works now? I will try later.

Previously, I had to choose between having the ability to bridge vs native IPv6. If they enabled native IPv6, they would lock me out of the "support" account, removing my ability to bridge. But recently, I discovered that they configured the ONR to use 6rd rather than native v6, leading to poor performance.

Previously I could not bridge the ZTE F8648P XGS-PON ONR but I got native IPv6 working (requested through Singtel).

Recently I got bridging working (no idea why bridging is working now) --> and the good thing is that Native IPv6 is working for my own router which is connected to the bridged port.

However, one thing changed compared to previous reports --> the device connected to the unbridged ONR ports no longer has Internet access (even the ONR has valid public IPv4 and IPv6 address).

I could bridge another ONR port (1G only) --> but I choose not to do that right now. I am worried that I may hit 4 IPV4 addresses limit in 24 hours whenever I change routers.

 

[yjwong]

Tried earlier, unfortunately -- when in bridged mode, my OPNsense router is still unable to obtain IPv6 prefix/address. Only a single port is bridged, nothing else connected to the ONR.

 

[bert64]

Wait, bridging and native IPv6 works now? I will try later.

Previously, I had to choose between having the ability to bridge vs native IPv6. If they enabled native IPv6, they would lock me out of the "support" account, removing my ability to bridge. But recently, I discovered that they configured the ONR to use 6rd rather than native v6, leading to poor performance.

As far as i'm aware they've only upgraded the infrastructure in some areas. Depending where you are and what node you're connected to you might have no option other than the 6rd tunnel for now.

The 6rd tunnel will perform terribly on the ONR because it's done in software and the CPU is very weak, native routing uses an ASIC. Tunnel protocols like 6rd are built for legacy network technologies like ADSL or dialup, the idea of using such a tunnel on a more modern GPON service is crazy so the router manufacturers don't really support it.
If you enable 6rd on your opnsense the performance difference compared to native will be fairly small, it adds maybe 2-3ms of latency due to the tunnel.
Also you will notice that the 6rd tunnel performs state tracking, so idle connections (eg an SSH session left open, or something like activesync) will get dropped after a few minutes.

You should run a packet capture (eg tcpdump -i IFACE -n ip6) to see if there's any router advertisements being sent out, or see if you get any responses when you ping6 the multicast address (ff02::1%IFACE). On a network where native v6 is enabled you should get a ping response from the router, and you should also see periodic router advertisement packets being sent out, on a legacy network you'll see nothing.

 

[xiaofan]

Tried earlier, unfortunately -- when in bridged mode, my OPNsense router is still unable to obtain IPv6 prefix/address. Only a single port is bridged, nothing else connected to the ONR.

Did you remove 6rd from the ONR? You may need to reboot both the ONR and the OPNsense as well.

6rd settings in the ONR did not work for me anyway. Once Singtel guy told me native IPv6 was enabled, the first thing I did was to remove 6rd and then reboot the ONR -- then I could confirm native IPv6 was working from the WAN Status.

It is indeed strange that you can only get either bridging or native IPv6 but not both. As mentioned, I could only get native IPv6 previously but now I get both.

 

[yjwong]

Yeah, there's no more active 6rd configuration on the ONR.

I think it's what @bert64 said -- probably the infrastructure hasn't been upgraded. Packet capture on the WAN interface shows only outgoing solicit messages but no response.

I do have another M1 fibre line where the same OPNsense instance gets v6 just fine.