IPv6 discussions

[xiaofan]

Are you trying to create a 2nd LAN on the first router, or a second router?

That debianct11r1 device is in the 2nd LAN? what does its interface config (ip addr list) and routing table (ip -6 route) show?

When you perform the pings, do they show up on the router's LAN interface? Are they subsequently forwarded out through the WAN interface?

2nd LAN of the main OpenWRT router (LAN2). I am using a virtual Linux bridge here as I do not have extra HW NIC (10G WAN, 10G LAN and one 2.5G Proxmox management port. Unfortunately the other 2.5G physical LAN port is not functional now).

debianct11r1 device is the LxC container connected to LAN2.

I will carry out more debugging later. Looks like it is a IPv6 gateway issue. The last one you mentioned seems to be the issue. It can reach the LAN2 interface but cannot go out through WAN.

root@debianct11r1:~# ip -6 route show default
default via fe80::xxxx:11ff:fe88:a29b dev eth0 proto ra metric 1024 expires 1764sec hoplimit 64 pref medium

root@debianct11r1:~# traceroute ipv6.google.com
traceroute to ipv6.google.com (2404:6800:4003:c00::66), 30 hops max, 80 byte packets
 1  2400:d802:de2:89ff::1 (2400:d802:xxx:89ff::1)  0.267 ms  0.242 ms  0.229 ms
 2  * * *
...
16  * * *
17  * * *^C

Supposedly the 2400:d802:xxx:8900::/56 route should cover both the default br-lan and LAN2 (eth2). eth1 is WAN port.

Not so sure why it does not work. It seems to me the Singtel side gateway somehow rejects the connection. I believe fe80::200:5eff:fe00:145 is the SingTel side IPv6 gateway.

root@OpenWrt:~# ip -6 route show default
default from 2400:d802:d18::x:xxx via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
default from 2400:d802:xxx:8900::/56 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium

root@OpenWrt:~# ip -6 addr
...
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:d18::x:xxx/128 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxxx:11ff:fe0c:e10d/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:89ff::1/64 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxxx:11ff:fe88:a29b/64 scope link
       valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:8900::1/60 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxx:11ff:fe2d:be5f/64 scope link
       valid_lft forever preferred_lft forever
...

 

[bert64]

2nd LAN of the main OpenWRT router (LAN2). I am using a virtual Linux bridge here as I do not have extra HW NIC (10G WAN, 10G LAN and one 2.5G Proxmox management port. Unfortunately the other 2.5G physical LAN port is not functional now).

debianct11r1 device is the LxC container connected to LAN2.

I will carry out more debugging later. Looks like it is a IPv6 gateway issue. The last one you mentioned seems to be the issue. It can reach the LAN2 interface but cannot go out through WAN.

root@debianct11r1:~# ip -6 route show default
default via fe80::xxxx:11ff:fe88:a29b dev eth0 proto ra metric 1024 expires 1764sec hoplimit 64 pref medium

root@debianct11r1:~# traceroute ipv6.google.com
traceroute to ipv6.google.com (2404:6800:4003:c00::66), 30 hops max, 80 byte packets
 1  2400:d802:de2:89ff::1 (2400:d802:xxx:89ff::1)  0.267 ms  0.242 ms  0.229 ms
 2  * * *
...
16  * * *
17  * * *^C

Supposedly the 2400:d802:xxx:8900::/56 route should cover both the default br-lan and LAN2 (eth2). eth1 is WAN port.

Not so sure why it does not work. It seems to me the Singtel side gateway somehow rejects the connection. I believe fe80::200:5eff:fe00:145 is the SingTel side IPv6 gateway.

root@OpenWrt:~# ip -6 route show default
default from 2400:d802:d18::x:xxx via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
default from 2400:d802:xxx:8900::/56 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium

root@OpenWrt:~# ip -6 addr
...
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:d18::x:xxx/128 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxxx:11ff:fe0c:e10d/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:89ff::1/64 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxxx:11ff:fe88:a29b/64 scope link
       valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:8900::1/60 scope global dynamic noprefixroute
       valid_lft 49721sec preferred_lft 49721sec
    inet6 fe80::xxx:11ff:fe2d:be5f/64 scope link
       valid_lft forever preferred_lft forever
...

So you have /60 directly assigned to br-lan which will cause problems, the interface itself needs to have /64, but i doubt it's the cause of your current issue.

Whats the "ip addr list" output from debianct11r1?

Containers btw share a networking stack with the physical host, so there can be strange things happening in some cases.

You only showed the default route, but your router has more than a default - it needs a route to its delegated prefixes, and it needs interface routes, my system is BSD based but you should get the idea:

Destination Gateway Flags Netif Expire default fe80::f60f:1bff:fe17:d400%pppoe0 UG pppoe0 64:ff9b::/96 2001:db8:db8:402::6464 UGS igc2 2001:db8:db8:400::/64 link#1 U igc0 2001:db8:db8:400::/56 link#23 U pppoe0 2001:db8:db8:400::1 link#6 UHS lo0 2001:db8:db8:400::240 link#6 UHS lo0 2001:db8:db8:401::/64 link#4 U igc3 2001:db8:db8:401::1 link#6 UHS lo0 2001:db8:db8:401::240 link#6 UHS lo0 2001:db8:db8:402::/64 link#3 U igc2 2001:db8:db8:402::1 link#6 UHS lo0 2001:db8:db8:402::240 link#6 UHS lo0 2001:db8:db8:403::/112 link#13 U igc1.22 2001:db8:db8:403::1 link#6 UHS lo0 2001:db8:db8:403::2 link#6 UHS lo0 2001:db8:db8:403::66 link#6 UHS lo0 2001:db8:db8:403::240 link#6 UHS lo0 2001:db8:db8:403:fe::/112 link#25 U ovpns2 2001:db8:db8:403:fe::1 link#6 UHS lo0 2001:db8:db8:403:ff::/112 link#24 U ovpns1 2001:db8:db8:403:ff::1 link#6 UHS lo0 2001:db8:db8:404::/64 link#14 U igc2.301 2001:db8:db8:404::1 link#6 UHS lo0 2001:db8:db8:404::240 link#6 UHS lo0 2001:db8:db8:405::/64 link#12 U igc1.120 2001:db8:db8:405::1 link#6 UHS lo0 2001:db8:db8:405::240 link#6 UHS lo0 2001:db8:db8:406::/64 link#10 U igc1.40 2001:db8:db8:406::1 link#6 UHS lo0 2001:db8:db8:406::240 link#6 UHS lo0 2001:db8:db8:407::/64 link#18 U igc3.78 2001:db8:db8:407::1 link#6 UHS lo0 2001:db8:db8:407::240 link#6 UHS lo0 2001:db8:db8:408::/64 link#15 U igc2.302 2001:db8:db8:408::1 link#6 UHS lo0 2001:db8:db8:408::240 link#6 UHS lo0 2001:db8:db8:409::/64 link#16 U igc2.303 2001:db8:db8:409::1 link#6 UHS lo0 2001:db8:db8:409::240 link#6 UHS lo0 2001:db8:db8:40a::/64 link#17 U igc2.304 2001:db8:db8:40a::1 link#6 UHS lo0 2001:db8:db8:40a::240 link#6 UHS lo0 2001:db8:db8:40b::/64 link#19 U igc2.101 2001:db8:db8:40b::240 link#6 UHS lo0 2001:db8:db8:40c::/64 2001:db8:db8:408::777 UGS igc2.302 2001:db8:db8:40d::/64 link#22 U igc3.79 2001:db8:db8:40d::240 link#6 UHS lo0 2001:db8:db8:40e::/64 2001:db8:db8:407:220:25ff:fee0:dda UGS igc3.78 2001:db8:db8:40f::/64 2001:db8:db8:407:220:25ff:fee0:dda UGS igc3.78 2001:db8:db8:410::/64 2001:db8:db8:407:220:25ff:fee0:dda UGS igc3.78 2001:db8:db8:4ff::/64 2001:db8:db8:402::73 UGS igc2

If you see the above, you will see the /64 link routes are assigned to every interface which has addresses assigned, and then i have those 4 /64 routes at the bottom which delegate prefixes to other routers - eg the last one 2001:db8:db8:402::73 is a router which has 2001:db8:db8:4ff::/64 delegated behind it.

If it's being blocked by singtel then a tcpdump on your wan interface (eth1?) will show the packets going out but no responses coming back.

What happens if you send traffic to the address of debianct11r1 from the outside? does it show up at every interface in tcpdump?

 

[xiaofan]

I have fixed the LAN interface assignment problem, using /64 for the LAN interfaces now, 8900::/64 and 89ff:64 now.

To avoid potential issues container, I use 8900::/64 for the LAN2 virtual interface and then 89ff::/64 for the default br-lan interface.

Now the two Linux LxC containers (LAN2 clients) will have no problem access Internet using IPv6. One of the LxC container (using DHCPv6 or "dhcp" as shown by Proxmox PVE, DNS server shows as the IPv6 address of LAN2) will have two IPv6 addresses. The other LxC container (using SLAAC or "auto" as shown by Proxmox PVE, DNS server shown as IPv4 address of LAN2 ) will have one IPv6 address only. But both Linux LxC containers have no issues with Internet access with IPv6.

Now physical host like my Windows laptop (br-lan clients) will get two IPv4 addresses, one with DHCPv6 and the other with SLAAC. But the Windows laptop will have no Internet Access using IPv6. I have also tried SLAAC only and it does not help.

The two LAN interfaces have exactly the same settings, other than the prefix.

I will carry out more degugging over the weekend.

 

[bert64]

I have fixed the LAN interface assignment problem, using /64 for the LAN interfaces now, 8900::/64 and 89ff:64 now.

To avoid potential issues container, I use 8900::/64 for the LAN2 virtual interface and then 89ff::/64 for the default br-lan interface.

Now the two Linux LxC containers (LAN2 clients) will have no problem access Internet using IPv6. One of the LxC container (using DHCPv6 or "dhcp" as shown by Proxmox PVE, DNS server shows as the IPv6 address of LAN2) will have two IPv6 addresses. The other LxC container (using SLAAC or "auto" as shown by Proxmox PVE, DNS server shown as IPv4 address of LAN2 ) will have one IPv6 address only. But both Linux LxC containers have no issues with Internet access with IPv6.

Now physical host like my Windows laptop (br-lan clients) will get two IPv4 addresses, one with DHCPv6 and the other with SLAAC. But the Windows laptop will have no Internet Access using IPv6. I have also tried SLAAC only and it does not help.

The two LAN interfaces have exactly the same settings, other than the prefix.

I will carry out more degugging over the weekend.

Linux will not pick up DNS resolvers from SLAAC/RDNSS unless you install rdnssd.
DHCPv6 is always in addition to SLAAC, unless you turn off the "autonomous" flag in your router advertisements, so you will always get 2 addresses.
Linux servers generally do not use privacy addresses by default, you can turn this on manually if you want - then you will get additional temporary addresses.

What is the routing table on the router? - the full routing table, not just the default route.
When you ping the windows laptop from the outside, how far does the traffic get when you view it with tcpdump (received on eth1? sent back out on br-lan?) and how about a ping in the opposite direction?

 

[xiaofan]

Linux will not pick up DNS resolvers from SLAAC/RDNSS unless you install rdnssd.
DHCPv6 is always in addition to SLAAC, unless you turn off the "autonomous" flag in your router advertisements, so you will always get 2 addresses.
Linux servers generally do not use privacy addresses by default, you can turn this on manually if you want - then you will get additional temporary addresses.

What is the routing table on the router? - the full routing table, not just the default route.
When you ping the windows laptop from the outside, how far does the traffic get when you view it with tcpdump (received on eth1? sent back out on br-lan?) and how about a ping in the opposite direction?

OpenWRT seems to use slightly different terminology.
https://openwrt.org/docs/guide-user/network/ipv6/configuration

wan6 settings

Not able to set to "server mode" for wan6.

br-lan IPv6 settings

I tried "server mode" or "relay mode" for lan as well.

I tried to use "M" or "O" or "M+O" as well. All do not help.

 

[xiaofan]

What is the routing table on the router? - the full routing table, not just the default route.
When you ping the windows laptop from the outside, how far does the traffic get when you view it with tcpdump (received on eth1? sent back out on br-lan?) and how about a ping in the opposite direction?

IPv6 routing table, no delegation to downstream routers in this case.

eth1 --> WAN
eth2 --> LAN2
eth0/br-lan --> LAN

tcpdum will be done later.

root@OpenWrt:~# ip -6 route
default from 2400:d802:d18::x:xxx via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
default from 2400:d802:xxx:8900::/56 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
2400:d802:xxx:8900:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:8900::/64 dev eth2 proto static metric 1024 pref medium
2400:d802:xxx:8910::1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910::785 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:36:be16:2c81:a36b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:127c:61ff:fedc:5021 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:8910:1c22:f0ff:fee9:2b2b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:2ef2:1238:e6c0:d4a1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:6982:7047:2536:4227 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:98ec:3e00:2544:e272 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:b4ed:77ff:fe7c:b1fe dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:b5ca:ddfe:b313:ae2 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:d822:21ff:fe05:2f6c dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::541 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::785 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:142:aac3:9b2f:d530 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:41f:d1f0:e00d:5596 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:127c:61ff:fedc:5021 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:1c22:f0ff:fee9:2b2b dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:2c44:2538:9b03:9a13 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:2d6d:716c:44af:85fd dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:3075:4b59:e3ad:191 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:3dd5:3e26:bd8e:3ca3 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:4081:eb32:fa33:d59c dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:585f:69e4:1f2a:1b53 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:59ad:3f83:5a0a:4137 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:59c3:724e:c63d:97d8 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:5deb:9fa0:daaf:de09 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:7c0b:368:4778:623a dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:84d2:aa2f:96c9:46c0 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:8603:4d5c:7e1c:e99b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:8c0c:384d:bc82:8566 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:8ddb:6296:4d62:5ad6 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b082:8dfe:673b:da2b dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b4ed:77ff:fe7c:b1fe dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b5b5:d393:6d6c:96f3 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:c02d:1057:3f70:ec06 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:c30e:d8a1:58cf:6c38 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:d822:21ff:fe05:2f6c dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:e48d:af10:db1c:f7f2 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:f17a:a0f9:3a16:9aaf dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2400:d802:xxx:8900::/56 dev lo proto static metric 2147483647 pref medium
unreachable fd5b:fdb2:a17d::/48 dev lo proto static metric 2147483647 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev tailscale0 proto kernel metric 256 pref medium
root@OpenWrt:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:d18::x:xxx/128 scope global dynamic noprefixroute
       valid_lft 74758sec preferred_lft 74758sec
    inet6 fe80::be24:11ff:fe0c:e10d/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:8900::1/64 scope global dynamic noprefixroute
       valid_lft 74759sec preferred_lft 74759sec
    inet6 fe80::be24:11ff:fe88:a29b/64 scope link
       valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:89ff::1/64 scope global dynamic noprefixroute
       valid_lft 74759sec preferred_lft 74759sec
    inet6 fe80::be24:11ff:fe2d:be5f/64 scope link
       valid_lft forever preferred_lft forever
13: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 state UNKNOWN qlen 500
    inet6 fe80::92d5:c9d0:bc1b:7653/64 scope link stable-privacy
       valid_lft forever preferred_lft forever

 

[bert64]

IPv6 routing table, no delegation to downstream routers in this case.

eth1 --> WAN
eth2 --> LAN2
eth0/br-lan --> LAN

tcpdum will be done later.

root@OpenWrt:~# ip -6 route
default from 2400:d802:d18::x:xxx via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
default from 2400:d802:xxx:8900::/56 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
2400:d802:xxx:8900:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:8900::/64 dev eth2 proto static metric 1024 pref medium
2400:d802:xxx:8910::1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910::785 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:36:be16:2c81:a36b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:127c:61ff:fedc:5021 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:8910:1c22:f0ff:fee9:2b2b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:2ef2:1238:e6c0:d4a1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:6982:7047:2536:4227 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:98ec:3e00:2544:e272 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:b4ed:77ff:fe7c:b1fe dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:b5ca:ddfe:b313:ae2 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:8910:d822:21ff:fe05:2f6c dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::1 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::541 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff::785 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:142:aac3:9b2f:d530 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:41f:d1f0:e00d:5596 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:127c:61ff:fedc:5021 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:1c22:f0ff:fee9:2b2b dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:2c44:2538:9b03:9a13 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:2d6d:716c:44af:85fd dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:3075:4b59:e3ad:191 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:3dd5:3e26:bd8e:3ca3 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:4081:eb32:fa33:d59c dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:585f:69e4:1f2a:1b53 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:59ad:3f83:5a0a:4137 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:59c3:724e:c63d:97d8 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:5deb:9fa0:daaf:de09 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:7c0b:368:4778:623a dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:84d2:aa2f:96c9:46c0 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:8603:4d5c:7e1c:e99b dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:8c0c:384d:bc82:8566 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:8ddb:6296:4d62:5ad6 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b082:8dfe:673b:da2b dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b4ed:77ff:fe7c:b1fe dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:b5b5:d393:6d6c:96f3 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:c02d:1057:3f70:ec06 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:c30e:d8a1:58cf:6c38 dev eth1 proto static metric 1024 pref medium
2400:d802:xxx:89ff:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:d822:21ff:fe05:2f6c dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:e48d:af10:db1c:f7f2 dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff:f17a:a0f9:3a16:9aaf dev br-lan proto static metric 1024 pref medium
2400:d802:xxx:89ff::/64 dev br-lan proto static metric 1024 pref medium
unreachable 2400:d802:xxx:8900::/56 dev lo proto static metric 2147483647 pref medium
unreachable fd5b:fdb2:a17d::/48 dev lo proto static metric 2147483647 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev tailscale0 proto kernel metric 256 pref medium
root@OpenWrt:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:d18::x:xxx/128 scope global dynamic noprefixroute
       valid_lft 74758sec preferred_lft 74758sec
    inet6 fe80::be24:11ff:fe0c:e10d/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:8900::1/64 scope global dynamic noprefixroute
       valid_lft 74759sec preferred_lft 74759sec
    inet6 fe80::be24:11ff:fe88:a29b/64 scope link
       valid_lft forever preferred_lft forever
6: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx:89ff::1/64 scope global dynamic noprefixroute
       valid_lft 74759sec preferred_lft 74759sec
    inet6 fe80::be24:11ff:fe2d:be5f/64 scope link
       valid_lft forever preferred_lft forever
13: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 state UNKNOWN qlen 500
    inet6 fe80::92d5:c9d0:bc1b:7653/64 scope link stable-privacy
       valid_lft forever preferred_lft forever

You might want to turn off source routing on WAN since you only have one upstream, it shouldn't break anything but does add some unnecessary complexity.

What is 2400:d802:xxx:8910:: on eth1? that another LAN prefix, or just a coincidence that the WAN interface is also :89xx?

Also weird that you have this on br-lan:
2400:d802:xxx:8900:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium

but br-lan has the 89ff:: prefix assigned, so it shouldnt be able to see 8900:: hosts in there

 

[xiaofan]

You might want to turn off source routing on WAN since you only have one upstream, it shouldn't break anything but does add some unnecessary complexity.

What is 2400:d802:xxx:8910:: on eth1? that another LAN prefix, or just a coincidence that the WAN interface is also :89xx?

Also weird that you have this on br-lan:
2400:d802:xxx:8900:c8d1:2b73:5db2:7b08 dev br-lan proto static metric 1024 pref medium

but br-lan has the 89ff:: prefix assigned, so it shouldnt be able to see 8900:: hosts in there

1. Looks there are are old information lingering. I have reboot the router and simplifed the stuff.

RA setting on br-lan (eth0) and LAN2 (eth2) --> changed to "M" + "O".

wan6 (eth1) --> using DHCPv6-PD, /128 for WAN interface, /56 IPv6 prefix delegation.

One Linux containers connected to br-lan (eth0) --> using dhcp, 8900::/64, with internet access using IPv6 (ping is successful)

One Linux containers connected to LAN2 (eth2) --> using dhcp, 8901::/64, NO internet access using IPv6 (ping is not successful).

From the below tcpdump, ping echo requests hit the WAN port for both clients. However, ping echo replies are only available for br-lan client (default 8900::/64).

2. Main OpenWRT router

root@OpenWrt:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:xxx::4:f26/128 scope global dynamic noprefixroute
       valid_lft 85434sec preferred_lft 85434sec
    inet6 fe80::be24:11ff:fe0c:e10d/64 scope link
       valid_lft forever preferred_lft forever
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8901::1/64 scope global dynamic noprefixroute
       valid_lft 85434sec preferred_lft 85434sec
    inet6 fe80::be24:11ff:fe88:a29b/64 scope link
       valid_lft forever preferred_lft forever
13: tailscale0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1280 state UNKNOWN qlen 500
    inet6 fe80::717:d2af:426b:11dc/64 scope link stable-privacy
       valid_lft forever preferred_lft forever
14: br-lan: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8900::1/64 scope global dynamic noprefixroute
       valid_lft 85434sec preferred_lft 85434sec
    inet6 fe80::be24:11ff:fe2d:be5f/64 scope link
       valid_lft forever preferred_lft forever

root@OpenWrt:~# ip -6 route
default from 2400:d802:xxx::4:f26 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
default from 2400:d802:de2:8900::/56 via fe80::200:5eff:fe00:145 dev eth1 proto static metric 512 pref medium
2400:d802:de2:8900::1 dev eth1 proto static metric 1024 pref medium
2400:d802:de2:8900::990 dev eth1 proto static metric 1024 pref medium
2400:d802:de2:8900::bb4 dev eth1 proto static metric 1024 pref medium
2400:d802:de2:8900::/64 dev br-lan proto static metric 1024 pref medium
2400:d802:de2:8901::/64 dev eth2 proto static metric 1024 pref medium
unreachable 2400:d802:de2:8900::/56 dev lo proto static metric 2147483647 pref medium
unreachable fd5b:fdb2:a17d::/48 dev lo proto static metric 2147483647 pref medium
fe80::/64 dev eth1 proto kernel metric 256 pref medium
fe80::/64 dev tailscale0 proto kernel metric 256 pref medium
fe80::/64 dev br-lan proto kernel metric 256 pref medium
fe80::/64 dev eth2 proto kernel metric 256 pref medium

root@OpenWrt:~# tcpdump -i eth1 icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:15:50.517101 IP6 2400:d802:de2:8900::ea8 > sf-in-f101.1e100.net: ICMP6, echo request, id 64984, seq 1, length 64
00:15:50.520629 IP6 sf-in-f101.1e100.net > 2400:d802:de2:8900::ea8: ICMP6, echo reply, id 64984, seq 1, length 64
00:15:51.305262 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor solicitation, who has fe80::200:5eff:fe00:145, length 32
00:15:51.306315 IP6 fe80::200:5eff:fe00:145 > fe80::be24:11ff:fe0c:e10d: ICMP6, neighbor advertisement, tgt is fe80::200:5eff:fe00:145, length 32
00:15:51.518338 IP6 2400:d802:de2:8900::ea8 > sf-in-f101.1e100.net: ICMP6, echo request, id 64984, seq 2, length 64
00:15:51.521781 IP6 sf-in-f101.1e100.net > 2400:d802:de2:8900::ea8: ICMP6, echo reply, id 64984, seq 2, length 64
00:15:52.519743 IP6 2400:d802:de2:8900::ea8 > sf-in-f101.1e100.net: ICMP6, echo request, id 64984, seq 3, length 64
00:15:52.523420 IP6 sf-in-f101.1e100.net > 2400:d802:de2:8900::ea8: ICMP6, echo reply, id 64984, seq 3, length 64
00:15:53.521364 IP6 2400:d802:de2:8900::ea8 > sf-in-f101.1e100.net: ICMP6, echo request, id 64984, seq 4, length 64
00:15:53.524919 IP6 sf-in-f101.1e100.net > 2400:d802:de2:8900::ea8: ICMP6, echo reply, id 64984, seq 4, length 64
00:16:05.348376 IP6 2400:d802:de2:8901::ea8 > sb-in-x66.1e100.net: ICMP6, echo request, id 15427, seq 1, length 64
00:16:06.379829 IP6 2400:d802:de2:8901::ea8 > sb-in-x66.1e100.net: ICMP6, echo request, id 15427, seq 2, length 64
00:16:07.403766 IP6 2400:d802:de2:8901::ea8 > sb-in-x66.1e100.net: ICMP6, echo request, id 15427, seq 3, length 64
00:16:08.427759 IP6 2400:d802:de2:8901::ea8 > sb-in-x66.1e100.net: ICMP6, echo request, id 15427, seq 4, length 64
00:16:10.180647 IP6 fe80::200:5eff:fe00:145 > ff02::1:ff0c:e10d: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe0c:e10d, length 32
00:16:10.180683 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe0c:e10d, length 32
^C
16 packets captured
18 packets received by filter
0 packets dropped by kernel

3. Linux containers connected to br-lan (eth0)

root@debian12ct22r1:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host noprefixroute 
       valid_lft forever preferred_lft forever
2: eth0@if35: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8900::ea8/128 scope global dynamic 
       valid_lft 43006sec preferred_lft 43006sec
    inet6 2400:d802:de2:8900:be24:11ff:fee6:f84c/64 scope global dynamic mngtmpaddr 
       valid_lft 85514sec preferred_lft 85514sec
    inet6 fe80::be24:11ff:fee6:f84c/64 scope link 
       valid_lft forever preferred_lft forever

root@debian12ct22r1:~# ip -6 route
2400:d802:de2:8900::ea8 dev eth0 proto kernel metric 256 expires 43003sec pref medium
2400:d802:de2:8900::/64 dev eth0 proto kernel metric 256 expires 85510sec pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::be24:11ff:fe2d:be5f dev eth0 proto ra metric 1024 expires 1599sec hoplimit 64 pref medium

root@debian12ct22r1:~# ping -c 4 ipv6.google.com
PING ipv6.google.com(sf-in-f101.1e100.net (2404:6800:4003:c03::65)) 56 data bytes
64 bytes from sf-in-f101.1e100.net (2404:6800:4003:c03::65): icmp_seq=1 ttl=102 time=3.72 ms
64 bytes from sf-in-f101.1e100.net (2404:6800:4003:c03::65): icmp_seq=2 ttl=102 time=3.70 ms
64 bytes from sf-in-f101.1e100.net (2404:6800:4003:c03::65): icmp_seq=3 ttl=102 time=3.98 ms
64 bytes from sf-in-f101.1e100.net (2404:6800:4003:c03::65): icmp_seq=4 ttl=102 time=3.79 ms

--- ipv6.google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004ms
rtt min/avg/max/mdev = 3.702/3.795/3.975/0.108 ms

4. Linux containers connected to LAN2 (eth2)

root@debianct11r1:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if29: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8901::ea8/128 scope global dynamic
       valid_lft 43076sec preferred_lft 43076sec
    inet6 2400:d802:de2:8901:be24:11ff:fea5:fadb/64 scope global dynamic mngtmpaddr
       valid_lft 86017sec preferred_lft 86017sec
    inet6 fe80::be24:11ff:fea5:fadb/64 scope link
       valid_lft forever preferred_lft forever

root@debianct11r1:~# ip -6 route
2400:d802:de2:8901::ea8 dev eth0 proto kernel metric 256 expires 43071sec pref medium
2400:d802:de2:8901::/64 dev eth0 proto kernel metric 256 expires 86012sec pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::be24:11ff:fe88:a29b dev eth0 proto ra metric 1024 expires 1667sec hoplimit 64 pref medium

root@debianct11r1:~# ping -c 4 ipv6.google.com
PING ipv6.google.com(sb-in-x66.1e100.net (2404:6800:4003:c01::66)) 56 data bytes

--- ipv6.google.com ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3079ms

 

[xiaofan]

Ping from external host to the two Linux containers, I can see ping echo request and echo reply hits the WAN port and also respetive br-lan and LAN2 interface.

However, pinging the br-lan client (8900::/64 interface) will be successful but not LAN2 client (8901::/64 prefix)

1. ping the two Linux containers from outside, tcpdump of WAN/eth1 port traffic

root@OpenWrt:~# tcpdump -i eth1 icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth1, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:52:05.535281 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:05.535309 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:05.535314 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:05.535317 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:05.535322 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:08.895284 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:10.277921 IP6 fe80::200:5eff:fe00:145 > ff02::1:ff0c:e10d: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe0c:e10d, length 32
00:52:10.277961 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe0c:e10d, length 32
00:52:10.815276 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor solicitation, who has fe80::200:5eff:fe00:145, length 32
00:52:10.816301 IP6 fe80::200:5eff:fe00:145 > fe80::be24:11ff:fe0c:e10d: ICMP6, neighbor advertisement, tgt is fe80::200:5eff:fe00:145, length 32
00:52:12.255281 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:18.895175 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:18.907857 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 48132, seq 1, length 64
00:52:18.908038 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 48132, seq 1, length 64
00:52:19.909452 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 48132, seq 2, length 64
00:52:19.909610 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 48132, seq 2, length 64
00:52:20.910125 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 48132, seq 3, length 64
00:52:20.910285 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 48132, seq 3, length 64
00:52:21.911654 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 48132, seq 4, length 64
00:52:21.911784 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 48132, seq 4, length 64
00:52:32.175281 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:47.615126 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor solicitation, who has fe80::200:5eff:fe00:145, length 32
00:52:47.616192 IP6 fe80::200:5eff:fe00:145 > fe80::be24:11ff:fe0c:e10d: ICMP6, neighbor advertisement, tgt is fe80::200:5eff:fe00:145, length 32
00:52:49.561695 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 42805, seq 1, length 64
00:52:49.561889 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 42805, seq 1, length 64
00:52:50.590324 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 42805, seq 2, length 64
00:52:50.590503 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 42805, seq 2, length 64
00:52:51.614231 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 42805, seq 3, length 64
00:52:51.614387 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 42805, seq 3, length 64
00:52:52.638346 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 42805, seq 4, length 64
00:52:52.638525 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 42805, seq 4, length 64
00:52:58.815291 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 127
00:52:58.815310 IP6 2400:d802:xxx::4:f26 > edge-fblite-tcp-p16-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 111
00:53:10.280684 IP6 fe80::200:5eff:fe00:145 > ff02::1:ff0c:e10d: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe0c:e10d, length 32
00:53:10.280720 IP6 fe80::be24:11ff:fe0c:e10d > fe80::200:5eff:fe00:145: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe0c:e10d, length 32
00:53:13.455287 IP6 2400:d802:xxx::4:f26 > edge-mqtt6-shv-01-sin2.facebook.com: ICMP6, destination unreachable, unreachable address 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 104
^C
36 packets captured
36 packets received by filter
0 packets dropped by kernel

 

[xiaofan]

2. ping the Linux container (br-lan/eth0 client) from outside, tcpdump of br-lan/eth0 port traffic, ping is successful.

root@OpenWrt:~# tcpdump -i eth0 icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:53:51.188216 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 5037, seq 1, length 64
00:53:51.188306 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 5037, seq 1, length 64
00:53:51.837773 IP6 fe80::cf1:88f7:b56f:ac8e > fe80::be24:11ff:fe2d:be5f: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe2d:be5f, length 32
00:53:51.837809 IP6 fe80::be24:11ff:fe2d:be5f > fe80::cf1:88f7:b56f:ac8e: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe2d:be5f, length 24
00:53:52.016461 IP6 fe80::be24:11ff:fe2d:be5f > ff02::1:ff16:9aaf: ICMP6, neighbor solicitation, who has 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 32
00:53:52.189522 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 5037, seq 2, length 64
00:53:52.189656 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 5037, seq 2, length 64
00:53:53.055269 IP6 fe80::be24:11ff:fe2d:be5f > ff02::1:ff16:9aaf: ICMP6, neighbor solicitation, who has 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 32
00:53:53.191864 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 5037, seq 3, length 64
00:53:53.191995 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 5037, seq 3, length 64
00:53:54.095274 IP6 fe80::be24:11ff:fe2d:be5f > ff02::1:ff16:9aaf: ICMP6, neighbor solicitation, who has 2400:d802:de2:8900:f17a:a0f9:3a16:9aaf, length 32
00:53:54.193871 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8900::ea8: ICMP6, echo request, id 5037, seq 4, length 64
00:53:54.194001 IP6 2400:d802:de2:8900::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 5037, seq 4, length 64
00:53:55.935169 IP6 fe80::be24:11ff:fe2d:be5f > 2400:d802:de2:8900:3961:3259:887c:2213: ICMP6, neighbor solicitation, who has 2400:d802:de2:8900:3961:3259:887c:2213, length 32
00:53:55.936999 IP6 fe80::cf1:88f7:b56f:ac8e > fe80::be24:11ff:fe2d:be5f: ICMP6, neighbor advertisement, tgt is 2400:d802:de2:8900:3961:3259:887c:2213, length 24
00:53:56.255281 IP6 fe80::be24:11ff:fe2d:be5f > 2400:d802:de2:8900::ea8: ICMP6, neighbor solicitation, who has 2400:d802:de2:8900::ea8, length 32
00:53:56.255438 IP6 2400:d802:de2:8900::ea8 > fe80::be24:11ff:fe2d:be5f: ICMP6, neighbor advertisement, tgt is 2400:d802:de2:8900::ea8, length 24
00:53:56.270801 IP6 fe80::be24:11ff:fee6:f84c > fe80::be24:11ff:fe2d:be5f: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe2d:be5f, length 32
00:53:56.270841 IP6 fe80::be24:11ff:fe2d:be5f > fe80::be24:11ff:fee6:f84c: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe2d:be5f, length 24
^C
19 packets captured
19 packets received by filter
0 packets dropped by kernel

 

[xiaofan]

3. ping the Linux container (LAN2/eth2 client) from outside, tcpdump of LAN2/eth2 port traffic, ping is NOT successful.

root@OpenWrt:~# tcpdump -i eth2 icmp6
tcpdump: verbose output suppressed, use -v[v]... for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), snapshot length 262144 bytes
00:54:13.667195 IP6 fe80::be24:11ff:fe88:a29b > ip6-allnodes: ICMP6, router advertisement, length 120
00:54:16.336392 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 56315, seq 1, length 64
00:54:16.336528 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 56315, seq 1, length 64
00:54:17.374224 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 56315, seq 2, length 64
00:54:17.374308 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 56315, seq 2, length 64
00:54:18.398476 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 56315, seq 3, length 64
00:54:18.398574 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 56315, seq 3, length 64
00:54:19.422263 IP6 2400:d803:7342:e2b6:447b:92ff:fea0:b95b > 2400:d802:de2:8901::ea8: ICMP6, echo request, id 56315, seq 4, length 64
00:54:19.422366 IP6 2400:d802:de2:8901::ea8 > 2400:d803:7342:e2b6:447b:92ff:fea0:b95b: ICMP6, echo reply, id 56315, seq 4, length 64
00:54:21.358703 IP6 fe80::be24:11ff:fea5:fadb > fe80::be24:11ff:fe88:a29b: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fe88:a29b, length 32
00:54:21.358747 IP6 fe80::be24:11ff:fe88:a29b > fe80::be24:11ff:fea5:fadb: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fe88:a29b, length 24
00:54:21.375264 IP6 fe80::be24:11ff:fe88:a29b > 2400:d802:de2:8901::ea8: ICMP6, neighbor solicitation, who has 2400:d802:de2:8901::ea8, length 32
00:54:21.375414 IP6 2400:d802:de2:8901::ea8 > fe80::be24:11ff:fe88:a29b: ICMP6, neighbor advertisement, tgt is 2400:d802:de2:8901::ea8, length 24
00:54:26.425251 IP6 fe80::be24:11ff:fe88:a29b > fe80::be24:11ff:fea5:fadb: ICMP6, neighbor solicitation, who has fe80::be24:11ff:fea5:fadb, length 32
00:54:26.425408 IP6 fe80::be24:11ff:fea5:fadb > fe80::be24:11ff:fe88:a29b: ICMP6, neighbor advertisement, tgt is fe80::be24:11ff:fea5:fadb, length 24
^C
15 packets captured
15 packets received by filter
0 packets dropped by kernel

 

[xiaofan]

Main OpenWRT router configuration (just removed the wireguard configurations.
https://openwrt.org/docs/guide-user/network/ipv6/configuration

root@OpenWrt:~# cat /etc/config/network

config interface 'loopback'
        option device 'lo'
        option proto 'static'
        option ipaddr '127.0.0.1'
        option netmask '255.0.0.0'

config globals 'globals'
        option ula_prefix 'fd5b:fdb2:a17d::/48'

config device
        option name 'br-lan'
        option type 'bridge'
        list ports 'eth0'

config interface 'lan'
        option device 'br-lan'
        option proto 'static'
        option ipaddr '192.168.18.1'
        option netmask '255.255.255.0'
        option ip6assign '64'
        list ip6class 'wan6'
        option ip6hint '0'

config interface 'wan'
        option device 'eth1'
        option proto 'dhcp'

config interface 'wan6'
        option device 'eth1'
        option proto 'dhcpv6'
        option reqaddress 'try'
        option reqprefix '56'
        option norelease '1'
        list ip6class 'wan6'

config interface 'LAN2'
        option proto 'static'
        option device 'eth2'
        option ipaddr '192.168.58.1'
        option netmask '255.255.255.0'
        option ip6assign '64'
        option ip6hint '01'
        list ip6class 'wan6'

root@OpenWrt:~# cat /etc/config/dhcp

config dnsmasq
        option domainneeded '1'
        option boguspriv '1'
        option filterwin2k '0'
        option localise_queries '1'
        option rebind_protection '1'
        option rebind_localhost '1'
        option local '/lan/'
        option domain 'lan'
        option expandhosts '1'
        option nonegcache '0'
        option cachesize '1000'
        option authoritative '1'
        option readethers '1'
        option leasefile '/tmp/dhcp.leases'
        option resolvfile '/tmp/resolv.conf.d/resolv.conf.auto'
        option nonwildcard '1'
        option localservice '1'
        option ednspacket_max '1232'
        option filter_aaaa '0'
        option filter_a '0'
        list server '/mask.icloud.com/'
        list server '/mask-h2.icloud.com/'
        list server '/use-application-dns.net/'
        list server '127.0.0.1#5053'
        list server '127.0.0.1#5054'
        option doh_backup_noresolv '-1'
        option noresolv '1'
        list doh_backup_server '/mask.icloud.com/'
        list doh_backup_server '/mask-h2.icloud.com/'
        list doh_backup_server '/use-application-dns.net/'
        list doh_backup_server '127.0.0.1#5053'
        list doh_backup_server '127.0.0.1#5054'
        list doh_server '127.0.0.1#5053'
        list doh_server '127.0.0.1#5054'
        option serversfile '/var/run/adblock-fast/dnsmasq.servers'

config dhcp 'lan'
        option interface 'lan'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv4 'server'
        option ra 'server'
        option dhcpv6 'server'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'

config dhcp 'wan'
        option interface 'wan'
        option ignore '1'

config odhcpd 'odhcpd'
        option maindhcp '0'
        option leasefile '/tmp/hosts/odhcpd'
        option leasetrigger '/usr/sbin/odhcpd-update'
        option loglevel '4'

config dhcp 'wan6'
        option interface 'wan6'
        option ignore '1'
        option ra 'relay'
        option dhcpv6 'relay'
        option ndp 'relay'
        option master '1'

config dhcp 'LAN2'
        option interface 'LAN2'
        option start '100'
        option limit '150'
        option leasetime '12h'
        option dhcpv6 'server'
        option ra 'server'
        list ra_flags 'managed-config'
        list ra_flags 'other-config'

 

[xiaofan]

You might want to turn off source routing on WAN since you only have one upstream, it shouldn't break anything but does add some unnecessary complexity.

Thanks. I turn off source routering on the wan6 interface and so far it does not seem to change anything as you expected.

Just wondering what you mean by "unnecessary complexity"? Thanks.

 

[bert64]

Thanks. I turn off source routering on the wan6 interface and so far it does not seem to change anything as you expected.

Just wondering what you mean by "unnecessary complexity"? Thanks.

Source routing uses multiple routing tables to ensure that traffic is routed out the correct WAN interface depending on the source address it has. It's intended if you have multiple WAN links (ie each with their own different prefix) to ensure that traffic goes out of the correct link that assigned the prefix, rather than just being sent via the default route (which will appear to work if the upstream doesn't filter spoofed packets, but you'll have asymmetric routing).

Since you only have one WAN link, no point using this feature as it will just add complexity that you don't need.

Looking at your tcpdump, the traffic is coming in and getting routed to the devices on LAN2 correctly, but the traffic doesn't flow back out. Have you checked the firewall rules? Only other thing that looks weird is routes for 8900:: on eth1/

 

[xiaofan]

Looking at your tcpdump, the traffic is coming in and getting routed to the devices on LAN2 correctly, but the traffic doesn't flow back out. Have you checked the firewall rules? Only other thing that looks weird is routes for 8900:: on eth1/

The Firewall rules are the same for br-lan and LAN2 (same Firewall zone as lan).

Pretty much the default now except the added rule for wireguard.

 

[xiaofan]

Looking at your tcpdump, the traffic is coming in and getting routed to the devices on LAN2 correctly, but the traffic doesn't flow back out. Have you checked the firewall rules? Only other thing that looks weird is routes for 8900:: on eth1/

Indeed the two entry for 8900:: on eth1 (WAN) is strange.

I reboot one of them (Linux LxC container) but it is still in the entry.

It is using SLAAC and not dhcp from the br-lan. And in fact it has issues with Internet access using IPv6.

More about the Linux LxC container.

root@ubuntu2204ct1:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if125: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8900::990/128 scope global dynamic noprefixroute
       valid_lft 37896sec preferred_lft 37896sec
    inet6 2400:d802:de2:8900:401b:78ff:fe25:66a4/64 scope global dynamic mngtmpaddr noprefixroute
       valid_lft 83229sec preferred_lft 83229sec
    inet6 fe80::401b:78ff:fe25:66a4/64 scope link
       valid_lft forever preferred_lft forever

root@ubuntu2204ct1:~# ip -6 route
::1 dev lo proto kernel metric 256 pref medium
2400:d802:de2:8900::/64 dev eth0 proto ra metric 1024 expires 83219sec pref medium
2400:d802:de2:8900::/56 via fe80::be24:11ff:fe2d:be5f dev eth0 proto ra metric 1024 expires 1646sec pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::be24:11ff:fe2d:be5f dev eth0 proto ra metric 1024 expires 1646sec mtu 1500 pref medium

root@ubuntu2204ct1:~# ping -c 4 ipv6.google.com
PING ipv6.google.com(sg-in-f100.1e100.net (2404:6800:4003:c1a::64)) 56 data bytes

--- ipv6.google.com ping statistics ---
4 packets transmitted, 0 received, 100% packet loss, time 3064ms

root@ubuntu2204ct1:~# nslookup ipv6.google.com
Server:         192.168.50.1
Address:        192.168.50.1#53

Non-authoritative answer:
ipv6.google.com canonical name = ipv6.l.google.com.
Name:   ipv6.l.google.com
Address: 2404:6800:4003:c1a::64
Name:   ipv6.l.google.com
Address: 2404:6800:4003:c1a::65
Name:   ipv6.l.google.com
Address: 2404:6800:4003:c1a::71
Name:   ipv6.l.google.com
Address: 2404:6800:4003:c1a::8a

Then I change it to use "dhcp" but it does not even get a proper IPv6 address in this case. Maybe this is a problem with the Ubuntu 22.04 LxC container.

root@ubuntu2204ct1:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host
       valid_lft forever preferred_lft forever
2: eth0@if149: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 fe80::401b:78ff:fe25:66a4/64 scope link
       valid_lft forever preferred_lft forever

root@ubuntu2204ct1:~# ip -6 route
::1 dev lo proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium

 

[xiaofan]

Another Debian 12 container does not have such issue -- just as a comparison.

It is not shown in main openwrt router "ip -6 route" output.

root@debian12ct1:~# ip -6 addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 state UNKNOWN qlen 1000
    inet6 ::1/128 scope host noprefixroute
       valid_lft forever preferred_lft forever
2: eth0@if129: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 state UP qlen 1000
    inet6 2400:d802:de2:8900::c9b/128 scope global dynamic
       valid_lft 38862sec preferred_lft 38862sec
    inet6 2400:d802:de2:8900:30d5:c7ff:feda:fbd0/64 scope global dynamic mngtmpaddr
       valid_lft 82895sec preferred_lft 82895sec
    inet6 fe80::30d5:c7ff:feda:fbd0/64 scope link
       valid_lft forever preferred_lft forever

root@debian12ct1:~# ip -6 route
2400:d802:de2:8900::c9b dev eth0 proto kernel metric 256 expires 38858sec pref medium
2400:d802:de2:8900::/64 dev eth0 proto kernel metric 256 expires 82890sec pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::be24:11ff:fe2d:be5f dev eth0 proto ra metric 1024 expires 1697sec hoplimit 64 pref medium

root@debian12ct1:~# ping -c 4 ipv6.google.com
PING ipv6.google.com(sm-in-f138.1e100.net (2404:6800:4003:c06::8a)) 56 data bytes
64 bytes from sm-in-f138.1e100.net (2404:6800:4003:c06::8a): icmp_seq=1 ttl=104 time=5.31 ms
64 bytes from sm-in-f138.1e100.net (2404:6800:4003:c06::8a): icmp_seq=2 ttl=104 time=5.83 ms
64 bytes from sm-in-f138.1e100.net (2404:6800:4003:c06::8a): icmp_seq=3 ttl=104 time=4.36 ms
64 bytes from sm-in-f138.1e100.net (2404:6800:4003:c06::8a): icmp_seq=4 ttl=104 time=4.46 ms

--- ipv6.google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3006ms
rtt min/avg/max/mdev = 4.361/4.990/5.831/0.611 ms

 

[bert64]

The Firewall rules are the same for br-lan and LAN2 (same Firewall zone as lan).

Pretty much the default now except the added rule for wireguard.

Try setting forward to accept, and also try turning off the offload options..

Can you view the raw ruleset with ip6tables/nftables from the cli?

 

[xiaofan]

Try setting forward to accept, and also try turning off the offload options..

Can you view the raw ruleset with ip6tables/nftables from the cli?

Not so sure what you mean in the first sentence. There is a rule to accept forwarding of the ICMPv6 messages. And I am not enabling any offloading options since they are not compatible with SQM which I have not enabled now but may use for experiments occassionally.

I will post the CLI output a bit later.

 

[xiaofan]

My OpenWRT firewall rules, more or less the same as default.

root@OpenWrt:~# cat /etc/config/firewall

config defaults
        option syn_flood '1'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'

config zone 'lan'
        option name 'lan'
        option input 'ACCEPT'
        option output 'ACCEPT'
        option forward 'ACCEPT'
        list network 'lan'
        list network 'wg_lan'
        list network 'LAN2'

config zone 'wan'
        option name 'wan'
        list network 'wan'
        list network 'wan6'
        option input 'REJECT'
        option output 'ACCEPT'
        option forward 'REJECT'
        option masq '1'
        option mtu_fix '1'

config forwarding
        option src 'lan'
        option dest 'wan'

config rule
        option name 'Allow-DHCP-Renew'
        option src 'wan'
        option proto 'udp'
        option dest_port '68'
        option target 'ACCEPT'
        option family 'ipv4'

config rule
        option name 'Allow-Ping'
        option src 'wan'
        option proto 'icmp'
        option icmp_type 'echo-request'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-IGMP'
        option src 'wan'
        option proto 'igmp'
        option family 'ipv4'
        option target 'ACCEPT'

config rule
        option name 'Allow-DHCPv6'
        option src 'wan'
        option proto 'udp'
        option dest_port '546'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-MLD'
        option src 'wan'
        option proto 'icmp'
        option src_ip 'fe80::/10'
        list icmp_type '130/0'
        list icmp_type '131/0'
        list icmp_type '132/0'
        list icmp_type '143/0'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Input'
        option src 'wan'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        list icmp_type 'router-solicitation'
        list icmp_type 'neighbour-solicitation'
        list icmp_type 'router-advertisement'
        list icmp_type 'neighbour-advertisement'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-ICMPv6-Forward'
        option src 'wan'
        option dest '*'
        option proto 'icmp'
        list icmp_type 'echo-request'
        list icmp_type 'echo-reply'
        list icmp_type 'destination-unreachable'
        list icmp_type 'packet-too-big'
        list icmp_type 'time-exceeded'
        list icmp_type 'bad-header'
        list icmp_type 'unknown-header-type'
        option limit '1000/sec'
        option family 'ipv6'
        option target 'ACCEPT'

config rule
        option name 'Allow-IPSec-ESP'
        option src 'wan'
        option dest 'lan'
        option proto 'esp'
        option target 'ACCEPT'

config rule
        option name 'Allow-ISAKMP'
        option src 'wan'
        option dest 'lan'
        option dest_port '500'
        option proto 'udp'
        option target 'ACCEPT'

config rule 'wg'
        option name 'Allow-WireGuard-lan'
        option src 'wan'
        option dest_port '51820'
        option proto 'udp'
        option target 'ACCEPT'