#StackBounty: #kvm #ipv6 #amazon-linux #tap #virtual-interface Unable to connect vm to IPv6 internet via virtual tap interface

Bounty: 50

I’m having trouble connecting a vm to the IPv6 internet through a virtual tap device on the host. I.e., I cannot ping ipv6.google.com or the public IPv6 host global primary interface address. Ex:

Edit: Simple topology for clarity:

router ———— host —-(tap device)—- vm

-bash-4.2$ ping6 ipv6.google.com
    PING ipv6.google.com(sea15s11-in-x0e.1e100.net) 56 data bytes
    From 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c icmp_seq=1 Destination unreachable: Address unreachable
    From 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c icmp_seq=2 Destination unreachable: Address unreachable
    From 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c icmp_seq=3 Destination unreachable: Address unreachable
    ^C
    --- ipv6.google.com ping statistics ---
    4 packets transmitted, 0

 received, +3 errors, 100% packet loss, time 3082ms

or to the host’s global ipv6 address, I get the same error.

It appears there is some issue with neighbor discovery on the host, when I tcpdump the tap interface from the host’s tap endpoint I receive the solicitation messages but nothing is returned:

[user ~]$ sudo tcpdump ip6 -vv -i tp-0gn-0000go-0    
tcpdump: listening on tp-0gn-0000go-0, link-type EN10MB (Ethernet), capture size 262144 bytes
    01:45:16.596378 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c > ff02::1:ff00:200e: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has sea15s11-in-x0e.1e100.net
          source link-address option (1), length 8 (1): 02:fc:80:d4:52:b6
            0x0000:  02fc 80d4 52b6
    01:45:17.610410 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c > ff02::1:ff00:200e: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has sea15s11-in-x0e.1e100.net
          source link-address option (1), length 8 (1): 02:fc:80:d4:52:b6
            0x0000:  02fc 80d4 52b6
    01:45:18.634402 IP6 (hlim 255, next-header ICMPv6 (58) payload length: 32) 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c > ff02::1:ff00:200e: [icmp6 sum ok] ICMP6, neighbor solicitation, length 32, who has sea15s11-in-x0e.1e100.net
          source link-address option (1), length 8 (1): 02:fc:80:d4:52:b6
            0x0000:  02fc 80d4 52b6

Note: I’m able to ping ipv6.google.com from the host:

[user ~]$ ping6 ipv6.google.com
PING ipv6.google.com(sea15s11-in-x0e.1e100.net (2607:f8b0:400a:808::200e)) 56 data bytes
64 bytes from sea15s11-in-x0e.1e100.net (2607:f8b0:400a:808::200e): icmp_seq=1 ttl=39 time=9.93 ms
64 bytes from sea15s11-in-x0e.1e100.net (2607:f8b0:400a:808::200e): icmp_seq=2 ttl=39 time=10.1 ms
64 bytes from sea15s11-in-x0e.1e100.net (2607:f8b0:400a:808::200e): icmp_seq=3 ttl=39 time=10.1 ms

It looks like there’s an issue with the neighbor discovery. I’m not sure if I’m facing issues with DAD, NUD, or something else, or potentially not a neighbor discovery issue at all?

I currently only have the router in ip -6 neigh show, but my impression of the neighbor discovery cache was just to be a cache, and that the routes should still be intact and discoverable otherwise (though this is my very limited understanding). Maybe I’m missing some neighbor discovery/advertisement kernel parameters?

[user ~]$ ip -6 neigh show
fe80::460:a1ff:fec3:9cb6 dev eth0 lladdr 06:60:a1:c3:9c:b6 router STALE 

I have a hunch that I’m missing some net.ipv6 kernel parameters here, but I’m not really sure where to start with modifying them. Any suggestions are much appreciated. Full network setup information can be found below. Note that I manually configured the vm global address so it is very similar to the host, one is :XXXb/128 and one is :XXXc/128.

VM endpoint –
interface:

-bash-4.2$ ip a s eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether 02:fc:80:d4:52:b6 brd ff:ff:ff:ff:ff:ff
    inet 169.254.18.177/30 brd 169.254.18.179 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 2600:1f14:680:xxxx:66a3:79d5:6c1d:14c/128 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::fc:80ff:fed4:52b6/64 scope link 
       valid_lft forever preferred_lft forever

and relevant VM routes:

-bash-4.2$ ip -6 r s
2600:1f14:680:6f00:66a3:79d5:6c1d:14c dev eth0  proto kernel  metric 256  pref medium
fe80::/64 dev eth1  proto kernel  metric 256  pref medium
fe80::/64 dev eth0  proto kernel  metric 256  pref medium
default dev eth0  metric 1024  pref medium

Host – the tap and primary interfaces look like:

[user ~]$ ip a s tp-0gn-0000go-0
2393: tp-0gn-0000go-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP group default qlen 1000
    link/ether d2:d5:4e:f3:de:ab brd ff:ff:ff:ff:ff:ff
    inet 169.254.18.178/30 scope global tp-0gn-0000go-0
       valid_lft forever preferred_lft forever
    inet6 fe80::d0d5:4eff:fef3:deab/64 scope link 
       valid_lft forever preferred_lft forever
[user ~]$ ip a s eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9001 qdisc mq state UP group default qlen 1000
    link/ether 06:b6:f7:16:ac:04 brd ff:ff:ff:ff:ff:ff
    inet 172.30.255.4/28 brd 172.30.255.15 scope global dynamic eth0
       valid_lft 2994sec preferred_lft 2994sec
    inet6 2600:1f14:680:6f00:66a3:79d5:6c1d:14b/128 scope global dynamic 
       valid_lft 405sec preferred_lft 105sec
    inet6 fe80::4b6:f7ff:fe16:ac04/64 scope link 
       valid_lft forever preferred_lft forever

and the relevant routes:

[user ~]$ ip -6 r s
2600:1f14:680:6f00:66a3:79d5:6c1d:14b dev eth0 proto kernel metric 256 expires 389sec pref medium
2600:1f14:680:6f00:66a3:79d5:6c1d:14c dev tp-0gn-0000go-0 metric 1024 pref medium
2600:1f14:680:6f00::/64 dev eth0 proto kernel metric 256 pref medium
unreachable 3ffe:ffff::/32 dev lo metric 1024 error 4294967183 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::460:a1ff:fec3:9cb6 dev eth0 proto ra metric 1024 expires 1798sec hoplimit 64 pref medium

ip6tables filter is allowing everything

[user ~]$ sudo ip6tables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination 

and host is amazon-linux, similar to centos/rhel/fedora, cat /etc/os-release:

NAME ="Amazon Linux"
VERSION="2"
ID_LIKE="centos rhel fedora"

Currently net.ipv6.conf.[interface].forwarding is disabled, but when enabled on eth0 I received the same errors.

Any suggestions are much appreciated. Let me know if I’m missing any necessary information or anything conceptually. Thanks in advance.


Update: Also I should note that I don’t get any tcpdump packets when listening on eth0 of the host and trying to ping ipv6.google.com from the vm. As can be seen in the first tcpdump, the packets are first being sent to the all node solicitation multicast address which should be routed through eth0 (based off the local routing table), but I never see the packets go through eth0 via tcpdump. I currently have net.ipv6.conf.all.forwarding=1, net.ipv6.conf.all.accept_ra=2, and net.ipv6.conf.all.accept_ra_from_local=1.


Update #2: I came across this article. I added net.ipv6.conf.all.proxy_ndp=1 and added a proxy neighbor, ip -6 neigh add proxy <host eth0 global ip6 addr> dev <tap device> which allows me to ping the host’s eth0 global address from the vm. Still no luck connecting to ipv6.google.com from the vm though I feel I’m getting closer.


Get this bounty!!!

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.