Configuration: NAT IPv4 and Routed IPv6 with Proxmox on Kumsufi

Getting IPv6 to work on OVH Kimsufi server is not easy. Configuring Routed IPv6 for NATed virtual machines is even a disaster. After about 5 hours of struggle I finally come up with a working config.

The goal of this post is to document how to go from a blank server into a proxmox server with NAT IPv4 and Routed IPv6 KVM virtual machines set up.

Preparation

[ ↑ ]

First thing first, you’ll need a server from Kimsufi. Here’s my KS-3C server specs for example.

CPU: Intel(R) Core(TM) i3-2130 CPU @ 3.40GHz

RAM: 8 GiB

Networking: 100mbps unmetered, 1x IPv4, 1xIPv6

Storage: 1x 2TB SATA

Please do note that, although only 1 IPv6 is assigned to the server, you can actually use (or steal) the entire /56 range (Daniele, 2016). I’d recommend using your own /64 space.

Kimsufi comes with an automatic Debian installer which is nice by the way, however it’s not a completelyvanilla installation as OVH will add some scripts and changes to the system. If you are not a paranoid like me, you can jump straight to first boot section and save a few minutes.

OS Installation

[ ↑ ]

I was basically following the blog post (Félizard, 2012) with a few changes to my own needs since It’s a proxmox server. I’d recommend installing Debian from the management panel first, as it writes OS type into its own database may avoid some issues afterwards. When the installation is done, you can then boot into rescue CD rescue64-pro.

Now, partition your hard drive. Wipe all partitions and recreate them according to your needs. Mine looks like this:

GPT: [BIOS Boot] [Root 40GiB] [Swap 8GiB] [LVM *GiB]    Device         Start        End    Sectors  Size Type  /dev/sda1         64       4095       4032    2M BIOS boot  /dev/sda2       4096   83886079   83881984   40G Linux filesystem  /dev/sda3   83886080  100663295   16777216    8G Linux swap  /dev/sda4  100663296 3907029134 3806365839  1.8T Linux LVM
# Wipe and recreate your partitions  fdisk /dev/sda    # Format root  mkfs.ext4 /dev/sda2    # Mount it  mkdir /target  mount /dev/sda2 /target    # Load the system  # Use debian.mirrors.ovh.net if your server resides in France  # Append packages you want on first boot  debootstrap --verbose --arch amd64 --variant minbase --include dialog,vim,ifupdown,netbase,net-tools,ssh,systemd,systemd-sysv,locales,wget stretch /target http://debian.bhs.mirrors.ovh.net/debian    # Chroot into target system  mount -t proc none /target/proc  mount -t sysfs none /target/sys  mount -o bind /dev /target/dev  chroot /target /bin/bash    # Pull in proxmox  echo "deb http://download.proxmox.com/debian/pve stretch pve-no-subscription" > /etc/apt/sources.list.d/pve-install-repo.list  wget http://download.proxmox.com/debian/proxmox-ve-release-5.x.gpg -O /etc/apt/trusted.gpg.d/proxmox-ve-release-5.x.gpg  apt update  apt install proxmox-ve  apt upgrade  rm /etc/apt/sources.list.d/pve-enterprise.list    # Set up other stuff, you know  vi /etc/fstab  vi /etc/hostname  vi /etc/hosts  vi /etc/ssh/sshd_config  passwd    # Set up your networking  vi /etc/network/interfaces

Networking is the tricky part. Debian Stretch uses so called Predictable Network Interface Names, which is unpredictable until you actually see it. Luckily, for KS-3C and KS-4C, it seems to be just eno1. For other types of server, please use systemd-networkd wildcard matching (Porquet, 2016) to obtain your initial access to the server.

# Example of interfaces file  auto lo  iface lo inet loopback  auto eno1  iface eno1 inet static    address  142.4.200.200    netmask  255.255.255.0    gateway  142.4.200.254

When everything is done, leave the chroot and reboot the server. Remember to switch back to HDD boot in your kimsufi management panel first.

exit  umount /target/proc  umount /target/sys  umount /target/dev  umount /target  reboot

First Boot

[ ↑ ]

As usual, many things to do like setting up swap and LVM, installing missing packages.

And of course, configur network bridges for VMs.

Note: bridging on eno1 is not absolutely necessary — use eno1 instead if you choose to do so.

# Example of interfaces file  source-directory /etc/network/interfaces.d    auto lo  iface lo inet loopback    auto eno1  iface eno1 inet manual    auto vmbr0  iface vmbr0 inet static    address  142.4.200.200    netmask  255.255.255.0    gateway  142.4.200.254    bridge_ports eno1    bridge_stp off    bridge_fd 0    iface vmbr0 inet6 static    address  2607:5300:66:6789::ffff    netmask  128    post-up sleep 5; /sbin/ip -6 route add 2607:5300:66:67ff:ff:ff:ff:ff dev vmbr0    post-up sleep 5; /sbin/ip -6 route add default via 2607:5300:66:67ff:ff:ff:ff:ff    pre-down /sbin/ip -6 route del default via 2607:5300:66:67ff:ff:ff:ff:ff    pre-down /sbin/ip -6 route del 2607:5300:66:67ff:ff:ff:ff:ff dev vmbr0    auto vmbr1  iface vmbr1 inet static    address  172.16.0.1    netmask  255.255.0.0    bridge_ports none    bridge_stp off    bridge_fd 0    iface vmbr1 inet6 static    address  2607:5300:66:6789::1    netmask  64

This is an example config for dual stack VMs with IPv4 NATed and IPv6 routed.

Internet <==> [(vmbr0) Host (vmbr1)] <==> [(ens18) Virtual Machine ]

Here the important part is netmask 128 on the vmbr0. The IPs are not important as longs as they are all different. I picked ::ffff for host and ::1 for subnet gateway. Feel free to choose IPs on your discretion. Same applies to private IPv4 subnet.

Check if you can ping to any IPv6 site like ipv6.google.com.

Edit /etc/sysctl.conf and enable traffic forwarding and ndp proxying.

# Example of sysctl.conf file  net.ipv4.ip_forward = 1  net.ipv6.conf.all.forwarding = 1  net.ipv6.conf.all.proxy_ndp = 1  vm.swappiness = 10  net.ipv6.bindv6only = 1

And reboot for networking to take effects.

NAT IPv4

[ ↑ ]

NAT IPv4 is achieved by iptables. Install iptables-persistent package using apt, and modify /etc/iptables/rules.v4 to properly NAT and forward ports for VMs.

# Example of /etc/iptables/rules.v4  *nat  -A POSTROUTING -s 172.16.0.0/16 -o vmbr0 -j MASQUERADE  -A PREROUTING -i vmbr0 -p tcp -m tcp --dport 10022 -j DNAT --to-destination 172.16.0.100:22  COMMIT

Load iptables with iptables-restore < /etc/iptables/rules.v4.

Routed IPv6

[ ↑ ]

Routing IPv6 on OVH network was such a headache and not much useful tutorial can be found at the other end of search engine. A typical strategy is to use ip -6 neigh add proxy command (Groarke, 2010) to manually add all IPv6 addresses into proxy list. It might be acceptable if you only have 1 or 2 VMs, but this is still tricky to accomplish (you have to write startup scripts, etc).

Luckily, there’s a small tool called npd6 which does this for you, intelligently and easily.

wget http://ftp.ca.debian.org/debian/pool/main/n/npd6/npd6_1.1.0-1_amd64.deb  dpkg -i npd6_1.1.0-1_amd64.deb  vi /etc/npd6.conf
# Example of /etc/npd6.conf  prefix = 2607:5300:66:6789:  interface = vmbr0  ralogging = off  listlogging = off

Restart the service and it should do its job. Whenever a new IPv6 is configured on the private side, npd6 will respond to its corresponding neighbor solicitation request for that VM. Thank you Sean for writing this!

Networking Config inside VM

[ ↑ ]

I tend to use systemd-networkd and it couldn’t be easier after someone else has done the heavylifting.

[Match]  Name=ens18    [Network]  Address=172.16.0.102/16  Gateway=172.16.0.1  Address=2607:5300:66:6789::102/64  Gateway=2607:5300:66:6789::1

Reference List

[ ↑ ]

  1. Daniele / Otacon22 (2016, Febuary 21). IPv6 setup in two hosting providers compared: awful (OVH) and awesome (Online.net). Retrieved November 27, 2017, from https://otacon22.com/2016/02/21/two-hosting-providers-ipv6-setups-compared-ovh-online-net/

  2. Félizard, C. (2012, November 05). How to install a vanilla Debian on a Kimsufi. Retrieved November 27, 2017, from https://infertux.com/posts/2012/11/05/how-to-install-a-vanilla-debian-on-a-kimsufi/

  3. Porquet, J. (2016, May 08). Howto install a vanilla ArchLinux on a Kimsufi/OVH server. Retrieved November 27, 2017, from https://joel.porquet.org/wiki/hacking/archlinux_kimsufi/

  4. Groarke, S. (2010, March 24). IPv6 – Proxy the neighbors. Retrieved November 27, 2017, from https://www.ipsidixit.net/2010/03/24/239/