Playing with Qubes Networking for Fun and Profit

The virtual interfaces in client VMs are called ethX, and are provided by the xen_netfront kernel module, and the corresponding interfaces in the Net/Proxy VM are called vifX.Y and are created by the xen_netback module.

Also, any VM-to-VM traffic, among the VMs connected to the same Net/Proxy VM is blocked by default.

Additionally, each Proxy VM enforces system-wide firewaling rules, specifically the rules for all the directly connected VMs. Those firewalling rules are centrally managed in Dom0 and exposed to each Proxy VM through Xen store. One useful application of this firewalling mechanism is to limit certain VMs to only specific type of white-listed traffic to minimize likelihood of user mistakes. A good example could be a work VM that might be limited to network connectivity only with the select corporate servers and denied all other traffic. This way, when the user receives an email message with an embedded http link (possibly leading to a malicious website) and accidentally clicks on it, nothing wrong happens.

The current infrastructure doesn't support IPv6 routing, but we will likely add this support in the upcoming Beta 3.

The default networking topology in Qubes OS

When you proceed with the default installation of Qubes Beta 2, then your initial networking topology looks like on the diagram below:

So, by default there is one Net VM, called 'netvm', that is automatically assigned all the networking devices in the system. There is also one Proxy VM, called 'firewallvm' that is directly connected to the default Net VM, and which provides networking to all other VMs in the system. This Proxy VM is used for firewall rules enforcement. Each such service VM consumes 200MB of RAM by default.

For some VMs it might be desirable to completely disconnect them from any kind of networking access. This can be easy done using the following command (issued from Dom0's konsole):

For example I have a 'vault' VM that I use for keeping my master PGP keys, and other secrets, and this machine is not connected to any network.

Using multiple Net VMs for physically isolated networks 

In some scenarios the machine might be connected to two or more physically separate networks (e.g. safe corporate intranet, reachable via ethernet cable on the user's desk, and the unsafe and evil Internet, reachable via WiFi card).

It is easy to use more than one Net VMs in Qubes, and assign different networking devices to different Net VMs, and also decide which VMs are connected to which Net VMs. The diagram below presents an exemplary such setup:

 

We could created such a setup using the following commands (issued in Dom0):

Currently qvm-create when used with the --net option automatically assigns all networking devices to the just created VM, so in the example above you would want to remove extra devices from each Net VM using qvm-pci -d, leaving only those you really want, e.g.: 

[dom0]$ qvm-pci -l netvm1 # to get a list of currently assigned devices

Now we should create the Firewall VMs:

... and connect them to proper Net VMs:

And now, for any other VM, just set the appropriate Net VM (either firewallvm1 or firewallvm2, or 'none), to get it assigned to either of the isolated networks, e.g.:

...

This configuration provides very strong isolation between the VMs belonging to network #1, and the VMs belonging to network #2. Specifically, this becomes significant if we fear about potential remotely exploitable bugs in the client code of the core TCP/IP stack (in this case the Net VM could potentially compromise all the connected VMs -- but the same problem applies to even physically separated machines that use the same network).

Let's now play a bit with Proxy VMs and see how we can use it to create a simple Tor proxy VM. Such a VM would provide anonymized networking to all its clients, so would allow to easily create VMs for anonymous Internet access. The simple setup we would like to prepare is depicted on the figure below:

The 'torvm' Proxy VM provides anonymized networking to 'anon-web' and 'anon-bitcoin' VMs. All the traffic generated by the VMs behind 'torvm' is either fed into the Tor network, or discarded. Furthermore, any app running in those VMs is not able to read any global system identifiers, such as the external IP, external MAC address, etc.

#!/bin/sh

killall tor

QUBES_IP=$(xenstore-read qubes_ip)

TOR_TRANS_PORT=9040

if [ X$QUBES_IP == X ]; then

echo "Error getting QUBES IP!"

echo "Not starting Tor, but setting the traffic redirection anyway to prevent leaks."

QUBES_IP="127.0.0.1"

else

/usr/bin/tor \

--SocksPort 0 \

--TransListenAddress $QUBES_IP --TransPort $TOR_TRANS_PORT \

--DNSListenAddress $QUBES_IP --DNSPort 53 \

--RunAsDaemon 1 --ControlPort 9051 \

|| echo "Error starting Tor!"

fi

echo “0” > /proc/sys/net/ipv4/ip_forward

/sbin/iptables -t nat -F

/sbin/iptables -t nat -A PREROUTING -i vif+ -p udp --dport 53 -j DNAT --to-destination $QUBES_IP:53

/sbin/iptables -t nat -A PREROUTING -i vif+ -p tcp -j DNAT --to-destination $QUBES_IP:$TOR_TRANS_PORT

/sbin/iptables -I INPUT 1 -i vif+ -p udp --dport 53 -j ACCEPT

/sbin/iptables -I INPUT 2 -i vif+ -p tcp --dport 9040 -j ACCEPT

/sbin/iptables -F FORWARD

echo “1” > /proc/sys/net/ipv4/ip_forward

#!/bin/sh

# Uncomment this if you would like to use a custom torrc file:

#rm -f /rw/config/log

#ln -sf /rw/config/torrc /etc/tor/torrc

chkconfig qubes_netwatcher off

chkconfig qubes_firewall off

/rw/config/start_tor_proxy.sh

#!/bin/sh

/rw/config/start_tor_proxy.sh

The web browser runs in the 'anon-web' VM that uses 'torvm' for networking access, and thus all the traffic generated by 'anon-web' is routed through the Tor network, or discarded if it's a different traffic than TCP or DNS.

My 'wififun' VM in action using a simple sslstrip attack, that surprisingly still works pretty nice...

A bit more advanced networking setup. The usbvm has a 3G modem assigned, and it is possible to dynamically switch between the Net VMs without restarting any other VMs.