faking services on an entire IP subnet

In a previous post, we introduced coprocessing and faking a TCP service on a single IP. The FAUCET coprocessor feature allows an external host to inject packets to a given VLAN (or even port). FAUCET ACLs are used to select traffic to be sent to the coprocessor (by default nothing is sent for coprocessing).

In this post we will develop the idea further – we will fake a TCP service on all hosts in a subnet, and we will do away with the need to deal with ARP.

Consider this topology. The two real hosts are at the bottom. We want to cause, from 192.168.2.5’s point of view, 192.168.2.1 to have an SMTP service it doesn’t really have. Further, from 192.168.2.5’s point of view, we want any host within 192.168.2.0/24 to appear to have that service.

The first step is to have FAUCET apply a port ACL to 192.168.2.5’s port, saying we want TCP port 25 (we’re faking SMTP) for all of 192.168.2.0/24 to be coprocessed. We could also use a VLAN ACL (pipette would need a small modification to allow it to receive tagged VLAN packets in that case).

Port 13 will have the host with 192.168.2.5 – port 18 is where the coprocessor is connected.

vlans:
  trusted:
    vid: 2
interfaces:
  13:
    acls_in:
      - coprocessssh
    description: 802.1x Pi
    native_vlan: trusted
  18:
    coprocessor:
      strategy: vlan_vid
    description: coprocessor NFV
acls:
  coprocessssh:
  - rule:
      dl_type: 0x800
      ip_proto: 6
      ipv4_src: 192.168.2.0/24
      tcp_dst: 25
      actions:
        output:
          ports: [18]
  - rule:
      actions:
        allow: 1

Now, on the coprocessor, we’ll need OVS and Ryu installed, and pipette. pipette patches TCP traffic in and out of a namespace where the fake service will run while maintaining the L2 illusion that the fake service is running on the real host.

# interface connected to FAUCET switch
NFVINT=enx0023565c8859
# used privately by the fake service.
FAKEHW=0e:00:00:00:00:66
FAKECLIENTHW=0e:00:00:00:00:67
# all 192.168.2.0/24 port 25 will be redirected to this fake IP.
FAKEIP=192.168.2.1/24
NS=copro

ip netns add $NS
ifconfig $NFVINT up
ip link add dev fake0 type veth peer name fakeovs0
ifconfig fakeovs0 up
ip link set fake0 netns $NS
ip netns exec $NS ifconfig fake0 hw ether $FAKEHW $FAKEIP up
for i in $NFVINT fakeovs0 ovs-system ; do
  echo 1 > /proc/sys/net/ipv6/conf/$i/disable_ipv6
done
ifconfig ovs-system 0.0.0.0

ovs-vsctl del-br br0
ovs-vsctl add-br br0
ovs-ofctl del-flows br0

for i in $NFVINT fakeovs0 ; do
  ovs-vsctl add-port br0 $i
done

ovs-vsctl set-controller br0 tcp:127.0.0.1:6653
ryu-manager pipette.py --verbose

Now we can run the fake SMTP service in the fake namespace. Note, even though the fake service is only running on 192.168.2.1, pipette is arranging for TCP traffic for the entire subnet to be mapped to that IP address as below.

# while /bin/true ; do echo completely legit SMTP service| ip netns exec copro nc -l 192.168.2.1 25 ; done

With everything ready, let’s try to connect to the fake SMTP server, apparently on 192.168.2.1, but in reality inside the coprocessor:

pi@pi8021x:~ $ telnet 192.168.2.1 25
Trying 192.168.2.1...
Connected to 192.168.2.1.
Escape character is '^]'.
completely legit SMTP service

How odd! What about 192.168.2.2?

pi@pi8021x:~ $ telnet 192.168.2.2 25
Trying 192.168.2.2...
Connected to 192.168.2.2.
Escape character is '^]'.
completely legit SMTP service

It appears there too! What’s going on? pipette is doing some TCP MAC and IP translation and mapping that into an isolated namespace, while also proxying the real host’s MAC address dynamically.

root@coprocessor:/home/pi# ovs-ofctl -OOpenFlow13 dump-flows br0
 cookie=0x0, duration=178.489s, table=0, n_packets=6, n_bytes=428, priority=1,in_port=enx0023565c8859 actions=goto_table:1
 cookie=0x0, duration=178.489s, table=0, n_packets=9, n_bytes=600, priority=1,in_port=fakeovs0 actions=goto_table:2
 cookie=0x0, duration=178.490s, table=0, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x0, duration=14.819s, table=1, n_packets=1, n_bytes=70, idle_timeout=30, priority=2,tcp,dl_src=b8:27:eb:d6:31:ca,nw_dst=192.168.2.1,tp_dst=25 actions=set_field:0e:00:00:00:00:67->eth_src,set_field:0e:00:00:00:00:66->eth_dst,set_field:192.168.2.1->ip_dst,output:fakeovs0
 cookie=0x0, duration=178.490s, table=1, n_packets=2, n_bytes=146, priority=1,tcp actions=CONTROLLER:65509
 cookie=0x0, duration=178.490s, table=1, n_packets=0, n_bytes=0, priority=0 actions=drop
 cookie=0x0, duration=14.819s, table=2, n_packets=1, n_bytes=66, idle_timeout=30, priority=2,tcp,nw_dst=192.168.2.5,tp_src=25 actions=set_field:aa:56:63:87:9e:e9->eth_src,set_field:b8:27:eb:d6:31:ca->eth_dst,set_field:192.168.2.1->ip_src,push_vlan:0x8100,set_field:4098->vlan_vid,output:enx0023565c8859
 cookie=0x0, duration=178.490s, table=2, n_packets=2, n_bytes=84, priority=1,arp actions=CONTROLLER:65509
 cookie=0x0, duration=178.490s, table=2, n_packets=4, n_bytes=280, priority=0 actions=drop