a FAUCET dev blog

Docker based systems are great, but if you live in New Zealand (and in rural New Zealand, like I do), repeated docker pulls to the Internet can be a little painful. Fortunately you can configure docker-registry to locally cache.

First install docker-registry and configure it as a proxy.

# apt-get install docker-registry

# vi /etc/docker/registry/config.yml

# cat /etc/docker/registry/config.yml 
 version: 0.1
 log:
   fields:
     service: registry
 storage:
   cache:
     blobdescriptor: inmemory
   filesystem:
     rootdirectory: /var/lib/docker-registry
   delete:
     enabled: true
 http:
   addr: :5000
   headers:
     X-Content-Type-Options: [nosniff]
 health:
   storagedriver:
     enabled: true
     interval: 10s
     threshold: 3
 proxy:
   remoteurl: https://registry-1.docker.io

# service docker-registry restart

Now configure docker to use the proxy.

# vi /etc/docker/daemon.json 

# cat /etc/docker/daemon.json 
 {
   "storage-driver": "overlay2",
   "registry-mirrors": ["http://localhost:5000"]
 }

# service docker restart

Now when you do a docker pull, the cache will be used if the latest image is available locally.

# docker pull faucet/gauge
 Using default tag: latest
 latest: Pulling from faucet/gauge
 Digest: sha256:26e2c56800df26deccc6a1d8377c697d7ca2a38cf2d8fc52739bb6b634028298
 Status: Image is up to date for faucet/gauge:latest
 docker.io/faucet/gauge:latest

# grep docker-registry /var/log/syslog|grep gauge
 Nov  5 22:29:21 finf-gw docker-registry[29624]: time="2019-11-05T22:29:21Z" level=info msg="Challenge established with upstream : {https   registry-1.docker.io /v2/  %!s(bool=false)  } &{{{%!s(int32=0) %!s(uint32=0)} %!s(uint32=0) %!s(uint32=0) %!s(int32=0) %!s(int32=0)} map[https://registry-1.docker.io:443/v2/:[{bearer map[realm:https://auth.docker.io/token service:registry.docker.io]}]]}" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=fb8d44dd-193a-4a18-8625-35dc3ea015eb http.request.method=GET http.request.remoteaddr="[::1]:49450" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 vars.name=faucet/gauge vars.reference=latest version=v2.6.2+debian
 Nov  5 22:29:23 finf-gw docker-registry[29624]: time="2019-11-05T22:29:23Z" level=info msg="Adding new scheduler entry for faucet/gauge@sha256:26e2c56800df26deccc6a1d8377c697d7ca2a38cf2d8fc52739bb6b634028298 with ttl=167h59m59.999997987s" go.version=go1.8.1 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
 Nov  5 22:29:23 finf-gw docker-registry[29624]: time="2019-11-05T22:29:23Z" level=info msg="response completed" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=fb8d44dd-193a-4a18-8625-35dc3ea015eb http.request.method=GET http.request.remoteaddr="[::1]:49450" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" http.response.contenttype=application/vnd.docker.distribution.manifest.v2+json http.response.duration=3.545613795s http.response.status=200 http.response.written=1582 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
 Nov  5 22:29:23 finf-gw docker-registry[29624]: ::1 - - [05/Nov/2019:22:29:20 +0000] "GET /v2/faucet/gauge/manifests/latest HTTP/1.1" 200 1582 "" "docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 \(linux\))"
 Nov  5 22:31:51 finf-gw docker-registry[29624]: time="2019-11-05T22:31:51Z" level=info msg="response completed" go.version=go1.8.1 http.request.host="localhost:5000" http.request.id=013a90e2-11e3-44d8-8875-03aca0c54be5 http.request.method=GET http.request.remoteaddr="[::1]:49702" http.request.uri=/v2/faucet/gauge/manifests/latest http.request.useragent="docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 (linux))" http.response.contenttype=application/vnd.docker.distribution.manifest.v2+json http.response.duration=2.237893853s http.response.status=200 http.response.written=1582 instance.id=9bdb53c3-1d48-4015-9708-8f1ddc816651 version=v2.6.2+debian
 Nov  5 22:31:51 finf-gw docker-registry[29624]: ::1 - - [05/Nov/2019:22:31:49 +0000] "GET /v2/faucet/gauge/manifests/latest HTTP/1.1" 200 1582 "" "docker/19.03.4 go/go1.12.10 git-commit/9013bf583a kernel/5.3.0-19-generic os/linux arch/amd64 UpstreamClient(Docker-Client/19.03.4 \(linux\))"

Google’s 2017 presentation at OVScon on FAUCET and SDN.

Tim Toole – Sandia National Laboratories
Given prior successes at SC18, a group at Sandia transitioned a small lab from using traditional switching and routing to near-100% OpenFlow 1.3 architecture using FAUCET. This talk discusses our use case, goals, and our journey from operating a traditional network to managing a multivendor SDN environment with FAUCET. SNL is managed and operated by NTESS under DOE NNSA contract DE-NA0003525.

Slides and video.

With the coprocessor feature, FAUCET allows an external NFV host to receive packets from the dataplane, inject arbitrary packets, or both. In particular, coprocessing allows you to inject services into the dataplane, or even override services on hosts already present.

In this example, we have a real host 192.168.2.1 (a router) which has an ssh service. We have a host on the network, 192.168.2.5. 192.168.2.5 may do anything it likes, but if it should try to connect to ssh on 192.168.2.1, we want to redirect that connection to a fake service running on the coprocessor server.

First we will need to tell FAUCET to intercept the service using an ACL, which will be applied to the test host’s port. Port 18 is the coprocessor port, and port 13 is the test host.

acls:
  coprocessssh:
    - rule:
        dl_type: 0x806
        actions:
          allow: 1
          output:
            ports: [18]
    - rule:
        dl_type: 0x800
        ip_proto: 6
        tcp_dst: 22
        actions:
          output:
            ports: [18]
    - rule:
        actions:
          allow: 1
vlans:
  trusted:
    vid: 2
dps:
  dp:
    interfaces:
      13:
        native_vlan: trusted
        acls_in: [coprocessssh]
      18:
        coprocessor: {strategy: vlan_vid}

FAUCET will now redirect port 22 to the coprocessor port. We will now need to set up an OVS switch on the coprocessor so we can start fake services, and allow ARP to work in parallel with the real host. The fake host will need to use the same MAC address as the real one, which won’t cause a conflict because FAUCET is intercepting just the TCP service and FAUCET knows not to re-learn the real host on the coprocessor port.

Note the OVS actions for setting VID to 4098 – that’s the VID for the trusted VLAN (2) or’d with the VID present bit (4096). That will cause packets from the coprocessor to have the right VID to be switched back into the trusted VLAN.

root@coprocessor:/home/pi# cat setupcopro.sh 
#!/bin/sh

NFVINT=enx0023565c8859
FAKEHW=aa:56:63:87:9e:e9
FAKEIP=192.168.2.1/24
NS=copro

ip netns add $NS
ifconfig $NFVINT up
ip link add dev fake0 type veth peer name fakeovs0
ifconfig fakeovs0 up
ip link set fake0 netns $NS
ip netns exec $NS ifconfig fake0 hw ether $FAKEHW $FAKEIP up
for i in $NFVINT fakeovs0 ovs-system ; do
  echo 1 > /proc/sys/net/ipv6/conf/$i/disable_ipv6
done
ifconfig ovs-system 0.0.0.0

ovs-vsctl del-br br0
ovs-vsctl add-br br0
ovs-ofctl del-flows br0

for i in $NFVINT fakeovs0 ; do
  ovs-vsctl add-port br0 $i
done

ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=1,actions=output:2"
ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=2,vlan_tci=0x0000/0x1fff,ip,tcp,actions=push_vlan:0x8100,set_field:4098->vlan_vid,output:1"
ovs-ofctl add-flow -OOpenFlow13 br0 "in_port=2,vlan_tci=0x0000/0x1fff,arp,actions=push_vlan:0x8100,set_field:4098->vlan_vid,output:1"

Now we will start a fake service on the coprocessor and connect to it from the test host.

root@coprocessor:/home/pi# cat fakessh.sh 
#!/bin/sh

while /bin/true ; do echo completely awesome ssh service| ip netns exec copro nc -l 192.168.2.1 22 ; done
root@coprocessor:/home/pi# ./fakessh.sh

pi@pi8021x:~ $ telnet 192.168.2.1 22
Trying 192.168.2.1...
Connected to 192.168.2.1.
Escape character is '^]'.
completely awesome ssh service
^]
telnet> close
Connection closed.

That’s it. We could run other services on the fake host – for example to broadcast CDP to impersonate a router, for example, or redirect other services.

I will be hosting a developer workshop (including how to get started with FAUCET) at Cyber Toa, 4pm, October 7th in Wellington. I’ll cover how to get up and running, an architectural overview of the FAUCET controller, and how to do development.

If you have any questions or topics you’d like me to include, please do send me email!

Please do register (depending on numbers we may use a larger venue).

This year we’ll be combining high availability (in the form of multiple controllers, and switches with multiple redundant links), 802.1X authentication, and distributed monitoring. You can find out more and follow developers on the PlugFest announcement list.

We’ll also be using a C64 with Ethernet interface as a test client – because as everyone knows, a network isn’t enterprise grade until you can get online with GeoLink.