Access to TCP header fields in MirageOS?

Learning
1

Hello,

I am currently writing a OCaml library that implements a firewall decision tree that is usable as a functor. Though the functor provided would be generic, the primary backend I intend to use would be MirageOS.

Currently I am writing the network connection tracking code which would require access to TCP header fields (flags, sequence number, etc). But at first glance they don’t seem to be available directly through MirageOS’s API.

While I am comfortable with writing a parser to extract the needed information from the buffer, I’d still want to check if the functionality is actually there and I just missed it.

Cheers,
Darren

2

Dear Darren,

This is most exciting work you are planning! I was just discussing with @dra27 the other day some plans we had about building some packet filtering unikernels for our infrastructure, and @hannes and @yomimono and @talex5 have been hacking on Qubes firewalls, NATs, new VPN implementations and other such uses in recent years.

To answer your question: the various header fields are not systematically exposed in the TCP interfaces, but can be accessed via the Tcp_wire protocol from the cstruct. This isn’t ideal – we should expose them as first-class functions to make it easier to build higher level libraries such as the firewall functor you are designing.

The best thing to do is to create an issue or a PR on the mirage-tcpip repo so that we can see what this should look like. @djs55 may also have some ideas as he is the lead maintainer of the vpnkit tool used in Docker that also does similar analysis on TCP/IP traffic.

3

Thanks very much for your quick response!

Ah nice! I have been thinking surely people have made effort on this front, but yet to find any repositories on it (EDIT : as in a generic library, the Qubes firewall was prominent obviously). You can see the repo here. Following text describes the current design in place if you’re interested.

Currently the firewall draws its design from iptables and pf, aiming to provide functionality such as filtering, load balancing, and NAT.

The basic units in the tree are “selector”, which picks the next path to follow, and “modifier”, which may modify the PDU, and “selecting modifier”, should the need of a node that both selects and modifies arise, though doesn’t seem likely. NAT can then be implemented as a modifier, load balancer can be implemented as a selector, filter is just a special case of selector where it picks the default outcome when condition is not met.

Users are free to implement their own selectors, modifiers and selecting modifiers obviously. To support rules which rely on knowing the output interface, a routing logic unit (RLU) is provided to selector etc that answers the output interface, next hop ip address and so on. Having this RLU accessible to each node allows easier handling with address translation stuff.

Ah gotcha, I’ll create an issue shortly then.

4

This sounds like really interesting work!

As @avsm said there’s some firewall-like code in the vpnkit tool which could possibly be updated to use your new library. vpnkit receives ethernet traffic on one side and proxies it to and from regular user space sockets. It provides Internet access for Docker containers running on Mac and Windows bypassing kernel networking (because routing tables and NAT rules can be accidentally messed up by corporate VPN policies)

vpnkit parses incoming ethernet frames into a simple data type: https://github.com/moby/vpnkit/blob/1c28c5d61033983ff3f32d48a675d270fbad55e6/src/hostnet/frame.mli

type t =
  | Ethernet: { src: Macaddr.t; dst: Macaddr.t; payload: t } -> t
  | Arp:      { op: [ `Request | `Reply | `Unknown ] } -> t
  | Icmp:     { ty: int; code: int; raw: Cstruct.t; icmp: icmp } -> t
  | Ipv4:     ipv4 -> t
  | Udp:      { src: int; dst: int; len: int; raw: Cstruct.t; payload: t } -> t
  | Tcp:      { src: int; dst: int; syn: bool; rst: bool; raw: Cstruct.t; payload: t } -> t
  | Payload:  Cstruct.t -> t
  | Unknown:  t

then the frames are pattern-matched in functions like this: https://github.com/moby/vpnkit/blob/1c28c5d61033983ff3f32d48a675d270fbad55e6/src/hostnet/slirp.ml#L573

let input_ipv4 t ipv4 = match ipv4 with

    (* Respond to ICMP *)
    | Ipv4 { raw; payload = Icmp _; _ } ->
      let none ~src:_ ~dst:_ _ = Lwt.return_unit in
      let default ~proto ~src ~dst buf = match proto with
      | 1 (* ICMP *) ->
        Stack_icmpv4.input t.endpoint.Endpoint.icmpv4 ~src ~dst buf
      | _ ->
        Lwt.return_unit in
      Stack_ipv4.input t.endpoint.Endpoint.ipv4 ~tcp:none ~udp:none ~default raw
      >|= ok

and they’re then injected back into instances of the Mirage TCP/IP stack, where each instance maps to one external IP address. For example there would be an instance (created on first use) for 8.8.8.8 which would accept incoming TCP connections and UDP datagrams and forward them to the real 8.8.8.8 address.

This is the second internal vpnkit design. In the first attempt I tried to use one single Mirage TCP/IP stack instance by extending it to have more information and control (e.g. to be able to set the from address of UDP packets). This was quite invasive and in the end I thought it was perhaps going in the wrong direction. This second design treats the Mirage TCP/IP “stacks” as being individual IP endpoints and not gateways or NAT boxes or routers.

Anyway I agree that you should create an issue or PR on the mirage-tcpip repo!

5

@avsm @djs55 I have created the issue

6

Thanks for the response and pointers!

Currently the firewall respects the OSI layer separation fairly strictly, which might incur performance issue. The simpler layout you showed and the GADT representation are both definitely things I need to look into.

Right now the design is still in a very early stage (only started implementating 3 days ago though on my mind for a fair while), so I’ll go through vpnkit’s code as well to make sure the design is sane in comparison.

The other objective I forgot to mention is the firewall rule tree is supposed to be readily testable by supplying a mock module to the functor. This means user can test the firewall rules in quickcheck etc, which I think is neat.

7

Some other links you might find useful:

1 Like
8

@djs55 Is there any particular reason for choosing GADT for the types in Frame?

9

Coincidentally I found out tree-based firewall is a thing as I’m renaming the project today (not surprising, just I didn’t actively look for one during the start of the project) : https://opus.lib.uts.edu.au/bitstream/10453/116911/2/02whole.pdf

I believe the current design and the design shown in the paper share the same intuition. The paper highlights one thing that I failed to realise is the reduction of packet processing time complexity from O(n) of list based firewall to O(lg n) of tree based firewall (assuming relatively balanced tree), so the performance issue may not be as great as I previously thought.

Progress wise, TCP and UDP connection tracking code has been added, the README of the repo now has a progress status section as well. I’m anticipating a MirageOS demo to be ready in 1-2 months, and publishing of the initial version of library afterwards if the demo goes alright.

1 Like