Mirage Cap'n Proto RPC library

mirageos
rpc
capnp
fuzzing

#1

Hi all,

I’m developing an OCaml implementation of the Cap’n Proto RPC protocol, and it’s at a point where early testers might like to try it:

https://github.com/mirage/capnp-rpc

The README contains a tutorial.

Cap’n Proto RPC is built on top of the Cap’n Proto serialisation format, which is similar to protobuf (an earlier system by the same author). You define your protocol in a schema file and can then compile it to generate typed bindings for various languages. The system makes it easy to upgrade the schema files in a backwards compatible way.

The RPC system provides a number of useful features, but the keys ones are:

  • As well as pure data, you can pass references to services (functions, objects, callbacks) in your messages too. Sending a reference to a service in a message automatically grants the recipient permission to use it (the Object Capability security model).

  • Messages can be pipelined. For example, you can ask for a service to be created, and then immediately invoke a method on it or pass it to another service, without waiting for the reply. This allows you to make clean APIs with simple functions that can be composed, rather than trying to create complex APIs to compensate for network latency.

  • Messages are delivered in E-order, meaning that if you send multiple messages to a capability then they will arrive in the order in which they were sent. This works even if the target was a promise that later turns out to be located elsewhere - pipelined messages will be forwarded and processed before any later messages sent directly. This avoids many race conditions, without greatly harming performance.

Since my last posting (on the mirage mailing list), I’ve fixed some bugs with out-of-order delivery, and made lots of changes to the capnp-ocaml schema compiler to improve the API of the RPC library. In particular, you can now attach capabilities directly to messages (previously, you had to export the capability explicitly and then attach its index to the message), and methods are now passed the request message directly rather than a wrapper containing the message and the index table. I’ve also moved some Unix specific code into a separate “capnp-rpc-unix” package, bringing us closer to being able to run it in a unikernel.

I’d be happy to hear from people trying it out. If you used the previous API, see the upgrade notes in:


#2

One of my favourite things about how @talex5 has been building this library is his use of Crowbar and afl-fuzz to test the corners of the library as it’s being built. Fuzz-driven development – fudd! https://github.com/mirage/capnp-rpc/tree/master/fuzz


#3

Ah, yes. It has found some subtle bugs, e.g. https://github.com/mirage/capnp-rpc/issues/59 (and, interestingly, it turned out that the C++ reference implementation didn’t handle this case correctly either).

I posted some details of how the fuzzing works at https://groups.google.com/d/msg/capnproto/_1kBtRSC51s/H6tvW0BICAAJ

Note however that it now uses afl directly, without crowbar, as that turned out to be easer for testing interleavings of actions (rather than data structures, which crowbar is good at). Also, crowbar isn’t released, which made depending on it problematic.