# \[ANN\] capnp-rpc 0.1
**URL:** https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698
**Category:** Community
**Tags:** announce, rpc, capnp
**Created:** [August 11, 2017, 7:10pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698 "2017-08-11T19:10:17Z")
**Posts on this page:** 13
**Page:** 1
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 11, 2017, 7:10pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/1 "2017-08-11T19:10:17Z")
I’m pleased to announce that the OCaml Cap’n Proto RPC implementation is now released and available via opam:
```
opam depext -i capnp-rpc-unix
```
All features of level 1 of [the spec](https://github.com/capnproto/capnproto/blob/master/c++/src/capnp/rpc.capnp) are implemented. There is a tutorial for the OCaml API here:
[https://github.com/mirage/capnp-rpc](https://github.com/mirage/capnp-rpc)
It shows how to create services and clients, connect them over a network, pass services themselves in arguments and return values, and use promises and pipelining for performance.
Please let me know if anything is unclear!
### Background
[Cap’n Proto RPC](https://capnproto.org/) is built on top of the Cap’n Proto serialisation format, which is similar to protobuf (an earlier system by the same author). You define your protocol in a schema file and can then compile it to generate typed bindings for various languages. The system makes it easy to upgrade the schema files in a backwards compatible way.
The RPC system provides a number of useful features, but the keys ones are:
- As well as pure data, you can **pass references to services** (functions, objects, callbacks) in your messages too. Sending a reference to a service in a message automatically grants the recipient permission to use it (the [Object Capability security model](https://en.wikipedia.org/wiki/Object-capability_model)).
- Messages can be **pipelined**. For example, you can ask for a service to be created, and then immediately invoke a method on it or pass it to another service, without waiting for the reply. This allows you to make clean APIs with simple functions that can be composed, rather than trying to create complex APIs to compensate for network latency.
- Messages are delivered in [E-order](http://erights.org/elib/concurrency/partial-order.html), meaning that if you send multiple messages to a capability then they will arrive in the order in which they were sent. This works even if the target was a promise that later turns out to be located elsewhere - pipelined messages will be forwarded and processed before any later messages sent directly. This avoids many race conditions, without greatly harming performance.
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 11, 2017, 7:14pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/2 "2017-08-11T19:14:09Z")
I should also mention that there is a new [v3.0.0 release of the OCaml schema compiler](https://github.com/capnproto/capnp-ocaml/releases/tag/v3.0.0) to go along with this.
Most of the changes are to support the RPC system, but there are some other nice improvements to the API too. In particular, it is now possible to write generic functions that can work on any type of builder or reader.
---
### Author:  [@avsm](https://discuss.ocaml.org/u/avsm)
#### Post date: [August 14, 2017, 10:35am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/3 "2017-08-14T10:35:50Z")
I’m just playing with the schema compiler and finding your README a very useful tutorial indeed!
One area I’m finding confusing is specifying type signatures for my own services. The generated `mli` file has something like
```auto
module Make(M : Capnp.MessageSig.S) : module type of MakeRPC(Capnp.RPC.None(M))
```
for the application of the protocol functor. Creating my own mli file involves exposing this module type again, and also pasting in the polymorphic variant that represents the unique id. Is there a best practises example somewhere (e.g. in `test-bin`) that also shows how to generate more succinct `mli` signatures for the associated protocol implementations?
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 14, 2017, 3:55pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/4 "2017-08-14T15:55:58Z")
You shouldn’t need to expose this again in most cases. You’ll need to include the unique type, but that’s just `[`Foo_123] Capnp_rpc_lwt.Capability.t` - no need to depend on the generated schema.
I should probably add some `.mli` files to the `examples` directory…
---
### Author:  [@avsm](https://discuss.ocaml.org/u/avsm)
#### Post date: [August 14, 2017, 4:19pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/5 "2017-08-14T16:19:26Z")
Ah yes, that’s indeed the right type. The inferred types were going through all the functor hoops, which confused me – `-short-paths` should do the trick hopefully. Thanks!
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 15, 2017, 10:29am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/6 "2017-08-15T10:29:32Z")
I’ve added some mli files in [#84](https://github.com/mirage/capnp-rpc/pull/84).
I also changed the recommended layout, removing the `Client` module and renaming `service` to `local`. This seems to work better - see [the new calculator API](https://github.com/mirage/capnp-rpc/blob/master/examples/calc.mli) for an example.
---
### Author:  [@avsm](https://discuss.ocaml.org/u/avsm)
#### Post date: [August 21, 2017, 10:15am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/7 "2017-08-21T10:15:54Z")
Thanks, the interface files are much clearer now. I had another broader architectural question about how best to use capability passing in Capnp, since none of the examples I could find online seem to address this.
I would like the interface to the OCaml build infrastructure to be as narrow as possible, with worker machines connecting and then receiving back capabilities to log their activities.
Consider this interface:
```auto
interface Log {
init @0 (label :Text) -> (id: Int64);
send @1 (id :Int64, msg: Text) -> ();
close @2 (id :Int64);
}
interface Build {
struct ProcessOutput {
stdout @0 :Data;
stderr @1 :Data;
exitCode @2 :Int32;
}
shell @0 (cmd :Text, log :Log) -> ProcessOutput;
}
interface Register {
worker @1 (hostname :Text, arch :Text, ncpus: UInt32, exec :Build) -> ();
}
```
The idea here is that each worker machine contacts the `Register` service, and passes in a `Build` callback that the registry can use to start off build jobs. When `Build.shell` is sent back to the worker, the registry includes a `Log` callback that can be used to send streaming updates about the build progress.
My question is: can the `Log` callback be specialised to the individual job that has been invoked, such that the particular callback can only be used for that build? Right now, the builder has to initialise a log and could in theory use the callback for other builds as well. Ideally, I would be able to synthesise a `Log` service for every single build and have each capability be unique.
I’m not sure where Capnp stops with its capability passing – it looks to me like the service model is more static than this use-case needs, and there should be a generic `Log` service that is shared by all the build jobs.
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 21, 2017, 10:39am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/8 "2017-08-21T10:39:00Z")
Yes, you should indeed export a fresh `Log` service for each job. There’s no problem with that (just remember to `Capability.dec_ref logger` when you’re done with it).
The registry service would do something like:
```ocaml
let log = Log.local new_stream in
Build.shell builder cmd log;
Capability.dec_ref log; (* If you don't need it; the builder has a reference now)
```
---
### Author:  [@avsm](https://discuss.ocaml.org/u/avsm)
#### Post date: [August 21, 2017, 10:42am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/9 "2017-08-21T10:42:52Z")
Ah excellent – that should be easy to do on the OCaml side and keep track of the log handling. How do you debug the capability reference counting? It makes me a little nervous to be doing manual reference counting in OCaml…
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 21, 2017, 10:47am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/10 "2017-08-21T10:47:24Z")
If something leaks it _should_ display a warning at the next GC. If you `dec_ref` too soon, it will complain next time you try to use it.
If you enable logging, it will display the current ref-counts (also with `Capability.pp`). You can also use `CapTP.dump` to dump out the state of the connection, which will show you what you’re currently importing and exporting over the connection. Turning on colour in the logging will help (see the `test-bin/calc.ml` example for how to turn on logging).
If you override your service’s `pp` method, you can include extra information in the output too.
But the simple rule is: any time you extract a capability from a message, you must eventually call `Capability.dec_ref` on it.
---
### Author:  [@Alex](https://discuss.ocaml.org/u/Alex)
#### Post date: [August 21, 2017, 9:06pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/11 "2017-08-21T21:06:23Z")
Thanks for sharing! Do you have a plans to implement levels 2-4 of the spec?
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 22, 2017, 8:57am UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/12 "2017-08-22T08:57:20Z")
I’m currently adding TLS support. That will allow encrypted communication, but will also give each vat a unique identity (public key). With vat identities, we should be able to support SturdyRefs (level 2).
---
### Author:  [@talex5](https://discuss.ocaml.org/u/talex5)
#### Post date: [August 23, 2017, 2:55pm UTC](https://discuss.ocaml.org/t/ann-capnp-rpc-0-1/698/13 "2017-08-23T14:55:16Z")
@Alex I have encryption and authentication working now. See [https://github.com/mirage/capnp-rpc#networking](https://github.com/mirage/capnp-rpc#networking) for the current state.
It’s not clear exactly what the persistence API should look like. I’ve asked on the capnp group at [https://groups.google.com/forum/#!topic/capnproto/d6uPbXf9e4E](https://groups.google.com/forum/#!topic/capnproto/d6uPbXf9e4E) for suggestions. If there isn’t a standard yet, I’ll define a basic SturdyRef structure and a restorer interface that can be used as your bootstrap service.