gRPC implementation in OCaml


#1

Anyone ever dabbled with gRPC with OCaml? E.g. building clients, servers, serializers, etc. I’ve come across piqi-ocaml and ocaml-protoc for serializations, but none for server/client implementations. I’m interested in exploring this, but would rather avoid duplication if there’s an effort or assessment already underway.


Advantages of OCaml over Rust
#2

I have some libgrpc bindings in progress that will (if finished) work together with ocaml-pb and ocaml-pb-plugin. They’re not online yet.


#3

The lack of gRPC support was one of the reasons why OCaml was not suitable for us at work. I opened an issue on the official gRPC issue tracker when I was researching OCaml.


#4

I am the author of piqi-ocaml. There are no plans for suppoting grpc. I do plan however to add support for Protobuf3 (currently piqi-ocaml supports only Protobuf2) which grpc relies on, and support for some kind of stubs generation for services. This way it would be usable for grpc serialization.

Those who want full grpc support would need to implement grpc protocol in OCaml, but this is strictly outside of the current piqi-ocaml project scope.


#5

That’s good to hear. I was also thinking perhaps binding to the core C libraries can prove to be more efficient while waiting for a pure OCaml implementation of the protocol (which may or may not happen). Sadly I’m currently not really familiar with C, but perhaps it’s all the more reason for me to also dig into it. Would you perhaps be able to share how far the progress have been?

Yeah I also came across this issue when looking around, great to be pointed to the interop tests.

Hi! Thank you for the nice lib.

I agree. I also think such grpc support should live on its own.


#6

Few months ago I had the crazy desire to do an ocaml native implementation (that way would be hopefuly possible to run on mirageos) of grpc, but the problems I’ve found were that there is still no http2 support for cohttp, so I tried to implement that, but then there was no support of ALPN for the ocaml-tls, and then I realised that I’m not very familiar with tls’ internals in order for me to submit a patch, and sadly I stopped.
I’m glad that this matter is being raised again.


#7

I tried going that path also but stopped when I realized that cohttp does not support HTTP/2. It’s concerning that the issue has been open for almost three years. Going forward, it’s crucial to have a reliable gRPC implementation since it’s becoming the defacto standard for microservices communication and cloud-native apps.


#8

Is this the “only” thing blocking cohttp from supporting http/2? It’s good to have some actionables so we can understand what’s currently missing.


#9

No. HTTP/2 is a major change to the HTTP protocol. I would not underestimate the difficulty of implementing reliably multiplexed streams.


#10

Well, it’ll be open until someone has the time to do the very significant chunk of work required :slight_smile:

The first step is probably to encode the http2 binary protocol using Angstrom and Faraday serialisers, and then work upwards from there towards the HTTP interfaces.


#11

I’m currently trying to implement ALPN support for tls:

(If you’re familiar with TLS, please take a look!)

I reckon something would need to be done with ssl as well but dealing with C is not exactly my forte. If protocol negotiation is done I think the next step is to make the clients (e.g. conduit-lwt-unix for cohttp) utilize that information.

I imagine it could go in parallel with the encoding of binary protocol as @avsm said, if anyone is interested to lend a helping hand :smile:


#12

Yeah, and the response from the Googler committer who fielded the discussion in that issue was pretty blunt:

@nathanielmanistaatgoogle
@shinzui: the magic of open source (and an open and unencumbered protocol) means that we can’t stop you from implementing gRPC for OCaml. [Maybe if you had a finished implementation to give us that passes all the interop tests…]

And when you pressed on that as if it were a real possibility…

[…] Let’s say that when an implementation passes the interop tests that’s when we start the conversation.

I won’t bother to quote the snarky dig at OCaml for not being SML/NJ.

I can’t say I’m terribly surprised by this response. Internal to Google, OCaml is regarded as something of a joke language, having been associated with a legendary troll who made a mess of himself evangelizing it (along with Scala and Haskell) internally for some months before flaming out in a rather inglorious flounce. It’s not possible to talk about it in a serious tone of voice there.

If we want a gRPC implementation in the OCaml community, we’re going to have to do it while having a lot of people at Google throwing bricks at us for trying it.


#13

If we want a gRPC implementation in the OCaml community, we’re going to have to do it while having a lot of people at Google throwing bricks at us for trying it.

I think that’s a bit pessimistic. When I submitted some code to the grpc project to make ocaml-ctypes bindings easier the response was encouraging:

Thanks again for your contribution and please keep us posted about your OCaml bindings - it would be awesome to see those posted on grpc-io mailing list.


#14

It may be overly pessimistic. I’d be thrilled to learn about any grounds for optimism that OCaml could have a future inside Google. From my perspective, there is none.


#15

The OCaml community really needs to tackle gRPC at some point.

Not having gRPC and full GraphQL support is going to be a significant roadblock for adopting OCaml in the future. An increasing number of technologies use one of the two for their APIs which means that those technologies would be inaccessible to OCaml developers.


#16

Google hasn’t shown itself to be much of a PL place so far. Aside from Go, which isn’t a shining beacon of language design, they tend to stick with the ‘tried and true’ languages from what I can tell. Unlike facebook, for example, or even Microsoft.


#17

It’s not a substitute for having first class OCaml support, (lack of a http2 library is a big blocker) but until that arrives you can use grpc-gateway with ocaml-swagger or atdgen.

Disclaimer: I don’t know if ocaml-swagger works with the grpc-gateway’s Swagger output.


#18

Predictable side effect of the monolithic repository tactic.


#19

But FB also uses the monorepo tactic, and they have a bunch of fancy PL stuff. And Jane Street does too (though I guess we but one fancy PL). Honestly, I don’t see the connection.

y


#20

It may be overly pessimistic. I’d be thrilled to learn about any grounds for optimism that OCaml could have a future inside Google. From my perspective, there is none.

Whether OCaml has a “future inside Google” is a separate (and not especially relevant) question.

You said that people at Google will “throw bricks at us” for building an OCaml gRPC. The response from the gRPC team to my PR shows that that’s not the case — in fact, the team actively encourages building OCaml bindings to gRPC, and has already merged some code specifically to support OCaml bindings.