Not all key-value stores are the same, take a more in-depth look at FoundationDB to see why so many people are excited about adding it to their infrastructure.
I just looked at Arakoon, it’s unmaintained, and even if it was still under development, the underlying store (tokyocabinet) is unmaintained because I used it at a previous job a very long time ago and we had to migrate away from it.
HTTP/2 is a big missing piece to OCaml to enable certain libraries like gRPC.
I’ve been thinking also that there should be a GCP library, I’d be happy to start working on one, despite the fact that it’d be my first OCaml program (outside of school)
HTTP/2 support would be definitely nice to have. I’m not very familiar with the new protocol internals, and i’ve only been using OCaml for a relatively short amount of time, but i’d be happy to contribute towards an effort to work towards this.
That being said, in my opinion what will help is an effort to split a single task of “Implement HTTP/2” into smaller more managable items. So far I can think about the following points as a start:
- Define the types (error types, settings, frame etc)
- Serializing/Deserialized the binary codec (Angstrom/Faraday might help here? )
HTTP2 types in OCaml
HTTP/2 support is really a painfully missing piece indeed, and grpc too. And it’s a big chunk of work.
That being said, if you just need to send http/2 requests you can just use curl.
Server side is more problematic though. I’ve been willing to try binding another language server with lwt, but that seems hard to do.
If it is production ready, ships with Debian and was used heavily in production in several places, it means tokyocabinet’s code base is stable and mature.
I do not care so much if it is unmaintained: this is open source in C.
Many people can have a look at it and change something if needed.
We migrated away because we had problems with tokyocabinet at scale and we lost data. That was a long time ago, but I don’t think the issues were fixed since the author started working on kyotocabinet as a replacement when we abandoned it.
Again, not all key value are the same. TokyoCabinet isan old read-optimized btree based keyvalue engine. I don’t know about FoundationDB internals, but you might want write optimized LSM based db for your use-case.
More importantly it does not look like a good idea to rely on dead unmaintained technology with no commit in the past few years to store your data while you can just use techs everyone else is using and debugging like rocksdb based distributed kv store or foundationdb
I agree. One step towards this is that we would need to track all these points. Perhaps a wiki post on this Discourse or on OCamlverse (cc: @bluddy)?
For instance, I’m currently working on implementing ALPN support which is required for negotiating http2 protocol over TLS on both ocaml-ssl and ocaml-tls. After (and if) this is merged, the next step is changing ocaml-conduit to take advantage of this, which in turn will enable cohttp servers and clients to negotiate “h2” protocol via TLS. I’m not sure about httpaf as it doesn’t seem to use these libraries.
But as has been said, this is unfortunately still a tiny piece of what enables HTTP/2, hopefully other efforts can be done in parallel.
(Btw this thread is getting a bit off-topic from Rust and deeper into missing OCaml libraries, I believe moderators can fork posts to a new topic, e.g. HTTP/2 support–or merge with this one?)
We definitely should change threads. Feel free to start one on HTTP/2 support. Also, feel free to add to the missing pieces on ocamlverse detailing progress and links with regard to getting http2 going. Once a section on this page gets too limiting, we can branch it off to another page.
Let me just say that I think it’s hugely in the community’s interest to switch to httpaf rather than keep using cohttp. cohttp is just not optimized enough at a low level to serve as the foundation for OCaml’s web services, while httpaf is designed from the ground up for performance.
“Arakoon is unmaintained”. Are you sure you’re looking at the correct repo?
There were some nasty bugs in tokyo-cabinet that were fixed, and these fixes have been incorporated in Arakoon.
Fwiw, the powers that be forced the creation of a private arakoon-ee repo, that replaced tc with rocksdb (complete with upgrade from tc to rocksdb). So it’s not unmaintained, it’s just that most of the current effort goes into the ee version.
Interesting discussion related to this thread: https://www.reddit.com/r/rust/comments/abm6hy/why_rust_is_successful_compared_with/