[ANN] valkey: modern Valkey client for OCaml 5 + Eio, on opam

Community
, , ,
1

Hi everyone,

I wanted to share that valkey 0.2.0 is now on opam:

The main thing I wanted with this project was to focus on the current Valkey stack with the newer OCaml stack.

So valkey is built around:

  • OCaml 5
  • Eio-native direct-style concurrency
  • RESP3 only
  • Modern Valkey features and real cluster behavior

A few parts I’m happy with:

  • Cluster support with a strong focus on durability and high availability:
    topology refresh, periodic background refresh, MOVED / ASK / CLUSTERDOWN handling, replica-aware reads, AZ-aware routing, and failover-aware sharded pub/sub replay

  • connection/runtime behavior:
    circuit breaker, reconnect handling, keepalive, TLS, and optional separation between socket I/O and parsing via Eio.Domain_manager, so one side of the system does not block the other

  • batch support:
    scatter-gather across slots, atomic single-slot flows with WATCH / MULTI / EXEC, multi-slot helpers, and in
    0.2.0 also WATCH guards for read-modify-write CAS plus cross-slot pfcount_cluster

  • command surface:
    typed helpers across a pretty broad set of commands, Client.custom /custom_multi for custom commands and multi-node execution, and named commands so you can register command templates once and reuse them later

  • scripting:
    local script caching and optimistic EVALSHA handling with automatic fallback/retry on NOSCRIPT, so callers don’t need to manage that flow themselves

There is also already a pretty good amount of validation around it:
integration tests, property tests, fuzzing, chaos testing, examples, and guides.

Performance-wise, it also came out pretty nicely: in several scenarios it gets to 90%+ of the C reference client.

opam update
opam install valkey eio_main

Current next steps on the roadmap are things like:

  • client-side caching
  • connection pools / blocking pools
  • IAM + mTLS
  • Valkey module support (JSON, search, bloom)

If there’s a feature people care about, I’d be very happy to reprioritize the roadmap around real interest.

I’d also really love feedback from the OCaml community on the API, ergonomics, docs, and general design.

I’m very familiar with the Valkey world, but OCaml is more of a side fun for me than my daily language, so feedback from people who really live in the OCaml ecosystem is especially valuable to me.

If you try it, I’d love to hear what you think.

6 Likes
2

Could you talk a bit more about Valkey? I have not heard about it and it appears to be a distributed key-value store. What is special about it relative to other key/value stores and what are domains where it is used?

1 Like
3

valkey forked from redis a few years ago, when redis went
source-available.

So it’s an in-memory key/value store with rich data structures (sets,
hashmaps, hyperloglog, streams, etc.)

1 Like
4

Yes, pretty much what was said above.

To be fair, the fork happened when they closed source; Redis became source available after a year of backlash.
After they announced they were closing the source, one of the 5 “board team” members who wasn’t from Redis and was forced out decided to fork and call it valkey. And the non-Redis employee contributors came after her.

Anyway, up until v7.2, it’s Redis 1 to 1; from then on, the ways split, and each delivers its own features.
Valkey from one side, is community-oriented, belongs to the Linux Foundation, and is cautious with the balance of the governance but is also backed by all the major companies in this field, like Google, AWS, Percona, and more.

It got massive improvements in the last two years and incredible growth. The performance and the new features are amazing and important.

And it has a very nice and welcoming community. :slight_smile:

5

More directly to your question, it is a feature-rich in-memory key-value store. Redis was the standard before, so Valkey is to some degree the same.

It’s used for almost everything, and while its primary usage is for cache, that’s the most well-known way to use it. But it’s used for search engines, streaming, gaming, LLM, recommendation systems, task queues, worker systems, distributed locks, money transactions, and many more. It’s straightforward to use, it promises atomicity, it’s very fast, and it has many types of data types, so users plumb it everywhere.
It has sets, strings, streams, hashes, and ZSets. If I’m not mistaken, those are the primitives, and on top of it, there are many others like geos, vectors, JSONs, and so on.
Functionalities like PubSub, clustering, TTL, search, replication, scanning, and so on.

It’s challenging to answer in what it’s different.

Just throwing the first names I have in my head - Memcached, redis and Dragonfly are the known names in the field. Memcached is very fast and basic; it deserves more respect than it has, but it doesn’t have many features.
DragonFly is Redis rewritten in CPP (it was before Rust :P) with some opinionated ways to do distribution.
Redis is the parent of Valkey. I’d say that currently Redis has a better module system in the variety (this was always closed source; companies used their modules internally or as source available), and that Valkey has better performance and a lower memory footprint. Since Google and AWS are now contributing all their developments to Valkey.io, the modules are also growing very fast, and the development in general. A year ago, Valkey announced 1M TPS; thanks to AWS, this year it’s 3M TPS, also thanks to AWS.

Do you notice that I’m personally involved with it? :sweat_smile:

4 Likes