[ANN] Bancos: A persistent Key-Value store based on ROWEX/P-ART

Community
1

I am pleased to announce the first release of bancos, our new persistent Key-Value store.

Bancos is an implementation of an Adaptive Radix Tree (ART) modified to support parallel reads and writes (ROWEX) directly on a file (based on the RECIPE/P-ART approach).

  • Our project is designed for indexing with file-backed storage.
  • We provide an implementation compatible with unikernels. It is currently being used by kevin, an HTTP/1.1 server unikernel that exposes routes for key insertion and lookup.
  • This work is part of our broader goal to index emails by their Message-ID. An extension of our tool blaze now allows searching through archives now (if you are interesting about our archive system, you can take a look here). This indexing also enables efficient email thread extraction using In-Reply-To headers.

While still experimental, the results — especially when paired with our Miou scheduler — are very promising. This release of bancos comes alongside several updates across our stack:

  • flux: Our OCaml stream library for Miou
  • carton: For manipulating PACKv2 archives (as used by Git)
  • bstr: A library for handling bigbytes and memory buffers
  • cachet: A library to cache mmap syscall results. Notably, this include significant work on an asynchronous write pipeline for unikernels on block devices

You can find the project here: GitHub - robur-coop/bancos: A simple KV-store

For a deeper dive into the implementation of bancos and its application to email indexing, we have prepared a detailed blog post here.

Happy hacking!

8 Likes