KV database (nosql) recommended for Mirage?

Ecosystem
1

Which KV database would you recommend for applications intended to build as Mirage Unikernels?

1 Like
2

Currently, when we need to make a KV-store with MirageOS, we use irmin as an high-level KV-store. Then, it provides some back-end like git. A tutorial can be found here. So, we currently prepare some releases of several packages around MirageOS and irmin, so you currently need to pin some packages.

Example of how to use Irmin with Git backend and MirageOS can be found here:

1 Like
3

What about mirage-kv in opam?

I don’t believe I need the overhead of the git store; can Irmin work without accumulating a history and any overhead of that?

4

So mirage-kv, as some others packages in Mirage, is just an interface. Then, you have some possibilities:

  • make your own mirage-kv implementation (which can be a simple hashtbl)
  • use something like irmin which respects this interface

Then, it’s mostly about what you want exactly. MirageOS does not have a proper file-system at the beginning and we use in some ways git to obtain persistence (where, when your unikernel will shutdown, you will lost contents). So, the idea is to have a KV-store inside the unikernel and a way to synchronize this KV-store with a remote repository - and, of course, we took the advantage of git, behind this idea (however, as I said, you can use/implement something else).

Then, with functoria you can easily orchestrate the compilation of your unikernel to choose which will implement the mirage-kv signature.

1 Like
5

Sorry I meant to ask what about mirage-kv-unix. Is that with a git backend? Doesn’t it have any history related overhead or is it a plain mutable RW backend?

6

there’s mirage-kv-mem which is not persistent, using a Map.t for storing data.

there’s as well mirage-kv-unix which is only available on unix (i.e. not if you’re running a freestanding unikernel), and uses the file system as persistent backing store.

both of them don’t contain a history and are fine to use in certain programs! :slight_smile:

(so, yes, mirage-kv-unix is persistent and without git)

2 Likes
7

Thanks hannes. So I’ll use mirage-kv-unix for now, till I get to building a unikernel. From there on I will have to pick irmin or some other back-end for permanent storage?

8

yes, at some point there should be a mirage-kv implementation which dumps on a block device (using a fat filesystem or wodan). :slight_smile:

9

There’s also:

The B-tree library is reasonably stable. The KV lib is still under development. Should not be too hard to link up to Mirage. If there is any interest, I can look into this.

7 Likes
10

I’m still finding my feet with mirage, so someone more familiar with the ecosystem should advise on this.

11

I wonder what the state of this is now. I see that GitHub - mirage/wodan: A Mirage filesystem library was worked on last about a year ago. It’s confusing whether it’s OK to use as a KV backend or if (as its docs suggest) it should be used in tandem with Irmin. If the latter, would that come with too many unnecessary features?