SandDB: A simple immutable database for the masses

Community
1

Hello everyone! I’m proud to announce my first opam package: SandDB

SandDB is a simple immutable database that I made, because I couldn’t really find a dead simple database (like TinyDB) for Ocaml and also because it’s fun to reinvent the wheel. :smiley:

SandDB is exciting, because it’s:

  • Simple: The code base is small and the db is easy to get started with. The basic building block of the database is also pretty simple, which is the record. A record consist only two element: id element(which is basically a randomly generated uuid) and data element, whose type you can define with atd.
  • Flexible: You can define your own record data type with atd, which also means that you can create both a json and biniou database thanks to atdgen.
  • Immutable: SandDB is built on the basic idea that it should be like an immutable stack, where you can only push elements. It’s implemented just by using file appended write, so SandDB is closer to a logger than a classical database.
  • Crud capable: Even though it’s an immutable database you still can update and delete records thanks to the shadowing insert, which works by using the old record’s id for the new updated or deleted record. The basic idea here was the let binding in Ocaml, which can also shadow another let binding if it has the same name.
  • Asynchronous: File operations are dead slow compared to others, so it was important that the db was async.
  • Error tolerant: Every commonly dangerous operation like parsing is covered by Result, so you can expect where things can go south.

I have to thank the developers and maintainers of atdgen,lwt,uuidm and base, because without them I couldn’t really have done this.

Next steps:

  • Using standard module naming(My_module) instead of my(My_Module).
  • Maybe adding lwt stream to the database so the range of the read could be controlled.
  • Start working on the next small project
12 Likes
2

Awesome! :smiley:

It could be used to implement abstraction layers such as an Event Store :open_book: or even a Time-Series Database :clock10: . Do you have any guarantees against the rewrite of entire history, e.g, a chain of hashes?

Nevertheless, keep that good work :+1: :clap: :ok_hand: ! I have been looking around for some sort of database, but all that I’ve found was the Irmin with its complex rules upon functor applications.

I would also suggest the replacement of UUIDs by CUIDs or ULIDs, 'cause they’re monotonically increasing (so we can optimize the indexing of keys by ordering them properly), and they also have less lack of entropy to generate collisions (as is the case of UUID v4).

3

Thanks for the compliment.

Well, I’m thinking about making a bigger database based on SandDB, which would be basically a manager of many SandDB instances, but right now I’m working on something else.

There is no guarantee against history rewrite, because id’s are random and the user of the library can easily modify the database file manually. Although you can put the database file under a version control system and you would have this guarantee.

I chose uuid, because it’s an industry standard, but you raise a good point about what if the user of the db want’s the id of the record to be increasing or something unique. Maybe the record id also could be customizable like the record data, but right now I think uuid is a good default choice.

Also I’m not entirely sure about cuid(which has ocaml implementation thanks to you), because in cuid the id contains a client fingerprint part, which just makes me uncomfortable, because I think a database id shouldn’t have this kind of information in it.