[ANN] Caqti 1.9.0 and Plans for 2.0.0

First I would like to announce the 1.9.0 minor release, see the release notes below for details. Caqti is an interface to multiple database client libraries, covering MariaDB, PostgreSQL, and Sqlite3.

There is also ongoing work in the caqti2 branch targeted for the next major release. If someone have an opinion on directions, we can discuss it here, or in the issue tracker (meta-issue), see my brief notes below.

I will attend parts of the ICFP 2022 virtually next week so there may be time to discuss over audio.

Thanks to the OSCF for sponsoring my time on this release.

Release Notes for 1.9.0

New features:

  • Allow unquoted semicolons in query strings in the new API. There are
    corner cases where it is needed, as reported in issue #87, and a parser
    which rejects semicolons are still available for loading schema files
    statement by statement.

  • Add support for MySQL and MariaDB configuration files, as a solution to
    issue #86.

  • Add a limit to the number of times a database connection is reused when
    pooling connections (#94). Thanks to Peter Mondlock for investigating
    resource usage server side motivating this addition.

  • Provide access to the raw SQLite3 connection handle for the purpose of
    defining custom functions (#56).

Fixes:

  • Add missing dune dependency on unix (GPR#85 by David Allsopp).

  • Documentation fixes (GPR#82, GPR#83, GPR#84 by Reynir Björnsson,
    GPR#88 by Jonathan Duarte, and GPR#92 by Jim Tittsler).

Deprecations:

  • Caqti_type.field was deprecated in favour of Caqti_type.Field.t.

Other:

  • Replace deprecated core_kernel dependency with core.

Notes on 2.0.0 Development

The main addition is pgx and mirage support. It is already functional, but not very useful for production, since it lacks TLS. The trick here is that PostgreSQL uses STARTTLS, so we can’t use conduit-lwt as-is.

Another thing in progress, but unpublished, is per-connection configuration. Up till now, configuration has only been possible through the connection URL or behind-the-scene via C libraries (now also for MariaDB). However, this will no longer be practical for delivering CA certificates to pgx. Two design issues which you may have an opinion about:

  • Driver specific options can be defined in the caqti package or in caqti-driver-* packages. In the former case, the configuration can be manipulated without depending on specific drivers, but the downside is that we will pull in dependencies on x509, domain-name, ipaddr and possibly tls and sexplib0.
  • My current sketch provides sexp-serialisation, a choice mainly motivated by the availability of such serialization for client configuration of tls, but I hope to find a more generic solution which allows easy embedding of Caqti configuration in application configuration independent of which format is used.

An example of how an sexp-formatted configuration might look like:

(connection 
 (pool
  (max-use-count 20)
  (max-idle-size 10)) 
 (driver postgresql)
 (endpoints
  (inet pg1.example.org)
  (inet pg2.example.org))
 (target-session-attrs read-write))

where the (pool ...) clause is driver-indepnedent and the (driver ...) clause determines which DB-specific options are valid. In the current draft, order does not matter despite this dependency.

(I could also mention plans of wrapping modules, but this will be done first as a forward-compatible module in parallel to the current modules preferably at the beginning of a major release cycle. The reason I haven’t written that main Caqti module yet, is that I would like to take the opportunity to tidy up the namespace to make it easier for newcomers to discover the main entry points.)

14 Likes

What is caqti? Where do I find it?

Caqti is a library that connect to relational databases: caqti-async 1.9.0 · OCaml Package

Good point, I’ve added a description and link to the first paragraph.

Sorry for the noise, I completely misunderstood.