We’re happy to announce the release of ppxlib.0.22.0, the fist release of ppxlib fully compatible
with OCaml 4.12.
The main and only feature of this release is the bump of the internal OCaml AST used by ppxlib from
4.11 to 4.12, allowing you to use 4.12 language features with ppxlib and any ppxlib-based ppx.
Note that ppxlib was compatible with the 4.12 compiler since 0.19.0 but that you couldn’t use 4.12
language features until now.
This is the third such AST bump release since we announced our plan to improve the state of the PPX
ecosystem here and we though it’d be a good time to report back to you and tell you
how things are going on this front.
For those of you who aren’t familiar with this plan, the goal is to upstream a minimal, stable,
ocaml-migrate-parsetree-like API on top of the compiler-libs called
Astlib. It will allow us
to keep ppxlib and any ppx based on ppxlib compatible with OCaml trunk at all time.
To allow better performances and a clear compisition semantic, all the ppxlib-based ppx-es need to use
the same AST (as opposed to ocaml-migrate-parsetree based ppx-es) so from a certain perspective, this
plan simply moves the breaking API up one step, from compiler-libs to ppxlib.
In order to greatly ease the maintainenance of ppx-es and to prevent opam-universe splits we decided
that everytime we cut a breaking ppxlib release, we will send patches to keep the existing ppx-es compatible with the latest version and therefore with the latest OCaml compilers and language features.
While this seems like a tremendous task and a huge amount of work, dune and other tools that raised
in its wake such as opam-monorepo incredibly simplified this kind of work.
Ahead of OCaml releases, we prepare a branch of ppxlib with the upgraded AST. We then fetch
opam-repository to gather a list of sensible reverse dependencies (i.e. packages whose latest version
depends on ppxlib and is compatible with ppxlib’s latest version) and assemble a dune workspace with
a clone of each of those reverse dependencies, our ppxlib branch and all of their dependencies thanks to
We then use dune to build all the packages we’re interested in and simply follow the compilation errors
until everything builds successfully with the new ppxlib.
What remains is to create PRs on the relevant repositories to upstream those changes, after which
maintainers have everything they need to cut a new compatible release.
Most of this process is automated using scripts but it still requires a bit of handiwork. We aim at
extracting tools to further improve this workflow and reduce the time and effort required but it has
been surprisingly smooth. Our experience with the 4.10, 4.11 and 4.12 upgrades so far is that
most reverse dependencies don’t need an upgrade and that it’s far less demanding for one person
to upgrade all the packages that need it than it would be for each individual maintainers to
understand the changes in the AST and do the upgrade themselves.
It’s worth noting that for this to work well, the ppx-es and all their dependencies have to build
with dune. We do maitain a separate opam-repository with dune ports of commonly used packages so in
practice most projects fall into this category but a few exceptions remain and they are therefore
not taken into account for this upgrade process.
We’re also trying to improve the tracking of the upgrade’s progress and for the 4.12 compatible
release we created a github project to have a list of all the packages we
considered and see where they are. We also keep track of the packages we had to exclude and why.
During this upgrade, we considered 80 opam packages, out of which only 4 needed to be patched and 6
had to be excluded from the process as we couldn’t reasonably get them to build in our workspace.
Once we have a better idea of what makes a package easy to upgrade we plan on releasing a set of
reasonable rules to follow to benefit from those upgrades, we’ll keep you updated on this!
All in all we’re pretty happy with this new process and although it needs to be refined, we’re
confident it can grow into something sustainable by creating tools and CI to support it. Hopefully
these will also benefit the wider community and help grow a healthier Opam universe.