I did think this is a somewhat harsh take on the dune maintainers, especially given your own recent post on core compiler bottlenecks – the same bottleneck applies here.
It might be useful to contextualise where we are with a historical perspective: it was just five years ago that we were struggling with absolutely abysmal build times and usability with the layered “combo attack” of ocamlbuild, camlp4, and oasis. This really was the worst of all worlds: you could run arbitrary code in the build system, and had to deal with the slowness of dynamic dataflow graphs (preventing good scheduling), and not particularly optimal build rules. I was recently looking over the epic ocamlbuild rules that @samoht and I wrote for MirageOS 2.0 back in 2011, and was sad for the wasted months of our lives we’ll never get back from that wrestling match.
Since then, dune (nee jbuilder) came onto the scene with one main innovation that changed everything: allow builds to be described as static files for the vast majority of builds (no arbitrary OCaml code), and make compatibility an important goal: since the static files are versioned, future versions of dune are able to build older build descriptions without breakage. Every single rule in dune is versioned, and with decent error messages if you use a new feature with an older dune. This is so much more user friendly than what came before, and to me a sign of maintainers that really do care about the build experience.
The last couple of years have seen a very rapid migration to dune (>91% in the user survey), and a proportionate burden of bug tails. Is it more important for large projects to successfully migrate to dune, or to facilitate usability improvements? There’s no good answer here, but I personally think it’s more important to ensure that projects that have already committed to dune are not blocked, and the team has done an excellent job there. It would, of course, also be useful to fix usability issues, and perhaps a sprint or hackathon of some sort would help kickstart such movement.
My own wishlist?
- Minor: fix dune#4121 so that I can finally vendor C libraries in my monorepos (and work with @yallop to merge the ctypes/dune port properly). This unblocks the ability to build large projects (like moby/vpnkit with millions of daily users in Docker on macOS and Windows) easily.
- Major: to have a standalone library to interpret
dunefiles and allow for the generation of rules, in the style of opam-0install-solver foropamfiles. This would help with integration into Bazel or other third-party build systems.