Following this thread, it became apparent that we need more maintainers of the opam package repository. The call for maintainers was very successful, and I am pleased to have expanded the team to include the following members: @perry, @djs55, @rgrinberg, @yomimono, @jpdeplaix, @hannes, @mseri and @hcarty.
So many contributors need a little more coordination, and luckily the OCaml 4.06.0 has set our opam world on fire, just in time for the newcomers to assist There has been a steady increase in the number of build failures, but the OCaml 4.06.0 release flipped the “safe-string” default into being active, resulting in 1000s of packages failing to build in the new release.
In order to keep up, I’ve built a prototype tool called “obi” (OCaml Build Infrastructure) which does tens of thousands of package builds on a build cluster hosted at OCaml Labs (for x86_64) and Packet.net (for arm64) and IBM (for ppc64le). The log outputs are available at: http://obi.ocamllabs.io/
Some things I want to draw your attention to:
- There are logs per compiler version that are rebuilt roughly daily across a suite of compiler releases. In particular, we build using 4.06.0 and the common variants (default-unsafe-string and flambda) to check for regressions there.
- There is a triage page which runs an analysis over the results to classify failures. The are two areas right now which need attention: fixing regressions due to the 4.06.0 safe-string, and failures with flambda.
- The safe-string triage has thousands of packages failing, which either require constraints being added (for older revisions), and ideally a release of the package that does use
safe-string. This is where we could use help from the whole community to bring our repository back up to speed.
As to where this is all going, it will be worked on at OCaml Labs over the next 12 months to improve the tooling considerably as OCaml continues to grow in popularity:
- Obi is still an experimental deployment, so all the URLs above are subject to change, so please don’t hard-link to it from anywhere important.
- All the metadata is available online as sexp files, and so Obi will also have a CLI tool that implements the workflow. I just wanted to get the HTML build logs up as soon as possible to allow our new maintainers to help fix problems.
- The non-x86 and non-Debian logs are available, but not yet rendered. If someone has a burning need to inspect them, let me know and I’ll push it up the priority queue.
- Eventually, the manual opam-repository fixes can be automated via the CLI, but that feature obviously isn’t there yet. It will likely land only when we complete the release of opam 2.0 and can take advantage of some of the automation features.
- Only Linux-based operating systems are being built right now, but we are expanding the cluster to include at least Windows, FreeBSD, OpenBSD and maybe OSX if we can figure out how to automate it. If anyone is particularly interested in helping with automation (e.g. something similar to containers) on the *BSDs in particular, that would be helpful.
For now though, let’s congratulate our new opam maintainers, and hopefully get to fixing some of these safe-string issues
Anil, your friendly opam-repository overseer