Thanks for the offer of help @Leonidas, it’s much appreciated! (I’m choosing to ignore @BrendanLong’s unkind and unconstructive message).
The situation with those containers is:
ocaml/opam : these were the first cut at building a container matrix, and were done from the first version of avsm/ocaml-dockerfile. They are built on the Docker Hub builder, but suffered from not having a good way to expire images. That meant that every time a new OCaml release comes out (including patch releases), we have to build an every-growing matrix that takes up 100s of VMs and close to a terabyte of images! We also started tracking opam2 in there, which made it a terrible burden to maintain as that keeps changing.
To solve this, enter…
ocaml/opam2-staging: this repository is generated from the latest version of ocaml-dockerfile (5.0 that I released yesterday). It solves some of the pressing maintenance problems of the original repository, and also focusses on supporting opam2 as best as possible. This includes some really nice new features: image sandboxing support, the images are multiarchitecture (ppc64, arm64) and automatically built using Buildkite and a large Docker machine cluster hosted at the Cambridge Computer Lab and Packet.net rather than ongoing cloud services (which I personally stuck on my credit card for far too long). In order to test this, @kit-ty-kate has been deploying it on the opam-repository CI for the past few months, and we are finally happy.
A few days, I recreated:
ocaml/opam2 - this is the place where the latest images for opam2 are pushed from the automated container builds, and will be used as the basis for ongoing CI. An important change to the naming layout here is that we drop patch releases from the container names (e…g ocaml/opam2:4.06) so that we don’t need to rebuild a huge container base. As distros expire, we can now easily push “do not use this image anymore” containers so that we can keep our build matrix reasonably sized.
Reconstructing the opam1 containers is tricky and difficult to contribute to – I’m planning to spend some time rebuilding and parking them as soon as we can shift the majority of opam-ci testing over to opam2. The important thing here is that opam1 will be tested beyond the opam2 release – however, the amount of testing will be reduced. For example, reverse dependency testing for newly submitted packages has always happened on opam2.
For new contributions, I’m going to archive the old repositories as you mentioned and am documenting the infrastructure. Stay tuned for a post about a cool new CI tool in another post today or tomorrow for more details
The new infrastructure is also satisfyingly maintainable – @dra27 has gotten Windows Buildkite workers, and I have Windows Docker containers working, and *BSD agents also activated. Hooking these in will happen over the next few months, and a fantastic area for new contributors to get involved in.
My apologies that your time was wasted looking into the myriad of repositories for various iterations of these containers – it has been a process of co-evolution as we fixed quite a few issues in the underlying Docker technologies over the past couple of years as we refined the OCaml container support The latest version of Docker with containerd is running incredibly smoothly, and I’m looking forward to integrating with the forthcoming BuildKit as well for even faster builds.