I just noticed that one of our Docker images based on ocaml/opam’s images “broke” due to the recent archival of dune 3.16.1, which was referenced inside.
Looking back at the message about stage 3 of the archival, it’s obvious that it would happen and I just didn’t think about it, because implicitly I somehow supposed that “well, it’s Dune, it’s used by a lot of people, and the release is less than one year old, so it won’t be archived”. And because these Docker images are only used once every few months, such issues go undetected most of the time.
This is not a big issue, but in the future, I’d like to know how to best handle opam package versions in our Docker images to minimize britleness and “unbuildability”:
Are there specific versions that are “longer-lived” ? (the x-maintenance-intent of dune only says (latest), so I don’t think so)
Should I enable the opam-archive repository by default in my Docker images?
Should I consider using my own opam repository for such images?
We are looking at having some dune “LTS” (quote because it’s not necessarily exactly what we might end up with, we don’t know yet) that are kept longer in the repo or something of this sort.
You can enable opam-archive repo. Alternatively, you can get opam to dump the package info for the content of your switch and then commit this dump to your repo (or some variation of keeping around the package info that you need).
You can make your own opam repo. That’s essentially like commiting the opam switch content but not necessarily tied to one repo so it’s probably better if you need to share the solution on several projects.
The ocaml/opam image has a checkout of opam-repo locally. This should deterministically stay working indefinitely, unless you are subsequently doing a “git pull” on the opam repository and an opam update.
Given the recent push to remove opam packages, a Dockerfile now needs to batch update its dependencies with a clearly pinned revision of opam-repository. Therefore, doing an “opam update” from within a Dockerfile is now a somewhat dangerous operation to include without an associated check to ensure that dependencies are still present.
In this specific case, I am somewhat surprised to see a dune release that’s less than a year old be archived. Is that the intent behind this new system, or an accidental side effect of the new x-maintainence metadata markers?
This is what I am doing for situations where stability is my primary concern (i.e., I don’t mind if this makes resolving slower, as long as it continues to build). Depending on the use case of your image (particulary if image rebuilds or post-build opam installs are relatively rare), I think it is a very reasonable choice.
The idea of hosting your own repository is also interesting, but would lose some benefits of using upstream. One thing that comes to mind is I pin some packages, but let others resolve automatically. The upstream repos adding new upper bounds to old opam files when incompatibilities arise is a big part of why this works.
FROM ocaml/opam:debian-12-ocaml-5.2 AS build
RUN cd ~/opam-repository && git fetch -q origin master && git reset --hard e5d038c1515b66851c8191c55500786cee950833 && opam update
RUN sudo ln -f /usr/bin/opam-2.3 /usr/bin/opam && opam option solver=builtin-0install && opam init --reinit -ni
You didn’t have to do this git pinning in the past since opam packages were never deleted except for occasional cases (and so a lockfile would work with the latest repository even with older packages), but that is no longer the case. The advantage of doing the git checkout is that the containers will tend to stay working and be cached, until you explicitly choose to run package upgrades.