Dear opam repository contributors,
You will all be familiar with the difficulty of contributing a large new set of libraries into modern opam, due to the need to establish upper bounds on third party packages that use your libraries that have made breaking changes to interfaces. This difficulty exists due to how successful we have been at importing in multiple versions of the same package into the opam-repository, and leveraging the opam solver to decide on the freshest versions of packages to install.
In order to make the contribution process a little easier, we are experimenting with a new release strategy for the latest large library set to drop into opam-repository (in this case, the new JS Core release). It is as follows:
- add upper bounds to all opam-repository packages that use your library in one PR. This prevents the newer libraries from being selected by the solver for existing packages.
- import the new libraries, knowing that they will not be selected by the solver for existing packages.
- relax the upper bounds on third party packages that have been tested to work with the new release.
This three-step process effectively allows the new versions of a library set to be staged in the opam repository without causing a massive interlock with third party packages. It also (via step 3) gives maintainers time to get used to a new library set without users being affected.
The main downside to this approach is that it may break development pins, since that metadata exists separately in each upstream repository and needs to be updated with upper bounds (or preferably, the code fixed to support the newer library).
The more minor downside is that it involves the maintainer of a library you depend on adding version constraints to “your” packages. Since the only change here is to add an upper bound to an existing dependency, we judge this to be acceptable to the long-term health of the opam-repository database. However, maintainers do have ample chance to comment on the step 1) repository if you really do not want anyone else making changes to your packages. In this case however, we expect the maintainers who have objections to step up and do some testing with the newer releases and/or propose any changes in good time.
I’d like to emphasise that this is an experiment at the moment and not firm opam-repository policy, but one that we feel is worth trying. If it does work, then we can improve the tooling around the 3 steps to make it as easy as possible for maintainers. In the meanwhile, you can see the first step of it in action in ocaml/opam-repository#13010 which adds upper bounds in preparation for the new version of Core.