One of the reasons for adding {build} to a dependency is to avoid rebuilding when that dependency updates/changes. That way each update to ocamlfind/jbuilder/ocamlbuild does not cause everyone to rebuild everything in their switches. In this case, unfortunately, {build} helped to mask a real issue.
1 Like
To distill the discussion above, there are two approaches to solving this issue of broken packages appearing in people’s opam feeds:
- Having a staging area gives some breathing time for large, interlocked changes such as a new release of Core or Lwt, or a base tool such as Jbuilder or ocamlbuild.
- In the event that we become aware of a breakage in the live system, then we should make the package uninstallable as soon as possible instead of removing it. This can be done by setting the
availablefield to false. In opam2, there is aninstall ----ignore-constraints-on[=package]which can help advanced users to override a failure if the package only fails in some circumstances (as was the case with jbuilder beta18).
I believe that does answer the original question of how to yank a package: we don’t want to remove versions unless absolutely necessary, so staging branches help to mitigate the risk of something broken hitting the live branch, and uninstallable constraints fix the emergency when something does inevitably get through the layers of protection.
4 Likes
I like this a lot. Are there any barriers to making this the official approach to broken packages discovered in opam-repository?
People have also proposed having a “known good, works on all mirageos targets” repo where new fancy, incorrectly packaged C code is not allowed.
This is also what we started doing recently, as we found out it was the best way to prevent everyone in the office hitting the same problem again and again. We take a frozen snapshot of opam repo, and someone is responsible of updating the snapshot every ~2weeks while ensuring all CI passes and adding dependencies constraints where it is needed. It works quite well
1 Like
sorry being late to the party. I could use some more detailed explanation (please point me to such an explanation if it already exists) about:
the main opam-repository is hosted on GitHub as a git project. this implies there’s always a history, and if someone is interested in old releases (such as software X which was developed for a paper in 2005), they can just git checkout the repository how it used to be in 2005, and will be able to install X.
having said this, I’d like to know what “absolutely necessary” means. I recently release mirage-unix.3.0.6, which installed fine, but was useless since its OS module didn’t meet the requirements from mirage-time-lwt, so I released 3.0.7, which installed fine, but linking lead to undefined references, so I released 3.0.8 – all within 4 days, and I removed the broken releases (3.0.6 and 3.0.7) – should I instead have set available: false in there (as I did for old and broken (due to security issue) mirage-net-xen releases)? why should we keep old and broken versions around in our repository if (as shown above), we don’t really want anyone to install these packages.
from a signing point of view, it is fine to remove releases of packages!
on the other hand, re-releasing the same version number with a different tarball / checksum is very brittle and likely leads to issues which are hard to debug (since on the client side, package X in version a.b.c is installed, and you can’t tell the difference between X and X’ (the re-released X on a.b.c with a new tarball) – we should avoid doing this (as mentioned in Package jbuilder.1.0+beta19 by rgrinberg · Pull Request #11604 · ocaml/opam-repository · GitHub as well, see discussion at package revisionism can break things · Issue #10531 · ocaml/opam-repository · GitHub and zarith-freestanding.1.6 + zarith-xen.1.6 by hannesm · Pull Request #10350 · ocaml/opam-repository · GitHub).
3 Likes
We had done this also, so we didn’t experience any problems in the office or in our CI system, unlike our users who did, because they weren’t using our stable repo 
This is a strict no-no! I think we should have a rule in Camelus that prevents such things.
Let’s not mix it. There are broken packages and there are old. We shall always keep the old packages, because someone may still use it. Concerning the broken packages, there are different ways of being broken.
-
code is semantically wrong, i.e., it works, but under some conditions it manifests some wrong behavior.
This is normal in software development, as nearly any new release will contain bug fixes, that are basically indicators that previous releases contain those bugs. We shall keep releases with bugs. -
code doesn’t compile, under no conditions. We shall not have such packages. We should even allow them to be merged into the upstream.
-
code that used to be compilable. This code worked, but now OPAM can’t find a solution for it, because:
3.1 OPAM uses answer set programming, that is undecidable in general case, so it is still a heuristic, at the end of the day, we may have a package that is perfectly fine, we just can’t find it. We should keep such packages since we can always instruct OPAM to install a particular version and ignore constraints.
3.2 OPAM can’t find it, because of the new information provided by other packages. That just means that our initial specification was improperly constraint. Or the new definitions of other packages is overconstrained. This should be fixed on either of the sides.
3.3 OPAM can’t install it because of a dependency that is now missing. And this is the particular reason, why we should keep packages. Because, if we will remove a package we may break our revdeps. And the main issue here, is that we can’t check this with Camelus because people do have their own opam repositories, starting from more than 500 forks of the main repo, and going to those that are not even clones or share any history with the main repo. So we need to stick to the open world assumption and assume that there is always a revdep that we may break when a package is removed. Unless, of course, the package was never compilable. But again, we shall not have those packages in the first place.
2 Likes
@hcarty I would intuitively have thought that {build} was there to allow computation of the set of libraries required for linking. I had not realized that {build} meant “I need this sometimes, but I don’t care about updates” - that really sucks, since I can’t think of any cases where I don’t want my code to profit from bugfixes in the compiler. Thanks for the tip, I’ll start removing the {build} flag from my opam files from now on!
@hannes
I agree with you; the main problem is when re-using version numbers, which is why I argue for the “bump minor version”-approach, since plainly deleting the offending version makes it very easy to accidentally re-use the version number, and it will be confusing to debug for people who have already installed the bad version when opam won’t alert them to updates being available.
The uninstallable feint seems unnecessary to me when there are more recent minor versions available, but that assumption probably rests on my idea of “no one will pin a specific minor version,” which is unlikely to be true. 
{build} is intended for things like build tools which are required at build/install time but not for normal use of a package. Before this flag was in common use, every update to ocamlfind, for example, would cause every opam user to rebuild everything under their active switch(es). So, while there is sometimes pain from using {build} it seems to be less overall than the constant, guaranteed pain caused by not including it.