Dropping some intermediate OCaml versions from CI

Ecosystem
1

@xavierleroy recently observed the madness of well intentioned open source crowds:

Right, we stopped supporting native-code compilation to 32-bit architectures in OCaml 5, but we’re good people, so we committed on maintaining OCaml 4 for the forseeable future, including all its 32 bit code generators.

Then, the OPAM-CI folks are good people, so they committed on testing ARM-32 for all packages.

Then, the OPAM repository maintainers are good people, so they duly report CI failures on ARM-32.

Then, all these good people end up wasting time reporting and fixing 20-year old oddities in OCaml’s ARM-32 code generator, despite the fact that it is completely irrelevant.

While there is now consensus on dropping arm32 and x86_32 from our test matrix, and I am dropping opam 2.0 as well, we can go yet further in our austerity measures.

We currently test every version from 4.08 → 5.4 in opam-repo-ci, and the intermediate versions only have a couple of uses:

  • to detect when a package is using some new stdlib function to put a lower bound on the CI. We can just figure this out from the OCaml changes file; no need to test on every single compiler.
  • find a historical bug. We mostly don’t care about these.

Therefore, I’m proposing to create a list of ‘significant’ releases that we will against:

  • 4.08 (the lowest bound)
  • 4.14 (the latest LTS)
  • 5.2 (used by Jane Street as their OxCaml base, so it seems important)
  • 5.3 (latest release - 1)
  • 5.4 (latest release)

This will greatly reduce our CI burden, which has just moved backed to the Cambridge Computer Lab where our air conditioning has a habit of falling over every summer if it works too hard.

I’m also staring at the compiler variants that we generate base images for, and may hack and slash some of those. Does anyone use flambda+fp? One seems to give performance, and the other taketh away.

Hopefully with this reduction we’ll stop generating annoying bug reports for obsolete features, but I’m also keen to hear from anyone who does use some of these images and would like for them to stick around!

9 Likes
2

I agree this is a good move, thanks!

(I think that upstream should probably clarify that this means that arm-32 (and probably x86_32) is not maintained anymore, even on the long-term 4.14 branch.)

2 Likes
3

… except on bytecode? I’m a little fuzzy on this matter, but 32-bit OCaml is used in JavaScript and wasm compilation right now.

1 Like
4

Right. I’d love to go 64-bit only, but currently and for the foreseeable future 32-bit processors are still supported by the OCaml bytecode compiler and runtime system, to help with Javascript and wasm compilation.

1 Like
5

Re: dropping some OCaml versions from OPAM CI, I think it’s a good idea to save on electricity and to make CI reports shorter and easier to process.

Concerning which OCaml versions to support, Debian is currently at 4.13 (oldstable), 5.3 (stable), and 5.4 (testing). RHEL/CentOS 8 is at 4.07 (too old!), 9 at 4.11, and 10 at 5.2. So, maybe you could add 4.11 to your list of significant releases, as a gesture towards old Linux distributions.

Concerning compiler variants, I agree with reducing the number of combinations. I think Flambda and FP support are largely orthogonal, so testing Flambda + FP may not be useful.

1 Like
6

Thanks! I’ll add 4.11 as you suggest and trim some of the compiler variant combinations (and perhaps remove things like flat-float-array as they are quite specialist).

7

I use this combination locally for some projects, but no need to keep it on my behalf. I take exception to the suggestion frame pointers takes away performance :slight_smile: it gives nice tools to improve OCaml performance. My benchmarking on ARM64 showed no difference between FP and no-FP, I don’t think anyone has done comprehensive benchmarking on AMD64 recently.

1 Like
8

Frame pointers clearly add some overhead; the question is whether or not the debugging utility outweighs the runtime cost. Brendan Gregg did an excellent analysis of frame pointers in 2024, which resulted in Ubuntu 24.04, Arch and (possibly) Fedora adopting them by default. Perhaps it is time to revisit this decision in OCaml as well.

9

looking at repology, it seems that Ubuntu 20.04 is the only distribution still using OCaml < 4.11 and since, by chance, Ubuntu 20.04 is actually getting EOL at the end of the month, i think it might be the perfect time to also bump that lowest bound for opam-repository and move the 4.08/4.09/4.10 requiring packages to the archive repository.

5 Likes
10

correction: EOL (for non-paying customers) was one year ago. Even more reason to do it

11

I do appreciate going to 4.11 as lower bound, and archiving packages that require earlier OCaml releases. If you like, I can get back to my scripts to prepare lists of packages that would be archived under this assumption.

It would as well be beneficial to state a policy in the opam-repository which OCaml compiler versions are supported, and when they’re going to change. One proposal I’ve heard several times is:

  • the version which the oldest LTS release of Ubuntu
  • the version which the oldstable debian uses

So, maybe the lower of the two? Maybe explicit “plus the latest version of OCaml 4 as long as the OCaml core team supports it”?

1 Like
12

I think focusing on a specific distro is a bit too restrictive. These things can change fast – you only have to look at what happened to CentOS.

Something more generic and future-proof like this would be better i think:

The lower-bound is set to the oldest OCaml version amongst the list of reasonably used distributions/OSs releases that are not EOL. Exceptions and common sense apply.

13

Taking 4.11 as the lowest OCaml version would result in the archival of: see https://discuss.ocaml.org/t/archival-ann-raising-the-ocaml-lower-bound-to-4-11-0-for-opam-repository/

14

oh, we actually have the following text:

This threshold is subject to change by the opam repo maintainers. The threshold is based on the oldest ocaml compiler version available in the “maintained” distributions. It determines the minimum compiler version used in tests for the opam-repository CI.

Where “maintained” has a footnote: The versions of ocaml shipped by linux distributions are accessible on repology. Both homebrew and macports are usually very fast to move to the latest available. Which LTS are still maintained isn’t explicit, but it can be found at endoflife dot software (e.g. https://endoflife.software/operating-systems/linux/ubuntu).

2 Likes
15

It may be sensible to advertise this more prominently in the README.md, maybe with a link to the opam-repository-archive!?

2 Likes