A workflow that takes seconds on my local machine takes 15 minutes or more on the GitHub CI because the whole world starting from the compiler is downloaded and compiled from sources. This is very different for a C project because the C compiler and many libraries come pre-installed or are installed from a binary distribution. So the OCaml CI looks wasteful in comparison. What are best practices to keep resource usage in check for OCaml projects? Is there a way to provide images where more opam packages are already pre-installed?
You can supply a Dockerfile that uses a fixed version of the opam-repository to allow image reuse. See, e.g. docker-base-images/Dockerfile at master · ocurrent/docker-base-images · GitHub
I’ve been wondering the same but without really taking the time to fix it properly.
It feels like, ideally, the ocaml-setup action should allow us to load docker images that:
- have an exact OCaml version
- have an exact opam-repo hash (for better caching, reproducibility, etc)
obviously this would require a gigantic set of docker images, so… I guess I need to pay for github’s docker repo, and build my own docker images?
Both of these have been autobuilt and published on the OCaml Docker Hub since 2015; see my earlier answer with a Dockerfile that uses it. (I’m not sure how to do this with setup-ocaml but Docker builds work fine with GHA).
Docker images certainly help reducing CI time, but still take significant time to load. The fastest Github CI runs I’ve seen with CompCert (a mixed Rocq/OCaml project) were obtained by using binary, precompiled Homebrew packages on macOS.
I always create my docker images using GitHub - mjambon/ocaml-layer: Make your own OCaml base image for fast CI jobs · GitHub
I tried to touch prebuilt images (from ocurrent or something else) and never succeeded. Either user name by default is weird and it is not obvious where to put stuff, or something else
Are you suggesting that people should do their CI inside of a dockerfile (with a FROM <exact dockerhub image> at the start), then? I’m not opposed to it at all but I thought setup-ocaml (which doesn’t do that, does it?) was the “official” way to do CI for OCaml projects.
I’ve been annoyed by the setup time for years so I’m very glad this thread exists! 
Sorry in advance for theorizing
Is the problem that we’re building everything from source, or is it that we’re not caching the results across CI runs?
For example, 4ward is a complex, multi language project (mostly Kotlin and C++; no OCaml) pulling in lots of heavy dependencies, yet it’s CI runs typically take 1-4 mins. And C++ has famously terrible compile times… How? Caching.
Specifically:
- Uses the Bazel build system, which specializes in hermetic builds with aggressive yet sound caching.
- Uses a (free) remote Bazel cache (buildbuddy.io)
- Uses GitHub actions cache as a fallback.
I wonder if we we could come close to something like this in OCaml?
I’m saying that if you’re willing to better specify your dependency needs, then you get faster build times as the layers can be cached. Whether or not you do that is up to you, as there’s more housekeeping involved in the CI then, or you need a coordinator (like ocurrent) to manage it for you. GHA is fairly poor at this, although setup-ocaml does have a bunch of options to help using GHA’s own caching layer.
The Docker Hub opam CI images are really for the CI because otherwise you get users like Kakadu calling the default username (opam) “weird”, or not being able to figure out where to put stuff (in the default WORKDIR). It’s easy enough to just create your own Dockerfiles.
For those on macOS, as Xavier notes Homebrew is also surprisingly good for latency; I put up notes on how to setup your own custom Homebrew tap here. Do note that using the Docker Hub will be slower from GitHub Actions than using GH’s own ghcr package registry (as traffic traverses AWS->Azure from the Docker Hub to GH).
I started using Nix in CI and now the CI time is between 2 to 7 minutes, whereas before it used to be 25 to 30+ minutes.
To add to the landscape of approaches, I believe it is worth mentioning the setup-dune project by Tarides.
Its approach is to access a pre-compiled dune binary, and then use dune pkg to download the dependencies and compile the project The goal is then to use dune-cache features combined with GH caches to improve hot re-runs performances.
In certain happy pathes I have witnessed this solution complete a CI job in the “less-than-a-minute” range (see some example e.g. here) - with other non-cache hit cases being not all that different from other approaches time-wise.
I haven’t used it enough yet, haven’t experimented with Docker approaches, and cannot offer a detailed comparison. I’ll simply say that it is looking to me like a potential solid approach for OCaml projects hosted on GitHub, and would recommend keeping an eye on it, as dune-pkg becomes more mainstream. I would be interested in further development and discussions on these questions. Thanks!
My gitlab.com CI for ocaml projects caches opam and _build files from one run to the next, and I get two-minutes builds.
(I decided that I was fed up with gitlab’s current business aims that are very far from being a good host for open-source projects, and am migrating away to Codeberg. No idea what the CI story is there.)
I can confirm that setup-dune runs much faster than opam-based workflows, and it’s now my default whenever I can. Unfortunately, my main projects are still unsupported by dune pkg.
A solution I use is to just cache the entire _opam folder in my CI workflows like a sagouin 
And reuse compiled artifacts across jobs as much as possible
I have a bit of that in certain workflows still, but reducing releases after releases. This sounds like more ways of doing things is coming up our ways soon!
We are in good company for I think the setup-dune alternative doen’t hesitate to roll up its sleeves and cache the entire _build for now.
My understanding is that we will eventually be all striving for a sweet spot (yet to be found?) to avoid caching too big artifacts that are otherwise fast to recompute. At the moment I don’t think any existing solution is located in any particularly good extrema with respect to this. For the case of setup-dune in particular, I had noted from a pr discussion that caching the entire _build was a way to avoid compiler rebuild, yet without queueing behind the integration of the relocatable compiler patches. Pragmatic, and effective as you confirmed!
Are the projects public? We’d be keen to see what is blocking support.
Yes, the issues in dune are tracked here in this issue: pkg: complex makefiles, cerberus-lib issue · Issue #13407 · ocaml/dune · GitHub
Indeed!
There was a stream of issues before we could get there (including the symbolic links which I ended up PRing out of one of our dependencies). Now this is in the way, and I’m hoping it’s the last issue that needs fixing.
Well, and to be fair the patch issue is still in the way and requires us to ask macos user to install additional dependencies otherwise
Alright, I gave it a go and look at this (35s for a format action!!).
I used a repo-specific parametrized dockerfile and a Makefile to instantiate it with versions of OCaml and base packages. 
An issue with dockerfiles is that they don’t allow for building artifacts macOS artifacts 