I wish that some of the heavy-ocaml-gitlab-users would team up to write a proper base setup for Gitlab CI for OCaml projects, that people could easily reuse and extend when they do their own project.
3 Likes
I think I initially misunderstood the caching used by @cemerick in basic OCaml gitlab CI config · GitHub : the cache is keyed on $CI_COMMIT_REF_SLUG; if I understand correctly this means that the cache is only maintained for several actions running on the same commit (possibly across several branches), and not from one commit to the next as I had in mind.
1 Like
Gitlab maintains a list of CI template files for many programming languages, we could contribute one for OCaml.
3 Likes
That’s fortunately not the case; $CI_COMMIT_REF_SLUG is the current git branch/ref being built, normalized according to certain rules. You can see examples of all of the variables present in gitlab CI runs (including this one) here: GitLab CI/CD variables | GitLab
I suppose it’s lagged w.r.t. OCaml version, but otherwise should work well. FWIW, this is representative of what I’ve been using for ~a year now:
test:
stage: test
only:
- branches
cache:
key: "$CI_COMMIT_REF_SLUG"
paths:
- _opam
- node_modules
script:
- '[ ! -d _opam ] || (rm -rf ~/.opam/4.08 && mv _opam ~/.opam/4.08)'
- opam switch 4.08
- eval $(opam env)
- opam switch
- if ! opam install --deps-only --locked -t -y . ; then opam repository add remote https://opam.ocaml.org && opam update && opam install --deps-only --locked -t -y .; fi
- npm install
- dune runtest
- mv ~/.opam/4.08 _opam
(The opam install/repository dance is there so that newer libraries that aren’t included in my base docker image can be picked up.)
I’ll be bumping the OCaml version across my key projects at some point in the next month or so once I finish the current milestone. I’ll see about either posting any updates here, or maybe polishing something for inclusion in the CI template file repo.
After a long ci-golfing adventure (83 tests), I got a .gitlab-ci.yml file that I think is reusable and useful for small projects / libraries:
- project: https://gitlab.com/gasche/gitlab-ocaml-ci-example
- configuration file: https://gitlab.com/gasche/gitlab-ocaml-ci-example/-/blob/main/.gitlab-ci.yml
Features:
- It is project-agnostic, so it should work unchanged for your own (simple) projects.
- It caches the opam dependencies.
- It builds the project, runs the tests and builds the documentation.
- Several compiler versions can be tested in parallel.
- It provides an easy way to upload the documentation as “Gitlab project Pages”.
CI times are satisfying: on very small libraries I observe a 11mn job time on the first run (or when cleaning the opam cache), and 2mn job time on following runs.
The expected usage-mode of this CI configuration is that you copy it in your own project. If you find that you need/want additional features, ideally you would try to write them in a project-agonistic way and contribute them back to the example repository.
This configuration does not use @smondet’s trick of generating a docker image on the fly. I think this would be an excellent idea to get more reliable caching, but it is too complex for me and I don’t see how to do it in a maintainable and project-agnostic way.
Current status
I wrote this CI configuration over the week-end, and have not used it much. I expect it to keep evolving somewhat before it stabilizes. Feedback from other people trying to use the configuration would be warmly welcome.
Aside on _build caching
I also implemented caching of dune’s _build data, inspired by the data-encoding example of @raphael-proust. I don’t need it for my small projects (dune build is 3s, compared to 1m setting up the Docker image), but I thought it would make the CI configuration scale better to larger projects.
When I tested this CI configuration, I discovered that caching the dune _build data does not work as well as I had expected. (Tracking issue: dune#4150).
I can tell because I am asking dune to tell me about what it is rebuilding (dune build --display short). I suspect that projects that cache the _build data without logging what dune (re)builds are also not caching as much as they think they are.
(But then maybe the use of a fixed-compiler OPAM image, as data-encoding is using, solves the issue.)
official CI template?
I considered submitting this CI configuration as an “OCaml Gitlab CI template” to go with the official list of “blessed” CI templates in the documentation. But reading the Development guide for Gitlab CI/CD templates convinced me that my CI configuration is nowhere ready to serve this role.
Gitlab developers apparently expect that users will be able to “include” those CI templates by pointing to their URL, and then tune it for their own use-case (without modifying it) by performing some (unreasonable?) inheritance tricks using whatever those configurations offers as abstraction/inheritance/extension/overriding mechanism. Let’s just say that this is next-level CI configuration writing, and that my script is not ready for this.
9 Likes
That config file looks great. I’ll probably be porting some of the libraries under the Nomadic Labs umbrella to use that.
I’d recommend:
- Making doc building optional (by detecting
with-docdependency in opam file or through the use of a variable). - Adding a step to check formatting (using
dune build @fmt) (optional step, controlled through a variable, or through a{with-test}dependency toocamlformatin the opam file). - Add a URL to the artifacts (using the
PIPELINEvariable to generate the correct URL) in the message about checking the artifacts.
I’ll try to add those soon if it hasn’t been done when I get around to it.
Thanks!
I’m not sure how to make steps “optional” by checking opam files: there may be several opam packages developed in the same git repository, and I would expect all of them to receive CI build/testing together. I just tried and dune runtest is a no-op if no tests are around, so there is no worry about failures in this case. Do you have in mind a scenario when we don’t want to run all tests as part of the CI, but only some tests (eg. “fast” tests?). Ideally the CI configuration would have a variable that let us describe what to test (everything by default, nothing if the variable says so, but also a subset if there is a corresponding way to tell dune about that). I never needed to specify only a subset of tests to run in Dune, so I would need to go check the documentation about that.
(I also have no experience with dune build @fmt.)
The PIPELINE suggestion is excellent, thanks.
The @fmt target for dune is available if the correct stanza is present in your dune-project file: (using fmt <version>).
The dune build @fmt command will return a non-zero exit code if some files are detected to not be formatted correctly. In interactive use you then issue dune promote command to get the formatting fixes on your working files. But in CI you can just rely on the exit code. (It also prints the diff of formatting on its output so it’s fairly obvious what went wrong.)
The main issues are:
- How to automatically detect whether we can run this? (That’d probably be from the
dune-project.) - How to automatically detect what version of
ocamlformatis needed? (That’d probably be from the.ocamlformatconfig file.) - How to allow users to disable it if they don’t want it running on the CI?
- How to deal with multiple-project single-repo?
According to Dune’s documentation on formatting, starting with (dune-lang 2.0) formatting is enabled, but ocamlformat requires an .ocamlformat file. I would thus:
- skip the
dune run @fmtstep if no.ocamlformatfile is present at the root - provide a CI variable to let users define the formatting taget (default
@fmt), which lets them
for example use weird stuff like@subdir/fmt, with the convention that an empty value means that formatting is disabled in the CI. This approach, “target alias or empty”, also works for@runtestand@doc.
I don’t think we need to do any work to guess the version of ocamlformat to run: this should be a {dev} depedency of the opam file, so it will have been built during the build-dependencies stage.
(Note: this discussion could also have happened on the gitlab-ocaml-ci-example issue tracker.)
2 Likes
@raphael-proust I implemented the following of your suggestions:
- show artifact URLs directly in the logs
- configurable targets for the build, test, doc stages (in particular, an empty variable skips the corresponding step)
I made some progress the _build caching issue, thanks to a tip from (my informer) @ejgallego who had the exact same issue for Coq. The permissions of some files are changed when restored from cache. Unfortunately the only way I can reproduce the issue is by using dune to build a problematic directory, any attempt at a simpler reproduction has failed so far. But this does seem to be a Gitlab issue.
The issue with dune _build caching is caused by a bug in gitlab-runner, which does not preserve file permissions for the targets of symlinks; I reported a bug upstream at https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27496. In the meantime we will have to work around this bug by caching not the _build directory itself, but a tar. I may not have managed to locate the issue without a helpful tip from @ejgallego who ran into the same issue with the Coq CI.
(It is also possible that this bug would be problematic for _opam caching, if we had packages within opam that installed (non-executable) files using symlinks. I don’t know of any package doing that, so I think the issue can be ignored for now, but please comment if you know of such a case.)
Edit: a Gitlab developer suggested a workaround (opt-in to an alternative zip implementation), so the problem is now solved (but the limitations of dune that made debugging too difficult remain). In case anyone is curious, the root cause of the bug is the frugality of Go’s archive/zip standard library package, which encodurages consumers to implement their own zip-file-extraction logic independently, resulting in buggy and often-insecure code.
1 Like
Hi @gasche
I just discovered your project. Why not try to include it in drom ? Currently, drom only targets Github, which is an issue as many projects prefer Gitlab. So, if you feel like contributing it, we can help 
After using it in “the wild”, I’m not satisfied with the “copy this large script and then hack on it randomly” model of the first version I released. My short-to-medium-term plan is to make the script “include-able by URL”, so that people can keep a short .gitlab-ci.yml script that points to the main script and tweaks a few variables of interest. See the tracking issue gitlab-ocaml-ci-example#2.
(I timed out on gitlab-CI hacking for a week or two, so this will not get done immediately.)
I think that it would be much easier to add Gitlab-CI support in project-starter tools such as drom after the script is reworked to be included by URL. For now, I am mostly hoping for feedback from established/experienced Gitlab users, that could try to adopt the script on their project (as a form of beta-testing) and report bugs or important missing features.
2 Likes
Ha, good work @gasche ! I too have been defining my own gitlab CI scripts some time ago, so here are my 2 cents.
This is very light-weight and quite bare-bones, but doesn’t rely on dune and may be fine for small projects.
build:
image: ocamlpro/ocaml:4.10
before_script:
- "[ -d _opam ] || opam switch create . ocaml-system --deps-only --locked"
cache:
paths:
- _opam
key:
files:
- foo.opam.locked
script:
- opam exec -- make all
- opam exec -- make test
This uses our light-weight alpine images with the beta of opam 2.1, which makes e.g. external dependencies handling much easier to handle. The image provides a system compiler at the right version, which avoids recompilations, and the dependencies are cached as long as you don’t change your .opam.locked file.
Of course, this is much more limited in many ways, in particular it’s just for one version of OCaml at a time.
Also I agree on the include workflow for Gitlab CI, it’s pretty nice.
EDIT: forgot to add, I like the include workflow better combined with extends though.
doesn’t rely on dune
Ah, you are relying on this shiny build system called make, but I don’t have a Makefile in my small projects of interest. (Maybe it’s a good practice to keep them for friendliness to non-OCaml-users?)
The script is nicely minimalistic, and indeed opam 2.1 makes depext handling much nicer. I prefer to not use 2.1 yet, to keep an environment that is close to what the users have at home, if they want to debug the script.
In my tests there is a large startup-time difference betwen the debian-testing-opam image I use for the build and the alpine:latest image I use for Pages deployment (about 1m20s vs. 20s). I guess I should to test alpine-opam images to see if I can reduce the load time a bit (assuming opam depext handles the alpine package manager correctly).
Ah, you are relying on this shiny build system called
make
Haha, that’s just an example, the point is that the script is so simple it should be trivial to customise for a given use-case; a more generic command could have been opam install ., but I wanted to hilight that you can use your own commands without going through opam. You’ll also need to specify your opam files for proper caching, unfortunately (foo.opam / foo.opam.locked …)
Alpine has been used for CI for a while, so it should be in the top-tier of supported distros for depext handling.
I’ve just used the yaml via copy-pasting on two projects and it just works. I’ll use the open MRs as opportunities to tweak things out, but it’s already much better than the homebrew stuff I had cooked up and reheated whenever I needed a new feature/test.
Sorry for offtopic, but do you have a link to docker-based instructions for GitHub?
Not sure what you mean by Docker-based instructions.
You can have a look at the Tezos project which does interesting things with Docker for its CI:
- Tezos / opam-repository · GitLab generates a Docker image with already installed opam dependencies. It actually generates a collection of them: one with only the runtime dependencies for packaging the binaries, one with runtime+build deps for building the binaries, one with runtime+build+test dependencies for running the tests.
- Tezos / tezos · GitLab pulls the different images for running the different parts of the CI.
It’s definitely over-engineered for small projects, but for bigger projects like Tezos it works quite well. Notably:
- it avoids rebuilding all the opam dependencies,
- it allows fine control over what is included in the released images,
- it allows fine control over the exact versions of all the opam packages that are used as dependencies,
- it makes it relatively easy to include non-opam/non-OCaml dependencies (e.g., python dependencies for tests and doc).
Do you have a more specific query? What kind/size of project are you looking at?
@raphael-proust I meant Github, not Gitlab, so I marked question as off topic.
At the moment github action for OCaml made by avsm is terrible because it can’t do caching. And I’m looking for the ways to use GitHub actions + Docker image. Maybe somebody already did that. …