OCaml CI and Opam Repo CI support for FreeBSD

The CI team at Tarides recently launched support for FreeBSD into opam-repo-ci and ocaml-ci, so now opam packages are tested against FreeBSD 13.2 before going into opam-repository and you can build your ocaml projects on FreeBSD 13.2. This feature is considered experimental at the moment but we expect to move to a supported setup over the next few months. Please try out the FreeBSD support and let us know what improvements you’d like to see.

Currently we are providing FreeBSD 13.2 on x86_64 with OCaml 4.14 and 5.0. When 5.1 is available we would move to providing that in place of 5.0. A sample FreeBSD build for ocaml-grpc showing a build on OCaml 5.1 :slight_smile:

We have also published a couple of blog posts detailing the improvements to OCaml-CI made this year and how the macOS support for CI was implemented. Expect a similar post on how FreeBSD support was implemented in the coming weeks.

Enjoy!

10 Likes

If you ever consider testing other BSD systems, I recommend starting from OpenBSD - due to some inherent differences, it often can catch some bugs that others systems didn’t, from my experience of C/C++ projects.

Thank you! Further progress with CI support for the long list of platforms supported by opam-depext.

Good point about OpenBSD, I am not a regular user of OpenBSD for OCaml work. To support it we would need to find a solution for file system snapshots, is something like ZFS available on OpenBSD, and then we would need a sandboxing implementation, ideally like jails or runc. Of course we’d welcome contributions on ocurrent/obuilder to add OpenBSD.

Our first focus is on supporting the Tier1 platforms for OCaml. Of those we are only missing Windows support, that is currently being worked on, and wider RISC-V support which is mainly blocked by availability of hardware to run builds on.

An initial run of opam-health-check for FreeBSD is available here. The results are quite promising out of a total 3511 packages built on 4.14.1 there are 222 failing to build with a further 472 whose dependencies failed to build.

Opam-health-check is a bulk CI system for testing the entire opam repository. It periodically rebuilds all the packages in opam-repository to check for broken packages. The current iteration on https://check.ci.ocaml.org runs on Linux and tests a range of OCaml versions.

We are working on making the FreeBSD version of checks run more regularly. If you use FreeBSD and would like to help fix broken packages, please checkout the opam definition for failing packages, make it work on FreeBSD and submit the change upstream to opam-repository.

3 Likes

Is there an opportunity to publish the FreeBSD packages, or a dump of the FreeBSD filesystem, so we know what is installed?

For example, I have scripts that searches for sha256sum (Linux), shasum (mac), etc. to compute SHA-256 sums … but the scripts can’t find sha256sum. I suspect there is a good alternative binary I could be using on FreeBSD. Example: https://opam.ci.ocaml.org/github/ocaml/opam-repository/commit/15f15de0cd88f79189ab9ccb4c8ae79af4c03dbc/variant/freebsd,freebsd-ocaml-5.0-amd64,conf-dkml-sys-opam.1

Ordinarily I’d spin up a VirtualBox, but for unrelated technical issues I can’t do that at the moment.

2 Likes

The base images are FreeBSD 13.2 with a minimum of packages (the ansible setup scripts are available on GitHub - ocurrent/freebsd-infra: FreeBSD Infrastructure for OCaml). It should be possible to provide a manifest of what is installed. I am not sure how easily we can export the jails, which is how the base images are implemented, but that might be another option. In theory you could recreate either the ocluster worker or the underlying jail on your own FreeBSD machine.

On that specific sha256sum not available problem. I’ll need to look into it.

it looks to be called simply sha256(1)

1 Like

Thanks all. It was /sbin/sha256 on FreeBSD.

(Don’t sweat about the file listing unless it is super easy)

I wonder if there is, or could easily be, an opam binary for FreeBSD? opam - Install (ocaml.org) / FreeBSD suggests installing from source, and that installs everything (that’s not installed yet) from source.

pkg install ocaml-opam - you have it in your ports tree with pre-build packages

1 Like

Is there a way to send, say, an opam file to a specific CI target, or otherwise access the target OS, so that I don’t spam opam-repository PRs?

One option is to enable https://ocaml.ci.dev on your GitHub repository and it will build against the same set of platforms as the CI on opam-repository. That will build on all platforms, rather than a specific platform. If you want a Linux platform you can replicate the build using docker on the other platforms like MacOS and FreeBSD that is not possible. FreeBSD uses jails and ZFS so it would be possible to provide a script that does the same commands in a jail, GitHub - ocurrent/freebsd-infra: FreeBSD Infrastructure for OCaml details the full setup. It would be a welcome addition to ocaml-ci if someone want to hack on that, I would happily show them how to get started on it.

ocluster does provide a way to submit jobs directly, however it requires a CapnP Capability to do so and we haven’t allowed for general access to the cluster. The Capability provides a wider range of access than is strictly necessary and it needs rate limiting to ensure all applications get fair access to the available compute.

1 Like