Hot on the heels of the 2022 User Survey results, I thought it a good opportunity to look back over the 2020 results (and summary) and look at some of the highlights of what the ocaml.org team of contributors did in the past year for our ecosystem, and to gather some inspiration on what to focus on next in 2023. As always, these recaps from me are my personal distillation of our community’s work, with me just reporting as best I can. Errors and omissions are mine, and credit to the individual hardworking maintainers!
At the start of 2022, I communicated three priorities to the OCaml.org maintainer teams when asked about what to work on, based on the work of the core development team and the feedback from the 2020 user survey:
- help the OCaml 5 release be a success
- launch a new OCaml.org web presence with documentation
- prototype new workflows for OCaml development
Help the OCaml 5 release be a success
The first thing I found notable (and reassuring) in the 2022 survey results is what wasn’t mentioned in the user responses: instability with the core system. We began 2022 drunk with relief from the multicore PR being merged, and began the year-long 5.0.0 release process. In one of the core developer meetings early in the year, I requested that the 5.0.0 release would be restricted to just the multicore runtime and effects features, since so much had changed there that we would have our hands full. This was promptly ignored and followed by a surge of PRs removing lots of legacy cruft that had built up over the course of the 4.x series. A recipe for release management disaster!
I’m glad to say that I was wrong, and (as you’ll see from the infrastructure report below) that the collective 5.0.0 release effort has been one of the most impressive I’ve witnessed. The core team signalled deprecations clearly, and the various tooling teams (such as opam and dune) in the ecosystem performed lots of differential builds and incremental releases to remove vestigial deprecated fragments that now broke builds. This was then followed by an engaged community releasing their various dependencies to the mainline opam repository, all in good time for the 5.0 release candidates to be cut and be usable.
We are, of course, still really early in the lifetime of the OCaml 5.x series, and some serious breakages may yet lurk and only be discovered as our users migrate. But we are at a point now where experimentation, prototyping and migration can be done in a controlled way across both OCaml 4.x and 5.x, so let’s pat ourselves on the backs for a moment about a job well done before moving on. I’ll ninja-edit this post if next year’s user survey is full of complaints about OCaml 5 
Launch a new OCaml.org web presence with documentation
The 2020 user surveys made it crystal clear that we needed to improve the state of art with documentation and OCaml. Accordingly, we started work in 2021 on a new site, previewed it in early 2022 and launched in April 2022.
This new website preserved older links (by redirecting them to an archived v2.ocaml.org) and provided a brand new centralised documentation site with package search and incremental rebuilds to ensure new packages get up there in a timely fashion. This was a complex task behind the scenes, since it requires ongoing bulk package builds of every version of every package in the opam repository, with consistent cross-linking.
It’s still by no means perfect, with some work needed on rendering glitches, missing sections, and the overall information architecture of how to present so much information to a range of users (beginner to advanced), but it’s a really solid foundation to work from (unlike the previous website, which was really showing its age). This year’s user survey continues to emphasise the need for advancing documentation, so I hope to see more contributions to the new website to ensure the content continues to improve. And of course, look to your own published libraries and ensure that your odoc markup is as good as you can make it.
Prototype new workflows for OCaml development
The other hot topic in 2022 was for us to figure out how to integrate better with modern, stateless workflows for code development. This is a complex topic for tool maintainers to work on, since we must also preserve backwards compatibility with existing workflows (witness the very high percentage of users in this year’s survey that use the opam cli as their primary mechanism of interaction). We also had a decent idea of who the various sorts of users are from discussion in 2019 with application developers, library authors and OS maintainers.
There have been a number of prototypes built this year to experiment with new workflows, but most of the effort from core maintainers has gone into the earlier priorities (releasing OCaml 5, especially). The prevalence of dune (>91% in the 2022 survey) means that one simple stateless workflow is to have all the source code available in one monorepo, and perform a dune build. This works because dune can scan all the dune files in the repo and build them in one pass. In this workflow, opam can be optionally used to assemble the source code (see the opam-monorepo) plugin, but our Nix-loving friends also have their own alternative mechanisms (1, 2, 3). While some projects like MirageOS and Real World OCaml are using these workflows, they are still maturing. Now that OCaml 5.0 is out and the new website is live, I hope to see more production quality workflows emerging in 2023.
There is also some concern that dune shouldn’t be a hard requirement on any workflow. This requirement has been successfully preserved to date, but is getting increasingly difficult to reconcile with the demand for a more opinionated, beginner-friendly workflow. With opam, our architectural answer to this is to separate the opam file format from the opam CLI, and make it easier to interpret opam repositories via external tools. The same discipline should work for dune files with alternative build systems.
OCaml.org and our decentralised future in 2023
How do we – as a community – figure out what will work for new workflows? That’s my segway into what I want to hear your thoughts on for 2023: how to incrementally improve community communication.
-
discuss.ocaml.org (Discussion forum): I setup this forum back in 2017 in response to a user request, and it has been a successful experiment. The number of OCaml users who report interacting via this forum has increased percentage-wise from the 2020 to 2022 results. I am also pleased that a small moderation team has been sufficient to deal with spam. Traffic-wise, we have had to upgrade the hosting capacity several times in the past 5 years as demand rises, and there has been a mild surge in page-views towards the end of 2022 with the release of OCaml 5.0.0.
I am still regretful that I had to sunset the mailing list service, but it will hopefully be back in 2023, especially if we find a volunteer to help configure and maintain it.
-
watch.ocaml.org (Video sharing): The Peertube-based service began in the 2020 lockdowns when we shifted our workshops online. Since then, it has been a resilient and useful resource to host videos about OCaml related topics. There are some advantages to hosting our own videos: they can be permanently archived and linked to, and we can integrate well with other “Fediverse”-based services (more on this later).
What I’d like to see in 2023 is more content being backfilled onto this site, so that we can have all the videos from the last few decades of OCaml conferences and workshops in one place! To that end, we will promote the service to non-beta status soon. It is a little tricky to use Peertube with multiple users (still involves password sharing), so we’re figuring it out as we go along and before finishing the promotion to production status. Please do leave comments and thoughts on that issue or here.
-
opam.ocaml.org (Package management): The surveys confirm that opam remains the dominant mechanism of installing and accessing the OCaml package ecosystem. In addition to regular releases of the opam tool itself, the backend infrastructure has been upgraded significantly so that the package archive should be more available and secure, and easier to mirror onto global CDNs in 2023 and also integrate with other software security supply chain software.
Alongside serving the package archives themselves, we maintain a significant multi-architecture cluster of machines that perform the bulk builds to ensure the health and integrity of the opam repository (the curated package database). These machines comprise of x86, ARM, PowerPC and RISC-V machines (yes, we did finally get rackmounted RISC-V boards this year!). The machines are variously hosted at the Cambridge Computer Laboratory, Inria, Scaleway and Equinix, and we are sunsetting our use of AWS for cost reasons. Individual machines have been generously funded by IBM, Tarides, Jane Street, the Works on ARM program.
The software driving this cluster continues to grow, with support for Windows and macOS builds going live in 2022. These are not yet hooked up to the live opam-repository-ci, but will hopefully be back in 2023. This marks our migration away from hosted CI services such as Travis and AppVeyor, and the backing infrastructure is open source and possible to deploy for yourself (e.g. in an industrial context).
-
www.ocaml.org (Website and Docs): I’ve talked extensively about the new website earlier, but I would like to emphasise the importance of receiving external contributions to the content of the website. The repository is open for PRs, and it has been a little quiet in the latter half of 2022 outside of a small maintainer team. If you’d like to get involved, then please feel free to open an issue and discussing your plans, or signal any blockers you encountered. Gabriel has already noted bottlenecks in the core OCaml distribution, and a similar story is playing out in the wider ocaml.org ecosystem. We need your contributions.
-
git.ocaml.org (not launched): a service that we have considered since 2019 and did not launch is a git mirror, or an alternative way of procuring OCaml ecosystem source code than from GitHub. There have been a small but steady stream of requests for this, with several motivations: availability (GitHub is a central point of failure), security (replicating ecosystem git branches is sound and secure practise), and privacy. The core OCaml team is firmly committed to GitHub at present, but launching a read-only mirror is in scope for 2023 if a maintainer is willing to step forward and survey available solutions (ideally not as heavy-weight as GitLab) for mirroring scripts. Once we have a robust read-only git mirror, we can begin to consider how to accept patch contributions (particularly to the opam repository) via email or other mechanisms, but no promises until we reach the first read-only milestone.
I’d really like to hear from industrial users who have stronger requirements for secure software supply chains here as well. I participated in a White House summit on software security earlier in the year, and it is clear that this is going to be an important topic for OCaml to keep up with in 2023, especially with our role in the formal verification ecosystem.
Should ocaml.org host more Fediverse services?
I’ve mentioned the Fediverse earlier, and could use a wider set of opinions. One of my concerns from the user survey is how much interaction happens on closed synchronous mediums such as Slack or Discord. I’m not against such platforms (1-1 and small team private chats are not replaceable), but there’s currently no way to then promote knowledge gathered from the closed systems into the public commons, where they benefit newcomers. And more recently, there has been drama around centralised services such as Twitter that throws its permanence into question. Our user survey indicated positive vibes about our current interactions, and I of course want to ensure any of our technical platform choices support this healthy growth.
The Fediverse itself is a fairly loosely arranged set of services that interoperate via two main protocols: ActivityPub for web-based services, and the Matrix chat protocol for encrypted 1:1 and group encrypted communications. Some potential services we could host are:
-
Mastodon is a micro-blogging platform which can be run on several domains. It exposes feeds via RSS as well as several open source clients. Fediverse clients can interoperate: a “boost” on a watch.ocaml.org video can be expressed in a Mastodon timeline, and a “favourite” of a video in Mastodon will increment the “like” counter on the videopage.
For ocaml.org, a simple service to run would be an activity feed (e.g. from the opam repository and the website blog) that would publish “Toots” and make them searchable across the wider network, but not allow user registration. This would sidestep the need for moderation and selection of blocklists. However, the hard work of the code of conduct team means that we have the basis for user registration provision as well (especially by using discuss.ocaml.org as a single-sign on backend). Opinions welcome – by default, I will select the conservative option of adding read-only ActivityPub to ocaml.org directly, as we do not currently have the moderation resources for a full Mastodon instance, and it can be upgraded at a later stage.
-
Matrix chat is already sitting alongside the venerable IRC as an open alternative. One of the nicest features of Matrix is that multiple servers can publish the same room, and the domain name is simply a namespace which can be replicated. For example, we have a chat room for the Eio library that is published as
#eio:roscidus.com(@talex5’s Matrix server) and#eio:recoil.org(my own). In the future, this could also be#eio:ocaml.orgsimply by publishing it as such. The value in an ocaml.org Matrix server is thus to act as a conveniently searchable directory, with the room contents being replicated in various other homeservers for availability.
There are still significant downsides to using the Fediverse as opposed to centralised services. Usability is patchy, availability can be variable as some servers go down while others remain, and moderation is never a fully solved problem that requires distributed maintenance of blocklists. We’ll need to be open to some experimentation and failures if we step further in this direction, but it is promising.
In this spirit of experimentation, the ocaml.org changes are all now being recorded on a blog (infra.ocaml.org, and at Issues · ocaml/infrastructure · GitHub), and I’ll begin discussions with ecosystem maintainers about how they feel about moving to slightly more open platforms. In the meanwhile, nothing stops independent initiatives. If you feel the urge to continue developing ActivityPub bindings (begun by Kate at a MirageOS retreat but in need of a new maintainer) or bringing the OCaml Matrix implementation to production quality, now would be an excellent time to do so!
None of these Fediverse services are intended to replace the excellent roundups often seen on these forums (such as Gabriel’s compiler newsletters) and via the Caml weekly news. If in doubt, feel free to step up with your own projects and post about them regularly here!
Finally, the absolute highlight of OCaml in 2022 for me has been the continued support for Outreachy from our maintainers (see posts and even a video roundup). This effort, along with the code of conduct process concluding, highlights the enthusiasm for bringing newcomers into our world. I encourage the senior members of our community to try to participate (even if just once), and get in touch with myself or the OCSF if the bottleneck is something we can help address (like funding).
I’ve never been more excited about the future of OCaml than I am heading into 2023; a whole new realm of systems programming has opened up with the release of multicore and effects, and it’s just really fun and a privilege being along for the ride with such talented collaborators. Happy new year everyone! (I’m currently snowed in somewhere very remote, and am only 50% sure this will make it through to the forum. Please please, don’t give me a HTTP error when I click on ‘create topic’) 
