This is a public announcement that we are experiencing a maintenance bottleneck in the development of the OCaml compiler distribution (the github/ocaml/ocaml repository).
Our development process naturally generates a fair amount of maintenance work to, among other things, discuss and integrate proposed patches, fix bugs, and react to feature requests. We don’t have enough people doing this maintenance work; currently the vast majority of this work is being done by about 5 people: David Allsopp, Florian Angeletti, Nicolás Ojeda Bär, Xavier Leroy, and myself.
Despair not! Bug fixes tend to be prioritized and handled quickly; I believe that the OCaml releases remain of satisfying quality. But other aspects are affected negatively, for example:
- our ability to react to proposed changes in a timely manner,
- the experience of people trying to contribute to the compiler codebase,
- various potential improvements that get stalled by lack of manpower to work on them.
The OCaml compiler distribution moved to Github in January 2014. Since then, maintainers have been constantly complaining that there are more people willing to submit changes/PRs than people willing to review them, creating a bottleneck on the reviewing side. (We point this out in the first section of our CONTRIBUTING.md document.)
But the effort to upstream Multicore OCaml has unfortunately made the situation worse, for at least two reasons:
Integrating the completely new Multicore runtime required a lot of review, integration and documentation work. We onboarded experienced Multicore developers as upstream maintainers and this helped smooth out the process, but we have still been less available with other maintenance tasks that piled up in the meantime.
The sequential glaciation indirectly reduced the maintenance workforce. In November 2021 we stopped merging non-multicore-related features in the development version; as a result, various maintainers and heavy contributors moved away from working on the main development branch, to do their experiments in separate repositories (which is completely fine), and also more or less stopped following issues and performing maintenance on the main branch (which aggravated the maintenance issue).
Now that OCaml 5 has been released, many contributors will be coming back with many exciting change proposals to upstream. At the same time, our users are playing with Multicore features and will soon find countless bugs to fix, limitations to lift, etc. It’s not easy to play OCaml maintainer right now.
Heavy contributors, in particular core developers but not only, should be expected to participate to this collective maintenance effort. We are having discussions right now about our expectations.
In my personal opinion, anyone who dedicates a substantial portion of their time to working on code intended for eventual upstreaming should dedicate a fraction of this time to collective maintenance of the upstream development trunk. (10%? 20%? Something like this.) This is the most healthy way to ensure that the volume of maintenance work scales with the volume of submissions. (If you are paid by someone to work on the compiler, please make sure that your pay also covers this maintenance fraction.)
Occasional contributors who would like to help with OCaml development should also consider whether they can help with this. (No pressure!) We have several instances of people helping with code reviews, triaging, helping make decisions on design questions etc. (I remember nice contributions in this direction from Daniel Bünzli, Gabriel Radanne, Nathanaëlle Courant, Favonia, Guillaume Munch-Maccagnoni and Kate for example.)
If you interact with the compiler distribution as a software project, please be mindful of the maintenance load that you generate.
If you send a Pull Request, make sure that its purpose/justification is explained very clearly, that it is easy to review; that the benefits of the change (explain those clearly) outweigh the long-term and also the short-term costs of integrating it.
Similarly for feature requests or enhancement proposals: now is the time to focus on the uncontroversial things that are clear improvements, and to justify, explain them very clearly.
It may not be immediately clear to people what “contributing maintenance work” means concretely. Right now I see three obvious approaches.
Subscribe to github/ocaml/ocaml notifications and jump in when you want.
Look at our issues (258 open as I write this) and see whether you think can help. Maybe some are out of date / irrelevant and could be closed – say it. Maybe some bugs could be fixed, or some enhancement requests could be fulfilled. If you can, give it a try. It’s best to start with issues where the desired outcome is consensual (a clear bug to be fixed, with no immediate downside; a small interface improvement that does not introduce much complexity and is well-justified; etc.), rather than work on some weird syntax proposal that will in turn require ample discussion and may be turned down in the end. (If you find a wonky proposal that failed to gather consensus and probably never will, it’s actually helpful to suggest closing the issue.)
Look at our pull requests (246 open as I write this) and try to see whether you can help. Again, it’s best to focus on PRs where there is a clear motivation/need. Look at the code, feel free to ask questions on things you don’t understand or comment on aspects you don’t like so much. If the PR is stale, maybe it should be rebased (would you like to give it a try?), or there isn’t much that can be reused and it could be closed – feel free to say so.
We have received the feedback that some people are still unsure what to do. In the upcoming weeks (probably in January) we will have more discussions about how to organize maintenance, to find more focused processes that encourage people to contribute in this way. I don’t think that there is a silver bullet, a magic process that will make it much easier, so I would encourage anyone interested to first try those three basic approaches above and see if one works for them.
In my experience people often self-censor and do not try to react to PRs or issues that are not in their area of expertise. But most of the compiler codebase is in only a very few people’s area of expertise, the rest of us (myself included) just make do with their imperfect understanding and try to help anyway. Do not hesitate to walk into issues outside of your comfort zone, it is a great way to learn about the compiler distribution codebase.