As we’ve been working on the various OCaml Platform tools such as opam, dune, odoc, merlin, ocamlformat and so on, a common question we have is the range of OCaml compiler versions that need to be supported. Often, the minimum OCaml compiler version is the minimum one that is packaged up in common OCaml distributions – for example, 4.01.0 has just last week rotated out of support (it was the default OCaml compiler in Ubuntu 14.04 which has just reached EOL on Apr 30th 2019). This was a compiler we released over five and a half years ago, so you can hopefully appreciate the depth of the support burden this is creating for the tooling developers.
In recent years, we’ve been putting significant effort into making it easier to separate the tasks of upgrading your codebases to newer versions of the compiler, by making it easier to cherry pick some features to older compilers. For example:
stdlib-shims provides newer modules in the stdlib for older compilers to help you migrate codebases
the ppx ecosystem has abstraction layers such as ppxlib and omp to cushion the AST changes for preprocessors
dune supports future syntax since 1.8 which applies shiny new syntax (e.g. let+) to older compilers, again to make it easier to separate the adoption of new features from upgrading compiler versions.
In the longer term, for tools such as dune we would really like to be able to support (e.g.) just the last three major revisions of the compiler, and perhaps have one version that is in something like Debian Stable as an LTS version that we support for longer. The data we need from you, and especially from users who use old versions of the compiler, is what is holding you back from upgrading.
Some examples of why some communities hold back have been:
Bucklescript has long used 4.02 as its base, although there are newer versions for 4.06 now.
MirageOS has had to manually upgrade some cross-compiled “freestanding” libraries on every OCaml release. The lag here is dropping as the compiler gains better cross-compilation support steadily.
So if you are a user (large- or small-scale, it doesnt matter), and are using a version of the compiler that is OCaml 4.02.3 or lower, we’d love to hear the reasons why you are still there. Your input will help us figure out if there is missing tooling we need to develop from the compiler side to aid with transition, or otherwise improve documentation and processes.
Note that we are not planning any immediate changes to dune or opam. This is a data gathering exercise in preparation for having a support policy for these tools, and one in which we could really use your input. If you want to drop myself or @jeremiedimino a private line for a non-public discussion, then please also feel free to do that.
Does merlin have support for this future syntax system? If not, I feel it is not really helping.
At Ahrefs we are between the latest version and 2 versions late. It depends on the rythme of the releases. As we don’t always have the time to test deeply the new version of the compiler. But it also depends on the stability of the compiler. We usually wait a few months after a release to be sure that most of the bugs are fixed.
future_syntax is targetted at people who develop using a recent compiler but want to make their software available to people using an older compiler. If you develop using an older compiler, then you are effectively not going to get a very good experience. Thinking about it, maybe the name is not so clear and should be something like syntax_shims instead
Releasing support for let+ before 4.08 was even released was a bit premature and effectively the whole ecosystem is not yet ready for it. However, we were really eager to have a let-syntax in dune because we use a lot of applicatives and monads. And we can’t use ppx in dune. So overall we were ready to give up merlin for a little while.
Once 4.08 has settled in and 4.08 is what most people use for their day-to-day development, future_syntax will no longer get in the way since it will be a no-op.
You are right that the lack of integration with ppxlib is a deal breaker if you use ppxlib. I have plans to develop future_syntax and integrate it with ppxlib. However, it is a non-trivial amount of work and we already have a packed todo-list, so it will take a bit of time.
Just to make things clear the only thing that this package provides for now is the Stdlib module to handle the deprecation of Pervasives.
After a discussion with @jeremiedimino we decided not to provide new modules (e.g. those introduced in 4.08) for now in order to entice the eco-system to move on to later versions of the compiler.
For the record, I’m trying to update from 4.05 to 4.07.1 and our build (see Test / Production module injection) fails to build in 4.06 already. I’m working it but this pattern prevents us to upgrade at the moment
In my experience the biggest issues with upgrading versions have been ppx_deriving (or its other plugins) not being up-to-date for the newest OCaml release (and even when master has a fix there is no release for a long time as happened with this commit).
So usually we are either on the current version or are one version behind. We don’t tend to use beta versions of the compiler since that basically guarantees that there will be some dependency that will prevent us from compiling.
I’m not held back. Orsetto dropped its dependencies on everything that isn’t yet ready for OCaml 4.08, and I’m adopting a new rule that PPX rewriting will never be required with whatever is the latest release of the compiler.
This time another big obstacle to upgrades is the many new deprecations that prevent code that uses Pervasives (or Yojson.t in a similar vein in the ecosystem) that kills normal dune projects in dev. So in my experience so far, I have to do some work for my librairies to compile cleany in dev mode on both 4.08 and older versions (using dune to generate shims, etc.).
The lab machines we use for teaching are based on Debian stable which means that the default OCaml compiler is currently 4.02.3 (soon to become 4.05.0, if I understand correctly). Installing opam to use a more recent version requires (1) an additional technical step and risk of confusion for beginners who sometimes already express a preference for a more familiar programming language and (2) a certain amount of disk space in student home directories mounted from a network share.
Maybe this does not matter. For such introductory courses, there is probably little need to use the latest compiler and related packages. Do other people have similar issues when using OCaml for teaching?
Unrelated to OCaml, but during teaching reverse engineering with radare2, Debian-based distributions are always a big pain, since always there is a someone who uses ancient version from the packages. Of course you can’t compare radare2 and OCaml infrastructure due to the sheer size difference. But at the same time, ignoring huge advancements in the language itself, tools and libraries will shy students away even further.
Yes, I could probably convince the admins to make that change. It’s not an insurmountable problem. I just wanted to add this consideration to the list and I wondered what other people thought. (I also know that maintaining many versions of software is time-consuming, thankless, and error prone!)
We’re switching to teaching in OCaml for our first year Foundations of Computer Science course in Cambridge, and I am putting together an OCaml/Jupyter Hub environment for our students. This setup means that there is no client-side software required by the students to start with, and then they can subsequently (after a few weeks of the basics) get a local environment running if the notebook environment isn’t powerful enough. Mostly though, we avoid any dependency on package management for student teaching and provide an explicit repository with all the build/exercise instructions.
I’ll share details of our Jupyter setup in a separate thread in a month or so once the quirks are ironed out. @jonludlam and I would like to get it to be a jsoo-based purely-client-side experience, but that may be out of scope for this year’s teaching. This used to with with iocamljs, but the Jupyter codebase has diverged quite a bit since @evilkidder wrote iocamljs sadly!
@avsm Have you considered using learn-ocaml? The Paris 7 OCaml course switched to it this year (I was TAing), and AFAICT it provides most of the jsoo-based purely-client-side experience you mentioned.
Additionally, having more teaching material written for the platform would be really nice for teaching OCaml even outside of this course.
