A Roadmap for the OCaml Platform - Seeking Your Feedback

Community
1

Dear OCaml Community,

Three years ago, @avsm and the OCaml Platform team presented a roadmap for the OCaml Platform. The vision we had of the Platform in 2020 was one of a seamless editor integration, where users could open their editors and instantly be productive with OCaml.

While there’s still a long road ahead to achieve this vision, we’ve made a lot of progress. The tooling landscape and developer experience with OCaml have changed quite a lot in the past 3 years and we’ve made significant progress on all the tools of the Platform:

In the meantime, things have also changed for the broader OCaml ecosystem. OCaml 5 was released with support for shared memory parallelism and effect handlers; and we saw a new version of OCaml.org going live with a centralised package documentation, a job board, an interactive playground, and more.

Following the three priorities we adopted in 2022, we now want to intensify our efforts on prototyping new workflows for OCaml development. It is time to reflect back on everything we’ve done and lay down a roadmap for the next three years.

In the past few months, we’ve been working on establishing a vision for the future of the OCaml Platform. We reviewed as much community feedback as possible, organised discussions with the maintainers of the Platform tools, met with industrial users of OCaml to understand their needs and pain points, and we collectively thought of what the ideal developer experience with OCaml ought to be.

We want the OCaml Platform to be driven by the community. So today, I’m excited to share the first parts of the roadmap to start collecting feedback and iterating on them.

You can see the work-in-progress roadmap on GitHub. The repository currently contains two documents:

  • Design Principles: The principles that will guide the desired OCaml developer experience.
  • Users: The different user personas within the OCaml ecosystem serve as a guide for the roadmap, ensuring that it addresses the unique requirements of each archetype.

Feel free to read through them and share your thoughts, in particular:

  • Do you agree with the Principles of the Platform?
  • Do you see yourself represented in the Users?

Once we’ve iterated on the feedback and are ready to adopt these high-level guidelines, we’ll refine the developer workflows we aim to build in the coming years. Similarly to the first sections, we’ll post them on Discuss to discuss them with the community and ultimately adopt a first version of the Platform roadmap.

I’m looking forward to everyone’s feedback and building this roadmap together!

37 Likes
2

By providing each tool with sensible default configurations, we can minimise the number of choices that users have to make and allow them to quickly start writing and experimenting with OCaml code

This is exactly what I was going to recommend–decision fatigue is a workflow killer, we need to minimize the number of decisions that need to be made by default. Glad to see this.

13 Likes
4

The Design Principles look solid to me.

(R3: Evolution) Tools evolve without breaking existing projects

This is essential but I don’t think it can be done perfectly at a reasonable cost. Acceptable breaking changes arise now and then despite best efforts. Ways to minimize the impact of breaking changes include:

  • marking projects as experimental;
  • showing deprecation messages when building a project in developer mode i.e. make sure the developer has a chance to update their code without bothering the end user who just wants to install the thing;
  • incrementing major versions or renaming the project to signal breaking changes. These are not solutions but mitigate the damage.

So, maybe I’d indicate that we must strive to minimize breaking changes rather than offering a 100% guarantee of no breakage and then getting blamed for not delivering it. Prefer “underpromise and exceed expectations”.

(R4: Sharing) Sharing code and docs is straightforward and secure

I would like to see efforts toward making the distribution of compiled code for multiple platforms easy. I’m thinking of cross-compilation and static guarantees that libraries and executables will work on other platforms without the need to test the code on these platforms. This would save manpower but also computational resources, benefitting R8 (sustainability).

6 Likes
5

Regarding Users:

I’m thinking of the following persona: the reluctant user

Maybe it’s in fact multiple personas. Here are the real-world characters I’m thinking of:

  • A user of an open-source OCaml application who wants to fix a bug and add a feature they need.
  • A programmer who doesn’t program in OCaml but needs to build the OCaml application that their colleagues develop because it’s part of the web service the company is developing.
  • A student who’s forced to use OCaml. This a variety of Newcomers.

My guess is that there are at least as many of these “reluctant users” as there are full-time OCaml programmers. This makes a lot of people who will form an opinion of OCaml based on actual experience.

12 Likes
6

Just want to say thank you to everyone that has been working on ocaml and the ocaml ecosystem to make it increasingly accessible and useful for the rest of us!

5 Likes
7

As someone that’s dabbled in writing (for personal use, a tiny bit of) tooling, I wonder if there’s a user persona, the “third-party tooling author,” that would benefit from more specified, tool-agnostic interfaces for communicating OCaml project information between tools. I’m thinking of things like the C/C++ compile_commands.json or Python’s specification efforts around setuptools functionality, migration to pyproject.toml etc – or just moving forward on the library linking proposal to simplify the notion of packages/libraries as implemented by dune/opam/ocamlfind.

The impression that I get is that the ecosystem is trending towards dune gluing every other tool together (there are clearly good reasons for this, and it’s been a huge improvement as a user, etc.) but imho it’s hard to square this with tools being “not overly dependent on each other” in R6. Does this section mean just that tools should have a best-effort standalone workflow (e.g. ocaml-lsp-server running with reduced functionality without a dune watch)?

Maybe that kind of specification is outside the scope of the OCaml platform project, low priority, or a explicitly non-goal, which would be understandable – it’s difficult and only really serves a small subset of users. Just curious about how the platform team sees this beyond what’s already written in the roadmap.

2 Likes
8

From the perspective of dune and ocamllsp, I welcome efforts for interoperability and would implement whatever standards the community converges upon.

Even so, I wouldn’t take the initiative to push any standards myself. I have little personal use for such standards because I just use the most popular tools that already interoperate quite well (no surprise there).

I can’t speak for the OCaml platform, so I can’t say it’s a non-goal, but I’d imagine such an effort would need to be led by people who are more motivated.

1 Like
9

In the name of R8 (Sustainability) I am always amazed by the difference between local compilation times and resources used by CI pipelines. It feels almost irresponsible to use the Github CI only to throw the result away. But maybe the feeling is wrong - it could be a first step to estimate the resource consumption (Energy) that a CI run takes and making it visible. I understand the problem that my local environment already has the libraries that the CI will need. But it’s typical to iterate on a branch and only small parts change. Could this be improved?

9 Likes
10

Indeed! Recently at LexiFi we switched away from cloud providers to hosted machines for our CI precisely because of this issue. Now, our builds are much faster and our costs lower. And of course it is much less wasteful in terms of energy, as we can reuse compilation artifacts, git clones, etc.

Cheers,
Nicolas

4 Likes
11

Definitely don’t break the ocamlopt / ocamldep / Makefile workflow. Not every project is all-OCaml code, and OCaml code must be buildable in the existing (and vastly more widely used) build systems.

12 Likes
12

So much wisdom and simplicity in only two sentences!
Let’s hope this obvious requirement will be taken into account in some [to-be-done] specifications for Ocaml’s core tooling (package&build).
But maybe I missed something somewhere in some documentation?

13

Last I checked the latest github action would build an entire compiler. Sure it’ll be cached in the repository 's action cache (for a while), but there are multiple repositories on github and compiling the same compiler over and over doesn’t really help (using an OS package where available might be better, or some global precompiled cache?).

Sharing a compiler and dune build cache at an organization (or individual user) level might be nice (sharing dune cache globally might have some security/reliablity concerns if the builds aren’t fully deterministic/reproducible). Dune cache works wonderfully with opam packages locally.

2 Likes
14

I think there should be another user category, maybe called “Modeler” or “Scientist”? The first three categories (App developer, Lib author, Distribution manager) seem to be for folks who write or maintain software that will (hopefully) be used for a long time.

People who write scientific models or do data analysis often write software for short-term use, perhaps reusing it later, but then with significant modifications for each project. OCaml can be a good language for this, especially with the availability of libraries like Owl. (One of the ideas motivating Owl, I think, was to make it easier for scientists of various kinds to move to OCaml from Python.)

This kind of user is kind of like App Developer, but most of the goals listed for that category are not relevant.

On the other hand, a Modeler/Scientist is obviously different from Newcomer (except when the modeler is a newcomer).

I suppose that sometimes Modeler/Scientist might overlap with Teacher when there is repeated need to train new lab members on OCaml.

Many of these people are academics, but not all are. Companies and government agencies have people doing this kind of research, too.

Neither “Modeler” nor “Scientist” seems like an ideal name for a category, but either might be good enough. Maybe someone else has a better idea. Some sorts of data analysis aren’t exactly modeling, but could still involve one-off or short-term projects. “Scientist” is pretty general, but there are people who do modeling and data analysis in the humanities, so they don’t think of themselves as scientists. (The first time I heard about Haskell when I came back to coding after years away from it, was when I learned that Haskell was being used to analyze relationships between Icelandic folk sagas.) However, it’s still relatively unusual for humanities folk to write code for any purpose.

[I’m in the humanities, but I use code for science-ey things—theoretical models related to scientific domains. I know several other people in my field who write code for their research, but most people in the category I describe above would be in the sciences.]

4 Likes
15

I would tend to call that category “Researcher”.
As you are searching, you need to wirte many programs, which are not necessarily useful for very long.
But of course, quite often you might still be able to define some libraries from your code that might be useful for a long time.
And you still care a lot about the stability of your dependencies, because it is very useful if you want to reuse code for a new project after not having touched it for a few years.

1 Like
16

That’s a good suggestion. A drawback might be that the “researcher” could also apply to someone developing a computer language, i.e. something large that hopefully won’t be abandoned, though it keeps changing.

Yes, that fits what I was thinking.

I agree that that could be an outcome, but often that wouldn’t happen for the kind of case that I have in mind.

Yes–this is an important point.

1 Like
17

The CI team at Tarides is in the process of collecting energy usage (carbon cost) of the build infrastructure used for the Opam Repo CI, OCaml CI, Docker Base Image Builder and Opam Health Check. We will post results here when we’ve compiled the numbers and written up a report on it. @patricoferris and @avsm have been leading that effort along with Mark Elvers and myself.

6 Likes
18

OCaml CI provides a more efficient solution to GitHub Actions, along with supporting more platforms like non-x86_64 linux platforms and macOS (arm64 and x86_64). The getting started guide should be enough to get you started. Feel free to get in touch if you need more support or details on how it works.

3 Likes
19

I just want typeclasses tbh. Seriously, typeclasses are insanely useful in terms of extending behavior for a type. More so than the module system. For instance, maybe you have an hlist type in OCaml and then you want to process the hlist in someway. Often, the best way to process the hlist is using typeclassses. You may have to manually build your processor function in some way.

20

Users section: add Tool Developer

1 Like