A Roadmap for the OCaml Platform - Seeking Your Feedback

Community
21

Thanks everyone for the great feedback!

I haven’t kept this thread alive as much as I would have liked (and should have), but we’ve continued to work on the Platform roadmap and I’ve now taken the feedback and updated the Users and Principles.

I actually rewrote the Principles entirely to be more focused on the guidelines that the OCaml Platform offers on user experience. The previous version was a mix of general principles and user experience guidelines. While the former is still useful to have, there was a lot more to say than what was in the document, so I found it useful to have a document that focuses on the user experience part exclusively.

That meant removing some of the principles, like openness and sustainability for now, and I plan to come back to these at a later stage, independently of the effort on establishing a roadmap for the Platform.

As recommended, I also added a Researcher persona for users who work on short-lived projects, often modelling or scientific tasks.

With the goal of adopting the documents rapidly, I have opened a PR to add the two sections on OCaml.org. Feedback on this thread is still very much appreciated, but also feel free to make suggestions on the Pull Request as well.

I am doing a last pass of review on the work-in-progress roadmap and will be sharing it here for feedback in the coming week.

22

I think that would be a subset of “Application Developer”. Do you see them as being different, or maybe there’s more to say in the Application Developer persona?

23

Two drive-by comments after reading principles.md and users.md.

users.md has a description of researcher that felt a bit odd to me:

Uses OCaml for short-term projects, often in scientific modeling or data analysis

The rest of the paragraph is written from the perspective of research involving scientific computing, data analysis and/or machine learning.

This is fine, but I know many people who are using OCaml in their research, and none of them are doing any of that. Well, almost none: I also know some people working on these fields, and I count the number of those using OCaml on the fingers of a hand. The researchers using OCaml that I know of are writing program analyzers, proof assistants, compilers, type systems, etc. This is a lively research community where OCaml and Haskell (and maybe soon Rust) are the dominant implementation languages.

I think this is a problem with this section. I don’t know how it should be fixed. The researchers I know using OCaml are probably not very different from your “application developpers” profile. (They probably don’t care about the “Security and Compliance” aspect of U1, but most application developers don’t either.) Maybe instead of “Researcher” the profile that you have in mind is “Data scientist” or “Using OCaml for experiments involving numbers”.

In the principles.md section, " (P5) Tools are independent, yet unified", it reads as if tools should know about each other, and in particular as if the umbrella tools providing a CLI interface and an editor interface to other tools should know about the other tools.

How does adding a new programming tool to the mix work in this scenario? Say I implement a cool new parser generator, how would people integrate it in their development environment in practice?

(This is also a problem that I perceive with Dune today: writing custom rules to support new tools is unwieldy. For example, using Menhir’s support for .messages file from Dune is difficult, and the only way for Menhir people to get a nice experience would be to contribute support for this within Dune upstream.)

Is some sort of auto-discoverability of other tools planned?

6 Likes
24

Thanks @gasche! I’m answering your questions on P5 only while I rework U6 based on your comments.

How does adding a new programming tool to the mix work in this scenario? Say I implement a cool new parser generator, how would people integrate it into their development environment in practice?

There’s nothing formalised at the moment. Something that has been happening organically is that tools first get integrated into Dune as “experimental”. From Dune’s perspective, that means that they are opt-in. For instance, Dune integrates with mdx, but you still need to add (using mdx 0.3) in your dune-project.

Then when the tools and their integration reach the necessary maturity level, they are promoted and users don’t need to opt-in to get the integration. That’s what happened with ocamlformat, for which you had to set (formatting enabled) until Dune 3.0.

The Platform also has the notion of maturity with its lifecycle: there are Incubated tools, and Active ones, with the latter being tools that are stable and ready for wide adoption. We could imagine recommending that Dune integrates with incubated platform tools as experimental, and promotes them when they reach the Active stage in the Platform.

(This is also a problem that I perceive with Dune today: writing custom rules to support new tools is unwieldy. For example, using Menhir’s support for .messages file from Dune is difficult, and the only way for Menhir people to get a nice experience would be to contribute support for this within Dune upstream.)

I’m aware of discussions to implement a plug-in system in Dune to solve this specifically. I don’t think there’s anything planned yet, but as the Platform starts incubating more tools and there is an increasing need for integrations with Dune, it seems that easing the integration process is going to become more and more important.

Is some sort of auto-discoverability of other tools planned?

I’m not sure what you have in mind by “auto-discovering” tools, but one thing that was discussed is that with Dune supporting package management, it could automate the installation of its dependencies when they are missing. For instance, running dune fmt could install ocamlformat in the background if Dune can’t find it. Again, no plan yet as far as I know, but I think that once Dune 4 is out, we’ll see a lot of opportunities for improving existing workflows by leveraging package management.

25

If I write a new parser generator:

  1. I don’t want to have to send an upstream PR to implement support in Dune. (This is too early, I don’t know if my project will work out.)
  2. I certainly don’t want to ask that my project be added as an incubated tool in the OCaml Platform. (At this point there is only one user, me.)
  3. I still would like prospective adopters to be able to give my tool a try without having to copy-paste obscure rules for each invocation of the tool in their project.

I’m aware of discussions to implement a plug-in system in Dune to solve this specifically.

Indeed, I participated to a retreat to discuss this topic in autumn 2017. To my knowledge nothing has been done since.

Note that I am not trying to suggest that solving this problem should be specifically the responsibility of dune. The “principles” document does not mention any one tool specifically, and some of its design principles are not currently a reality, they are more forward looking. You propose the principles that tools should be “unified”, and I point out a difficulty that I see with going in this direction: there seems to be a tension between “existing tools are unified” and “it is easy to integrate new tools in my development environment”. I think this may be worth thinking about in principle, independently of dune.

I’m not sure what you have in mind by “auto-discovering” tools.

Suppose I am implementing a new linter for OCaml code. If the platform vision is that all tools should be usable directly from vscode without going through the command-line, I would like to be able to use the same approach with my linter, instead of feeling like a second-class citizen all the time. Is there some metadata that I can add to a given project to enable an option to run my linter in the GUI? (Again, this might be solved using dune, but the question is not dune-specific.) This is what I have in mind as auto-discovery: future tools, that are not part of the platform and probably never will be, have a way to become integrated in the development experience by declaring themselves (somehow).

3 Likes
26

future tools, that are not part of the platform and probably never will be, have a way to become integrated in the development experience by declaring themselves (somehow).

That would amount to defining an interface for each “kind” of tool (e.g. build system, documentation generator, linter, etc.). To take the example of the linter, for an automated integration to be possible, both Dune and OCaml LSP would need to know how to communicate with your tool without having that integration hardcoded into either of them.

Easing the integration of tools into the platform is a good goal to have, and in that sense, it’s useful to think of defining interfaces between the tools that can be generalised. However, I don’t think the platform should have the goal of supporting every tool “that are not part of the platform and probably never will be”. So I am not immediately convinced that this should be in the principles.

It not being stated as a principle for the platform doesn’t mean that it’s not an effort worth pursuing though, and for what it’s worth, Dune is already doing this to some extent. For instance, the instrumentation stanza is a good example of an interface between instrumentation tools and Dune.

When the first linter is incubated into the platform, it will certainly be useful to think about making the integration general enough that linters can be swapped.

27

We should a serious discussion about this, and personally I would support adding a principle in this direction. If the platform defines the development experience for OCaml, I wish the development experience design would give a thought about integrating external tools in my development environment. Currently the discourse reads a bit as “the platform is the set of good OCaml tools, we will make it easy to co-opt new good tools if they get adoption”. (And between the line you could facetiously read: “nobody needs to care about the other tools”.)

You are in fact making a choice in this direction in the current principles, by describing “unified tools” with a single frontend. In the interest of having a self-contained post, let me quote here the relevant principle:

(P5) Tools are independent, yet unified

Following on P4, we appreciate the need for tools to evolve independently, making the Platform resilient to future changes.

Despite this, and in the interest of creating a great user experience, the Platform offers a unified experience and strives to ensure cross-compatibility among tools.

This unified experience implies the existence of a single CLI that serves as a frontend for the Platform tools. On the editor, the implication is that development workflows should be available directly from within the editor, avoiding the need for users to resort to the command line.

I think that this idea of having unified tools and a single frontend brings benefits for user convenience, but it also increases the barrier of entry for external tools – they would feel like second-class citizen that cannot benefit from the nicer user experience, but they could even become actually harder to use than before this unification principled went into effect.

I think that being mindful of external (non-platform) tools gives a good sort of flexibility. Many tooling transitions in the OCaml ecosystem have happened relatively quickly, in a few years, because people found that a new tool was good and adopted it. For example the transition from GODI to opam was almost immediate, the transition from ocamlbuild to Dune was also very fast for newer projects. (To my knowledge, there wasn’t such a user-chosen transition from ocamldoc to odoc for most users, it is just that dune only supported odoc and people were fine either way.) ocamlformat, Merlin were also adopted rather quickly by a large userbase. I think that one key enabler for such smooth transitions was that it was trivial for people to integrate things in their existing development environment. And here I am only talking about famous projects that succeeded in getting wide adoptions, but many other tools were experimented with over the same period, most of which never gained traction, but still it was easy for users to give them a try; and some of these projects remained niche but solve real problems for their niche userbase. The higher the barrier of entry for external tools to be added to a development project, the least likely these positive outcomes in the future.

1 Like
28

I agree that this should be added to the principles somehow.

We added an extension mechanism to opam since the beginning for the same reason – to make it possible to extend the CLI in a fashion decoupled from the core CLI. This is the opam plugin flag; in the event of seeing an unknown command (e.g. opam anil says hello), it will search the PATH for a binary opam-anil, try to install a package opam-anil if marked with a plugin flag if not found, and pass it the arguments. This extensibility has been useful in practise – the opam depext plugin was developed independently like this, and then integrated into opam 2. I use others locally fairly often for my own workflows.

We should be thinking about how to do this more systematically to allow for further tool innovation; the rest of the field is moving fast with copilots, refactoring tools, security scanners, and many other things which we could benefit from if there were suitable hooks in our tools to reduce the friction.

5 Likes
29

I’ve added a sixth principle for the extensibility of the Platform:

(P6) The Platform is cohesive, yet extensible

While striving for a unified experience (P5), we aim to support users who want
to use tools that don’t belong to the Platform.

Users who want to use other tools in their workflows should be able to do so
without feeling like second-class citizens.

This can be achieved through plugin systems or implementations that stay
general enough to support different tools.

My main (only, really) concern above was in committing to making the integrations general enough that the underlying tools can be swapped without effort. That would require substantial effort and I wouldn’t want to see the development of the new workflows being impacted by a constraint.

Stepping back, though, I can only agree that supporting users of tools that don’t belong to the Platform has been a principle of the Platform and we should continue to follow it.

I also renamed the Researcher persona to Data Scientist, it sounds more representative.

Let me know if the text above looks good to you, or if you have any other feedback.

If there’s no blocking feedback, I’ll merge Partially import the OCaml Platform roadmap by tmattio · Pull Request #1488 · ocaml/ocaml.org · GitHub in the coming days to move on to the review of the actual roadmap.

2 Likes
30

I have merged the first version of the User Persona and Guiding Principles in OCaml.org, they are available in the Platform Tools section of the Learn area.

Their being merged doesn’t mean the feedback phase is over, however! They are living documents that will continue to evolve alongside the OCaml Platform. If you have any comments or suggestions, don’t hesitate to share them here or open a PR.

And now, I’m really excited to share the second part of the roadmap: the goals and development workflows we plan to build in the coming three years.

You can see the document here and on the accompanying OCaml.org’s PR.

The roadmap is the result of many discussions with the Platform projects maintainers, careful reviews of OCaml survey results, synthesis of various Discuss threads and blog posts, and user research with both industrial users and individual developers alike.

We expect it to represent the community’s needs relatively well, however, this is the first public draft. We expect to go through a few iterations before it is ready to be adopted.

So now is the time to share your feedback once again! Here are a couple of questions you can try to answer while reading the roadmap:

  • Would you use these development workflows? If so, which ones? Is there any missing workflow you would expect the OCaml Platform to provide?
  • Would you use (all or parts of) the OCaml Platform as specified in the roadmap yourself, in your organization? If not, what are the blockers? How would you measure success for Platform adoption in the community or in your organization?

Looking forward to taking everyone’s insight into account and iterating on the roadmap!

4 Likes
31

Offline feedback on the roadmap thus far has been largely positive!

It’d be great to get some feedback here as well.

We’ll keep the roadmap in review for another two weeks, and if no major concern is raised, we should be ready to adopt the initial version.

1 Like
32

I have a couple comments on the roadmap file:

  • In W5:

    Additionally, the opam repository contains a mapping of libraries in each package. This allows Dune and other opam clients to provide hints to users when a library isn’t available in the current workspace. By looking at the list of packages and their libraries, Dune is able to suggest installing specific packages to use a library.

    I’m curious to know what you’re thinking of doing specifically. To me it seems to be pretty hard to get that map accurately and in a cross-platform manner.
    Of course it can be done on a best-effort basis, that would probably be enough for 90% of cases but still.

  • In W9:

    Dune reports linting failures as warnings during the build, and these errors are reported to the Editors, thanks to the integration of OCaml LSP and Dune RPC.

    The focus on “warnings” in this sentence feels to me like it’s implying the default for dune as a whole might switch away from its well argued over “warnings-as-errors opinionated default”. That default might also be in contradiction to U4 and especially U6’s goals that i read prior so I’m wondering if I might be seeing too much into that sentence and context, or not.

  • In W10, is the specific REPL used “up to the implementation” or is there a specific one in mind? If it’s the former, what are the prerequisites for it?

  • In W22:

    Dune users don’t need to create (or even commit) a .opam file to their repository; however, Dune knows how to synthesise this opam file from a Dune project and can generate a tarball containing the source code and metadata needed to release a package on the opam-repository. The generated tarball only contains the files that opam requires.

    Without a change in opam to understand dune projects, this will break the opam pin workflow for everyone. Could there be a mention of said work needed in opam too?

    While we’re on the subject of this section (opam-repository integration with dune), there is no mention of one of the pain-point for new users publishing their packages to opam-repository: the syntax doesn’t map between dune and opam. I feel like most new users use (generate_opam_files) for ease of use (having to learn only one syntax and edit only one file), however when the time comes to publish on opam-repository and for opam-repository maintainers to do a review and ask users to do some changes they still have to learn the opam syntax on the spot, translate that into dune syntax and work out the difference of behaviour, possible bugs, etc… I don’t think this is a great experience for newcomers.

    If the opam syntax is not going to be the syntax of packaging in this roadmap, there needs to be at least a story on either making this translation automatic (open the PR in dune syntax and automatically translate it upon merge) or change the opam syntax in opam 3.0 or something (it’s more work for an already extremely sparse opam dev team of only 1 full-time dev but it’s always a possibility)

1 Like
33

At a certain point I made this proposal. I’d be interested too, my tools also generate opam file and it’s a chore to keep them in sync.

34

Any proposal that keeps opam from depending on dune is a better proposal than one that establishes that dependency.

35

Thanks for the write up @tmattio , it gives a great basis for discussion. I like the ideas for first class support of benchmarking and generating installers (something I didn’t know I wanted until just now :slight_smile: )

My questions after a first read through:

  • Should enable other build tools like nix, buck, bazel to integrate with OCaml, this document focuses heavily on Dune and could give the impression that other build tools will be excluded. In a mixed language environment where I am using OCaml with other things like Rust or C I have found it better to use Nix as the tool to build everything. Working alongside other programming languages should be a goal.
  • (W5) Manage Dependencies. Given the opam file is the basis for the package index, managing dependencies should run off that rather than a dune-project file. Nothing new to add from me that hasn’t already been raised by @kit-ty-kate. I personally would like a better install / update dependencies experience, trying to write a tutorial where you start from scratch is embarrassingly complicated at the moment.
  • I assume things like (W9) Lint Code and (W8) Format Code will be made pluggable so other options are able to be used rather than just the default choice. What are the thoughts about how to support plugins?
  • (W12) Compile to JavaScript, I am surprised WASM isn’t being mentioned here. I know it’s not ready just yet but within the timeframe of this three year roadmap it would be.
  • What about having first class support for building with profiling similar to W13?
  • (W21) Generate Documentation - Better support for tools like md2mld to build comprehensive documentation for libraries.
  • (W20) Literate Programming fully working on Windows and not just on Unix platforms.
4 Likes
36

(W5) (manage dependencies): I disagree, writing .opam files is a chore.
The problem is that you have a whole file for each package in your
project, with a lot of redundancy. With dune I can just edit all that in
one place.

(W18) (formal verification): this is very exciting. I hope it will get
its own flavor of :with-proof or something, so that people just
interested in running/testing don’t have to install SMT solvers, why3,
etc. But it’d be amazing to be able to prove parts of projects easily
(especially if there’s integration with LSP).

Overall, I personally like the roadmap. Dune shouldn’t be the only way
to compile OCaml, but it’s fine if it’s the main/“normal” way, just like
cargo for rust. It’s good enough for 90% of projects imho.

That said, special care should be taken to not bake-in too much some
tools; @gasche, a while ago, mentionned parser generators, but it should
also be possible to use one’s own testing tools, benchmarking, doc
generation, verification system etc. with the price of being a bit more explicit.

6 Likes
37

Listing all the libraries is indeed a little too simple. But we can improve this in the following ways:

  • Allow packages to specify the prefix for all the libraries they can provide. This is similar to dune’s convention of only allowing package foo to contains libraries that match foo(\.[^.]+)*
  • We could also allow packages to mark libraries as optional.

This is also a quite important issue for building packages with dune. Currently, dune must build all package dependencies and only then build something in the workspace. These two targets cannot be built concurrently because dune doesn’t know how to resolve an external library until every single package is built. As you can imagine, a lot of opportunity for concurrency is lost this way.

38

EDIT: I misread your negation but the point remains as to why I think that baking to much of the exprience through/in dune is not a very good idea.

That’s not a very good idea. Beyond terrible usability which was discussed more than once, there is also the problem of power.

Discussions on this forum already made it clear that basically nothing can really be done in dune that wouldn’t suit Jane Street – even something as simple as reverting to the default warning set of upstream.

That’s fine with me, after all they started it all, but people should bear in mind that the way they use OCaml is not the way everyone uses OCaml. Aka “you are not Jane Street”. You don’t have a tools & compilers team ready to fix and upgrade your infrastructure, their pace may not be your pace, their way of organizing and building code may not be yours, etc. Large technological companies with a single “product” are few and have different needs compared to the multitude of software companies providing software services to clients.

And if you see dune as the final word on build systems that’s fine aswell but then I’m kind of sorry for you :–)

5 Likes
39

Regarding W17:

The editors also integrate with Dune to provide a UI to explore and execute tests. Typically, the VSCode extension provides a test explorer similar to the official Python extension.

For this to work (without hacks like stopping the watching Dune instance) two prerequisites must be met:

  • there must be a way to run any test of any test runner and get the test output from the dune in watch-mode
  • there must be a way to generate inline test runners without having to know the name of the test runner’s executable (or a way to get it from a given source file) or running all tests. This is to add newly written tests to the test list without having to parse source files for tests (which is error prone and slow).
40

That’s fine with me, after all they started it all, but people should bear in mind that the way they use OCaml is not the way everyone uses OCaml.
Aka “you are not Jane Street”.

And yet I find dune useful, as many other people do. It’s more flexible
than you give credit for, I think. For example the flags situation is
fixable with a few lines, in a project for example I have:

(env
 (_
  (flags :standard -w -30-49 -warn-error -a+32+33+39+27+8 -strict-sequence)))

and I’d have changed the flags anyway, so that’s not really an issue?

And if you see dune as the final word on build systems that’s fine awell but then I’m kind of sorry for you :–)

In the world of OCaml build systems, yes, it’s the best we’ve seen in
the last… forever. You might have a better design in a hidden repo but
I’d rather not wait 10 more years for a better design to come along.

I’ve used ocamlbuild and oasis (and a bit of make a long time ago) and
dune is just much better than all of these. And much, much faster, too.
What’s your solution?

6 Likes