A Roadmap for the OCaml Platform - Seeking Your Feedback

Community
41

Everybody hates existing, working code. Everybody loves hypothetical, unwritten, greenfield code that will be perfect when it’s done :slightly_smiling_face:

4 Likes
42

I want Bazel.

I already ditched my OMake library, not because it didn’t work well enough, but because OMake itself has been mostly abandoned to unpaid caretakers who are always busy with other things. OMake was excellent. Until it wasn’t.

Bazel has excellent cross-language support, apart from its lack of OCaml rules (although @mobileink is making progress), and its user community dwarfs the entire community of OCaml programmers by at least an order of magnitude.

I do not understand why we would intentionally cripple our cross-language build support by standardizing on a single build tool with such terrible support for multi-language projects.

1 Like
43

First, I’d like to say that I really appreciate the simplicity of dune exec my-cli to get OCaml code to run, regardless of platform and regardless whether the user has already installed the compiler/opam/etc. That is sorely missing in today’s ecosystem.

Some of my opinions:

  • (G1) Dune is The Frontend of The OCaml Platform. Users will type dune <cmd>. They will call it and think about it as “Dune”. It is weird to see people get worked up about The OCaml Platform, because new users won’t call it The OCaml Platform. // I learned this myself when I initially was calling my Windows stuff Diskuv OCaml and then had to teach users both the words “Diskuv OCaml” and the second word (dkml) … that in hindsight was silly. The product is what the user types.
  • I’d love to see either of the following to allow other tools (metabuild tools like Bazel/Buck2 and CMake, benchmarking tools, test tools, etc.) to interoperate with Dune:
    • (W24) Forward Compatibility. The dune-project, dune and other build metadata files will conform to a sexp schema language (to be designed) that supports forward compatibility. The semantics of the build metadata will be documented (this is done today, although today all the hierarchical schema elements are collapsed into a linear table of contents) and will have an independent conformance test suite. For tools that have not yet been integrated into Dune, local extensions to the sexp schemas (ex. <projectdir>/dune-project.ext.schema) will be allowed to aid the LSP (and the user).
    • (W24) Plugin Extensibility. Bytecode or compile-on-demand plugins, where a plugin implements module type DUNE_PLUGIN, will be supported in a manner similar to Gradle (Using Gradle Plugins).
1 Like
44

Do people conflate Rust and Cargo? I’m not sure. The OCaml Platform is a general concept. I think it happened to you with Diskuv OCaml partly because users had no idea what Diskuv refers to. Given the alternative between 2 seemingly random words, they chose the shorter one that appears more often.

Very happy about this roadmap. It shows a lot of vision and planning, as well as looking at what our competitors are doing and learning from it.

1 Like
45

Just name it “the dune platform” I guess :joy:. I’d be fine with that personally.

4 Likes
46

Me too probably. That’s how I’m going to start referring to it anyway.

47

That’s exactly the kind of thinking I’m alluding to.

For one thing if I take only my free software repos I have approximatively 40 repos to maintain (without including the private ones I have both for myself and for clients). Adding that line in 40 repos is a maintenance burden I don’t want to have. It just doesn’t scale.

I always need to remind people that a “simple” five minute routine task only on each of my free software repositories is already 3 soulless hours of my time gone.

Of course that’s not a problem you have when you have a single product developed in a monorepo.

Dealing with multiple clients with each their own idiosyncratic infrastructure, distributing software releases to non-technical users with webpage manuals, etc. are very different things for which I’m happy to develop my own software because trying to convince people who don’t have your problems, don’t understand them and are generally interested in satisfying the wishes of the people who pay them (fair enough but that’s not community) who also don’t have these problems is a pure waste of my time (and I speak out of experience).

I’m an extremely happy user (and developer) of my own solutions[1] b0 and brzo. I manage to get things done quickly build and infrastructure wise which keeps both my costs and my client’s costs low – not to mention my sanity.

I’m perfectly fine with people using and standardizing on dune if it suits them. Just don’t think that it’s the solution for everyone and just don’t lock everything on it.

  1. Not yet ready for general consumption sorry. ↩︎

2 Likes
48

It can still be a problem you have when your single product is developed in a monorepo that isn’t yet using OCaml as one of its principal programming languages.

Dune is designed explicitly not to work well in that kind of environment. It’s a feature. Making Dune into “the front end of the OCaml programming language” is how you make OCaml useless in large multi-language products.

2 Likes
49

My understanding is that Jane Street use their own, very custom, build rules, so it should be entirely possible to reconsider what the defaults are in upstream Dune. I’ve opened ocaml/dune#8645 to begin this discussion with some low hanging fruit. The complexity lies more in backwards compatibility considerations, but that can be handled by the versioned dune-project files allowing the change to be opt-in for projects as they upgrade.

I agree with this entirely. My own uses of OCaml tend towards the esoteric cross-compiling on embedded architectures :wink:

And if we’re diving into strong opinions, I have a strong negative reaction to introducing a dependency on the JDK in order to build my OCaml project on some RISC-V board with tiny amounts of memory. I’m glad you have some use of Bazel, but it would be an entirely inappropriate default. But as I noted in another thread, your usecase isn’t mine, and I’m entirely supportive of seeing such alternatives in the OCaml documentation, if they pass a reasonable threshold of coverage and adoption.

7 Likes
50

How do you make sure that anyone who builds one of your repos is able to build it with the same set of warnings that you’re using?

1 Like
51

A yes of course it took @xavierleroy to notice…

Not exactly sure what you are mentioning here, people build repos with the build system of the repo. Besides a warning never inhibits compilation of a project, it simply spits an error message on stdout.

52

People build the repo with the build system of the repo which has to fetch set of warnings from somewhere. You’ve mentioned that’s painful to maintain the set of warnings consistent across projects - and I agree. How do you then deduplicate this warnings across your projects? I assumed that you’ve found a way to avoid this since you’ve said:

Adding that line in 40 repos is a maintenance burden I don’t want to have. It just doesn’t scale.

2 Likes
53

For now I simply use OCaml’s default set which in contrast to dune’s one is sensitive[1] and non irritating – but then if your system is really compositional solving this problem is just a matter of being able to import configuration defaults from a library.

  1. Except IIRC for useless-record-with which doesn’t make sense from a code evolution perspective but I got over it. ↩︎

1 Like
54

To reinforce this point: its not about intentions. No matter how irenic and welcoming the Dune devs are, they are finite. Presumably they do not work for free, at least most of them don’t. What happens when the economy takes a nose dive and their budget gets cut? Or their company simply changes priorities, and Dune devs get transferred to teams addressing core business?

55

I have no problems with experimenting with a cargo-style unified CLI tool for OCaml. But I am extremely concerned that existing tools that we know and love could disappear, or be neglected, or lose their CLIs as they are turned into libraries usable only from the unified CLI. This could happen because of limited resources being shifted to the development of the unified tool, or as a way to force adoption of the unified CLI.

I’d really like the roadmap to commit in the strongest words on keeping essential tools like the core compilers, OPAM, interfaces with code editors, documentation generators, etc, usable independently of the unified CLI tool, with their own CLIs to support existing and future alternate workflows. (For the core compilers I can vouch for that, but not for the other tools.)

19 Likes
56

I don’t think it’s critical to keep the different cli tools – it’s fine if dune is the frontend to all major cli commands. What I think is important is to keep the internal data interchange format between different functionalities of dune textual. This is a pain from the perspective of the efficiency loss - using data structures in memory is much easier and type-safe etc. But this allows anyone to hook in to the functionality they need for their particular tooling. And since the tools are currently communicating textually (for the most part), there’s no loss relative to today – there’s just less of a gain.

57

What would be the point of maintaining an interchange format between the internal libraries of a single unified command line driver if there were no other independent tools in active maintenance that use that format?

58

Maybe something like:

PUT /exec/bin.exe HTTP/1.1
...

:grin:

59

Indeed I agree with @xavierleroy that the more modular the setup is, the best; however I’d like to remark that there are some big wins due to tool integration that often outweight the loss of CLI modularity. Two examples:

  • a large amount of time is spent in Coq marshalling back .vo files, fcc, a tool that integrates the coqc compiler with a build system similar to Dune can share the .vo de-marshalling across the compilation of a whole Coq theory; gains in time a memory are large.
  • something similar can be observed in for example opam and build caches, it is not easy for Opam to improve sharing and build caching without basically reimplementing their own build system, or creating a new protocol which is hard to design and maintain

I’ve observed that in general users do much prefer integration, but that’s a HCI question that should be answered formally IMHO.

1 Like
60

I thought this was another update but it’s from May!

Thanks for making OCaml so easy to use. @tmattio @sabine (and others that I don’t recognize - Dune folks). As a lurker and experimenter, all the improvements in the tooling has made it feel much more like Go, Rust, npm where I can spin up a script in seconds. I keep coming back to OCaml for…idk a multitude of reasons despite always testing the waters with f#, scala, haskell. :sweat_smile: Not that I share any open code to help others because they’re all too specific and garbage.

2 Likes