It seems like there are a bunch of different build-related tools that currently used for building and packaging OCaml systems. Off the top of my head, I can think of ocamlbuild, omake, oasis, jenga, and jbuilder. Though I’ve been using OCaml for several years, I’m not certain of the differences between these tools. Personally, I’ve mostly used oasis (which in turn uses ocamlbuild and Makefiles). I’m sure there are good reasons for them all to exist, and be used. And as somehow who writes research code, my needs are probably pretty basic compared to industrial OCaml users.
However, given that the OCaml community is standardizing around OPAM as a package manager and repository, is there value in standardizing around a single build tool? If there is, is there an existing candidate for such a tool?
There would be value in standardizing one build tool, but that’s a decentralized decision to make. I suspect ocamlbuild (and probably oasis) will be alive for a long time because lots of projects use it and migration is never enjoyable. New projects might start using jbuilder more and more though.
I’m not expecting existing projects to migrate wholesale to a new build system overnight. However, for new projects, or even projects interesting in changing, it would be nice to have a good resource for deciding which system to use. I think that right now there isn’t a good document that summarizes the states of the various build systems. The Compiling OCaml Projects page has some notes about some of the older build systems (not jenga or jbuilder though).
Would someone who’s fluent in these systems be willing to write a comparison between the different systems?
I think with opam there actually isn’t a need to centralize around a build tool. When I do a change to a repo in opam I usually don’t care about how it’s built, I just check out the code and do opam pin add foo ./ and every time I want to check my change I do opam reinstall foo and it syncs the latest changes and builds.
That being said, if your needs are simple, maybe check out pds which I have posted about in this category earlier this week.
Then there is a script in it called genesis (see the README) which takes some parameters and turns that generic repo into the specialized one with a default pds config. After that, just do make and make test. pds is meant to make the simple case simple, hopefully it’s good at that
Exactly; the problem is documentation, not changing what existing projects use.
The community needs to document the various prominent build systems so that developers can make informed decisions and also learn quickly how to utilise a tool to achieve their goal.
As you observe, the OCaml community historically has been plagued with many build systems. I am seeing a lot of momentum behind jbuilder and would use it for new projects. The reason that I would recommend it is that (unlike all previous build systems) it is quite comprehensive, is not an assemblage of tools (modular but hard to understand), designed around Opam, and extremely fast. I also like Ocamlbuild but its limitations triggered the creation of Oasis and you end up with a quite tall stack of tools that are hard to understand and not portable to Windows. I see a lot of potential with jbuilder to converge on a build system.
jbuilder looks promising, but before it can completely displace oasis+ocamlbuild I think it needs to provide a super set of the features (in particular generic rules and plugins, for cppo, ocamlify and the likes).
The jbuilder story really has been extraordinary. The initial goal of jbuilder was to facilitate our public release process, but given the speed and enthusiasm with which it’s been adopted, it now seems destined now to become the default OCaml build system.
While Simon is right that jbuilder isn’t yet fully general, it’s a good bet that it’s flexible enough for your next project. It’s already used for all Jane Street packages, many Mirage and Docker packages, and js_of_ocaml and atdgen, all of which attests to its range. I’d guess jbuilder can handle the needs of 95% of packages.
In the meantime, the functionality is quite good. It builds things 5x faster than ocamlbuild, it supports Reason, compiles to Javascript via js_of_ocaml, emits .merlin files, supports test targets, and will soon have support for doc generation. There’s even a manual.
The Jbuilder migration is going very well indeed – most of our big repositories like DataKit (see dev reports) have been ported, and see here for notes on the Mirage/jbuilder efforts.
I also find jbuilder interesting but haven’t had the time to play with it yet. Currently it seems to lack, as @c-cube pointed out, a mechanism for sharing build recipes among users (as opam or ocamlfind packages). So, say, Menhir and ocamlyacc are well supported (because their support is hardcoded in the implementation), but if someone wants to build a new parser generator they are out of luck – they will not be able to provide the same user experience to users. I think that, long term, it’s important for the tools we use at the base to support an open world, by accommodating new tools coming up in the future with as little an entrance barrier as possible.
Of course this is also in tension with the simplicity and robustness of the tool (which explains tools like obuild trying to cut down on extensibility), and an alternative is to have the developers of a centralized tool just be very reactive in adding built-in support for new things.
Agreed. I expect jbuilder will eventually move in that direction, but it’s getting surprisingly far by just being light on its feet in terms of supporting things of key importance to the OCaml community.
Thank you to Jane Street for solving this problem. Kinda funny how simple the solution is, at least on the front end. I’d personally have preferred fewer parentheses aka something like JSON or YAML, but I’m happy with anything that ‘just works’.
JSON is worthy of consideration but YAML is a disaster (see spec).
@bobzhang mentioned that a JSON frontend allows for easy integration with tooling if you specify your schema with something like http://json-schema.org/
I think for that reason an alternative for jbuilder is something that should still be considered. Keeping the existing sexp frontend of course.
Aww, come on people! Can’t we just all get down with our functional
programming roots? S-expressions have a long and storied past. And unlike
Json, they have a comment syntax…
Tooling: vscode can provide validation and semantics based auto-completion, as a 12 years Emacs users who wrote hundreds of thousands of lines Lisp code, the user experience can not be beaten by an Emacs mode, S expression will be never popular and it looks weird to new users …
Standardize: so that third party tools don’t need dig into jbuilder details
I like s expression more than JSON too, and Emacs is my editor.
When I made BuckleScript build system, the first front-end language is S expression as well, later I decided to move it into JSON, I think it is a wise choice.
If we do want to make jbuilder or OCaml more mainstream, we should not use S expression, people do care about those superfcial issues.
For Item 1, most editors support comments, item 2 is subjective, item 3: I wish jbuilder won’t add computation in configuration …
jbuilder’s own sexp format has its own multiline comment syntax. No reason why something similar can’t be added to json.
Agreed but the familiarity of the majority of programmers towards json is so overwhelming that in practice it doesn’t matter.
Why is that? Is it because it’s harder to write valid json as strings?
Compared to that having completion for an editor that most OCaml programmers don’t use does not seem worth the cost.
My experience does not confirm this. Almost every single younger programmer that I’ve seen is using vscode or atom. It’s imperative to support these editors properly. Note that they already have OCaml support on par with Emacs. So it shows you that many people do use and care about them.
JSON really wasn’t designed to be used as a configuration language, and should not be used as such. This is XML in the 2000s all over again.
While that’s the case, this position does not benefit the wider adoption of OCaml. If using sexp as a configuration language is indeed a bottleneck for people to get started with OCaml, this should be fixed regardless of how superior sexp is to json.
