Ah yes “let’s just add comment to json”, nobody though about that one, except the 300 projects that all did it differently. If you really want a non-lispy configuration format that supports comments, please follow the Rust example and use toml instead. 
That being said, I think jbuilder’s syntax is fine, and this is all a huge pile of bikeshedding.
That’s not really the case. A lot of projects are using http://json5.org/
The download numbers speak for themselves
Rust’s cargo uses toml. It looks pretty good. Is it well supported?
BTW one of the things I’m grateful for with ReasonML is that we seem to have a serious influx of young programmers into the language. Finally I feel like we’re releasing OCaml from the grips of the ‘get off my lawn’ crowd and we can actually talk about introducing modern changes without getting the inevitable ‘I was perfectly happy with my PDP-10, ergo we don’t need this new thing’ response. (BTW I don’t mean you, Yaron 
)
I’m not sure who you are trying to insult but this feels a little bit circular since it seems that this is the very myth reason people like to propagate about the OCaml eco-system.
Now I’ll reveal something which explains everything, I saw Jérémie live today: he has a freaking beard.
Personally though I’d vote for XML.
I’m all for modernity, but maybe we should learn a different lesson from jbuilder. I think it has started to take off because of the well-designed workflows and the top-notch engineering that went into it.
Perhaps this suggests that the fundamentals of the system matter more than the choice of syntax for the config files.
It’s not that I think syntax is fundamentally unimportant. I’m not sure Reason is the next big thing for OCaml, but there’s at least a plausible story that the syntax of the programming language itself is a material part of its usability. But the concrete syntax of the jbuilder config files is focusing on what is perhaps the least important part of the system. And having multiple syntaxes for the files will surely add to the complexity of the system. Maybe there are better hills to charge?
y
Personally, I’m just glad there’s a fast, well-documented, build system that is gaining steam and eliminates most of the boiler plate for 95% of use cases. As long as we have that, I don’t really care what jBuilder requires for its config.
I much prefer s-expressions to JSON in general, but all the data we’ve collected demonstrates that (edit: all else being equal) using a more approachable(edit: familiar) syntax has a strong effect on reported approachability and adoption. The effect is not small but once you become comfortable with a new syntax, it’s harder to imagine anyone else not being comfortable with it. Core language syntax is far more important than build config syntax so I’m not sure it matters too much in the case of jBuilder config.
I’d like to learn more about the meta-programming features that jBuilder plans on adding. I’ve always thought that if your config becomes that expressive, maybe it makes more sense to just encourage dropping down to OCaml entirely? I think JSON(with comments) would provide a small but easy-to-achieve win for adoption though I’m not sure if that’s worth it if metaprogramming becomes much more awkward (and mandatory for interesting builds).
I’m really excited about jBuilder in general!
I’ll probably post a more detailed feedback topic about my experience once I can create a sample project to share, but I’m looking at converting an ocamlbuild project to jbuilder and it has more boilerplate. Specifically around subfolders, which I like to use to conceptually separate parts of my code even though logically they are flat.
In ocamlbuild I can list all subfolders in a single _tags file but jbuilder seems to require that each subfolder have a jbuild file with library specification and the list of other subfolder “libraries” it uses. The lack of a . or _ prefix also means they mix in with my code files alphabetically.
This is all an interesting contrast to the bucklescript bsb tool, which takes the ocamlbuild style and flattens it even further; all source folders are treated as a big flat collection of files.
I believe that was the rationale for the capital “M” in Makefile. Modern locales tend to do case insensitive sorts, but an ls-alias refined from LC_COLLACTE=C ls would help. So allowing capitalised Jbuild and Jbuild-workspace* might be sufficient. In any case, I don’t think it’s a good idea to hide build instructions with a .-prefix.
OT: Thanks for the tip! I have been happily using config-file, but when I need something more cross-platform, json5 is my favourite suggestion here so far.
I don’t think it’s important for jbuilder though; S-expressions are fine. A cheat-sheet section in the --help page would probably improve productivity more than the change of syntax.
For complex cases, an plugin system like the ocamlbuild’s -plugin-tag would have the advantage, over encoding rules in the jbuild file, of providing a real programming language, and of utilising already present packaging and distribution facilities.
I think you can have only one jbuild file if you want, by being explicit about which modules belongs to which library/executable, see the [jbuild specification chapter](http://jbuilder.readthedocs.io/en/latest/jbuild.html?highlight=(modules) of the manual.
Here is the vscode extension to make working with sexps/parens a bit easier.
https://marketplace.visualstudio.com/items?itemName=2gua.rainbow-brackets
I don’t want to hide them, just have them alphabetically separated from my code.
That would be nice, but I don’t see any obvious way in that document to have it use source files from a subfolder. That’s pretty much my entire concern with jbuilder.
See Subdirectory compilation · Issue #109 · ocaml/dune · GitHub. You have to copy the files to the root first. You probably want to step-in in that issue to give your use-case and opinion
So I spent a bit of time looking in more details at jbuilder today, and thought that I may as well share some comments here. In general, it is as I expected: the core design is minimalist, solid and, well done (this is as I expected from Jérémie), but the current lack of extensibility means that features are currently hardcoded in the tool in a relatively invasive way (Merlin and Reason for example show up all over the codebase).
This works well for now, but I don’t know how well it scales to the needs of the community – after how many hard-coded features would the codebase become a mess? Jbuilder’s implementation is already larger than ocamlbuild’s today. So I think a lot of the viability of the tool rests on Jérémie being able to find a good extensibility story, and that is not easy. The internal build action language is at its second iteration if you count Jenga as a first, third if you count ocamlbuild actions are an inspiration as well; how many iterations to we need to get build system extensibility?
At a more high-level: if a tool is proposed as a “standard” build system for the OCaml community, the following questions come to mind:
does it do the job for most people?
is there adoption / chance of adoption?
is there a contributor base?
is it well-documented?
is the design solid?
is the implementation solid?
is it flexible enough to meet the needs of our users?
what will it look like in five years?
(Those questions are not very clearly answered in the existing jbuilder documentation, in my opinion). Here would be an improvised attempt at answering them briefly:
doing the job: Right now jbuilder seems able to cover the need of 60% of OCaml packages. On the other hand, most of the “big old programs” (Coq, Frama-C, SLAM, etc.) may very well be difficult to port as they don’t respect some of jbuilder’s assumption. Similarly, the restriction on ppx usage means that many ppx extensions (and programs using them) currently won’t work – it’s probably a good thing to force the community to adopt ppx drivers.
adoption: It looks like it there is adoption; the Mirage people are in the process of adopting it, as well as a wider vocal group of early-adopters.
contributions: Yes, there is an active community of users that implement new features. (On the short term it is better than jenga, omake, ocamlbuild, obuild and ocp-build in this respect, although enthusiasm might change over time.)
documentation: The documentation of the currently supported jbuild language and general interface is fairly good. The documentation of the internals, on the other hand, is minimal. (But then, few build system have good internal documentations, although omake and ocamlbuild are decent.)
design: The internal arrow datatype of build actions is fairly nice (I already quite liked this aspect of jenga, although I hear it was not completely satisfying). The idea of reimplementing an action DSL (in s-exp syntax) suggests that jbuilder configuration file will soon become Turing-complete, but I guess that is how life is – it is probably less painful overall to use a Turing-complete language interpreted in OCaml than bash. I’m not sure the module discovery logic is very robust, but the community may be able to respect certain conventions to make it a non-issue (I think that if we assume that there exists a file of the form <foo.*> for each compilation unit , whatever preprocessing steps are in place, the current logic can adapted.). I don’t understand how jbuilder would work in real mixed-language projects (where we have to interact with the build system of the other language, possibly be called by it), but then I don’t know of existing OCaml build systems that solve that problem well.
implementation: Very nice! The fact that external contributors were able to come and implement stuff suggests that the coding style is not destabilizing. In particular, a pitfall from the ocamlbuild that was avoided here is the over-reliance on global state, that is replaced by context-threading – except, oddly, for Clflags, that remains a bunch of global mutable state.
extensibility: Right now that is the major question about jbuilder. It was not really designed for flexibility. For example, I don’t think it is currently possible to specify different warnings for one of your .ml files (they may be set globally or on a per-library per-executable basis, even though most jbuilder adopter just adapt to its default choice of warning flags and fix their code during the migration). That may be problematic if a source generator produces code that warns.
in five years: time will tell
Good stuff gasche, that’s a fair and balanced assessment. Note that the relative lack of extensibility and rigid conventions are one of the things that I’ve learned to really like about jbuilder. I find it liberating that I can just start hacking on any jbuilder project out there and not have to understand the peculiarities of each project’s build system. It’s not an abstract concern either, as I’ve had to port ppx_bisect’s nightmarish ocamlbuild/makefile setup to jbuilder for example.
I was already used to this when hacking haskell (cabal/stack) and java (maven). I welcome not having to think about build systems when programming in OCaml.
I’d also like to focus on the unique feature that jbuilder brings, which is the reason why we’re adopting it so eagerly: multi-directory builds and workspaces. As we are porting more libraries, we can just git clone multiple opam packages and run a single jbuilder build/test invocation to run them all, without requiring any opam or other external tools at all.
I cannot emphasise how much this improves our day-to-day productivity when developing code that straddles multiple opam packages. This feature, combined with the performance improvements, justifies almost any of the other tradeoffs.
I agree with rgrinberg about the lack of extensibility being a positive feature and not a drawback.
Yep, the support for multiple package and the support for building on multiple OPAM switches is nice. I think that right now it is not a feature relevant many OCaml users (it’s not that common to have a large amount of inter-dependent package as in the Mirage world), but for those affected it can be a big boost in convenience (and speed, I guess).
There is some self-selection bias here in your analysis
The reason that we have so many large monolithic repos in existing projects is because such intertwined development is difficult. Jbuilder unlocks the full potential of opam to have many relatively small, standalone packages, but to still conduct day-to-day development as if those splits didn’t exist.
opam pin gave us the initial impetus to scale to 3-5 pins, but jbuilder makes it possible to have 10+ without any effort at all beyond cloning them via opam.
I agree with Rudi’s point here. One of the advantages of Jbuilder is that it is an opinionated build system: the rigidity is a key part of what’s good about this model: it encourages people to adopt uniform standards in how the structure OCaml projects, which overall simplifies the ecosystem. And because it’s exporting the model we use internally at Jane Street, we know it’s one that scales pretty far, supporting 5m or so lines of code covering a thousand or so projects.
While I believe extensibility is going to eventually be necessary in some form, I think we should approach it with some caution. Spending time building out good defaults that cover a wide range of what’s necessary for the community is I think a virtuous process, and so the current way effort is being spent seems proper to me.
y
Extensibility vs conventions is a false dichotomy.
In practice when you are working on applications there’s always other things you need to integrate and for this you want a good extensibility story so that it doesn’t become a pain.
Having a extensible system doesn’t mean that you can’t provide top-notch built-in support and conventions. For example this is something ocamlbuild failed at providing where compiling something with C immediately lead to pain; this could have been aleviated by its extensibility but it also failed at giving a good story on that front in my opinion.
