It’s very simple to get the confirmation. Just watch a newcomer struggle to figure out how to use dune.
1 Like
Wow, back when I was at Google, the major inhibitor was senior leadership. They made it very clear that the only general purpose languages allowed were Python, Java, and C++, and that we should not waste time advocating for more, because there would be huge costs associated with supporting each additional language. Then they added Go and made it clear that that was a rare exception and that we should still not be advocating for other languages. (at least for everyone below Rob Pike’s level)
Are you really talking about using OCaml at Google? Have they abandoned the idea that every engineer should be able to understand any code, or that every basic service should support every blessed language?
As opposed to what?
What’s the build system out there that’s so clear and simple that newcomers don’t get confused?
Bazel solves problems that are not even on Dune roadmap, sure, but would a newcomer be more at ease if the prerequiste to do a simple OCaml project was the understanding of Bazel?
I’ve watched newcomer struggle to figure out how to use ant/maven/graddle/sbt (and that’s only because I’ve doing Java professionaly, I’m sure the list would have been longer if I worked in more ecosystems)
I’m only doing Javascript once in a blue moon, and if the random magic string I’ve copied out of the internet doesn’t do the trick, I’m confused.
From my experience, build tools are complex and confusing and the only real remedy is better docs, start-up guide, books, blogs, etc.
I’m sure there are plenty of ways to make dune better or simpler, but if the goal is the newcomers not being confused, that’s a wild goose chase.
I was thinking more about the humble open source dev deciding which language to use, or which build system to use for OCaml.
Not me, I don’t have any connection to the Google. Then again, if its good enough for Facebook…
No more than if the prerequisite was to understand Dune.
And tools that make it easy to get started, like a “new” command to generate the stuff you need to get started. Ideally somebody learning a new language should not need to spend any time (at first) dealing with a build language too. But that’s easily addressed with some tooling and simple instructions.
OTOH, dealing with the toolchain (including tools like debuggers, ocamlobjinfo, etc.) is an inescapable aspect of learning any language. For any but the most trivial project so is dealing with a build system. No point in writing code if you cannot compile and link it. Build systems often are “complex and confusing”, but that’s largely because the problem space itself is complex and confusing. There’s no getting around that.
A major difficultly with OCaml (in my view) is the complexity and opacity (under-documentation) of the build protocol itself, independent of any build system. Dune goes to considerable lengths to hide it, to the point of sometimes misrepresenting it (in my view; see “virtual libraries”). OBazl takes pains to expose it and make it understandable. I think minimizing opportunities for confusion here is a realistic goal.
Gregg
This doesn’t only apply to learning. It also applies to prototyping, hypothesis generation and testing.
That’s the reason why I built brzo which I hope I’ll be able to release at some point (still needs a good design review and changes to the OCaml strategy since it assumed we were moving towards a model that didn’t happen in the end – namely the library linking proposal, I’d also like to add more languages to the mix but that could wait).
None of my projects do not start with brzoing these days and the hassle free build experience is exhilarating.
Note however that this is largely accidental complexity due to the fact that compilers work in idiosyncratic ways for what build systems need in order to do their incremental and parallelization business.
They are still stuck in a world where people would invoke their compiler manually at the cli level or specify the invocations themselves in a Makefile.
In fact if it were not for the actual tools and the (lack) of information they give us, build is in fact an excessively simple problem.
More specific to OCaml, the compiler clis have an insane amount of quirks and the whole system greatly suffers from an underspecified linking model. Basically it was not a good idea to let that be defined by a third party tool, if only so that you can actually talk about libraries in error messages from the compiler.
2 Likes
I left Google over three years ago. (For entirely other reasons.) If I were still there, I would not try to advocate for OCaml adoption at Google.
I know it’s still unreleased but I briefly considered it anyway as an alternative to OMake for building my projects that go into OPAM packages. I couldn’t figure it out well enough, but I’m interested to see where it goes. I hope its composability and suitability for polyglot projects turns out better than with Dune. I now see that it was never designed to be suitable for building software distributions. Grmf.
I’m still very intrigued by OBazl because it promises to help make introducing OCaml 5.0 as an experimental language at my workplace a possibility to consider.
Are you mixing with b0 maybe ?
As mentioned in the docs brzo is totally unsuitable for building distributions, it works with heurisitics and the build outcome depends on the state of your environment.
I wouldn’t use it but I’m very glad OBazl exists, when you are a small language it’s good to allow yourself to be easily used in larger systems.
1 Like
That’s another one of yours I haven’t been able to figure out yet. Will try again.
Doing it now would be a waste of time. While I’m using it as my main build tool I don’t think it’s ready for general consumption, it needs a few more design rounds and bolts to make it usable for building distributions.
I get that, and I’m planning to limp along with OMake until it becomes even less tolerable to do so than it is now. So by the time I’m ready to jump, maybe B0 will be ready for public consumption?
One thing that really bugs me about Dune is that it refuses to allow multiple libraries to be packaged in the same OPAM package. Which is why there are ten thousand Tezos packages in the OPAM repository and not five. And why you have to automate the process of making a release to OPAM because there needs to be a package for each library name. I’d rather avoid any tool that adopts the same pattern as Dune in this respect.
1 Like
I don’t think that’s the case. See the package field here.
1 Like
You don’t have do that if you don’t want to though? See https://dune.readthedocs.io/en/stable/concepts.html#libraries for more details, but you can always choose to name your public libraries package_name.library_name, and have multiple said libraries all part of the same package_name opam package.
Shows what I get for not bothering to keep up with the release notes. This was not an option last I checked. Dunno if it’s enough to sway me to reconsider. There are so many other opinions Dune feels so strongly about that I’m still wary.
Probably first on the list is that I really strongly believe that build logic needs to be written in a programming language with at least the level of espressivity found in OMake, and Starlark is magnitudes better. Dune doesn’t offer anything like that.
1 Like
So while I’m offering marginally solicited feedback about build tool features that appeal to me… (this may be of interest to both @mobileink and @dbuenzli)…
A thing that three years of working in a shop heavily invested in Bazel has taught me is that Bazel really got something very very right by defining a syntax for hierarchical names for build targets and a query language for searching the build graph. The experience was convincing enough that I launched a major refactor of my Conjury library for OMake to make a crude approximation of its virtues to only the most marginally satisfying effect. What I learned from that is Bazel is really that much better than any Make variant is ever likely to be, and I should finally give up on them.
Accordingly, if OBazl can be made useful as the build tool for my OPAM packages, then it would meet all my needs.
Likewise, if B0 were to take a cue from Bazel and define a systematic path hierarchy for naming build units (maybe it already does?) and a query language for searching the dependency graph, then it might develop into a better tool for use in my personal projects. (It already looks like I could use it now if its library were stable. I might even be able to live with it having an unstable interface.)
Anyway, I really like Bazel’s path hierarchy, the query language, the composability of its toolchains and build configuration. The Starlark language is a reasonable compromise. The “aspect” feature is brilliant. I think there may be many things about Bazel with which B0 could strive to achieve parity.
Indeed that’s a practical requirement. Nobody publishing stuff to OPAM is going to be very interested in a tool that can’t do that with relative ease. So it’s definitely on the to-do list.
1 Like
I’m not sure what this refers to exactly but basically in b0 each B0.ml file defines a flat namespace called scope in which you perform definitions. It also defines a scope directory which is the directory where the B0.ml file is defined.
Whenever you include a B0.ml in another you name the included scope which results in a hierarchical path to access the names defined in the scope you included. This is explained in this part of the manual, which I believe is mostly up-to-date.
The tool makes it easy to gather and scope existing B0.ml files in a larger one via b0 file gather. Something I use for example when I need to do bulk releases of packages when there’s a new Unicode version. Note that inclusion is not tied to file system layout and for example it’s easy to have one project’s B0.ml file included by two different independent projects.
Since B0.ml files are meant to be defined at the root of your project, the notion is also quite handy to bulk operate VCS or perform operations on all the projects you gathered in a B0.ml file. The b0 scope command provides a few subcommands to iterate on scope directories and execute commands (think find -exec). Since I manage quite a few (> 30) repo-wise independent packages in opam I find it quite handy to have a B0.ml that gathers all of these.
Gregg,
First, thank you for doing this work, and for sharing it. I’ve started: got bazel installed, built the C++ tutorials, and am about to start on OCaml. Um, where would be the right venue for asking questions about OBazl, but also about Bazel itself? For instance:
How do I see what was actually run during a build? That is, I build a two-target (say, "lib’ and “main”) project, then modify “lib”, and I’d like to see that only “lib” gets recompiled, then “main” get re-linked. Is there someplace I can see the trace of commands? Kind of like the dune log file?
I see there’s a “command_log” but that doesn’t appear to contain the actual build actions.
That’s an example.
Again, thank you for doing all this. I am hopeful that I can switch everything I do to Bazel, bit-by-bit. Blaze/Bazel was one of the many impressive tools I used during my time at Google. I’m glad that I can now use it for OCaml.
I’ve set up a discord server, https://discord.gg/wZCSq2nq6y. That’s probably the best place if you’re worried about annoying people who are not interested in Obazl. I monitor the OCaml discord and this list. The Bazel Discuss mailing list is usually pretty responsive.
To respond to your example: just use the aquery subcommand instead of build; that will not run the build, but it will show you the command that would be used, along with lists of all inputs and outputs. To see the actual build, pass --subcommands=pretty_print on the command line. You can put that in a .bazelrc file to keep it on. The command_log will log that stuff, but be aware that it gets rewritten with every command, so after you run bazel info to find its path, it will contain the output of that very command. You need to store the path somewhere (an alias maybe) and view it after you run a build.
The docs at The OBazl Book include a fair amount of info like this. Also take a look at the .bazelrc (and other dotfiles) at https://github.com/obazl/demos_obazl/tree/main/rules_ocaml.
Thanks, HTH,
Gregg