Why what? If you’re asking about the relative merits of build systes, let’s postpone that - this thread is already too long, and before the end of the month I expect to make available some tools for automating conversion of dune projects. I’ll post a message here when I do.
2 Likes
I don’t think Bazel is interesting personally, but Buck2 looks like it’s written in Rust and supports OCaml (because Facebook uses it to build OCaml stuff)
1 Like
if your point is that it’s easy, then I agree with you. I actually started Rust with some tasks around features and it’s so well explained and everything makes sense and you can quickly do what you want to do without spending hours trying to write custom Makefiles. I have yet to find another programming language so well integrated with its build systems and with such a system of features/cfg flags that just works and doesn’t look ugly.
I can do something complex pretty easily with a few lines in a Cargo.toml, doing the same with opam/dune would require days of hacking and figuring out how to make the pins and the Makefiles (sorry dune files) work
I have skipped all steps nobody bothered to mention, as usual. Saw cargo seems to be a thing (at Installation), so I followed their install procedure. I am in no way interested in rust toolchain maintenance at the moment, I just wanted to get this editor built. If the build instructions target rust developers, I am rightfully out for now.
On the other hand, there’s what I like so much about this community here, it’s a place to ask and I used to get quick and useful answers. I usually prefer that to formal, compliance-driven documentation.
So you probably installed rust via your O.S. ? I went down that path using both the Ubuntu-installed Rust, and the Rust that came with Anaconda; both ways, it was a mess. Then I went and learned (it was pretty easy) how to install Rust via rustup, and everything worked smoothly.
2 Likes
Haha, that was supposed to demonstrate how not easy it is. You say it’s easy; I’m going to take your word for it since you actually use the stuff. I hate it when that happens.
Seriously, my mistake was forgetting that cargo is for rust programmers and it clearly works great for them. We don’t have such a well-integrated build system for OCaml. Dune is a noble effort but equally clearly isn’t quite there. Then we have general-purpose build systems like Bazel, Buck2 (thanks for the ref), etc. Almost none of which offer really good OCaml support, presumably because the OCaml build protocols are so… [insert adjective here]. There the question is whether an OCaml-specific customization of a general engine can match up with cargo/rust in terms of power, flexibility, ease of use, aesthetics, etc. In principal the answer is obviously yes (it’s just software after all); in practice we’ll find out, at least with respect to Bazel. I will say that lots of problems I see people having with Dune are just eliminated with Bazel.
Just to repeat here what has been said many times in other venues: as I understand it, the main advantage Cargo has over Dune is the tight integration between the package manager and the build system. This integration results in a much better user experience. In the OCaml world these two tools are separate for historical reasons (Opam came before Dune); and this adds a lot of complexity to the user experience (especially for beginners). Of course, there are other issues that people have with Dune: some are flatly opposed to the general approach taken by Dune, others prefer their “old” tools, etc, but I think that the lack of integration with the package manager is one (if not the main) issue for beginners.
But, luckily for us, work is ongoing to try to fetch/build Opam packages directly with Dune which should help with this exact issue: Issues · ocaml/dune · GitHub.
Cheers,
Nicolas
5 Likes
My guess would be that those build systems were designed for large companies and not small open-source projects, and are predominantly adapted to the software stacks used in those large companies. It’s probably a matter of community and effort to get good support for those tools, and few people (you are a notable exception) have spent the effort to provide support for OCaml.
On the other hand, I find your description of the OCaml compilers borderline insulting, and I don’t understand why you are so negative about them. Sure, they could always be documented better, but my impression (as stated previously) is that they are in fact reasonably simple and well-documented (with corner cases and advanced options, as any other tool). You complain because you try to make them do things that they were not designed to do, and it doesn’t work for reasons you find obscure. My experience is that most language implementations (including in much more mainstream languages with order-of-magnitudes more people improving them) have their quirks, especially if you use them outside their intended usage modes.
I believe that your work is valuable and I hope to someday your favorite build tools get good OCaml support, but I don’t understand why you are so dismissive.
5 Likes
I think the core is that Cargo does much much much less. I’m pretty sure I can use Dune as a Makefile replacement. I can not use Cargo as a Make replacement.
Because of Cargo’s restrictive nature, 99% of ‘cargo errors’ I have run into are of the form:
- syntax error, simple to fix (it tells me which line / column error is on)
- forgot to add a feature flag (some package complains about lacking package)
- pkg A depends on libFoo >= x1; pkgB depends on libFoo < x1 – generally solved by bumping everything up to latest version
===
In contrast, learning Dune is like learning a separate language with a separate library with a separate idiom. I’m not complaining about Dune; I’m quite happy with the power/flexibility Dune.
The point I’m trying to push is that (1) a hypothetical Dune can be just as simple as Cargo if we limited Dune to just (libraries ...) clause, and (2) the simplicity of Cargo, imho, comes from the fact it does so little.
Phrased another way, imho, the complexity of Dune is fundamental, not accidental. Any system more complex than Cargo is likely to be harder to use than Cargo; and any system easier to use than Dune is likely to be less flexible than Dune.
2 Likes
My own take, in addition to what has been said before (it’s more configuration based rather than makefile-like, and it’s deeply integrated with a package manager):
- it uses convention over configuration. One cargo.toml file of a few lines can be enough to structure an entire project of hundreds of rust files (because folders with mod.rs are modules, src/lib.rs is expected from a lib, src/main.rs is expected for the main binary, etc.)
- most/all of the complicated things that you can do with dune are easy to do with cargo because they just rely on you writing a build.rs and writing that logic in Rust (not in a different language, and it’s not sandboxed)
- cargo is usually installed and managed by rustup, a version manager, which solves a huge amount of issues by itself (no need for global and local switches, it just detects which version you need via a rust-toolchain.toml file)
- cargo is also packaged by default with a formatter (which nobody seems to tweak because the defaults are good), an amazing linter (clippy), and other useful commands (and you can easily create cargo commands as well to extend cargo)
4 Likes
It’s not just that, it’s also that rust, the language, is more strict and has stronger conventions in its handling of modules and paths. An OCaml tool can’t entirely compete on that: you simply can’t replicate the model rust uses to structure the tree of files, because OCaml’s syntax has nothing similar (you use modules that are magically there, you don’t declare mod foo; to specify there’s a foo.rs or foo/mod.rs hierarchy). OCaml also lacks conventions that were there from day 0 in Rust.
The point about integration is of course still a very good one. Another thing Cargo does is feature flags (which integrate directly with feature flags in packages, and in the language itself via #[cfg(feature="log")] to enable the following item only if the “log” feature is enabled).
Anyway, it’s not just tooling, it’s conventions and language features that can’t be retroactively added to OCaml. On the other hand, OCaml does some things that very few languages do at all, namely separate compilation with the possibility of cross module inlining; separate compilation is responsible for all these separate .cm*** files and it works remarkably well imho.
4 Likes
there is no package on my raspbian
$ apt-cache search rustup
$
(this is my last OT post in this topic)
Not quite magically–they are there because they’re linked in to the compilation unit. And that is a build system decision. I think I’ve suggested before that making these dependencies explicit would be good for newcomers. E.g. have a #require "foo"; statement at the top of the file to link in that library. It would save people from the song and dance routine of looking up the libraries listed in the dune file. They could just refer to the top of the OCaml source file. And I think it would also simplify the build system.
2 Likes
I’m not so sure the examples you cite are so impossible in OCaml.
-
one could pack the files in a dir into a module (the only trick would be picking a name, but gosh, that doesn’t sound insurmountable).
-
some simple PPX rewriter could do the
#[cfg(...)]stuff you describe
I actually think that if what we wanted was something like cargo, it wouldn’t be that tough to build. Maybe I’m missing something, and would love to be disabused of this notion.
I personally use Makefiles, and don’t see why they’re not perfectly cromulent. They don’t seem to be any more complicated than for your typical significant C/C++ project, either. But hey, to each their own.
Markus Mottl’s OCamlMakefile used to do this: a header comment of the form
(**pp arg1 arg2 arg3 *)
that got added to the ocamlc -c invocation. This didn’t help for the link-step, but was already quite effective in reducing Makefile size and complexity. I adopted it for my own Makefiles via camlp5-buildscripts/ya_wrap_ocamlfind.ml at main · camlp5/camlp5-buildscripts · GitHub
1 Like
In general I recommend people spend more time with Rust to understand the hype and why people enjoy cargo. It’s hard to describe how perfect the tooling is without experiencing it yourself.
1 Like
It’s possible I haven’t spent enough time with Rust. I worked with it for 5mos, in a moderate-sized project ( GitHub - chetmurthy/qrusty: Rust Quantum Computing Library ) that use Rust, Python, and a bunch of sparse-matrix libraries, and heavy multi-thread parallelization. Sure, it’s quite nice. But I don’t quite see where the obstacle is, to building something very much like that for OCaml. Maybe I’m just not seeing a key problem or two.
Dependencies. opam+dune. I’ve given up on ocaml multiple times in the past because I can’t figure out how to work with opam. I thought I’d gotten it down, but the new flow I decided should work does not: add package with a version to dune-project, run dune build to generate the opam file, run opam install . --deps-only --working-dir to install the new dependency. Nope - opam doesn’t install anything. And the documentation does not help me figure it out. It’s also not clear to me why opam cares about source control, i.e. why --working-dir. I just want to update my project dependencies in a declarative manner and code! I really don’t understand what the intended flow is here.
2 Likes
Did you commit the changes you made to the .opam file? opam install ignores uncommitted changes to the project.
1 Like