So the part I actually have issue with is rather the reference than the tutorials. What each option does is not described precisely, there is too little structure to the way they are introduced, there are too few examples and cross-references. In practice it is difficult to get a general picture of what can be done or not, you end up going through everything asking yourself “does this help?” and experimenting to find out what it does exactly.
I mean yes you can change them, but looking up the magical line to do so in every single project is annoying, and I don’t understand why dune has different defaults than the compiler in the first place.
In the issue I linked, it seems everyone actually agreed the defaults are too strict and would be aligned with the compiler eventually. So two years later, this is not the plan anymore?
Yes, some support for scripts would be nice (although I don’t know if this has to be part of dune’s mission). I actually just discovered dune ocaml top-module which sort of lets you use the main file of an executable as a script but it’s not very easy to use.
But even for regular executables, my impression has been that in order to install them with dune you need to define packages, public names etc., all sorts of boilerplate I do not care about for small utilities, and that it will also install meta files that I don’t want. In order to just get the actual executable and reproduce the workflow you’d have in a traditional make-file set-up where it would but the file in root/bin, you need to promote it, and then have an action to rename it and place it somewhere sensible or an external script to grab it (in particular, to get rid of the .exe). Otherwise you need to use dune exec but you need to be inside the project and it’s a lot to type every time.
You need very little boilerplate for simple executables, see the dune home page for an example: https://dune.build/
In order to just get the actual executable
After running dune build the executable is in the build directory, you can just grab it from there. This is not really different from say a C project where the executable output of a build is in some directory and you grab it from there…
Uv design influence from Rust/Cargo does has some rough edges for larger “monorepo” setups.
Global workspace transitive dependencies. In Rust, you can have different versions of serde as transitive dependencies, which include symbol mangling with version numbers.
UV and Python test runners with workspaces is also quite the pain with no linking step/verifiction of dependencies.
Other than that UV is a huge improvement to anything before it in python.
If you run dune install with the project from the example it doesn’t do anything unless I’m missing something. You need to name your package etc.
The exact location is not documented and presumably changes from version to version and perhaps with other parameters, e.g. the profile (I have no idea). I don’t think this is a good basis to build on and I don’t think it’s the intention of the design that you should be doing such things, the choice of name _build is rather clear in terms of intent. If you want to actually get the executable reliably you need to promote it, which you apparently do with (promote (into ...)) (seems to be new, last time I looked into it it was (mode promote) instead).
This will create a directory called _build and build the program: _build/default/hello_world.exe . Note that native code executables will have the .exe extension on all platforms (including non-Windows systems).
and presumably changes from version to version and perhaps with other parameters, e.g. the profile
It has never changed.
I don’t think it’s the intention of the design that you should be doing such things, the choice of name _build is rather clear in terms of intent
I disagree, I don’t read that much intent into it other than ‘this is an output directory, don’t commit it in the repo’.
If you want to actually get the executable reliably you need to promote it
I’ve always been able to reliably get the executable from the _build/default/... directory.
Reread the comment you replied to. I was making a dichotomy, either you try to use dune install or you don’t, and then what happens in each case.
I’m just not convinced at all by what you say about the _build directory, sorry. To me it’s very clear it’s not intended to be user-facing. Both the initial underscore and the unexplained “default” part (what are the other possible values for “default” even? looking through some projects on my system I also find install/default sometimes). And to me a sentence in the “Quickstart” tutorial is not really documentation, it’s just “here is what happens under the current version after this exact sequence of steps”.
If you say it is in practice stable I believe you, but first how would I know, second even forgetting about stability I just don’t find it satisfying to have to fish my executables from a sea of source files and build artifacts two directories deep from the project root, so I use promotion instead (which works fine, but it’s annoying to have to go through these steps).
The traditional C set-up would have some ways of affecting the -o option of the compiler so things are built where you want them under the name you want them, not in the middle of source files, not with forced extensions, not necessarily in the build artifact subdirectory. Dune forces you to rename and move things at the end. It’s not the end of the world, you can work around it, sure. And the other conveniences it brings are worth it relative to makefiles. But this specific aspect is less practical than it could be, that’s all I’m saying.
Personally, I think dune is awesome. And given the prevalence of AI, which will only grow, I think the focus on reinventing build systems at this point is mistaken. AI eats build system configuration stuff for breakfast. For the first time I’m not afraid to dive into and alter even complex cmake configurations.
It seems pretty clear to me that the _build/ directory is intended to be user-facing.
first how would I know
Because it’s documented in multiple places throughout the dune docs.
I just don’t find it satisfying to have to fish my executables from a sea of source files and build artifacts
I understand your preference but this stuff is clearly documented and predictable, it’s not black magic
Dune forces you to rename and move things at the end.
Every build system forces this. You can’t just leave the artifacts wherever they are output, you have to do something with them if you want to distribute them or install them somewhere.
with forced extensions
The .exe extension doesn’t matter in Unix and ensures it works on Windows. Almost every Unix system has auto-complete for executables. So in practice you don’t even have to rename the output, you can just use it as is. Plus it predictable, you know the output executable will always have the extension and can easily find it either by eyeballing it or just ls *.exe.
I’m not actually seeing in what you linked a documentation of what the default means (let alone that I can trust it to remain stable across versions). It mentions a default alias but doesn’t say if it’s the same default, then there is an example where other values for default are given and from there I would infer it seems it’s actually the switch? But my switch is not called default? So maybe they defined some aliases that have the names of ocaml versions for the example? Or default is when it picks up the switch from the environment? But if so it does seem I cannot trust it will be called default all the time? I don’t know what to make of all this.
Edit: to save everyone time, I’m not actually asking for an explanation. If I really needed to know, I would find out from more careful reading or from experimenting. For this particular use case, I settled with the promotion mechanism.
At any rate let’s stop this discussion. My general sentiment is based on the fact that I only use ocaml semi-occasionally. Whenever I do, I re-discover that dune does things a certain way and have a very painful time finding out how to get around this or that behaviour I don’t like. And part of this experience is that I’m not going to read the entire doc from start to finish every time, so usually I go to the reference and start looking through the definitions, but apparently this is wrong of me to do as I’m getting from our discussion that important things are only documented in passing somewhere in a tutorial, I’ll try to remember this for the future. I don’t doubt that frequent users end up knowing their way around the tool and around the documentation, it’s understandable that from their point of view my complaints are absurd or wrong-headed, but it remains that my experience using the tool has been consistently not nice, and you’re not going to change that experience by telling me you got used to it.
And then there are many things where I totally get why things are a certain way, yeah it’s practical for some workflows or some preferences. Just not mine. The .exe is very much in this category. I just don’t like it, why does this one tool in all the world create .exe files on linux? Again, I still use dune regardless, it does very useful things and it’s great it exists. But I don’t like the experience, that’s all.
The discussion looks more nuanced than conclusion of a simple consensus, on my read. First, there is agreement that coordination with the compiler is needed, spawning Warnings: documentation, UI, choice of defaults, synchronization with Dune · ocaml/ocaml · Discussion #12557 · GitHub (not yet resolved, afaict), then there is this dissenting opinion, currently the last in the thread, warning 9 enabled by default in dev mode · Issue #8645 · ocaml/dune · GitHub . But I agree that issue has not seemed to move forward despite mostly encouraging remarks. It seems to indicate that intersection of available devs and urgency of need have not well aligned . Contributing your perspective on issues about things you care about is a great way to make needs more visible. Meanwhile, I’ve cross linked the discussion.
An easy way to get executables built by dune (within its current configuration requirements) available for use on your system is to do an opam install project_dir from the global switch on your systems. Any public executables will get installed system wide.
But it is also common, and generally safe AFAIK, to grab out the needed executable from the build dir and put it where you want. _build dirs can be safely inspected, and files taken out of them, but users generally shouldn’t alter the contents or write content into them.
There is significant existing work in flight on OxCaml, LLDB, and GDB to support DWARF 5 and source debugging. The plan is to upstream this, but the diff is large and it will take some time.
It seems dune decided to treat to treat the structure of _build as part of its API. Personally I don’t find that to be a very wise move.
In general in my build systems (yes I have more than one :–) I add a --path option to the moral equivalent of dune exec -- tool ARG…. When invoked with the option it prints (rather than exec) the invocation with tool absolutely resolved in the build directory: /path/to/_build/tool ARG…
This is quite useful for a number of reasons:
It treats the _build folder as an abstract data type but still provides a resolution mecanism for end-users (e.g. if your build system allows side-by-side build variants selected via a cli flag). For example if you want to extract an executable artefact you can simply do:
cp $(dune exec --path -- tool) ~/.local/bin/
More generally it easily allows you to invoke other tools on the build artefact like system debuggers, profilers or time or hyperfine without timing the build:
This does not work if the global opam switch on your system already has packages installed with incompatible versions of some dependencies. Or if the global opam switch on your system already has a different version of the same package (removes previous version).
This goes back to the original topic of the thread:
I find it difficult in ocaml to install software and tooling globally. I want opam-grep and odds and other stuff on my system rather than on my switch.
I find it painful that the dev tooling for a project polutes the dependency cone constraints of the project itself. Like I want to use ocamlformat but I don’t want to force re >= 1.10.3 on my dependency constraints, I want to use ocaml-lsp-server but I don’t want base in my opam switch.
I think that when I work on a project I want an opam switch for the project’s code itself and then separate opam switches for each of the tools I want to install. The tools’ switches should just expose the binaries from the tools I installed explicitely.
And similarly for installing binaries globally. I think it makes sense that opam isolates stuff into switches for the purpose of building. But then it’s hard to bridge that to global installation. (Related issue from a long long time ago: Allow installation of binaries in .opam/bin · Issue #489 · ocaml/opam · GitHub)
Just in the interest of not letting a question hang. default is what is known as a build context. And I agree that it could be better documented. You can find more information on contexts here context - Dune documentation in the documentation. I should warn you however they aren’t something that most users need to think about.
The install directory is a special context which contains a 1-for-1 install layout for your project. You will see the install layout for the default context as _build/install/default. When it is actually built, dune install will just copy its contents to the various directories you configured it to target. Installation layouts are configured with install - Dune documentation and the various options to dune install.
If you are a developer and interested in hacking around with project, the installation layout isn’t something you need concern yourself much with unless you are interested in packaging through opam or otherwise. Dune will already pick some sane defaults for the layouts that most users seem to be happy with.
Regarding build contexts in general and why have them: They allow you to run parallel build setups and are useful for people interested in testing multiple versions of a compiler / other package or those interested in cross compilation Cross-Compilation - Dune documentation. Contexts can either be defined on their own or have particular opam switches attached to them.
So far as concerns a compiled executable, this point can be avoided by including a promote field in your executable stanza. ‘promote’ operates by reference to the source tree, so a field (promote (into .))will promote into the directory in which the dune file is situated.
