But I don’t think this is the same. First, in my development tree, I prefer my dependencies to be open-ended, for example sedlex>=3.0 , so I can upgrade, rebuild and test without editing the versions of each dependency a priori. But then I want to reproduce the versions exactly when building in a CI framework. Second, I actually prefer separating the concerns of dune and opam, that is I want to use dune only as a build system. There are a few reasons for that, one can be gleaned from my recent question here The upshot is that I don’t like to specify prerequisite packages in the project file at all - that is a packaging concern, hence better left to opam. If you proceed as you indicated, I’ll be stuck.
I have some tests that need to be run as superuser, would be nice to have support for it with sudo and doas. Could be simple enough, but it might require a settable timeout for the test, in case things were not engineered for sudo to not ask for a password and it’s not entered.
The first contact with dune is horrible.
The second contact as well. etc.
(re)giving dune a trial from scratch shows many ambiguities and makes the experience very uncomfortable or horrible. You may wonder what is rotten: the documentation, the sw or the programmer himself? Just with a basic program with a basic library module (lib)
Why calling bin/ what should be called source/??!! Why not crunch/ or foo/?…
Being forced to handle a bunch of dune files in the hierarchy is a trap. See herebefore how easy it is to fail.
Why is/are dune file(s) not generated from the program structure, defined by the main program and its modules (internal « libraries ») and (« external » depext) libraries?
There is often…no feedback. Or it’s cryptic.
Obviously, this software has not been designed from the user standpoint… while it’s supposed to be a key part of the OCaml infrastructure.
It also prevents beginners to do and understand what is compilation and linking of .mli/.ml and their subtleties, which are hidden by declarations (sexp).
I don’t see advantages over make (handwritten or generated Make files, which are readable and simply… work). Being faster is not enough; make is still a good choice until dune becomes usable.
Btw, where is the specification of that software? And the documents on which was made that spec.
Where is the roadmap?
Iirc, I had similar questions as @Luc_ML on my first starts with dune in regards to the naming conventions of bin/lib/test and why files in subdirectories couldn’t immediately compile. I remember speedrunning through the Examples and https://dune.readthedocs.io/en/latest/quick-start.html to try out some lib, missing some key details here and there haha.
But maybe a respective readme should be generated with each dune init templates because dune’s quick start docs explain a handful of these questions really early on.
Maybe include_subdirs should be default for new init’ed templates. But now we unknowingly introduce users to the rules of include_subdirs. We do know the dune team is working on (include_subdirs qualified) - thread. So once this is shipped, defaulting new projects to have (include_subdirs qualified/unqualified) may be a great help.
I’m on team-make-startup-as-easy-as-possible because it makes experimentation for new users really easy and lets them continue down the ocaml rabbit hole. I expect software developers to be of the patient, debugging, doc-reading type, but having to dig into build docs when you want to experiment with a library is always a hassle.
@jbeckford generating dune files without touching the source tree is something we’re working on. We should have something ready with 2023 but with other limitations. For example, you might not get the friendliest error messages if the script you’re using for the generation will itself depend on dune file it is generating.
I’m curious about the comparison of dune and ninja though. The dune dsl is designed to be very high level so I’m not sure why you think it’s a good idea to wrap it. I understand there are some complicated builds that require dynamically generated rules, but this isn’t a common use case I imagine.
@nobrowser We’re not going to break the workflow for any existing users without at least providing an opt out so there’s no reason to worry. Note that the feature doesn’t require you to write package dependencies in the project file either. Dune can read .opam files just fine as well.
@haesbaert Yes, that would be quite useful. Something more general would be to add a timeout action. So you could write something like (with-timeout 2.0 (run my-test.exe)).
@Luc_ML Please read my initial post in this thread. I’m interested in feedback but not in low effort rants.
There is no specification for dune. I don’t think that spending time writing one would be a good use of time. A lot of software remains without a spec and is solely defined by an implementation - some interesting programming languages come to mind. I don’t see anyone complaining about those.
There’s no official roadmap and development is done at the discretion of the maintainers. In practice, we heavily rely on feedback from both industrial and open source users to decide what to work on.
In the same way that, when writing OCaml code, I don’t want to have to read the doc of a particular module, just to learn about its function names. I may do just fine just by poking around and calling an obvious function by its name.
So some kind of auto-complete in the official vscode plugin (which beginners should use) would be very nice. This would enable dune maintainers to add, move or remove (and maybe deprecate?) options with minimal user friction I would presume.
I have a couple of other nitpicks with dune:
Builds seem to be slow overall and frequently hang at the late stage of my build processes. Sometimes for up to 10s on my computer, even for very simple projects.
I faced a few bugs here and there that required me to do a dune clean. I don’t quite remember the details but basically I was getting really weird error messages from my tooling and cleaning the build was the not very obvious solution. I suppose dune is a little buggy here.
This is an interesting example actually. Normally when we have s-expression syntax and a Lisp-like semantics we expect forms to work automatically with variable numbers of arguments, e.g. this is touted as one of the clear advantages of the syntax:
(+ 1 2)
; works the same way as
(+ 1 2 3 4)
So it’s surprising to have plural forms in dune file syntax.
I would love to see (include-subdirs qualified) become the default (with default file names like _module_.ml so it requires minimal thinking to see where the base module is described).
It’d also be nice if we could integrate dune file contents into dune-project (perhaps in a dune section with a base directory entry). This way, with one file (and include-subdirs), you have all your configuration done for you in a simple project and you don’t need to touch anything else.
Thanks @rgrinberg for the initiative! It’s a cool idea and brave
My main gripe with Dune is that its usability is sometimes poor (both the experience of using it, and of reading the documentation to use it), and I think that the root cause is that none of the heavy contributors to Dune really cares about it. I don’t think that you are evil people intentionally writing confusing software, but when you decide what to work on, you repeatedly pick something else. This is what I describe as “you don’t care” – you don’t care enough to make a real effort. My wish for 2023 is that some of the frequent dune contributors start caring about usability – ideally all of them!
To give some examples:
last August I suggested that the list of stanzas in the reference manual be sorted in alphabetic order ( manual: list stanzas in alphabetic order · Issue #6037 · ocaml/dune · GitHub ). I am continually surprised that it is not done yet. How many OCaml programmers does it take to sort a list in alphabetic order? (Actually I first suggested this in February 2021, but in the middle of another issue.)
More generally, it looks like I have reported 10 issues against Dune in 2020 and 2021, the vast majority of it being actionable usability issues, and basically none of them were fixed. I’m not of course suggesting that my issues should be given priority treatment, I believe the lack of prioritization applies to all usability issues reported. (But it’s hard to tell because you don’t have a “usability” label in your issue tracker. Please add one!)
I mostly stopped reporting usability issues when I encounter them with Dune or Dune’s documentation, because right now it feels like a lost effort.
(Note: I already made a comment to this general effect in March 2021 on the bugtracker, so this should not come as a complete surprise to the Dune team. This point remains at the top of my “Dune wish list”.)
I agree with many of the criticisms here, but I still find @gasche’s message a bit harsh. As far as I know, Dune has only two developers working on it as their main job: @rgrinberg and @snowleopard. @snowleopard (who works at Jane Street) focuses on the low-level “engine” of Dune and his priorities are aligned with those of his employer, which leaves only one general-purpose developer working full-time on Dune for all the other aspects (@rgrinberg), and his time is shared with other important projects for the community (eg OCaml-LSP).
There is a small group of people who contribute more or less regularly, but mostly on specific areas of expertise (@Alizter and @ejgallego for Coq, @hhugo for Jsoo, @dra27 for Windows, @bobot, myself from time to time) and Dune’s development is not their main job.
Given the limited developer capacity available, I think it is not surprising and rather understandable that documentation/usability issues have not been given top priority.
So it’s good to provide constructive criticism (and I do think that better documentation is sorely needed and the most obvious candidate for improvement that is coming out of this thread so far), but at the same time I feel it is not realistic to expect such a small team to focus on documentation and usability above all else (fixing bugs, implementing new features requested by the community, etc).
Ideally, more people would step forward to help with these issues, especially given that many of them can be fixed without much technical expertise on Dune’s internals. However, the fact is that this has not been happening so far.
Regardless of the above, I think the discussion in this thread will probably influence how Dune’s dev team prioritizes usability/documentation issues, but one should keep in mind that there are some structural manpower issues at play in the current situation, as well.
I would really like to use root_modules but can’t because of this issue and the fact every single one of my projects use Logs. I’m surprised I’m the only one hit by that
I often pin packages to a semantic version range to ensure upgrades don’t break compilation, which ends up reading: (dependencies (some-package (and (>= 1.15.0) (< 1.16.0)))). I would like to be able to shorten this to (dependencies (some-package 1.15.x)) or (dependencies (some-package 1.15._)). Happy to submit a patch myself if there is some interest as this looks simple enough.
Nitpicking as it looks a bit tricky to implement given the benefit, but I often (include dune.inc) and it does not regenerate dune.inc before interpreting it, nor does it work if it is initially missing (the manual explicitly states this is not supported). Alternatively some templating/rule generation mechanism could alleviate the need for such includes, but this has already been mentioned in this thread.
Thanks for the great work on dune and happy new year !
@benjamin-thomas the fact that you’re asking this question is a flaw in our documentation. The executables form is quite useful and is not there just to save you a bit of typing. In your example, my_exe and my_exe2 are allowed to use the same shared modules. For example, if you add a.ml, you will be able to access it from my_exe.ml and my_exe2.ml. If you were to use the singular form of executable stanza, you will need an intermediate library for module A to share it.
I’d love to add auto complete to dune but it’s quite a hard problem unfortunately. We have a parser that works like a compiler, but it’s hard to turn that into editor tooling.
Could you benchmark your build with --trace-file. That’s the best way to get to the bottom of slow builds.
Yes, doing incremental build correctly is hard. I would say most problems stem from incorrect build rules written by the user (missing dependencies are quite common). We have a sandboxing mode that we intend to make the default one day to make it easier to catch such bugs.
@bluddy_module_.ml was already discussed. Hopefully there will be a mode to enable it this year. I proposed adding a subdir stanza to allow users to write dune files inside the dune-project file. E.g.
@gasche Thanks for understanding. We are indeed not apathetic and would like to improve UX. Unfortunately there are resource constraints and organizational issues in the real world. I think the manual stagnates because the entire team agrees that it’s “bad”, and therefore has no incentive to improve something that will be discarded eventually anyway. We should probably adopt a more gradual plan for improving the manual.
I’ll try to look over all your other tickets this week and at least respond with an estimate of how hard would it be solve these issues.
Regarding compiler errors, to separate individual errors newlines would require us to parse compiler output, no? Sometimes the compiler emits more than one error/warning per compilation unit.
@Kakadu Yes, custom templates would be quite useful. I’m afraid they’re not important enough for me to tackle without additional help unfortunately.
oh dear, I wasn’t aware of that either; for the longest time I assumed that you couldn’t share modules between executables at all, which was indeed quite annoying. Great to know that it’s actually possible…!
Perhaps the error message for the case where several executable clauses share a module could be tweaked. The current wording points towards using a modules clause and reads as being a bit too definitive:
To fix this error, you must specify an explicit "modules" field in every
library, executable, and executables stanzas in this dune file. Note that
each module cannot appear in more than one "modules" field - it must belong
to a single library or executable.
I would tend to read this as asserting that there is just no way for executables to share a module.
Perhaps the message could, as a hint, suggest using executables?