[ANN] Dune 2.0.0

Community
,
1

On behalf of the dune team, I’m delighted to announce the release of dune 2.0. This release is the culmination of 4 months of hard work by the dune team and contains new features, bug fixes, and performance improvements . Here’s a selection of new features that I personally find interesting:

  • New boostrap procedure that works in low memory environments
  • (deprecated_library_name ..) stanza to properly deprecate old library names
  • (foreign_library ..) stanza to define C/C++ libraries.
  • C stubs directly in OCaml executables

Refer to the change log for an exhaustive list.

We strive for a good out of the box experience that requires no configuration, so we’ve also tweaked a few defaults. In particular, $ dune build will now build @all instead of @install, and ocamlformat rules are setup by default.

Lastly, dune 2.0 sheds all the legacy related to jbuilder and will no longer build jbuilder projects. This change is necessary to ease maintenance and make it easier to add new features down the line. There are a few other minor breaking changes. Refer to the change log for the full list. We apologize in advance for any convenience this might cause.

Happy Hacking!

2.0.0 (20/11/2019)

  • Remove existing destination files in install before installing the new
    ones. (#2885, fixes #2883, @bschommer)

  • The action field in the alias stanza is not available starting lang dune 2.0. The alias field in the rule stanza is a replacement. (#2846, fixes
    2681, @rgrinberg)

  • Introduce alias and package fields to the rule stanza. This is the
    preferred way of attaching rules to aliases. (#2744, @rgrinberg)

  • Add field (optional) for executable stanzas (#2463, fixes #2433, @bobot)

  • Infer targets for rule stanzas expressed in long form (#2494, fixes #2469,
    @NathanReb)

  • Indicate the progress of the initial file tree loading (#2459, fixes #2374,
    @bobot)

  • Build .cm[ox] files for executables more eagerly. This speeds up builds at
    the cost of building unnecessary artifacts in some cases. Some of these extra
    artifacts can fail to built, so this is a breaking change. (#2268, @rgrinberg)

  • Do not put the <package>.install files in the source tree unless -p or
    --promote-install-files is passed on the command line (#2329, @jeremiedimino)

  • Compilation units of user defined executables are now mangled by default. This
    is done to prevent the accidental collision with library dependencies of the
    executable. (#2364, fixes #2292, @rgrinberg)

  • Enable (explicit_js_mode) by default. (#1941, @nojb)

  • Add an option to clear the console in-between builds with
    --terminal-persistence=clear-on-rebuild

  • Stop symlinking object files to main directory for stanzas defined jbuild
    files (#2440, @rgrinberg)

  • Library names are now validated in a strict fashion. Previously, invalid names
    would be allowed for unwrapped libraries (#2442, @rgrinberg)

  • mli only modules must now be explicitly declared. This was previously a
    warning and is now an error. (#2442, @rgrinberg)

  • Modules filtered out from the module list via the Ordered Set Language must
    now be actual modules. (#2442, @rgrinberg)

  • Actions which introduce targets where new targets are forbidden (e.g.
    preprocessing) are now an error instead of a warning. (#2442, @rgrinberg)

  • No longer install a jbuilder binary. (#2441, @jeremiedimino)

  • Stub names are no longer allowed relative paths. This was previously a warning
    and is now an error (#2443, @rgrinberg).

  • Define (paths …) fields in (context …) definitions in order to set or
    extend any PATH-like variable in the context environment. (#2426, @nojb)

  • The diff action will always normalize newlines before diffing. Perviousy, it
    would not do this normalization for rules defined in jbuild files. (#2457,
    @rgrinberg)

  • Modules may no longer belong to more than one stanza. This was previously
    allowed only in stanzas defined in jbuild files. (#2458, @rgrinberg)

  • Remove support for jbuild-ignore files. They have been replaced by the the
    dirs stanza in dune files. (#2456, @rgrinberg)

  • Add a new config option sandboxing_preference, the cli argument --sandbox,
    and the dep spec sandbox in dune language. These let the user control the level of
    sandboxing done by dune per rule and globally. The rule specification takes precedence.
    The global configuration merely specifies the default.
    (#2213, @aalekseyev, @jeremiedimino)

  • Remove support for old style subsystems. Dune will now emit a warning to
    reinstall the library with the old style subsystem. (#2480, @rgrinberg)

  • Add action (with-stdin-from ) to redirect input from
    when performing . (#2487, @nojb)

  • Change the automatically generated odoc index to only list public modules.
    This only affects unwrapped libraries (#2479, @rgrinberg)

  • Set up formatting rules by default. They can be configured through a new
    (formatting) stanza in dune-project (#2347, fixes #2315, @emillon)

  • Change default target from @install to @all. (#2449, fixes #1220,
    @rgrinberg)

  • Include building stubs in @check rules. (@rgrinberg, #2530)

  • Get rid of ad-hoc rules for guessing the version. Dune now only
    relies on the version written in the dune-project file and no
    longer read VERSION or similar files (#2541, @jeremiedimino)

  • In (diff? x y) action, require x to exist and register a
    dependency on that file. (#2486, @aalekseyev)

  • On Windows, an .exe suffix is no longer added implicitly to binary names that
    already end in .exe. Second, when resolving binary names, .opt variants are no
    longer chosen automatically. (#2543, @nojb)

  • Make (diff? x y) move the correction file (y) away from the build
    directory to promotion staging area. This makes corrections work with
    sandboxing and in general reduces build directory pollution. (#2486,
    @aalekseyev, fixes #2482)

  • c_flags, c_names and cxx_names are now supported in executable
    and executables stanzas. (#2562, @nojb)
    Note: this feature has been subsequently extended into a separate
    foreign_stubs field. (#2659, RFC #2650, @snowleopard)

  • Remove git integration from $ dune upgrade (#2565, @rgrinberg)

  • Add a --disable-promotion to disable all modification to the source
    directory. There’s also a corresponding DUNE_DISABLE_PROMOTION environment
    variable. (#2588, fix #2568, @rgrinberg)

  • Add a forbidden_libraries field to prevent some library from being
    linked in an executable. This help detecting who accidently pulls in
    unix for instance (#2570, @jeremiedimino)

  • Fix incorrect error message when a variable is expanded in static context:
    %{lib:lib:..} when the library does not exist. (#2597, fix #1541,
    @rgrinberg)

  • Add --sections option to $ dune install to install subsections of .install
    files. This is useful for installing only the binaries in a workspace for
    example. (#2609, fixes #2554, @rgrinberg)

  • Drop support for jbuild and jbuild-ignore files (#2607, @jeremiedimino)

  • Add a dune-action-plugin library for describing dependencies direcly in
    the executable source. Programs that use this feature can be run by a new
    action (dynamic-run …). (#2635, @staronj, @aalekseyev)

  • Stop installing the ocaml-syntax-shims binary. In order to use
    future_syntax, one now need to depend on the ocaml-syntax-shims
    package (#2654, @jeremiedimino)

  • Add support for dependencies that are re-exported. Such dependencies
    are marked withre_export and will automatically be provided to
    users of a library (#2605, @rgrinberg)

  • Add a deprecated_library_name stanza to redirect old names after a
    library has been renamed (#2528, @jeremiedimino)

  • Error out when a preprocessor_deps field is present but not
    preprocess field is. It is a warning with Dune 1.x projects
    (#2660, @Julow)

  • Dune will use -output-complete-exe instead of -custom when compiling
    self-contained bytecode executables whenever this options is available
    (OCaml version >= 4.10) (#2692, @nojb)

  • Add action (with-accepted-exit-codes <pred> <action>) to specify the set of
    successful exit codes of <action>. <pred> is specified using the predicate
    language. (#2699, @nojb)

  • Do not setup rules for disabled libraries (#2491, fixes #2272, @bobot)

  • Configurator: filter out empty flags from pkg-config (#2716, @AltGr)

  • no_keep_locs is a no-op for projects that use lang dune older than 2.0. In
    projects where the language is at least 2.0, the field is now forbidden.
    (#2752, fixes #2747, @rgrinberg)

  • Extend support for foreign sources and archives via the (foreign_library ...)
    stanza as well as the (foreign_stubs ...) and (foreign_archives ...) fields.
    (#2659, RFC #2650, @snowleopard)

  • Add (deprecated_package_names) field to (package) declaration in
    dune-project. The names declared here can be used in the (old_public_name)
    field of (deprecated_library_name) stanza. These names are interpreted as
    library names (not prefixed by a package name) and appropiate redirections are
    setup in their META files. This feaure is meant to migrate old libraries which
    do not follow Dune’s convention of prefixing libraries with the package
    name. (#2696, @nojb)

  • The fields license, authors, maintainers, source, bug_reports,
    homepage, and documentation of dune-project can now be overriden on a
    per-package basis. (#2774, @nojb)

  • Change the default modes field of executables to (mode exe). If
    one wants to build a bytecode program, it now needs to be explicitly
    requested via (modes byte exe). (#2851, @jeremiedimino)

  • Allow ccomp_type as a variable for evaluating enabled_if. (#2855, @dra27,
    @rgrinberg)

  • Stricter validation of file names in select. The file names of conditional
    sources must match the prefix and the extension of the resultant filename.
    (#2867, @rgrinberg)

  • Add flag disable_dynamically_linked_foreign_archives to the workspace file.
    If the flag is set to true then: (i) when installing libraries, we do not
    install dynamic foreign archives dll*.so; (ii) when building executables in
    the byte mode, we statically link in foreign archives into the runtime
    system; (iii) we do not generate any dll*.so rules. (#2864, @snowleopard)

  • Reimplement the bootstrap procedure. The new procedure is faster and
    should no longer stack overflow (#2854, @dra27, @jeremiedimino)

  • Allow .opam.template files to be generated using rules (#2866, @rgrinberg)

24 Likes
2

Hello dune devs,
seems the simple way of using js_of_ocaml like in the following CLI interaction has gone in dune 2.0 :

philou@air:~/mysrc/tmp/foo$ dune build
philou@air:~/mysrc/tmp/foo$ ./_build/default/foo.exe
hello from js

philou@air:~/mysrc/tmp/foo$ dune build ./foo.bc.js
Error: Don’t know how to build ./foo.bc.js

What is now the proper way to use js_of_ocaml from dune?
THAAAANK YOU!

3

ahh in the dune file of project foo :
(executable (name foo)
(modes js)) <<---- was missing, now seems required

4

Congrats on the release!

Can you remind me folks what’s the compat story for opam users, Dune 2.0, and compilers less than 4.06.0 ?

As far as I can see it is not possible yet to depend on Dune >= 2.0 and get a working binary from OPAM for compilers older than 4.06.0, right?

5

At the moment it’s indeed impossible. Perhaps in the future, it will be possible to install the dune binary in opam switches with an older compiler. This will require some additional support from opam.

6

Pity, that may prevent some projects from moving to 2.0.0 at least for a while; I guess that hacking Dune’s makefile to build its own OCaml would be too fragile, right?

7

Yeah, that seems a bit drastic.

To be honest, I have some suspicion that our advancement in OCaml versions is too rapid. However, the option of users staying with an older version of dune is fine in my opinion. At this point, dune is fairly mature software and 1.11 should be perfectly usable for most people’s needs.

9

Hey, this is great news, thanks! Is there an upgrade guide? Our builds are getting dune 2.0 automatically through dependencies and broken with something about no inline_tests backends found.

1 Like
10

Congrats! Can’t wait to try it. However I could not install it due to the following:

The following packages are not being upgraded because the new versions conflict with other installed packages:
  - dune.2.0.0
    ∗ jbuilder.transition is installed and requires dune < 2.0

Any suggestions on how to fix that? I could not uninstall jbuilder package because a buch of packages like core depends on it.

11

Lastly, dune 2.0 sheds all the legacy related to jbuilder and will no longer build jbuilder projects. This change is necessary to ease maintenance and make it easier to add new features down the line.

You’ll probably be unable to upgrade until the projects that depend on jbuilder upgrade to dune.

12

Which version of inline_tests are you using?

There will be a migration document soon.

13

The project in question here already upgraded to dune in later versions. I suggest that you finish upgrading core and then attempt upgrading to the latest dune.

14

Dune’s bootstrap actually supports building with a secondary compiler :slight_smile: @dra27 added support for this. We would just need to release this secondary compiler in opam to make it possible to build dune 2.x in older switches.

That said, I agree with @rgrinberg that sticking to the 1.11 language is a fine choice as well.

And in any case, it’s also important for OCaml projects to upgrade to new compilers. All these new OCaml features are meant to be used and relied upon!

15

This already works, I just need to update and submit the package for it to opam-repository:

dra@eoan:~/dune$ git checkout 2.0.0
HEAD is now at baab63c6 Update release date for 2.0
dra@eoan:~/dune$ ocamlc -vnum
4.02.3
dra@eoan:~/dune$ ocaml bootstrap.ml
…
ocamlfind -toolchain secondary ocamlc 2>.duneboot.ocamlfind-output

ocamlfind -toolchain secondary ocamlc -custom -w -24 -g -o .duneboot.exe -I boot unix.cma boot/libs.ml boot/duneboot.ml
./.duneboot.exe --secondary
dra@eoan:~/dune$ 

dra@eoan:~/dune$ ./dune.exe --version
2.0.0
1 Like
16

Would it be reasonable to create a new jbuilder.transition package that is the previous version of dune but only installing the jbuilder program so it does not conflict with dune 2.0?

17

Actually, the original jbuilder packages (1.0+betaXX) are co-installable with Dune 2.0.0. It’s just that they are not compatible with OCaml >= 4.08, so if you are using a recent compiler you cannot co-install Dune 2.0.0 and jbuilder.

18

and here is an attempt to get a 4.08 compatible standalone jbuilder: https://github.com/ocaml/opam-repository/pull/15427

19

Attempted to install dune 2.0.0, but was blocked by core_kernel. (v0.12.0)

The following dependencies couldn't be met:
  - core_kernel → dune < 2.0.0
Your request can't be satisfied:
  - No available version of dune satisfies the constraints
20

resolved by “core_kernel.v0.12.3”

now blocked by “re2.v0.12.0”

will go through these and send final notes…

21

Couldn’t finish due to some older packages still using jbuilder (bitstring)