[ANN] Dune 1.10.0

Community
, ,
1

On behalf of the dune team, I’m pleased to announce the 1.10.0 release of dune. This release is packed with bug fixes, but it also introduces a few interesting features. I’ll highlight one important feature that we’ve introduced and plan to improve in future versions:

Dune allows you to specify package metadata in the dune project file and generate an appropriate opam file. This is advantageous to writing opam files manually as dune will correctly fill in some boilerplate such as the build command:

Here’s a snippet from dune’s own project file as a demonstration:

(generate_opam_files true) ;; necessary to enbale the generation

(license MIT)
(maintainers "Jane Street Group, LLC <opensource@janestreet.com>")
(authors "Jane Street Group, LLC <opensource@janestreet.com>")
(source (github ocaml/dune))
(documentation "https://dune.readthedocs.io/")

(package
 (name dune)
 (depends
  (ocaml (>= 4.02))
  base-unix
  base-threads)
 (conflicts
  (jbuilder (<> "transition"))
  (odoc (< 1.3.0))
  (dune-release (< 1.3.0)))
 (synopsis "Fast, portable and opinionated build system")
 (description "<redacted>"))

Generating the opam file is done via $ dune build @check (the @install, and @all aliases will also work).

Later, we plan to have dune do some sanity checks on the depends field. In the more distant future, we might even skip the generation step altogether by having opam invoke dune to get the generated opam file directly.

As usual, the change log is replicated here for your convenience:

1.10.0 (04/06/2019)

  • Restricted the set of variables available for expansion in the destination
    filename of install stanza to simplify implementation and avoid dependency
    cycles. (#2073, @aalekseyev, @jeremiedimino)

  • [menhir] call menhir from context root build_dir (#2067, @ejgallego,
    review by @jeremiedimino, @rgrinberg)

  • [coq] Add coq.pp stanza to help with pre-processing of grammar
    files (#2054, @ejgallego, review by @rgrinberg)

  • Add a new more generic form for the promote mode: (promote (until-clean) (into <dir>)) (#2068, @jeremiedimino)

  • Allow to promote only a subset of the targets via (promote (only <pred>)). For instance: (promote (only *.mli)) (#2068, @jeremiedimino)

  • Improve the behavior when a strict subset of the targets of a rule is already
    in the source tree for projects using the dune language < 1.10 (#2068, fixes
    #2061, @jeremiedimino)

  • With lang dune >= 1.10, rules in standard mode are no longer allowed to
    produce targets that are present in the source tree. This has been a warning
    for long enough (#2068, @jeremiedimino)

  • Allow %{…} variables in pps flags (#2076, @mlasson review by @jeremiedimino and
    @aalekseyev).

  • Add a ‘cookies’ option to ppx_rewriter/deriver flags in library stanzas. This
    allow to specify cookie requests from variables expanded at each invocation of
    the preprocessor. (#2106, @mlasson @jeremiedimino)

  • Add more opam metadata and use it to generate .opam files. In particular, a
    package field has been added to specify package specific information.
    (#2017, #2091, @avsm, @jonludlam, @rgrinberg)

  • Clean up the special support for findlib.dynload. Before, Dune would simply
    match on the library name. Now, we only match on the findlib package name when
    the library doesn’t come from Dune. Someone writing a library called
    findlib.dynload with Dune would have to add (special_builtin_support findlib_dynload) to trigger the special behavior. (#2115, @jeremiedimino)

  • Install the future_syntax preprocessor as ocaml-syntax-shims.exe (#2125,
    @rgrinberg)

  • Hide full command on errors and warnings in development and show them in CI.
    (detected using the CI environment variable). Commands for which the
    invocation might be omitted must output an error prefixed with File. Add an
    --always-show-command-line option to disable this behavior and always show
    the full command. (#2120, fixes #1733, @rgrinberg)

  • In dune-workspace files, add the ability to choose the host context and to
    create duplicates of the default context with different settings. (#2098,
    @TheLortex, review by @jeremiedimino, @rgrinberg and @aalekseyev)

  • Add support for hg in dune subst (#2135, @jeremiedimino)

  • Don’t build documentation for implementations of virtual libraries (#2141,
    fixes #2138, @jonludlam)

  • Fix generation of the -pp flag in .merlin (#2142, @rgrinberg)

  • Make dune subst add a (version ...) field to the dune-project
    file (#2148, @jeremiedimino)

  • Add the %{os_type} variable, which is a short-hand for
    %{ocaml-config:os_type} (#1764, @jeremiedimino)

  • Allow enabled_if fields in library stanzas, restricted to the
    %{os_type}, %{model}, %{architecture}, %{system} variables (#1764,
    #2164 @jeremiedimino, @rgrinberg)

  • Fix chdir on external and source paths. Dune will also fail gracefully if
    the external or source path does not exist (#2165, fixes #2158, @rgrinberg)

  • Support the .cc extension fro C++ sources (#2195, fixes #83, @rgrinberg)

  • Run ocamlformat relative to the context root. This improves the locations of
    errors. (#2196, fixes #1370, @rgrinberg)

  • Fix detection of README, LICENSE, CHANGE, and HISTORY files. These
    would be undetected whenever the project was nested in another workspace.
    (#2194, @rgrinberg)

  • Fix generation of .merlin whenever there’s more than one stanza with the
    same ppx preprocessing specification (#2209 ,fixes #2206, @rgrinberg)

  • Fix generation of .merlin in the presence of the copy_files stanza and
    preprocessing specifications of other stanazs. (#2211, fixes #2206,
    @rgrinberg)

  • Run refmt from the context’s root directory. This improves error messages in
    case of syntax errors. (#2223, @rgrinberg)

  • In .merlin files, don’t pass -dump-ast to the future_syntax preprocessor.
    Merlin doesn’t seem to like it when binary AST is generated by a -pp
    preprocessor. (#2236, @aalekseyev)

  • dune install will verify that all files mentioned in all .install files
    exist before trying to install anything. This prevents partial installation of
    packages (#2230, @rgrinberg)

23 Likes
2

Is it normal to still have the warning:

Warning: .merlin generated is inaccurate. Cannot mix preprocessed and non preprocessed specificiations.
Split the stanzas into different directories or silence this warning by adding (allow_approximate_merlin) to your dune-project.

with dune 1.10.0 and dune-project 1.9 ?

3

It still normal. The warning says that dune cannot generate an accurate .merlin for that directory. This is due to a limitation of the .merlin format. There’s not much dune can do.

1 Like
5

I’ve just started incorporating OPAM package specifications into my dune-project files. So far that seems to be working really well. I believe it’s going to be another step towards making packaging a simpler and more coherent experience.

But before making new releases based on this specification, I’d prefer waiting for OPAM or dune-release support for this new kind of format. I’d rather not have redundant specifications in a project tree. It’s just error-prone when making releases, e.g. forgetting to regenerate OPAM files, or editing the wrong file.

@samoht Are there any plans to depend on dune-project for generating OPAM package specifications?

6

There is some plans to use dune in order to generate opam files. You’ll find in theses open issues the relevant discussions.

1 Like
7

Thanks, if I understood these discussions correctly, it will be necessary to keep a pre-generated opam file in the distribution for the foreseeable future. Among other things, so that users of older OPAM installations can work with these distributions.

8

That’s correct, you need to keep opam files around for now. However, dune will generally promote and regenerate those opam files pretty systematically if they get out of sync. Have you had any problems with them not being promoted after you make a change to dune-project and call a dune build?

9

Not really.

Note that in general you don’t need to provide opam files in your distribution tarballs (except the one you install for odig but that one can be happily generated and installed by your build system).

For now you need to keep opam files in your VCS repository only if you want to let your users opam pin your repo. If this proposal gets implemented you will then only need to keep a single static opam file that instructs how opam can generate them when the repo is opam pinned.

At that point you may want to keep older generated file in the repo only if you wish older opam versions to be able to pin your repo – but that’s really only for pinning your repo so I wouldn’t mind requiring from those users that they use the a version of opam that has the proposal implemented.

2 Likes
10

Good point, Dune apparently regenerates the OPAM file if necessary for most build commands anyway. It seems almost impossible to accidentally forget about regenerating it before making a release.

11

I’ve read the proposal and hope it will make it into OPAM. This will probably still take a while. For now it doesn’t seem too much of a hassle for Dune users to let it update their OPAM file before final commits when making new releases.