Changes of Jbuilder's user interface

Ecosystem
,
1

Hi,

The next release of Jbuilder will change a few things in the way Jbuilder runs and reports information to the user, so I thought it was important to communicate about it before the release.

Report all errors

The first change is that Jbuilder now continues and reports all errors instead of stopping as soon as an error is encountered. The old behavior wasn’t deterministic; several consecutive run of Jbuilder without changing the code might have reported different errors. This was especially annoying when running a big testsuite.

For instance, consider the following project:

$ cat jbuild
(library ((name foo)))
$ cat x.ml
let x = 1 + "e"
$ cat y.ml
let y = List.hd "foo"

Both x.ml and y.ml contain a typing error. Jbuilder until version 1.0+beta17 would report either:

  • the error in x.ml
  • the error in y.ml
  • both

depending on the selected number of jobs and other things such as the position of the moon.

With Jbuilder starting from 1.0+beta18, both errors will always be reported. Note that the order in which they will be reported is still non-deterministic except with -j1.

Display modes

Until now, Jbuilder only supported two display modes:

  • the default one, where Jbuilder would print one line per command being run
  • the verbose one, selected with --verbose, where Jbuilder would print the full command lines with lots of colors

There was a request to support a more quiet mode and reporting all errors made it even more relevant. This is because errors ended up being drown in a sea of commands being run.

So we added two new display modes to Jbuilder:

  • a quiet mode, where Jbuilder only reports errors
  • one where Jbuilder displays a status line showing:
    • the number of goals reached
    • the total number of goals to reach to build the requested targets
    • the number of jobs currrently running in parallel

The latter one is the new default. It is possible to change the display mode with the new --display command line option:

  • use --display short for the old default
  • use --display verbose for the verbose mode
  • use --display quiet for the quiet mode
  • use --display progress for the new default

Configuration file

Since things such as the display mode are often a global user setting, we added support for a configuration file.

The configuration is ~/.config/dune/config on Unix systems and Local Settings/dune/config in the user directory on Windows systems.

You can put the two following settings in the configuration file:

;; same as --display MODE
(display MODE)

;; Same as -jN
(jobs N)

You can get more information with jbuilder help config.

Other news

We have settled on what the new files and project layout will be for Dune, you can look at this ticket for more details. As a result we should be able to start working on the migration from Jbuilder to Dune relatively soon.

We also have a much better idea of how plugins will work, so we should be able to start working on adding support for plugins as soon as the migration to Dune is complete. The main concern being to provide a clean, easy to use, well documented and future proof enough build API for users to write plugins.

The main topics we are working on at the moment are good support for inline and expectation tests, as well as doing the necessary changes in Jbuilder to support tools such as ppx_import or features such as menhir --infer.

15 Likes
2

Thanks @jeremiedimino – the change to add a quiet mode is really useful for Real World OCaml, since our examples all run jbuilder and record its output via a script. The working copy of the book will be much improved when we upgrade the scripts to beta18!

3

Is there going to be a generic interface for testing libraries? I’d love to have qcheck or qtest supported for inline tests…

2 Likes
4

For now we focused on adding support for ppx_inline_test and ppx_expect, since these are non-trivial to set up and require cooperation from the build system.

I don’t know much about qcheck and qtest, but if you could open an issue describing briefly how they work and what support would be needed from Jbuilder, we can look into what can be done.

5

Hmm I can see why ppx tools need some help. qtest uses a custom lexer(!) to extract tests from comments with a special header (*$T and (*$Q. qtest parses a bunch of ml files and generates a test module that can be compiled and run like any other executable.

Right now I can already generate and run these tests with dune, even though it’s a bit manual: https://github.com/c-cube/ocaml-containers/blob/master/qtest/jbuild .

1 Like
6

The issue I see with the way you’ve setup qtest is not the amount of boilerplate (which seems quite reasonable), but the amount of wasteful test running and rebuilding that it does. I think you’d much prefer to have a single qtest binary for every single library that you define. This will save you time both rebuilding the tests that are relevant to a particular lib, and similarly only run test binaries that have changed on jbuilder runtest. However, that kind of setup would indeed impose a lot of boilerplate.

@jeremiedimino supporting qtest is basically the same thing as supporting ppx_inline_test but without the ppx part. I also think that it only support testing exported functions.

7

Out of curiosity, why use comments rather than attributes? The latter would give you the same benefit as putting tests in comments and editing plain code is nicer than editing code in comments.

8

qtest is relatively old, and is used in projects like Batteries where compatibility pre-4.02 is still important. Of course if you use jbuilder that is a moot point now.

As for me, I could move to ppx tests, but I’d have > 1000 tests to migrate… :laughing: