Dune Cram tests: preloading aliases/scripts

Ecosystem
,
1

We are considering large-scale adoption of Cram tests, but we are missing some engineering features that are needed to ensure we don’t have to manually update hundreds of files when we need to apply some global changes.

For instance, in several of our tests, we need to build a small OCaml module before running tests with it. So we currently have several tests which start with:

$ dune build --root . --disable-cache _build/default/foo.cmxs

We’d like to “wrap” some of this in an alias or shell function, to ensure that all tests uniformly include the same options and future changes will be automatically propagated to all of them. To sum up, we want a preamble of some sort.

For instance, we could define a cram_preamble.sh file and then run:

$ source cram_preamble.sh

In fact, it looks more like this:

$ source ../../../../../../../cram_preamble.sh

Because we are starting at build/.sandbox/57632bdb88b19d3ee68c697d605883ba/default/tests/foo/mytest.t.

Besides the ugliness, one of the issues is that our Cram tests are stored in several different directories, with different nesting levels, so the number of ../ will vary per test.

Overall, this seems doable, but I wonder if I’m missing a simpler way to do things. Maybe I can do something with the cram stanza in a dune file? Or using the runtest alias?

1 Like
2

In my project, I use this trick to add a test library in the path:

My dune file contains this code:

; Add the test library in the path

(env
 (_
  (binaries %{project_root}/test/tests.exe)))

; And depend of it. The binary will be available in the
; cram tests under the name "tests"

(cram
 (deps %{bin:tests}))

This is the part of the dune documentation which defines the trick:

(binaries <binaries>), where <binaries> is a list of entries of the form (<filepath> as <name>). (<filepath> as <name>) makes the binary <filepath> available in the command search as just <name>. For instance, in a (run <name> ...) action, <name> will resolve to this file path. You can also write just the file path, in which case the name will be inferred from the basename of <filepath> by dropping the .exe suffix, if it exists. For example, (binaries bin/foo.exe (bin/main.exe as bar)) would add the commands foo and bar to the search path.

1 Like
3

The dune team has been running into this problem for a while now as well, and we’ve recently sought a solution. Starting from version 3.21, you’ll be able to write:

(cram
 (setup_scripts cram_preamble.sh))

And have this shell script be sourced in every single cram test this stanza is applied to.

Here are the docs, but there’s not much more to it than what I’ve just described.

4 Likes
4

Thanks! It will be very useful when we can force our users to upgrade to dune 3.21.

Until then, I found out that using $DUNE_SOURCEROOT, which I didn’t know exist, allows setting up a not-so-ugly path:

  $ source $DUNE_SOURCEROOT/cram_prelude.sh

But it’ll definitely be better to use the setup_scripts stanza.

5

I think, if you don’t define a dependency on this file, then tests won’t re-run when the file changes. That can be confusing and non obvious to debug.

2 Likes