Running subset of alcotests / without running tests for dependencies

Learning
,
1

If I download dune-starter and run the tests like this, I get some output, as expected:

$ dune runtest -f 
   run_tests alias test/runtest
Testing `proj'.
This run has ID `57543AFA-7473-4F1C-8C1F-8247B4380034'.

  [OK]          Sub1.A          0   time.
  [OK]          Sub1.B          0   string.
  [OK]          Sub1.B          1   string, hasty.

Full test results in `~/dev/dune-starter/proj/_build/default/test/_build/_tests/proj'.
Test Successful in 0.000s. 3 tests run.

On the other hand, if I run it with a directory, which the docs say is supposed to run just the tests in that directory, I get no output:

$ dune runtest -f sub1/test

In fact, the second command does not run the tests in that directory: I can break them, but it still exits successfully and prints nothing.

How can I run just the tests in a particular directory? I ask because I want to run just my own tests, not those of opam dependencies (which alcotest/dune seems to do – for example if I add sqlite3 as a dependency, it runs the tests for that package).

1 Like
2

Agh, as soon as I post I see that the run-one-directory thing is a dune feature, but I think dune knows very little about alcotest (it’s just a “custom test”).

So I guess I need to use alcotest more directly to do this…

Edit: I see I can run just the tests for my project (rather than its dependencies) using:

dune runtest -f -p proj

(where proj is the name of the dune project as defined in file dune-project in the name s-expression)

I’d still like to know how to run just a subset of alcotest tests – for example, a single test file or a single test. Tantalisingly there is this: How can you build the test executable in dune without running the executable - #7 by JohnWu – but the gist is gone :frowning:

Edit again: What I was looking for turns out to be this:

dune exec test/run_tests.exe -- --help
1 Like
3

It seems you’ve found the solution for yourself, but just for the record (in case others come along), it’s possible to run test subsets via:

dune exec path/to/test.exe -- test <regex> <numbers>

where <regex> matches the test case names and <indices> are the test indices to run (as shown in the Alcotest output). The help page generated for each Alcotest binary is also included in the top-level of the development repository. I’m currently working on some better documentation / tutorials for Alcotest which will hopefully make these features easier to discover in future :slight_smile:

And indeed: Dune knows nothing about Alcotest’s API, so generally one has to either use dune exec <test_binary> -- <options> or ALCOTEST_OPTION=foo dune runtest.

2 Likes
4

Another option, if the subset is pretty static, is to link different parts of your test suite into different harnesses in different directories (test/componenta/test.exe and test/componenta/test.exe). That way you delegate partitioning and parallelism to dune instead of alcotest: you can run dune runtest test/componentb, and when you run dune runtest the tests will run in parallel.

2 Likes