Alcotest does not run all my tests

Ecosystem
,
1

I am using alcotest with dune and I am encountering a strange (buggy?) behavior where dune
runs twice the same test and fails to run another test. I’m not sure if the problem
comes from dune or alcotest. It will be easier to explain on a concrete minimal example:

$ tree
.
├── bin
│   ├── dune
│   └── main.ml
├── dune-project
└── test
    ├── a.ml
    ├── a.mli
    ├── b.ml
    └── dune

2 directories, 7 files
$ dune runtest --force
Testing `a.ml'.                    
This run has ID `5ZVAJR9B'.

  [OK]          azer          0   azert.

Full test results in `xxx/_build/default/test/_build/_tests/a.ml'.
Test Successful in 0.000s. 1 test run.
Testing `a.ml'.                    
This run has ID `62AWYKQ2'.

  [OK]          azer          0   azert.

Full test results in `xxx/_build/default/test/_build/_tests/a.ml'.
Test Successful in 0.000s. 1 test run.

As you can see, a.ml is run twice but not b.ml. This is because the latter uses a value from the former. Here is the full project:

$ cat dune-project
(lang dune 3.15)
(package
 (name alcobug2_name)
 (depends ocaml dune
  (alcotest :with-test)))
$ for i in $(ls **/*); do echo "(******* $i *******)"; cat $i; done
(******* bin/dune *******)
(executable
 (public_name alcobug2_public)
 (name main))
(******* bin/main.ml *******)
let () = print_endline "Hello"
(******* test/a.ml *******)
let x = 1
let () = Alcotest.run "a.ml"
  [ ("azer", [ Alcotest.test_case "azert" `Quick (fun () -> ()) ]) ]
(******* test/a.mli *******)
val x:int
(******* test/b.ml *******)
let _ = A.x
let () = Alcotest.run "b.ml"
  [ ("bazer", [ Alcotest.test_case "bazert" `Quick (fun () -> ()) ]) ]
(******* test/dune *******)
(tests
 (names a b)
 (libraries alcotest))

In my use case, I am of course not reusing an integer A.x, but helper functions for my tests.

Does anyone have an idea of where this problem comes from and maybe of a workaround?

2

I think you are asking for trouble using an executable (the a.ml tests) as also a library.

Since b is linked with a, when b is executed to run tests it will first need to execute the code in its dependency a, and this includes running Alcotest.run. I would guess Aloctest.run has an exit in it that terminates the program in a.ml, so it never gets to the run in b.ml.

I suggest putting shared code between your two tests in a separate library which both depend on.