Junit reports from various testing frameworks, libraries?

Ecosystem
1

Is anybody outputting junit reports and integrating them with some CI system? If so, which OCaml testing libraries are you using?

2

I set up alcotest and junit_alcotest, hereā€™s the main test code: re-web/ReWebTest.ml at 7b407c1b98ddfac32a81c32481c40cc535c7452a Ā· yawaramin/re-web Ā· GitHub

CI tools like like Azure Pipelines (which Iā€™m using for this project) can pick up the junit.xml file that gets produced during the build and, and report the results. Iā€™ve verified this personally, although sadly Azure has lost past test run results due to some flakiness.

2 Likes
3

I use oth, which needs a refresh in opam, which outputs TAP

1 Like
4

I used to do that, using OUnit directly, I think. The junit library
can also be useful to output results by hand.

1 Like
5

Iā€™m using junit_alcotest and Github Actions

the reports get written to _build/default/tests/<filename> which can be specified in the test like Junit.to_file report "<filename>"

the action looks like:

    steps:
      - uses: actions/checkout@v2

      - uses: ocaml/setup-ocaml@v2
        with:
          ocaml-compiler: 4.12.x

      - run: opam install . --deps-only --with-test
      - run: opam exec -- dune test --force

      - name: Publish test results
        uses: EnricoMi/publish-unit-test-result-action@v1
        if: always()
        with:
          check_name: Dune unit test results
          comment_on_pr: false
          files: _build/default/tests/junit-*.xml

test failures are reported as ā€œannotationsā€ in the actions UI

2 Likes
6

Out of curiosity, when JUnit format could be useful?

7

CI tools like GitHub/etc. understand JUnit format and present it in their UIs with nice rendering.

2 Likes
8

Ah, OK. I forgot that running all tests on local machine may be costly and biased.

9

It is also not great if you want to accept PRs, then you have to check out the branches locally that they pass the tests and so must all contributors remember to run the tests. Generally a rather fragile setup prone to someone forgetting and then having broken code committed.

3 Likes