[ANN] "Cram tests: a hidden gem of dune" and "Snapshot tests for your own ppx"

Community
,
1

Hi, I wrote 2 blog posts about cram tests and It’s a good idea to share them together.

Cram tests: a hidden gem of dune

I’m a strong advocate of unit tests, I can confidently say that it has saved me from introducing regressions countless times. Today I want to share one of the hidden gems of OCaml and their testing story with dune, cram tests.

Snapshot tests for your own ppx

When building preprocessor extensions (ppx) in OCaml, testing is crucial. You want to ensure your ppx works correctly and continues to work as you make changes. After experimenting with different approaches, I’ve found that cram tests fit well for the task.

Let me know what you think, and if there’s a need for more :smiley:

17 Likes
2

Regarding the “Make your executable available under cram tests” section, there’s a fourth way that I really like. Namely, using (env ...) with (binaries ...): env - Dune documentation.
The benefit is that you can make certain executables available in tests, as if they’re public/installed, without actually making them public/installing them. All while avoiding a bunch of relative path business.

This is good for having some cram test utility executables (e.g. for stripping some custom output to make it deterministic). You wouldn’t want to make them public/install them, but referring to them by a long relative path through some cram test folder hierarchy is also annoying. This solves both problems.

4 Likes
3

I do something similar except without cram: fun-sql/test_ppx/dune at main · yawaramin/fun-sql · GitHub

This has three components:

  1. The executable which runs the PPX driver in standalone mode
  2. The rule which generates the output OCaml code test.actual.ml from the input test.ml
  3. The test which diffs the files
1 Like
4

This was I did previously on a few projects and moved away from it

5

TIL, thanks!

Do you have any project where this is being used? I would love to explore and add it to the blog post.

6

Personally I’ve used it in Goblint.
But grepping for (binaries in some sources I have on my machine also comes up with ppxlib, odoc and vscode-ocaml-platform, although I haven’t looked closely at their uses.

1 Like
7

Thanks for the suggestion, I tried and it’s a great trick.

added here: Add env binaries as an option · davesnx/sancho.dev@a6c3b4c · GitHub

also, I added a link to this post, but let me know if your github or twitter is more suited to credit you :stuck_out_tongue:

8

Apparently cram requires sh which is not portable to Windows CRAM tests on windows · Issue #10872 · ocaml/dune · GitHub

Not relevant for everyone of course but good to know I guess.