Automatic generation of PPX debug artifacts

Learning
,
1

Some PPX regressions are very difficult to catch by conventional testing - for example, changes in how Location data is propagated. In order to make these regressions more visible, I would like for dune to generate certain debug artifacts using the PPX binaries and promote these artifacts to the source tree.

When debugging interactively, I find it quite handy that the preprocessors support a standard set of flags for this purpose. For example:

$ _build/default/.ppx/0924ad344eb146e54f1637fccb0792d7/ppx.exe --help
ppx.exe [extra_args] [<files>]
...
-dparsetree                 Print the parsetree (same as ocamlc -dparsetree)
...

However, I’m finding it hard to automate these debug runs in a robust way. In a large project, there may be many PPX binaries under _build/default/.ppx with names that are opaque and presumably unstable, so it’s not clear how to write a dune rule that dispatches to the correct one.

The only way I know to map a source file to its PPX binary is by scraping dune’s log. And since dune truncates the log frequently, it may be necessary to force a rebuild to capture the output at the moment when I need it.

Is there a better way to accomplish this?

2

One approach that comes to mind: use CRAM tests to build small test executables and compile them using the PPXs that you want to test and also passing -dparsetree using the (flags) field of the (executable) stanza. That way you will get the “parsetree” dump directly in the CRAM test and you will be able to use the usual “promote” workflow to accept them, etc.

Makes sense?

Cheers,
Nicolas

3

Thanks @nojb . I like the insight that one can always access the PPX via a single-purpose binary with custom build flags instead of invoking the PPX directly. That’s already a big improvement over the log-scraping hack.

However, I haven’t quite found the right recipe for the automation. Based on your description, I think you meant something like:

dune

(executable
 (name my_parsetree_dumper)
 (public_name my_parsetree_dumper)
 (modules my_parsetree_dumper)
 (preprocess (pps my_ppx))
 (flags (:standard -dparsetree)))

(cram
 (deps %{bin:my_parsetree_dumper}))

my_parsetree_dumper.t

  $ my_parsetree_dumper 2>&1

I do indeed get a verbose stderr from the PPX during the build phase, but nothing in the CRAM output.

The other wrinkle here is that I would like to auto-promote the output if there’s a diff, but with CRAM it looks like I would need for all consumers of the code to use dune build -–auto-promote, whereas a Rule would allow me to specify (mode promote).

Any thoughts?

4

What I had in mind is to also build your my_parsetree_dumper as part of the CRAM test, so you will get the verbose output in the CRAM test as well.

Cheers,
Nicolas