Ppx in the toplevel: how does it work?

Learning
,
1

I would like to understand what happens precisely when loading a ppxlib-flavoured ppx in the toplevel. For instance:

$ocaml -noinit
# #use "topfind";;
[...]
# #require "ppx_sexp_conv";;
[...]
# type t = A [@@deriving sexp];;
[...]

More precisely, when interpreting a phrase, Toploop.loop calls Toploop.preprocess_phrase that calls Pparse.apply_rewriters_str. This function invokes PPX rewriters stored in Clflags.all_ppx which is a list (reference) whose elements are strings corresponding to file system paths of executables. So I can only imagine that at some point after calling #require somebody updates Clflags.all_ppx with some value. This is this “somebody” and that “some value” that I’m after.

I looked into ocamlfind, ocaml-migrate-parsetree and ppxlib but I couldn’t find anything useful, although I really suspect the magic happens in ocamlfind…

2

It indeed happens in findlib:

  • the directive #require triggers a call to Topfind.load_deeply, which in turn calls Topfind.load
  • Topfind.load queries a package for a ppx and ppxopt property (which are part of the META file of the package)
  • if available their values are used to construct and call a #ppx directive, which is part of the standard directives
  • the ppx directive sets the value of Clflags.all_ppx, no more no less.

I started digging on this because I couldn’t make the toplevel work in presence of ppx rewriters when calling the functions from Toploop inside a program rather than using the CLI of the toplevel (ocaml or utop). I ended up understanding that there was absolutely no need no know about all this, I just had forgotten to call Toploop.preprocess_phrase before calling Toploop.execute_phrase. And here I was, wondering why the ppx transformations were not performed :sweat_smile:.

3 Likes