Another question on PPX rewriters and functors

Learning
1

Hello,

IIUC, PPX rewriters must be registered at the very beginning of the compilation process.

Does this mean that such rewriters can not depend on an module that would be defined by the program itself, typically by a functor ?

For example, suppose i have a module Expr parameterized by a signature CONST, decriving the litteral constants that can occur in such expressions:

module type CONST = sig ... end
module type EXPR = sig ... end
module Make(C: CONST) : EXPR

and, instead of writing expression values as (for ex):

let e1 = EBinop ("+", EConst 1, EConst 2)

i would like to denote with in concrete syntax using a dedicated parser, called by a PPX rewriter:

let e1 = [%expr "1+2"]

It seems that i cannot write such a rewritter (in expr_ppx.ml) because the actual definition of the parser is not known when registering this rewriter : it is sth like

Expr.Make(MyConst).parser

where MyConst, the actual type of constants, in only known by the program.

Am i right or is there a way to “defer” the registration of PPX rewritters such that can have some modules defined by the target program in scope ?

I suspect that the answer in no, but would appreciate any confirmation…

Jocelyn

2

I don’t completely follow your description, but I think the answer is “no”. Think of PPX rewriters as complete programs that preprocess your code. So there’s a “T diagram” you can draw of the various source and binary artifacts involved, and questions of scope don’t enter into it (naturally) because a preprocessor must be completely compiled-and-linked, before it can be run on your source code.

3

If think you’re right (although i don’t really understand what you mean by “T diagram” here). It’s quite obvious if you think of PPX rewriters as pre-processors.

However, i think the sentence

because a preprocessor must be completely compiled-and-linked, before it can be run on your source code.

is a bit misleading.
Consider, for example, this definition of a PPX rewriter for a non-parameterized Expr module:


open Ppxlib

let name = "expr"
         
let expand ~loc ~path:_ (s:_) =
  let _ =
    try Expr.of_string s
    with Expr.Parse_error -> Location.raise_errorf ~loc "cannot parse expression: \"%s\"" s in
  let e = Ast_builder.Default.estring ~loc s in
  [%expr Expr.of_string [%e e]]

let ext =
  Extension.declare
    name
    Extension.Context.expression
    Ast_pattern.(single_expr_payload (estring __))
    expand

let () = Ppxlib.Driver.register_transformation name ~extensions:[ext]

where the function Expr.of_string here implements the parser.
This one is perfectly legal and works.
The only difference is that the Expr module can be compiled the PPX rewriter, although, in the original case, it can’t - IIUC…

4

See this wikipedia article.

5

Regarding your example code, one problem is that people use dune, which obscures that your little .cmo file is linked into an executable, which itself is used to preprocess source code. But that’s what’s really going on, as you can clearly see in the _build/log file.

6

@dbuenzli
@Chet_Murthy
Thanks for the tips !