PPX rewriter for static expressions?

Ecosystem
1

I searched around a bit, and couldn’t find a PPX rewriter for static code blocks. I wondered if anybody else had done something like this (before I do it myself). The idea is really simple: in ocaml code, if someone writes

..... [%static e] ....

then this rewriter would

  1. replace the extension with a generated variable, e.g. __static_0001__
  2. place the str_item let __static_0001__ = e at the innermost module-scope that is itself only in a path of modules to the outermose scope. That is, a nested sequence of struct...end, no functors, functor-application, let. And immediately prior to the str_item that contained the %static.

That’s it! I can’t offhand think of any problems that might come up, but … well, perhaps somebody else has already attacked this problem, and thought of other issues.

1 Like
2

Well, some problems are pretty obvious. How would you compile let f x = [%static x] ? How would you deal with shadowing when one of these values escape (for instance when module Foo has no mli and module Bar contains let open Foo in just before a [%static] expression) ?

That said, the feature is something that makes sense to me. The hard part is going to find how to implement it correctly/safely.

3

There’s no way to implement it safely until after the typechecker runs. So that means, don’t do that – implement it unsafely. B/c it’s too useful. But you bring up an important point: the variable names need to include some sort of hash of the AST, so that they’re not usable via open. Good point!

ETA: if only there were a way to know the type of the expression, one could get a cheap-as-dirt implementation, by using a file-level array of Obj.t: each [%static ...] gets a number, and the slot gets set at first-reference, then reused thereafter. For the particular use I have in mind, I know the type, so I think I’ll do that.

4

There’s this one for very simple conditional compilation ppx FWIW: GitHub - mcclure/ppx_const: A PPX syntax extension for the OCaml programming language. Adds a compile-time "if" statement.
and by very simple I mean it thinks 1000.0 = 1e3 is false.

5

I think I was wrong about it being hard to implement this safely. I have an implementation here: GitHub - camlp5/pa_ppx_static: PPX Rewriter for static code blocks (lifted to file-level)

It works like this: you write [%static e] and it expands to (where NNNN is the md5 signature of the filename):

Pa_ppx_static.Runtime.Static.get __static_NNNN_0__

and, at the top of the file, the str_item

let __static_NNNN_0__ =
  Pa_ppx_static.Runtime.Static.mk (fun () -> e)

The function Static.mk creates a cell with the “create function” and a cache location; the function Static.get looks in the cache location, and if it’s empty, runs the create function to get a value to cache there. By putting the code at the top of the file, we avoid any ability to look at any local variables, opened modules, etc, etc. By putting the MD5 checksum of the filename into the variable-names, they’re effectively unusable outside of the module (b/c their names aren’t known).

In the use-case that I’m working with (regexps for GitHub - camlp5/pa_ppx_regexp: Camlp5-compatible PPX rewriters for Perl idioms ) my test has many repetitions of the same regexp. Which leads to the idea that maybe pa_ppx_static should hash the expressions it’s given, and only have one instance of each expression. B/c after all, two expressions that are structurally equal, even if in different spots in the file, are still going to evaluate to the same thing (assuming no side-effects in any modules they call, which seems reasonable).

6

Not sure if this is relevant, but have you also considered wrapping the static declaration in an open to avoid it being accessed from external modules?

Something like:

open (struct
let __static_NNNN_0__ =
  Pa_ppx_static.Runtime.Static.mk (fun () -> e)
end)
1 Like
7

Oh, that’s a lovely idea! I’ll use it! Thank you!

8

It seems like it might be worthwhile to add an extension element at the beginning of the file, which would carry a series of open statements, that would be available inside that struct. So only for the statics, not for the rest of the file. Like [%%static_preamble open Foo open Bar open Buzz].

9

I made GitHub - paurkedal/ppx_static: Syntax extension to lift constants out of abstractions some time ago as a proof-of-concept for myself before writing GitHub - paurkedal/ppx_regexp: Matching Regular Expressions with OCaml Patterns. The former is very short and unsophisticated, and I can see I’ve adjusted the corresponding code of the latter which puts static code into a private module inline open (thanks @Gopiandcode) at the top of the compilation unit.

2 Likes
10

Thank you! I had searched but somehow didn’t find your package. It seems I ended up walking down the same path you did, so that’s good! And pretty much all the same decisions, too!