Using ppx for code generation

Learning
1

Hello,
I would like to parse some file format and generate code at compile time. One way would be to just stitch formatted string of ocaml syntax. I was wondering if i could make use of ppx for this. Say.
let _ = () [%parse_n_gen “foo.xml”];;
Which should generate several modules baz, bar and so on by parsing that file at compile time. I am not sure if there is any facility for quasiquotation. Also how would i generate new structs and sigs from this? So far, i have seen ppx rewrite the same module by adding some more expressions.

Any pointers on this would be appreciated.

2

The dummy let = () binding can be dropped by using [%%parse_n_gen "foo.xml"]. To avoid the source file altogether, I think this may be a reasonable solution:

open Migrate_parsetree
open Ast_406
open Parsetree

let ocaml_version = Versions.ocaml_406
module Convert_to_current = Convert (OCaml_406) (OCaml_current)

let () =
  let loc = Location.none in
  let str = [%str
    let rec fact n = if n <= 0 then 1 else n * fact (n - 1)
  ] in
  let ppf = Format.std_formatter in (* or add an -o option *)
  Pprintast.structure ppf (Convert_to_current.copy_structure str)

Link and preprocess with ocaml-migrate-parsetree and ppx_metaquot.

3

A primitive script would work? Please don’t use ppx just because you can, it’s a huge API surface area where no ABI is guaranteed

4

My experience with generating OCaml bindings from the vulkan specification file is that manipulating OCaml AST with the help of ppx_metaquot scales better beyond a certain size. Depending on your aims, it may not be necessary to use the ppx API proper: the ppx API would be useful to insert generated content inside preexisting OCaml source file; but generating standalone OCaml source files can be done directly using the compiler-libs library and printing the generated AST with Pprintast.

2 Likes
5

I will share my recent finding into this topic, but since I am quiet new to Ocaml and its ecosystem there must be a better way.

The relevant official documentation on Inria site is at "Chapter 27 The compiler front-end". Ast_helper, Parsetree, Pprintast, and Asttypes are the most relevant.

To view parsetree of any Ocaml file, you can either use ocamlfind ppx_tools/dumpast <filename> or use -dparsetree option of ocamlc/ocamlopt.

To generate code, using Ast_helper and Parsetree directly is possible but very tedious, the following block of code generates [5] :

let e_const = Ast_helper.Exp.constant (Pconst_integer ("5", None))  
let loc = { Asttypes.txt = Longident.Lident "[]"; loc = Location.none }
let my_tuple = Ast_helper.Exp.tuple [e_const; Ast_helper.Exp.construct loc None]
let concat_id = Longident.Lident "::"
let concat_loc = { Asttypes.txt = concat_id; loc = Location.none }
let concat_construct = Ast_helper.Exp.construct concat_loc (Some my_tuple)

‘ppx_tools/ast_convenience’ simplify the above code to:

Ast_convenience.list [(Ast_convenience.int 5)]

Though it does not seem to have abstraction for all parts of Parsetree, so familiarity with Parsetree is probably needed even when using ppx_tools. I have not experiment with ppx_metaquot, so I can’t comment on this topic.