Creating a ppx that transforms record updates

As part of my recent attempt to write a soft-realtime application (game) in OCaml, I’ve come across a pattern that I think is valuable. When a record contains internal data structures, it’s useful to do a physical equality both when recreating the record immutably and when updating the record mutably. For example, suppose I have

type t = {
  mutable a: a;
  b: b;
}

let update_mutable (v:t) a =
  if v.a != a then v.a <- a;
  v

let update_immutable (v:t) b =
  if v.b != b then {v with b} else v

Both update_mutable and update_immutable are always worth using rather than doing the updates as-is. The immutable case is more obvious, since the comparison helps you avoid an allocation. The mutable case is also valid, because by checking for physical equality, you avoid hitting the write barrier in cases where the inner value hasn’t changed. Basically, you always want to do this.

Unfortunately OCaml has no way of lifting the v.a record access and assignment (v.a <- a), which means there’s no easy way to write a generic function to do this stuff. I would therefore like to create a ppx to do it. Since I’ve never written a ppx rewriter, I’d appreciate some advice. How would you go about creating this ppx rewriter?

1 Like

Maybe you can find what you are looking for here; at least it should help you to figure out how to write your own. But don’t, it’s obvious, you are just trying to find a way to procrastinate on your project :–)

6 Likes

Here’s a relatively small ppx-rewriter that I posted a while back that provides anaphoric lambdas ([%a it + 1] becomes fun it -> it + 1):

open Ppxlib

let name = "a"

let expand ~loc ~path:_ expr =
  match expr with
  | expr ->
    [%expr fun it -> [%e expr]]

let ext =
  Extension.declare name Extension.Context.expression
    Ast_pattern.(single_expr_payload __)
    expand

let () = Driver.register_transformation name ~extensions:[ext]

To implement the functionality you want, you can probably just modify the match inside expand.

1 Like

Shameless plug: it can be a nice example of application for opam - metapp!

The following example declares two “meta” functions update_mutable and update_immutable, and then shows a short example for their usage.

Edit: to use it, you just have to add (preprocess (pps metapp.ppx)) in the dune file.

[%%metadef
let extract_field_access (e: Ppxlib.expression):
      Ppxlib.expression * Ppxlib.Ast_helper.lid =
  match e.pexp_desc with
  | Pexp_field (expression, field) -> expression, field
  | _ ->
      Location.raise_errorf ~loc:e.pexp_loc "field access expected"

let update_mutable field_access =
  let record, field = extract_field_access field_access in
  [%e
    (* Binding [record] first to evaluate it only once *)
    let record = [%meta record] in
    fun value ->
      if [%meta Ppxlib.Ast_helper.Exp.field [%e record] field] != value then
        [%meta Ppxlib.Ast_helper.Exp.setfield [%e record] field [%e value]];
      record]

let update_immutable field_access =
  let record, field = extract_field_access field_access in
  [%e
    (* Binding [record] first to evaluate it only once *)
    let record = [%meta record] in
    fun value ->
    if [%meta Ppxlib.Ast_helper.Exp.field [%e record] field] != value then
      [%meta Ppxlib.Ast_helper.Exp.record [field, [%e value]]
        (Some [%e record])]
    else
      record]]

type a = int and b = bool

type t = {
    mutable a: a;
    b : b;
}

let () =
  let r = { a = 0; b = false } in
  ignore ([%meta update_mutable [%e r.a]] 1);
  assert (r.a = 1);
  let r' = [%meta update_immutable [%e r.b]] true in
  assert (r'.b = true)
4 Likes

First of all: WOW! This is amazing! It’s exactly what I’ve been looking for.

How many countless projects have I ended up procrastinating on because of the endless bureaucracy it takes to setup ppxs in OCaml.

I can think of several projects off the top of my head with repetitive code that I want to experiment with removing with this.

Second question: Could you comment a bit on how this differs from GitHub - stedolan/ppx_stage: Staged metaprogramming in stock OCaml? I remember coming upon ppx_stage a while back, but ended up not using it because a) it wasn’t released on opam and b) it didn’t seem to be actively maintained.

1 Like

I confess I did not know about ppx_stage, thank you for the reference!

As far as I understand, here are the differences I see:

  • By using ppx_stage, a program can generate and execute other programs at runtime (by invoking the compiler and dynamic loader at runtime). By using metapp, a program can contain portion of codes that are generated at compile time by executing other programs (by invoking the compiler and dynamic loader during the preprocessing phase).

  • With ppx_stage, each piece of code that is manipulated is checked to be well-typed: a 'a code value is equivalent to a unit -> 'a closure, with an efficient composition (by compiling). With metapp, meta-programs directly manipulate the parse tree, with the only constraint that the parse tree produced at the end leads to a well-typed program: this is more error prone, but that allows to describe syntax extension as in the example I gave in this thread, where we can manipulate as first-class values some piece of syntax like record fields, that are not first-class citizens in the language.

3 Likes

This looks amazing! Does it coexist nicely with other ppx plugins, such as ppx_deriving and ppx_import?

metapp registers itself as a ppxlib preprocess phase: it should cooperate nicely with ppxlib transformations that only register rules, such as opam - ppx_import and “native” ppxlib derivers such as opam - ppx_compare, opam - ppx_show, opam - refl. But metapp cannot cooperate with opam - ppx_deriving because ppx_deriving registers itself as a preprocess phase as well, mainly to maintain backward compatibility because the syntax of ppxlib rules do not follow the syntax chosen by ppx_deriving (if I understand correctly, ppx_deriving maintains legacy derivers: I believe new developments should use native ppxlib derivers, that are compatible with metapp). metapp cannot cooperate with opam - ppx_optcomp neither. However, I believe it should be possible to register metapp as ppxlib rules (and not preprocess phase) to overcome these incompatibilities in a future version if needed.

1 Like

Thanks for explaining. This is very important information that I don’t exactly have a grip on. We need a list of recommended ppxs for basic functionality (show, enum, compare, etc), and I was under the impression that ppx_deriving were compatible.

Why not propose this as the default semantics for record update ?

In the immutable case, this should be the default only when the record has no
mutable field because the user may want to copy the record to break sharing?

I was going to do so, but then I realized that with OCaml 5.0 and Atomics, comparing the inner pointers could be expensive, not to mention that a comparison at time t may be invalid at time t+1 due to multithreading, so you’d really need a mutex on that comparison ie. it’s just not appropriate.

That’s an interesting point. The semantics of mixing immutable data structures (with referential transparency) and mutable ones (without it) are tricky.

An atomic test and set for record update could do the job. Is this available in the 5.0 runtime ?