Hi, I’m relatively new to OCaml but have been digging in deep for the past 6 months (thanks to ReasonML). OCaml has turned out to be a wonderful language with a lot of great features, but one feature that is sorely missing for me is extensible records.
In this post I describe how implementing extensible variants is a worthwhile effort, covering:
- Why extensible records?
- The ideal solution
- The interim solution
Why extensible records?
Extensible records shine most when writing and working with frameworks. Take a web server framework for instance. It’s common for frameworks to support composable, reusable middleware that you can easily plug into your app. See how the following authenticate_user
middleware function is used:
let me_route =
get("/me")
>>> authenticate_user
>>> fun req ->
User.json req.ctx.user |> Response.send(200)
In our pretend web framework, a route has a context. This context happens to be an extensible record, allowing middleware to extended it with arbitrary fields. In this case, the authenticate_user
function extends req.ctx
with a new user
field.
This is a powerful pattern, evidenced when you start working with multiple middleware functions:
let update_team_route =
patch("/teams")
(* Adds req.ctx.team *)
>>> param(id => { team: Team.get(id) })
(* Adds req.ctx.user *)
>>> auth_user
(* Checks req.ctx.user has access to req.ctx.team *)
>>> auth_team_admin
(* Adds req.ctx.body *)
>>> json_body
>>> fun req -> ...
Extensible records take some of the best aspects from both the dynamic and static programming worlds. Even though our functions add arbitrary fields, OCaml will statically disallow you from composing middleware that try to read fields that don’t exist.
The Ideal Solution
In an ideal world, OCaml would support extensible records directly. Due to how current “normal” records are optimized for speedy constant access, this would have to be separate from that.
I have two ideas on how to approach this, but I don’t want to take too much focus away from the interim solution.
See the ideas
Idea 1: New Datatype
As a new, built-in datatype, extensible records would only need (I think) minimal syntax to introduce into the language. For starters, an extensible record can be marked by a backtick (giving a nod to polymorphic variants), whereas updating a record can be marked using a pipe character, a with
keyword, or nothing at all:
let x_rec = `{ x = 10 }
let xyz_rec = `{ x_rec | y = "hmm"; z = "nice" }
(* or *)
let xyz_rec = `{ x_rec with y = "hmm"; z = "nice" }
(* or *)
let xyz_rec = `{ x_rec; y = "hmm"; z = "nice" }
Field access will probably require more bikeshed, but here’s one way to do it:
let x_rec = `{ x = 10 }
let n = x_rec`.x
Whichever operator is chosen, type inference will of course be as streamline as the rest of the language:
let f record = record`.x + record`.y
Here f
is inferred as having the type `{ x: int; y: int; 'r } -> int
Idea 2: Extensible Objects
When it comes to typing, OCaml’s object system is already very close to extensible records. The only missing feature is OCaml cannot extend an arbitrary object with a new method. This might look something like:
let obj2 = object extend obj1 with method y = 20 end
A major downside is the mismatch where how objects have internal state while extensible records do not. This isn’t a blocker to the feature, but it might be a blocker to optimization. A way to mark an object as “pure” might solve this problem.
The Interim Solution
My primary motivator for wanting extensible records is to use it with BuckleScript. This is why I present a small addition that could be implemented and used today, without blocking any potential future implementation of our extensible friends.
The following code is already valid OCaml:
type t = < m : int >
type u = < n : int; t; k : int >
However, the following is not:
# type 'a add_x = (< .. > as 'a) -> < 'a; x : int >;;
Error: The type < .. > is not an object type
If something like this could be supported in the type system, we could immediately start using it in BuckleScript external types compiling to JavaScript objects. In other words, this would be useful even without a native OCaml extensible records implementation.
Conclusion
Extensible records are a powerful feature that allow writing APIs that are both succinct and type safe. I really, really want extensible types for the web framework I’m writing in BuckleScript!
I’m new to the OCaml community, so I have to ask: Is this feasible? How do features get approved to implement? I’m willing to contribute and/or help start a fund for this feature. Obviously I’m very excited for this
(special thanks to @octachron for patiently answering my workaround questions)