# Type Trick: Polymorphic variants for Extensible Modular Messaging

**URL:** <https://discuss.ocaml.org/t/type-trick-polymorphic-variants-for-extensible-modular-messaging/8855>\
**Category:** Learning\
**Created:** [November 21, 2021, 7:53am UTC](https://discuss.ocaml.org/t/type-trick-polymorphic-variants-for-extensible-modular-messaging/8855 "2021-11-21T07:53:22Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Gopiandcode](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/gopiandcode/32/5346_2.png) [@Gopiandcode](https://discuss.ocaml.org/u/Gopiandcode)\
**Post date:** [November 21, 2021, 7:53am UTC](https://discuss.ocaml.org/t/type-trick-polymorphic-variants-for-extensible-modular-messaging/8855/1 "2021-11-21T07:53:22Z")

</div>

One place I’ve often found myself trying to use Polymorphic variants in has been in implementing modular messages.

The general idea is that the components of your system operate in terms of messages, and update their state upon receiving specific messages.

```auto
module Volume = struct 
      (* state of this subsystem *)
      type state = int

      let update msg self = match msg 
          | `Incr -> 
           (* received an incr message from somewhere, don't care where *)
           (* update self *) 
           self + 1 
          | _ -> self (* ignore other messages *)
end

```

Once we have a collection of modules defined in this way, the goal would be to hook them up all together with an event bus that propagates messages between the systems:

```ocaml
let deliver msg systems =
    List.map (fun (state, system) -> 
             (* for each system *)
             let module M = (val system : SYSTEM) in
             (* update the state according to the message *)
             let state = M.update msg state in
             (state, system)) systems

```

In this way, the individual systems can be modular, loosely coupled to each other and easy to understand/test (the behaviour of a system depends only on its state and the message it receives).

By encoding the messages as a polymorphic variant, it means that we don’t need to explicitly create a monolithic message type (It also gets around some cyclic dependency problems that you might run into if certain systems depend on the same message (and this can’t be solved by extensible types either)).

The problem that I’ve kept on running into with this is that functors can’t capture this behaviour:

```ocaml
module type SYSTEM = sig 
   type state (* okay, no problems *)
   val update : [>] -> state -> state (* doesn't work - or not in the way one intends it to *)
end

```

Judging from previous questions that have been asked on this forum, I believe I’m not the only one who has tried and run into this wall in the past.

> [@Can I generically extend Classes and Polymorphic Variants via Functors?](https://discuss.ocaml.org/t/can-i-generically-extend-classes-and-polymorphic-variants-via-functors/2170):
>
> I’d like to do something akin to this module type SINATURA = sig type ('a) msg (\*this should be constraint to a (any) polymorphic variant \*) type ('a) action (\*this should be constraint to a (any) class \*) end module Z (M1: SINATURA) = struct type real\_msg = [`some\_id | string M1.msg] class real\_cls = object inherit int M1.action end end obviously (sadly?) I get error on both real\_msg and real\_cls

Anyway, I recently realised the solution to my problem - I was trying to use the module system to represent something I should have done in terms of OCaml values instead.

We start with a GADT to represent a system:

```ocaml
type 'b system =
    Mod : ('a * (([>] as 'b) -> 'a -> 'a * 'b list)) -> 'b system

```

(The existential type `'a` represents the state of the system, and `'b` represents the final type of messages accepted by the system.

We can then define our systems by encoding them not as modules, but as instances of this GADT:

```ocaml
module Background: sig
  val system: [> `Incr | `LIncr] system
end = struct
  let system =
    let init = 0 in
    let incr = function
      | `Incr -> fun n -> n + 1,[`LIncr]
      | _ -> fun n -> n,[] in
    Mod ((init : int), (incr :> [>] -> int -> int * [>] list))
end

module Volume : sig
  val system: [> `FIncr | `Incr] system
end = struct
  let system =
    let init = 0. in
    let incr = function
      | `Incr -> fun _ -> 0., []
      | `FIncr -> fun n -> n +. 1., []
      | _ -> fun n -> n, [] in
    Mod ((init : float), (incr :> [>] -> float -> float * [>] list))
end

```

Finally, with this, we can write a message bus function to deliver messages:

```ocaml
let update msg system = match system with
  | Mod (vl, fn) ->
    let vl', msgs = fn msg vl in
    Mod (vl', fn), msgs

let rec update_all (acc, msgs) msg = function
  | [] -> List.rev acc, List.concat msgs
  | hd :: tl ->
    let hd, omsg = update msg hd in
    update_all (hd :: acc, omsg :: msgs) msg tl

let update_all = update_all ([],[]) 

```

The beauty of this is that the type system will track the shapes of polymorphic variants accepted by the systems exactly as we would want:

```ocaml
let example = update_all `Incr [Background.system;Volume.system]

```

---

<div class="post-metadata">

**Author:** ![progman](https://avatars.discourse-cdn.com/v4/letter/p/8797f3/32.png) [@progman](https://discuss.ocaml.org/u/progman)\
**Post date:** [November 22, 2021, 7:07pm UTC](https://discuss.ocaml.org/t/type-trick-polymorphic-variants-for-extensible-modular-messaging/8855/2 "2021-11-22T19:07:01Z")

</div>

What do you do about the open row variable - a weak global variable here that prevents  
compilation?  
Is it necessary to convey such values as state threaded through one’s program?  
Is it foolish to want a way to signal the compiler to close the row variable (after all  
constructors have been seen) so that storage at global level is possible?  
This seems not unlike how OCaml doesn’t do finalsation of object types  
but might benefit from explicit instruction to do so (ie. in course the of generating an executable)

---

<div class="post-metadata">

**Author:** ![Gopiandcode](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/gopiandcode/32/5346_2.png) [@Gopiandcode](https://discuss.ocaml.org/u/Gopiandcode)\
**Post date:** [November 23, 2021, 2:38am UTC](https://discuss.ocaml.org/t/type-trick-polymorphic-variants-for-extensible-modular-messaging/8855/3 "2021-11-23T02:38:28Z")

</div>

> [@progman](#):
>
> What do you do about the open row variable - a weak global variable here that prevents  
> compilation?  
> Is it necessary to convey such values as state threaded through one’s program?

Yeah, good point. I was mainly thinking of a design in which all components are encoded in terms of these modular systems, but when you start adding other global state into the mix, then explicit anotations will be needed I guess.

The threading is a little annoying - you can use a mutable type to mitigate some (but not the issue of being able to have global state) of the issues there - i.e the type of systems would be:

```auto
type 'b system =
    Mod : ('a * 'b Queue.t -> (([>] as 'b) -> 'a -> 'a)) -> 'b system

```