Versioned Serialization PPX

bluddy · July 8, 2025, 5:53am

I’m getting close to finishing my classic game remake, and I have to start thinking about a stable save file format. So far, I’ve just dumped parts of the state to a JSON file using ppx_deriving_yojson, and obviously that doesn’t provide any backwards compatibility, which is necessary post 1.0.

Since handling serialization formats is painful (and duplicated work), I had the idea of making a serialization ppx that allows for annotating different fields with versions. For example, suppose that a label of my record only appeared at version 1.1. By annotating this label with the version number (e.g. [@version "1.1"]) it would know to use a default value function when loading this field. Of course, this could only tolerate minor adjustments rather than complete modifications, but one could imagine a few other minor instructions like renames, deletions etc. The point is not to destroy save files on every minor change.

Does anyone know of such a feature, or of something similar?

R_Huxton · July 8, 2025, 8:37am

Alternative option. Always assume the latest format of your save-file, but have a separate upgrade_save_file function that will convert v1 to v2,3,4 and run that at startup.

Won’t really work if people share files around a lot, but if you just want to handle upgrades it keeps all the “fiddly stuff” in one place and only requires one version number at the top-level.

Otherwise most of the versioned-format stuff I can think of is protocol-related (protobuf, flatbuf etc) - lots of alternatives in that space.

smuenzel · July 13, 2025, 6:29am

I think GitHub - janestreet/ppx_stable: Stable types conversions generator is very similar to what you’re looking for, although the syntax is a bit more verbose (since the entire type is versioned).

An example of usage is here:

github.com/janestreet/bonsai

src/protocol/bonsai_protocol.ml

90209249f


      
            [@@deriving bin_io, sexp, stable_variant]
          
            let%expect_test _ =
              print_endline [%bin_digest: t];
              [%expect {| b17892a0948ebd34a0b716278484df52 |}]
            ;;
          end
          
          module V2 = struct
            type t =
              | Graph_info of Graph_info.V2.t
              | Performance_measure of Entry.V1.t
            [@@deriving bin_io, sexp, stable_variant ~version:V1.t ~modify:[ Graph_info ]]
          
            let%expect_test _ =
              print_endline [%bin_digest: t];
              [%expect {| acf03a1188bfb7efeb8af957c2f31a09 |}]
            ;;
          
            let of_v1 t =
              of_V1_t t ~modify_Graph_info:(fun graph_info ->

along with

github.com/janestreet/bonsai

src/graph_info.ml

90209249f


      
              [%expect {| 127c33aa3895110d2fdd1b3c356bf5c3 |}]
            ;;
          end
          
          module V2 = struct
            type t =
              { node_type : String.V1.t
              ; here : Core.Source_code_position.Stable.V1.t Option.V1.t
              ; id : Int.V1.t
              }
            [@@deriving sexp, compare, bin_io, stable_record ~version:V1.t ~remove:[ id ]]
          
            let%expect_test _ =
              print_endline [%bin_digest: t];
              [%expect {| da0e8001d1f5692d5a56c26e18c81a01 |}]
            ;;
          
            let to_v1 t = to_V1_t t
            let of_v1 t = of_V1_t t ~id:(-1)
          end

Topic		Replies	Views
Version control of records Learning serialization , records , s-expression	9	821	March 30, 2023
[ANN] ppx_protocol_conv 5.0.0 Community	0	951	April 21, 2019
[ANN] ppx_deriving_protocol 0.8 Community announce	3	1357	November 17, 2017
[ANN] Ppxlib: Support for future compilers Ecosystem compiler , ppx , ppxlib	9	475	October 31, 2025
Update on the big ppx refactoring project Community ppx	6	2160	September 30, 2019

Versioned Serialization PPX

Related topics