# Ppx\_deriving\_yojson : nested \`Assoc

**URL:** <https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118>\
**Category:** Learning\
**Tags:** json, yojson, ppx\_deriving\_yojson\
**Created:** [May 7, 2023, 3:11pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118 "2023-05-07T15:11:10Z")\
**Posts on this page:** 13\
**Page:** 1

<div class="post-metadata">

**Author:** ![EliasGit2017](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@EliasGit2017](https://discuss.ocaml.org/u/EliasGit2017)\
**Post date:** [May 7, 2023, 3:11pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/1 "2023-05-07T15:11:10Z")

</div>

Hi everyone,  
I am currently trying to serialize/deserialize Json values using ppx\_deriving\_yojson.  
my json looks like this :

```auto
type t = {
    id: string [@key "Id"];
    names: string list [@key "Names"];
    image: string [@key "Image"];
    image_id: string [@key "ImageID"];
    command: string [@key "Command"];
    created: int64 [@key "Created"];
    ports: Port.t list [@key "Ports"];
    size_rw: int64 option [@default None] [@key "SizeRw"];
    size_root_fs: int64 option [@default None] [@key "SizeRootFs"];
    labels: (string * string) option [@default None] [@key "Labels"];
    state: string [@key "State"];
    status: string [@key "Status"];
    host_config: Container_summary_host_config.t option [@default None] (* [@key "HostConfig"] *);
    network_settings: Container_summary_network_settings.t option [@default None] (* [@key "NetworkSettings"] *);
    mounts: Mount_point.t option list [@default []] (* [@key "Mounts"] *);
} [@@deriving yojson { strict = false }, show];;

```

The problem here is with the `labels` key which is a **json object too** (an `Assoc` of Yojson.Safe).  
How can I serialize / unserialize the whole type **t** knowing that `labels` is a simple key-value pairs where the key is user defined and can take different values ?

---

<div class="post-metadata">

**Author:** ![Frederic\_Loyer](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/frederic_loyer/32/3472_2.png) [@Frederic\_Loyer](https://discuss.ocaml.org/u/Frederic_Loyer)\
**Post date:** [May 7, 2023, 3:38pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/2 "2023-05-07T15:38:30Z")

</div>

I have tried to compile and run your examples (minus some field where I have no detail of the actual type, like `host_config`).

And it works well. The following sentence:

```auto
let () = print_string @@ Yojson.Safe.to_string @@ to_yojson a

```

prints:

```auto
{"Id":"aa","Names":["ff"],"Image":"","ImageID":"ee","Command":"ee","Created":8,"Labels":["a","b"],"State":"","Status":""

```

Then the labels are correctly serialized. What do you expect more?

---

<div class="post-metadata">

**Author:** ![EliasGit2017](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@EliasGit2017](https://discuss.ocaml.org/u/EliasGit2017)\
**Post date:** [May 7, 2023, 4:04pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/3 "2023-05-07T16:04:37Z")

</div>

To give more details : I can post the json which I am trying to unserialize  
Here is the output from utop where i am testing it

```ocaml
Openapi.Container_summary.of_yojson (Yojson.Safe.from_string {|{
    "Id": "ad1ad195fe314991b364e124573eb9240093a2718586932789086f9c960cce88",
    "Names": [
      "/dazzling_goldberg"
    ],
    "Image": "alpine",
    "ImageID": "sha256:b2aa39c304c27b96c1fef0c06bee651ac9241d49c4fe34381cab8453f9a89c7d",
    "Command": "/bin/sh",
    "Created": 1683414855,
    "Ports": [],
    "Labels": {
      "this_is_a_label": "good_otter",
      "another_label" : "nice_cat"
    },
    "State": "exited",
    "Status": "Exited (0) 15 hours ago",
    "HostConfig": {
      "NetworkMode": "default"
    },
    "NetworkSettings": {
      "Networks": {
        "bridge": {
          "IPAMConfig": null,
          "Links": null,
          "Aliases": null,
          "NetworkID": "9b8d4180e3a28d08cd055ef460dfdb369e5f669795068508b2f7483d513fd100",
          "EndpointID": "",
          "Gateway": "",
          "IPAddress": "",
          "IPPrefixLen": 0,
          "IPv6Gateway": "",
          "GlobalIPv6Address": "",
          "GlobalIPv6PrefixLen": 0,
          "MacAddress": "",
          "DriverOpts": null
        }
      }
    },
    "Mounts": [
      {
        "Type": "bind",
        "Source": "/home",
        "Destination": "/home/elias/Desktop/camelwhale/camelwhale",
        "Mode": "",
        "RW": true,
        "Propagation": "rprivate"
      }
    ]
  }|});;
: Openapi.Container_summary.t Ppx_deriving_yojson_runtime.error_or =
Result.Error "Container_summary.t.labels"

```

As you can see, the labels field is causing some issues  
PS : The json is the result of a simple `docker container list`. I am trying to build an SDK for the Docker Engine deamon.

---

<div class="post-metadata">

**Author:** ![geoffder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/geoffder/32/3310_2.png) [@geoffder](https://discuss.ocaml.org/u/geoffder)\
**Post date:** [May 7, 2023, 4:12pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/4 "2023-05-07T16:12:29Z")

</div>

In your posted code, labels has type `(string * string) option` (not a list).

---

<div class="post-metadata">

**Author:** ![Frederic\_Loyer](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/frederic_loyer/32/3472_2.png) [@Frederic\_Loyer](https://discuss.ocaml.org/u/Frederic_Loyer)\
**Post date:** [May 7, 2023, 4:24pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/5 "2023-05-07T16:24:55Z")

</div>

Note also that

```auto
("a","b")

```

(which should be writed `["a", "b"]` in JSON)

Is not a synonymous to

```auto
{ a="b" }

```

Then you can also have some issue when match the last one with a `(string*string) option`. And like goeffer said, an option is not a list. You can’t twist the OCaml typing system to give `labels` two pairs.

---

<div class="post-metadata">

**Author:** ![Leonidas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/leonidas/32/4039_2.png) [@Leonidas](https://discuss.ocaml.org/u/Leonidas)\
**Post date:** [May 8, 2023, 12:39pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/6 "2023-05-08T12:39:01Z")

</div>

Also, `ppx_deriving_yojson` doesn’t really handle maps where the _keys_ are variable too. So you’d need to specify the type as `Yojson.Safe.t` which will give you the object as-is, and from there you can process it.

Alternatively declare it as custom type `labels` and implement `labels_of_yojson` which will parse an `Yojson.Safe.t` into your `labels` type, e.g. some kind of `Map.t` with string keys.

---

<div class="post-metadata">

**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\
**Post date:** [May 8, 2023, 8:43pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/7 "2023-05-08T20:43:04Z")

</div>

You can also implement a type of “maps with string keys,” viz.

```auto
type 'a _stringmap = (string * 'a) list[@@deriving yojson { strict = false }, show]
type 'a stringmap = (string * 'a) list[@@deriving show]

let string_to_yojson j = Pa_ppx_runtime.Runtime.Yojson.string_to_yojson j

let stringmap_of_yojson sub1 (j : Yojson.Safe.t) = match j with
    `Assoc l -> _stringmap_of_yojson sub1 (`List (List.map (fun (s,v) -> `List [string_to_yojson s; v]) l))

```

[NOTE that the functions to/from string above are from `pa_ppx`, so probably will need to be recoded for ppx\_deriving\_yojson`]

Then you can modify your type to

```auto
type t = {
    id: string [@key "Id"];
    names: string list [@key "Names"];
    image: string [@key "Image"];
    image_id: string [@key "ImageID"];
    command: string [@key "Command"];
    created: int64 [@key "Created"];
    ports: Port.t list [@key "Ports"];
    size_rw: int64 option [@default None] [@key "SizeRw"];
    size_root_fs: int64 option [@default None] [@key "SizeRootFs"];
    labels: string stringmap [@default []] [@key "Labels"];
    state: string [@key "State"];
    status: string [@key "Status"];
    host_config: Container_summary_host_config.t option [@default None] (* [@key "HostConfig"] *);
    network_settings: Container_summary_network_settings.t option [@default None] (* [@key "NetworkSettings"] *);
    mounts: Mount_point.t option list [@default []] (* [@key "Mounts"] *);
  } [@@deriving of_yojson { strict = false }, show];;

```

and your JSON input works unmodified.

Notice the type of labels is now `string stringmap`.

P.S. I only implemented the direction “yojson → t”, not “t → yojson”. The second direction should be straightforward.

---

<div class="post-metadata">

**Author:** ![EliasGit2017](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@EliasGit2017](https://discuss.ocaml.org/u/EliasGit2017)\
**Post date:** [May 8, 2023, 10:09pm UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/8 "2023-05-08T22:09:18Z")

</div>

Thank you, this looks promising, I was going for a something like (as proposed by @Leonidas ):

```auto
module StringMap = Map.Make(struct type t = string let compare = compare end);;
let yojson_of_stringmap m = StringMap.bindings m |> [%to_yojson: (string * string) list];;

```

And was working on the %of\_yojson part.  
But your solution seems to fit better. I am trying to implement it but I get a

```auto
 dune utop . ─╯
File "_none_", line 1:                  
Error: Module `Pa_ppx_runtime is unavailable (required by `Openapi__Container_summary')

```

even though I did add pa\_ppx to my dune file.  
Any idea why this is happening ?

---

<div class="post-metadata">

**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\
**Post date:** [May 9, 2023, 12:47am UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/9 "2023-05-09T00:47:45Z")

</div>

Ah.

1. Don’t use `pa_ppx`. Continue to use the `ppx_deriving` (or whatever it was) that you were using before. `pa_ppx` is part of a suite of PPX derivers based on Camlp5, and they are incompatible (in the sense of don’t interoperate) with the standard `ppxlib`-based infrastructure. They provide the same function, but … well, they’re not standard.

2. When I get to a stable location, I’ll write the “of\_yojson” code. It’s not hard.

---

<div class="post-metadata">

**Author:** ![EliasGit2017](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@EliasGit2017](https://discuss.ocaml.org/u/EliasGit2017)\
**Post date:** [May 9, 2023, 12:49am UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/10 "2023-05-09T00:49:06Z")

</div>

Thank you, it really helps me a lot as I am trying to build a Docker Engine SDK for Ocaml and I might encounter this problem again

---

<div class="post-metadata">

**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\
**Post date:** [May 9, 2023, 1:43am UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/11 "2023-05-09T01:43:29Z")

</div>

OK, here’s a new set of `map` stuff:

```auto

type ('a,'b) _map = ('a * 'b) list[@@deriving yojson { strict = false }]
type 'b stringmap = (string * 'b) list[@@deriving show]

let string_to_yojson j : Yojson.Safe.t = Pa_ppx_runtime.Runtime.Yojson.string_to_yojson j
let string_of_yojson msg j = Pa_ppx_runtime.Runtime.Yojson.string_of_yojson msg j

type maprow = string * Yojson.Safe.t[@@deriving yojson { strict = false }]

let stringmap_of_yojson sub1 (j : Yojson.Safe.t) = match j with
    `Assoc l -> _map_of_yojson (string_of_yojson "string_of_yojson") sub1 (`List (List.map maprow_to_yojson l))

let stringmap_to_yojson sub1 (l : 'a stringmap) : Yojson.Safe.t =
  let l = List.map (fun (k,v) -> (k, sub1 v)) l in
  `Assoc l

```

BTW, you have other type errors, e.g. in `Container_summary_network_settings`. The “networks” field is not a list of Network.t objects. There’s too little data for me to tell what you want to do, but perhaps you want a map from string to Network.t (where “bridge” is a key).

---

<div class="post-metadata">

**Author:** ![EliasGit2017](https://avatars.discourse-cdn.com/v4/letter/e/9de0a6/32.png) [@EliasGit2017](https://discuss.ocaml.org/u/EliasGit2017)\
**Post date:** [May 9, 2023, 2:00am UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/12 "2023-05-09T02:00:23Z")

</div>

Thank you for your help,  
I do need to go deeper and debug all the nested objects for many json outputs (sent by the docker deamon). “bridge” will be a key containing

```auto
"NetworkSettings": {
    "Networks": 
{
    "bridge": 
    {
        "NetworkID": "7ea29fc1412292a2d7bba362f9253545fecdfa8ce9a6e37dd10ba8bee7129812",
        "EndpointID": "88eaed7b37b38c2a3f0c4bc796494fdf51b270c2d22656412a2ca5d559a64d7a",
        "Gateway": "172.17.0.1",
        "IPAddress": "172.17.0.8",
        "IPPrefixLen": 16,
        "IPv6Gateway": "",
        "GlobalIPv6Address": "",
        "GlobalIPv6PrefixLen": 0,
        "MacAddress": "02:42:ac:11:00:08"
    }
}

```

To my knowledge, this is the generic output containing all the description of a docker network.  
I will look into it as soon as I manage to get the labels working. Your solution fits well, I just need to fix the `pa_ppx_runtime` issue by using something compatible with ocaml 5.0 . The goal is to have a complete Docker Engine SDK for Ocaml and give users the ability to manage the deamon locally (swarm mode and remote access is another “story”).

---

<div class="post-metadata">

**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\
**Post date:** [May 9, 2023, 2:11am UTC](https://discuss.ocaml.org/t/ppx-deriving-yojson-nested-assoc/12118/13 "2023-05-09T02:11:06Z")

</div>

There should be equivalent runtime entrypoints available for `ppx_deriving_yojson`: when I wrote my `pa_ppx` version, I mimicked the former as much as possible – that is to say, the only difference is _how_ the two PPX rewriters are coded, not so much the code they generate.

P.S. A sugggestion: it is very much worth using `-dsource` to inspect the output of the PPX rewriter `ppx_deriving_yojson` to see what the code looks like. For instance, to see what code it generates for the type `string`.