# Deserialize a numeric JSON field into a variant using ppx\_deriving\_yojson

**URL:** <https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438>\
**Category:** Learning\
**Tags:** ppx\_deriving\_yojson\
**Created:** [September 6, 2022, 1:31am UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438 "2022-09-06T01:31:59Z")\
**Posts on this page:** 8\
**Page:** 1

<div class="post-metadata">

**Author:** ![cdaringe](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cdaringe/32/2419_2.png) [@cdaringe](https://discuss.ocaml.org/u/cdaringe)\
**Post date:** [September 6, 2022, 1:31am UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/1 "2022-09-06T01:31:59Z")

</div>

# Problem

```ocaml
module Num = struct
  type t =
    | NInt of int
    | NFlt of float
  [@@deriving show, yojson]
 (* snip *)

```

My incoming JSON payload looks like `{ "number": 1 }` or `{ "number": 1.1 }`, wherein I expect int and float JSON values. I need to get these basic JSON values _into_ my Num type.

`ppx_deriving_yojson` lets you specify some serializer and deserializers. I’ve tried a variety of fns and syntaxes, and though a bunch of things compile, they just _don’t work_. The docs are a bit light on the matter, so I’ve been going thru the ppx tests: [ppx\_deriving\_yojson/src\_test/test\_ppx\_yojson.ml at e030f13a3450e9cf7d2c43fa04e709ef608486cd · ocaml-ppx/ppx\_deriving\_yojson · GitHub](https://github.com/ocaml-ppx/ppx_deriving_yojson/blob/e030f13a3450e9cf7d2c43fa04e709ef608486cd/src_test/test_ppx_yojson.ml). The tests are also a bit light in the custom [de]serialization offerings.

# Discussion

- How can I deserialize JSON int & float values gracefully to `Num.t`?
- How can I serialize `Num.t` back to untagged JSON, much like the inputs came in (vs tagged JSON output)?

I considered sharing my failed attempts, but at this point I think it would distract more than help 🙂. Thanks!

---

<div class="post-metadata">

**Author:** ![yawaramin](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/yawaramin/32/3384_2.png) [@yawaramin](https://discuss.ocaml.org/u/yawaramin)\
**Post date:** [September 6, 2022, 2:41am UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/2 "2022-09-06T02:41:23Z")

</div>

The payload here is different enough from the OCaml data type that I think you’re justified to write your own codec. E.g.,

```auto
module Num = struct
  type t = NInt of int | NFlt of float [@@deriving show]

  let to_yojson = function
    | NInt i -> `Int i
    | NFlt f -> `Float f

  let of_yojson = function
    | `Int i -> Ok (NInt i)
    | `Float f -> Ok (if Float.is_integer f then NInt (int_of_float f) else NFlt f)
    | json -> Error (Yojson.Safe.to_string json)
end

```

Now you can type your payload as:

```auto
type payload = { number : Num.t } [@@deriving show, yojson]

```

---

<div class="post-metadata">

**Author:** ![cdaringe](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cdaringe/32/2419_2.png) [@cdaringe](https://discuss.ocaml.org/u/cdaringe)\
**Post date:** [September 6, 2022, 3:07am UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/3 "2022-09-06T03:07:00Z")

</div>

Amazing. I didn’t realize i could just … _implement_ those fns! Thanks @yawaramin!

---

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [September 6, 2022, 2:38pm UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/4 "2022-09-06T14:38:06Z")

</div>

> [@cdaringe](#):
>
> Amazing. I didn’t realize i could just … _implement_ those fns!

I was flummoxed in using ppx’s non-trivally for a while as well, until I had the same revelation, that ppx-emitted code was just looking to refer to definitions with names that followed implicit conventions, as available in the lexical scope where the ppx is used.

Where this is is very handy is that you can trivially customize behaviour in different contexts. e.g. say you have a `Foo` that is used widely in your program, and you use yojson converters for basic IPC:

```ocaml
module Foo = struct
  type t = ...... [@@deriving yojson]
end

```

Fine, very convenient. But when you want to present `Foo`s to e.g. users of an external API, the naive implementation won’t be so great:

```ocaml
module API = struct
  (** The type of response provided for [Foo] GET requests *)
  type foo_get =
    { id : Uuidm.t
    ; name : string
    ; foo : Foo.t
    }
  [@@deriving yojson]
end

```

With this, API consumers will be exposed to yojson’s rendering of whatever `Foo.t` is, which is surely not something they’ll be expecting, and which will probably leak all kinds of internal changes over time, thus churning that external API’s compatibility. And, we don’t want to change `Foo.to_yojson`, as that will impact our IPC usage.

The solution is to just make an API-local `Foo`, and implement whatever encoding of it makes sense in that context:

```ocaml
module API = struct
  module Foo = struct
    include Foo
    let to_yojson foo = ....
  end

  (** The type of response provided for [Foo] GET requests *)
  type foo_get =
    { id : Uuidm.t
    ; name : string
    ; foo : Foo.t
    }
  [@@deriving yojson]
end

```

Now my API-local `Foo` module with its own purpose-built `to_yojson` is what is lexically in scope for the code in the derived `foo_get_to_yojson` function, and my internal IPC usage of the default `Foo.to_yojson` is undisturbed.

* * *

(BTW, yojson does support using an annotation for customizing the serializer used for just one field, e.g.

 

```ocaml
module API = struct
  let api_foo_to_yojson foo = .......

  (** The type of response provided for [Foo] GET requests *)
  type foo_get =
    { id : Uuidm.t
    ; name : string
    ; foo : Foo.t [@to_yojson api_foo_to_yojson]
    }
  [@@deriving yojson]
end

```

 

This is great if you really only need said customization in a single location, but quickly becomes tedious otherwise, so I find myself defining local copies of modules much more frequently.)

---

<div class="post-metadata">

**Author:** ![emillon](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/emillon/32/1206_2.png) [@emillon](https://discuss.ocaml.org/u/emillon)\
**Post date:** [September 6, 2022, 2:59pm UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/5 "2022-09-06T14:59:19Z")

</div>

These are good tips. As a third option, you can also use this to avoid `[@to_yojson]`

```ocaml
  type api_foo = Foo.t
  let api_foo_to_yojson foo = .......
  let api_foo_of_yojson = Foo.of_yojson

  type foo_get =
    { id : Uuidm.t
    ; name : string
    ; foo : api_foo
    }
  [@@deriving yojson]

```

---

<div class="post-metadata">

**Author:** ![cdaringe](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cdaringe/32/2419_2.png) [@cdaringe](https://discuss.ocaml.org/u/cdaringe)\
**Post date:** [September 6, 2022, 8:37pm UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/6 "2022-09-06T20:37:36Z")

</div>

Hey all, maybe you’ll kindly spare me a bit more advice here. It’s disjoint from the original problem statement, but i realized I need `int64` not `int` in my type, for which the ppx has a [bunch of custom processing for](https://github.com/ocaml-ppx/ppx_deriving_yojson/blob/38715864513e6619916e5e42b70e9182ea1f5376/src/ppx_deriving_yojson.ml#L112-L113), as there is no `'Int64` poly variant in Yojson.Basic.t. Perhaps I need to emulate what the PPX is doing to cram my int64 into something that Yojson.Basic.t accepts, as a cheeky little fake out. However, it’s not clear to me what that `Exp.variant` expression is up to, so I’m not clear how to emulate it full well.

---

<div class="post-metadata">

**Author:** ![yawaramin](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/yawaramin/32/3384_2.png) [@yawaramin](https://discuss.ocaml.org/u/yawaramin)\
**Post date:** [September 6, 2022, 9:10pm UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/7 "2022-09-06T21:10:50Z")

</div>

Technically, JSON specifies the ‘number’ type as IEEE754 double precision. Which corresponds to OCaml `float` type, so you can read the data as float and try to convert it to int64. However, since int64 has a lower range than float, it will be tricky to avoid underflow/overflow.

```auto
let max_int64_flt = Int64.(to_float max_int)
let min_int64_flt = Int64.(to_float min_int)

let of_yojson = function
  | `Int i ->
    Ok (NInt (Int64.of_int i))
  | `Float f ->
    Ok begin
      if f < min_int64_flt || f > max_int64_flt || not (Float.is_integer f) then
        NFlt f
      else
        NInt (Int64.of_float f)
    end
  | json ->
    Error (Yojson.Safe.to_string json)

```

Encoding as JSON should be simpler of course because `Int64.to_float` is guaranteed to not underflow/overflow.

---

<div class="post-metadata">

**Author:** ![emillon](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/emillon/32/1206_2.png) [@emillon](https://discuss.ocaml.org/u/emillon)\
**Post date:** [September 7, 2022, 7:54am UTC](https://discuss.ocaml.org/t/deserialize-a-numeric-json-field-into-a-variant-using-ppx-deriving-yojson/10438/8 "2022-09-07T07:54:08Z")

</div>

`ppx_deriving_yojson` is intended to be used with `Yojson.Safe.t` (so that’s the parser you’re supposed to run). Is there a reason you’re using `Yojson.Basic.t`?

If you’re stuck with `Yojson.Basic.t`, you can’t encode numbers that do not fit into an `int`. If you want this behavior but still have an `int64` in the record (probably a bad idea), you can use something like:

```ocaml
type small_int64 = int64
let small_int64_of_yojson j = Int64.of_int ([%of_yojson: int64] j)
let small_int64_to_yojson n = [%to_yojson: int] (Int64.to_int n)

```

… and use `small_int64` in your type. (`[%of_yojson: t]` expands to the derived function for type `t`).