[ANN] Jsont 0.1.0 – Declarative JSON data manipulation for OCaml

Your code review comments made sense to me. Thanks!

I wrote a proof of concept of this for jsont.

With new helpers Jsont_record_builder and Jsont_variant_builder defined as:

module Jsont_record_builder = struct
  module F = struct
    type ('a, 'e) t = ('e, 'a) Jsont.Object.Syntax.schema

    let map a ~f = Jsont.Object.Syntax.( let+ ) a f
    let both a b = Jsont.Object.Syntax.( and+ ) a b
  end

  include Record_builder.Make_2 (F)

  let mem jsont f =
    field
      (Jsont.Object.Syntax.mem
         (Fieldslib.Field.name f)
         jsont
         ~enc:(Fieldslib.Field.get f))
      f
  ;;

  let define ?kind ?doc creator =
    Jsont.Object.Syntax.define ?kind ?doc (build_for_record creator)
  ;;
end

module Jsont_variant_builder = struct
  let case jsont (variant : _ Variantslib.Variant.t) cases =
    let map = Jsont.Object.Case.map variant.name jsont ~dec:variant.constructor in
    let case = Jsont.Object.Case.make map in
    (fun d -> Jsont.Object.Case.value map d), case :: cases
  ;;

  let define ~kind make_matcher =
    let enc_case, cases = make_matcher [] in
    Jsont.Object.map ~kind Fn.id
    |> Jsont.Object.case_mem "type" Jsont.string ~enc:Fn.id ~enc_case cases
    |> Jsont.Object.finish
  ;;
end

You can then write the Geometry_variant of the cookbook as follows:

module Circle = struct
  type t =
    { name : string
    ; radius : float
    }
  [@@deriving fields]

  let jsont =
    let open Jsont_record_builder in
    define ~kind:"Circle"
    @@ Fields.make_creator ~name:(mem Jsont.string) ~radius:(mem Jsont.number)
  ;;
end

module Rect = struct
  type t =
    { name : string
    ; width : float
    ; height : float
    }
  [@@deriving fields]

  let jsont =
    let open Jsont_record_builder in
    define
      ~kind:"Rect"
      (Fields.make_creator
         ~name:(mem Jsont.string)
         ~width:(mem Jsont.number)
         ~height:(mem Jsont.number))
  ;;
end

type t =
  | Circle of Circle.t
  | Rect of Rect.t
[@@deriving variants]

let (jsont : t Jsont.t) =
  let open Jsont_variant_builder in
  define
    ~kind:"Geometry"
    (Variants.make_matcher ~circle:(case Circle.jsont) ~rect:(case Rect.jsont))
;;

It’s a slightly different aesthetic, with a bit more automation.

The record part is built with @art-w’s scheme, while the variant part is based on the existing jsont.

Seeing how jsont exports its low level representation, it may be possible to re-work this purely outside of the lib, I am not sure.

I also experimented with Jsont in a project of mine and I couldn’t resist starting to write an ad-hoc PPX… I was using ppx_yojson_conv before that, so the new ppx is compatible with its annotations.

It’s still quite rough and incomplete but was sufficient to remove a lot of boilerplate in my project. For anyone interested in trying it or contributing, the repo. is hosted here: GitHub - voodoos/ppx_deriving_jsont: A ppx deriver for Jsont descriptions

(Jsont descriptions that require tailored encoding / decoding, like handling numbers, still need to be written by hand, my goal was really to remove the bulk of the grunt work. Error reporting is also probably quite lacking compared to hand-written descriptions.)

2 Likes

I’m really liking the design of Jsont so far, but I’m wondering what’s currently the right way to deal with untagged cases.

For example, bidirectional JSON-RPC messages roughly map to the following:

type message =
  | Request of {
      method' : string;
      params : structured_json;
      id : id option;
    }
  | Result_response of {
      result : Jsont.json;
      id : id;
    }
  | Error_response of {
      error : error_obj;
      id : id;
    }

But there isn’t a case field at all: checking the unambiguous presence of a method, result or error member is the only way to select which case to decode.

Ah protocols made by dynamically typed languages programmers working in languages with poor data type definition abilities :–)

Indeed casing on the presence of fields is not part of the patterns that are explicitely supported; see section 3.5 of the paper for those that are. There is however always an ultimate escape hatch which is Jsont.map. This should be mentioned in the cookbook I opened this issue to track this.

So I suspect you already found out but the way to go here is to parse them as a single object with all fields optional and then sort things out via a map where you can use the full power of OCaml to do so (use Jsont.Error.msgf to error, see also these functions if you need to convert standard OCaml erroring interface to raise Jsont.Error ):

type untagged = 
{ method' : string option; 
  params : Jsont.json option; 
  result : Jsont.json option; 
  id : string option }

let untagged_jsont : untagged Jsont.t = Jsont.Object.map … 
let message_of_untagged : untagged -> message  = …
let message_to_untagged : message -> untagged = … 
let message_jsont = 
  let dec = message_of_untagged and enc = message_to_untagged in 
  Jsont.map ~kind"JSON RPC message" untagged_jsont ~enc ~dec
1 Like

That being said I just has a read of JSON-RPC, I’m not sure I fully agree with your OCaml modelling.

First I think you should keep requests and responses separate. Second errors only happen in responses and are distinguished by the mutually exclusive result or error JSON members. So I would rather model response with:

type response =
  { jsonrpc : jsonrpc;
    value : (Jsont.json, error) result;
    id : id; }

Now the mutually exclusive result and error logic can be done in the constructor given to Jsont.Object.map. Based on a quick read of the spec I added a full modelling of JSON-RPC along these lines to the examples in the repo.

1 Like

Thanks for your feedback and the example! I had solved it with map indeed, but I was unsure about whether raising errors from the mapping function is the correct way to validate.

Note that your implementation is a bit more lenient than the spec: params must be array or object only.

I specified bidirectional because, despite JSON-RPC clearly being a client-server protocol, some others built on top require using it as peer-to-peer, sending requests and responses on the same channel, possibly on the same batch. This extension to the spec is completely unspecified anywhere AFAIK, specially when it comes to mixed batches, yet it is taken for granted in some specifications such as the Language Server Protocol.

So I do need a bare message type, although your encoding is more truthful to the actual specification.

I also prefer the value : (Jsont.json, error) result encoding for responses, I simply thought the three cases would explain the untagged example better without having to point to the spec or a TypeScript schema.

1 Like

Well in fact my first answer was not that good. You can solve the problem in the object constructor given to Jsont.Object.map. I should also mention that’s it’s ok to raise errors there (in fact it should be ok to raise errors from any function you tuck into Jsont.t values).

Thanks! fixed by this commit.

What kind of framing do they use ?

1 Like

Thanks for the update to the example, I was about to implement that very same codec.

Yes please, my only head-scratcher with the library has been how to report errors. This snippet from your latest commit wouldn’t have ocurred to me, for example:

let meta = Jsont.Meta.none in
let fnd = Jsont.Sort.to_string (Jsont.Json.sort j) in
Jsont.Error.expected meta "object or array" ~fnd

For LSP? An HTTP header knock-off which doesn’t convey any metadata about the JSON-RPC payload aside from the length; even the Content-Type is ignored by the reference implementation.

1 Like

Note this has not been released. But you can use Jsont.Error.msgf with the rest.

It’s a HTTP-style set of CRLF-terminated headers, one of which indicates
the content length of the following json value. At least for LSP.

1 Like