Combinator library for extracting data for s-exps?

Community
41

Glad you found decoders useful.

Re the extra level of list wrapping: this is not intrinsic to Decoders itself, but it is intrinsic to the uncons combinator. uncons peels off the head of the list, but the tail is still a list.

To solve this I’d probably use uncons twice - once for the kind, as you already have, and again for the exec_decoder/lib_decoder.

You could define a let operator for uncons to make this look nicer. I’d add this to Decoders but I’m hesitant to add a whole barrage of cryptic operators.

Something like this:

let ( let*:: ) x f = uncons f x

let nil =
  list value >>= function
  | [] -> succeed ()
  | _ -> fail "expected an empty list"

let entry_decoder kind =
  match kind with
  | "executables" ->
      let+ v = exec_decoder in
      Some v
  | "library" ->
      let+ v = lib_decoder in
      Some v
  | _ ->
      succeed None

let entry_decoder =
  (* pop the first element off the list *)
  let*:: kind = string in
  (* now pop the second *)
  let*:: entry = entry_decoder kind in
  (* optional - assert we have nothing left to decode *)
  let+ () = nil in
  entry

let entries_decoder =
  list_filter entry_decoder |> map List.flatten

Note there is still a List.flatten. This is not due to Decoders, but due to the shape of the sexp and the shape of the desired result. In the source sexp the separate executables and library stanzas contain lists that we just want to concat together.

1 Like
42

Hi @jnavila : I think the combinators you are looking for are variant, field and repeat_full_list. I pushed your example as a test on the repository.

And I also agree that it is a pity that gitlab.inria.fr is not open by default to external contributors. I can ask to open an account for you, if this is what you want.

43

I see thanks (technically there’s a version available through the b0 package but don’t use it) – given the way you wrote your message I thought you had hit some kind of expressiveness issue.

44

I agree with these points wholeheartedly.

PPX is great where the producer and consumer are both under your control, in an internal codebase. For everything else, I want the expressivity of writing parsing code.

Decoders tries to make this as easy as possible. Often, composing decoders is mechanical and mirrors the shape of your types. But where it doesn’t, it’s easy to adjust the decoder, and the adjustment is transparent to other people reading your code.

For example, handling versioned data is trivial with decoders. Just use one_of: try the latest version first, and fallback to the old version if it fails. Or, if you’re lucky enough to have a version field in your data, decode it, and switch on it to choose how to decode the rest.

In the current released version (0.7.0), Decoders tries to treat everything as being shaped like JSON. As such, it has to make a decision on how “objects” are represented in S-expressions (it follows Dune - see the note at the top of Decoders_sexplib.Decode · decoders-sexplib 0.7.0 · OCaml Packages).

In the next (unreleased) version, we are exposing a lower-level ('i, 'o) Decoder.t type (see decoder.mli). This is useful wherever you are decoding a type 'i into a type 'o with some possibility of error. I hope this will pave the way for Decoders interfaces to non-JSON-like formats, such as XML (see xml.ml).

1 Like
45

Another tip: since there is nothing in here specific to S-expressions, you could write it as a functor using the Decoders.Decode.S interface:

module Decode(D : Decoders.Decode.S) = struct
  open D

  ...
end

Then you can instantiate it with module Sexp_decode = Decode(Decoders_sexplib.Decode).

The benefit is you can now decode JSON, CBOR, msgpck, YAML, of the same shape for free.

Might not be all that useful for your use case, but we use this pattern a lot so we can instantiate our backend decoders with Decoders_yojson.Basic.Decode and our frontend decoders with Decoders_bs.Decode (for Bucklescript/Melange).

If you’re writing a library, it also leaves your users free to chose their favorite ocaml JSON library (Yojson, Jsonm, jsonaf, etc).

3 Likes
46

Yes, this can be used to skip some fields, but the reason I’m not using it is that I find it more useful to explicitly capture and ignore all fields. This acts as a safety net when parsing responses from something that might change its schema (a bit like warning 9), and gives an opportunity to quickly change the type of the field when realizing later you need it.

47

Waa! Thanks a lot for the head start! Now, I have no excuses to procrastinate :laughing:

2 Likes
48

Hi, years ago I was not able to use Jane-St’s solution (I didn’t found any example of use, I didn’t found how to use it to read s-expression the same way than XML from the .mli, and no one answered me when I asked on the forum/list about it), so I made my own solution:

It’s very small, and with no deps, you can easily include it into your project.

2 Likes
49

Personally I just pattern match it when there are only 1 or 2 parameters, in your example it will be:

  | Expr [Atom "circle";
        (Expr [Atom "center"; Atom cx; Atom cy]);
        (Expr [Atom "radius"; Atom radius]) ] ->
     (* convert cx, cy and radius to int or float here, and use it *)

If there are more than 2 parameters and if I want that these parameters can be provided in any order, I use getters like:

  | Expr (Atom "circle" :: circ_attrs) ->
      let cx, cy = get_circle_center circ_attrs in
     (* convert cx, cy and radius to int or float here, and use it *)

I would also simplify your input (stroke (width 0.2032)) into (stroke_width 0.2032), which is what we have in CSS and SVG.

Also all your primitives “circle”, “arc”, “polyline” are at the same level, which means that you can just process all these in a very simple way with any iterator from the List module of the stdlib.

1 Like