How to use [@@deriving bin_io] with ucommon types

ppx_deriving
#1

I have assumed that I could use ppx_bin_prot to generate functions for serializing, into bin_prot formats, structures containing bin_prot-defined types. If this is correct, I haven’t been able to figure out what I have to load or open to get it to work:

# #require "ppx_bin_prot";;
# open Bin_prot.Std;;
# type foo = {dims : int array; buf : Bin_prot.Common.buf} [@@deriving bin_io];;
Error: Unbound value Bin_prot.Common.bin_shape_buf

I can get [@@deriving bin_io] to work for basic OCaml types. int array doesn’t cause a problem, for example. I’ve tried opening various Bin_prot submodules before the type definition, but they haven’t helped.

Do I need to define some functions myself in order for deriving to generate functions for a record type containing Bin_prot.Common.buf? (Or can I use a different type as the target/source for bin_prot serialization functions?)

#2

Sorry–I think wasn’t thinking about this correctly. Withdrawing the question. Maybe more later. Apologies to anyone who’s getting this by email.

Oops, can’t figure out how to delete the thread or the original post in it. :frowning:

#3

What I probably should be doing is this:

type serialized = 
  {dims : int array ;
   bigarray1 : (float, CamlinternalBigarray.float64_elt, CamlinternalBigarray.fortran_layout) Bigarray.Array1.t
  } [@@deriving bin_io]

since that bigarray type is what I want to serialize. But that produces a similar error:

Error: Unbound value Bigarray.Array1.bin_shape_t

I am guessing that I am going to have to learn more about bin_prot or ppx_bin_prot to make this idea work. Perhaps it will be easier to simply serialize/unserialize the dims and thebigarray1 separately.

(Sorry for the public display of what hasn’t been properly thought through. Since I couldn’t delete the thread, I figured I might as well correct my original idea, but didn’t want to revise the original post to make it say something very different since some people had already read it.)

#4

[@@deriving bin_io] generates (de-)serialization code for a given type, and it has knowledge of how to do so for basic OCaml types. However, if the record type you’re trying to use it on contains a field with a type that it doesn’t know how to (de-)serialize (e.g., Bin_prot.Common.buf).

Since it just looks for the bin_*_t identifiers in the module of the type name you specify, and Bin_prot.Common.buf = Core.Bigstring.t, you can use Core.Bigstring.t as the type of your record field instead. Core.Bigstring includes the bin_shape_t and other identifiers that ppx_deriving was looking for.

Unfortunately, Core does not define (de-)serialization protocols for bigarrays more generally. There are type aliases in Bin_prot.Common named vec* and mat* that might apply to your case, but I can’t figure out how to get them to work with @@deriving, short of doing:

type vec = Bin_prot.Common.vec

let bin_read_vec = Bin_prot.Read.bin_read_vec
let bin_shape_vec = Bin_prot.Shape.bin_shape_vec
let bin_size_vec = Bin_prot.Size.bin_size_vec
let bin_write_vec = Bin_prot.Write.bin_write_vec

type t =
  { a : int array
  ; b : vec
  }
[@@deriving bin_io]
#5

Thanks very much @bcc32! This solves my problem, I think. vec is actually all I need as the type of the second element. I’ll see if I can figure out the aliases, but I wouldn’t have known to define my own aliases as you did, and using that with @@deriving adds in the other functions that are needed. Using your approach, I was able to define the record with the vec element containing data, and then serialize it and unserialize it without problem.

Good to know, also, that Core.Bigstring contains the required functions for that type.

#6

@bcc32’s solution and comments allowed me to read the bin_prot docs with a bit more insight than I had had. This led to the discovery that opening Bin_prot.Common as well as Bin_prot.Std allows one to avoid the extra aliases that @bcc32 defined:

# #require "ppx_bin_prot";;
# open Bin_prot.Std;;
# open Bin_prot.Common;;
# type foo = {a : int array; b: vec} [@@deriving bin_io];;
type foo = { a : int array; b : vec64; }
val bin_shape_foo : Bin_prot.Shape.t = <abstr>
val bin_size_foo : foo -> int = <fun>
val bin_write_foo : buf -> pos:int -> foo -> int = <fun>
val bin_writer_foo : foo Bin_prot.Type_class.writer0 = {Bin_prot.Type_class.size = <fun>; write = <fun>}
val bin_read_foo : buf -> pos_ref:pos_ref -> foo = <fun>
val bin_reader_foo : foo Bin_prot.Type_class.reader0 = {Bin_prot.Type_class.read = <fun>; vtag_read = <fun>}
val bin_foo : foo Bin_prot.Type_class.t0 =
  {Bin_prot.Type_class.shape = <abstr>; writer = {Bin_prot.Type_class.size = <fun>; write = <fun>};
   reader = {Bin_prot.Type_class.read = <fun>; vtag_read = <fun>}}

I think this works because Bin_prot.Common defines the type vec, while Bin_prot.Std defines aliases for the corresponding helper functions (which are defined in other modules). That’s how it looks to me, anyway; I’m not sure I understand everything involved.