Hand-writing bin-prot writers/readers for cyclical data structures?

Learning
1

As per the the README at https://github.com/janestreet/bin_prot cyclical data structures cause non-termination. Where non-termination may even mean segfault due to stack overflow.

Is hand-writing the converters to work around this limitation an exercise in suffering, or do people do it? Or both?

EDIT: to be clear, I’m wondering if there are projects that have hand-written bin_prot writers (and readers). Would be nice to have something to slide into all of the places that normally expect/produce bin_prots.

2

For the sake of anyone interested in this in the future, it’s not too much suffering to meet the interface bin_prot expects.

Simplified, for a given t you need to provide a bin_size_t function that tells how many bytes t would take to represent in encoded form, a bin_write_t function that writes the encoded t into a bigstring of size bytes, and a bin_read_t function that given a bigstring returns a t.

You can even use the generated by bin_io functions for more primitive underlying types to help.

That all said, it was easier still and offered good enough performance to write a function that traverses the cyclic data structure to yield an associative list of immediate values and use bin_prot on that.

1 Like
3

I have a vague memory from working on CORBA marshallers, that in the past there have been IDLs where you marked the types that could “enjoy” cyclic references, and the IDL-compiler generated code to check for shared-pointers at those points. I almost have a memory that maybe some Microsoft IDL did this? It seems like this might be an easy thing to add to a type-deriver for bin_prot? That is, assuming that bin_prot has the necessary wireline information slots …

4

Is that better or worse than offering a deriver called bin_io_slow (you normally use bin_prot by saying deriving bin_io) and use that on types where you want to support cyclic references (or shared values).

5

Oh sorry, I should have Ibeen clearer: there is always overhead for this (tracking sharable pointers) support, but since it only gets triggered by markings on the IDL type-tree, you don’t pay for it unless you actually use it. For types that aren’t thus marked, the generated code should e the same as with current bin_io.