What is the best practice for exporting data from one OCaml program to be read or incorporated by another OCaml program? The data transfer is through a file.
A. Write a %a printer to output the data in an OCaml readable fashion. Advantage is that the output is human readable and editable without introducing external dependence. And the data can be incorporated at compile time.
B. Marshaling the data through some web standards (like JSON). Encoding/decoding could require more work except for the most basic data structures.
C. output_value is pretty magical but the output is not human readable or editable. Does OCaml retain names of constructors at runtime or is it completely based on matching memory layout? Are there any potential pitfalls?
answering this implies omniscience. Knowing all options and requirements including the future. That’s utopian, this will lead you nowhere. Focus on the properties you care about and use what suits you.
It depends on your use-case, but doing this would only solve the “encoding” part and you would need to write a parser to decode data written on this format.
No, no such “type” information is kept in the marshalled data. The data is only guaranteed to be read back in a program compiled with the same version of OCaml (and of course against the exact same type definition that was used when marshalling).
It depends on your use-case, but doing this would only solve the “encoding” part and you would need to write a parser to decode data written on this format.
I was actually thinking that the format would be compiler readable (a poor man’s version of meta-circulating compilers) so that the parser is free too.
Wow! That is really cool! This is exactly the kind of answer I was looking for that I didn’t even know how to pose the question for, hence the more general open-ended lede I had to resort to (@mro).
to add to @nojb’s comment, there is also a PPX deriver for Google protobuf format, and of course, as part of Thrift, a type-based de/marshaller-compiler that can generate to OCaml for Thrift types, so you can use the Thrift format.
Another interesting option could be the Umarshal module that was recently shipped with the Unison file sync tool. I took a look at the code and it looks fairly self-contained to me; could probably be extracted pretty easily. The cool thing about it is that it aims to be an almost drop-in replacement for OCaml’s Marshal module, except type-safe and stable across OCaml versions. It can also handle variant types, which few other serialization systems can.
A lot of people are happy with sexp.
The files are written in readable ASCII s-expressions, but don’t include the field names of ocaml structures. It doesn’t require a lot of work while it read and write from the type definitions that are annotated.