I’m working on a proof assistant that read and rewrite cache files produced with Marshal module. The file size ranges from 20K~900K.
I wonder if it could benefit from applying compression on these cache files, like what Agda does on
.agdai files. Tested with LZ4 command line tools (with default compression configuration) it can achieve about 70% or more file size reduction on my cache files.
However I do not know if there’s an effective way to integrate compression with OCaml’s IO, which seems can not be easily extended without hacking the C runtime. And there did exists an OCaml binding to LZ4 or various implementation of other lz77 family of compression algorithms, and most of them support only compression on
bytes data type.