[ANN] first release of lz4-chans

Dear OCamlers,

It is my pleasure to announce the first release of lz4_chans:

You can use them as drop-in replacement for the channels provided
by the stdlib:

(** open/close binary channels, with LZ4-compression
    happening in the background, using a separate process and a named pipes *)

val open_in_bin: string -> in_channel
val open_out_bin: string -> out_channel

val close_in: in_channel -> unit
val close_out: out_channel -> unit

val with_in_file: string -> (in_channel -> 'a) -> 'a
val with_out_file: string -> (out_channel -> 'a) -> 'a

It was fun to write this little system programming thing.

Performance tests on my computer:

2019-05-17 16:38:40.483 INFO : plain_fn: /tmp/lz4_chans_test_1b1d6a.bin
2019-05-17 16:38:41.847 INFO : plain output: 7335450.09 floats/s
2019-05-17 16:38:42.670 INFO : plain input: 12191301.78 floats/s
2019-05-17 16:38:42.686 INFO : lz4_fn: /tmp/lz4_chans_test_8b6517.bin.lz4
2019-05-17 16:38:45.348 INFO : lz4 output: 3757097.68 floats/s; eficiency: 0.51
2019-05-17 16:38:46.518 INFO : lz4 input: 8557598.32 floats/s; efficiency: 0.70


1 Like

Would be awesome at some point to use decompress library in the future, if and when LZ4 will be implemented there.

Contributions are welcome.

But, the fun thing with the pipe hack and using a system command is that you
can easily change to whatever real-time compressor is available on your system.

You just need to change the (de)compression commands, run the test (make test) and you are good to go! :smile: