Writing bigarray to file

Community
1

Dear camlers

There is no way to write a bigarray to a file (except in C or element by element) ? In my case, the file is a socket hence map_file is not an option.

Cheers,
Christophe

3 Likes
2

Indeed, this operation is not available in the standard library. If you are dealing with a “char” bigarray, there are third-party libraries that will do what you want, eg ocaml-bigstring:

If you are dealing with a general bigarray, then you will need to work a bit harder by writing some C code yourself.

Cheers,
Nicolas

3

GitHub - inhabitedtype/bigstringaf: Bigstring intrinsics and fast blits based on memcpy/memmove is the maintained alternative to the ocaml-bigstring. I use it in production and it works great, can recommend.

4

I second bigstringaf, it’s better than bigstring :slight_smile:

1 Like
5

Maybe npy is the answer for float bigarrays.

6

You could still try marshaling the bigarray to file then send that file over the socket.

Mmapping a bigarray to file is several orders of magnitude faster than marshalling w/ Marshal, in my experience.
Here is a code snippet:

  assert(n = BA1.dim g.arr);
  let fd = Unix.(openfile fn [O_RDWR; O_CREAT; O_TRUNC] 0o600) in
  (* create mmapped bigarray *)
  let dst =
    BA.array1_of_genarray
      (Unix.map_file fd BA.Float32 BA.c_layout true [|n|]) in
  (* copy existing bigarray to the (new) one mapped to file *)
  BA1.blit g.arr dst;
  Unix.close fd;
2 Likes