Could the Marshal module from the stdlib support marshal to/from a Char Bigarray?

Community
,
1

That would be a useful feature, since memmapped bigarrays are the de facto shm
on Unix-likes for multi process programs.

When marshalling, we will need to be able to pass an offset into the bigarray buffer.
The function should return how many bytes were written.
Probably, the current Marshal.to_buffer has the right interface.
Also, bytes.t cannot be memmapped AFAIK.

I think, the current Marshal.from_bytes does not have the right interface, the length
that was read from the (future) bigarray buffer should also be returned.

Thanks,
F.

PS: currently in user-space I can marshal to a string then copy that string into the bigarray; but this involves unnecessary data copy and shms are meant to be used in a high performance setting…

1 Like
2

This used to be available in core. See Bigstring_marshal

1 Like
3

Thanks, that’s useful.
But one problem is that if you rely on Jane Street Core, there are like 142 dependencies that need to be installed…

PS: time opam install -y core; 142 opam packages downloaded… 2min50s on a 24 cores computer.