Is janestreet bin_prot library [https://github.com/janestreet/bin_prot] compatible with mirage unikernels? i.e. it doesn’t seem to have a dependency on Unix module so I gather it is okay to use in mirage unikernels. Correct?
I don’t know any reason why it wouldn’t be compatible with mirage. There’s a
xen directory that appears to have mirage-specific C stubs.
Thanks for confirming.
It seems true for Xen but it should fail for Solo5/KVM (freestanding) In such case, you probably need to follow the same layout than bigstringaf to correctly provides C stubs for any targets.
Ah. Okay. In that case does the standard Marshal module work correctly then?
Marshal should be ok, I never use it but I would like to say that the idea of a file-system (and by extension a
file-descriptor) does not really exist into a Mirage (only an Mirage with UNIX should be able to play with
out_channel as we expect).
For example, for some operations such as
open_in, due to the use of
spt targets (Solo5), they always raise an error. It’s the reason of some projects such as:
ocaml-crunchto copy some files into the unikernel (read-only)
irminto have an access to a external database (read-and-write)
More generaly, if your case is to serialize some data and be able to retrieve them outside your unikernel, the best is to use
With regards to
bigstringaf, is this required/recommended when building mirage unikernel? I don’t have experience with this lib and after looking at its api, it seems to offer an alternative api to stdlib
Buffer data structure along with getting/setting bytes of BigEnding/LittleEnding integers. Stdlib with ocaml version 4.08 seems to provide these API too in its Bytes module. Is
bigstringaf more performant than these stdlib APIs?
So the problem is not precisely
bigstringaf but “the required layout for a library to be compatible with MirageOS”. Generally, an usual library in OCaml can be used with MirageOS only if it does not depends on
dune comes with a mechanism,
forbidden_libraries to ensure that you don’t depend on
C stubs and MirageOS
However, for a library which has some C stubs, the requirement is a bit more complex. In fact, MirageOS can compile your librairies with something else than the usual standard C library. If you want to make a the
hvt unikernel (Solo5/
ocaml-freestanding), your C stubs must be compiled with some extra flags and some specific headers.
The usual layout in such case (which appears for
mirage-crypto) is to provide into the distribution cross-compiled C artifacts for MirageOS. The example of
bigstringaf is better than others because the library is small.
Finally, the library provides:
libbigstringaf_stubs_freestanding.ato be linked with a Solo5 unikernel
libbigstringaf_stubs_xen.ato be linked with a Xen unikernel
Both are compiled with specific rules:
bigstringaf.freestandingwhich is a compilation of the copied
bigstringaf_stubs.cwith specific flags. Currently these flags can be get by
It’s the same than
freestandingbut with Xen’s flags
Then, the library must provides the place where are located these specific C libraries. For that, we use the
META file and we add these informations:
xen_linkopts = "-l:xen/libbigstringaf_xen_stubs.a" freestanding_linkopts = "-l:freestanding/libbigstringaf_freestanding_stubs.a"
With all of that, you are able to use these C stubs with MirageOS! It’s a bit complex than usual but we must respect that to be able to correctly link and use your C stubs in any proposed targets by MirageOS.
The same layout exists for
bigarray-overlap or any libraries largely used by MirageOS which contains C stubs. To come back to your initial case,
bin_prot seems to provide the Xen target (eg.
xen directory) and it follows the same pattern:
- a sub-library
- a call to
pkg-configto get extra flags
- a modification of the
So, as far as I can tell,
bin_prot is compatible with MirageOS only for Xen targets. However, it should be easy to extend it to Solo5 unikernel (and, again, follow the same layout).
bigstringaf was ported to MirageOS mostly to facilitate the port of
httpaf to MirageOS (as one of its dependencies). It is about an issue from the OCaml community to have a decent API to use a
bigstring (or more specially, a
(char, unsigned_int8_elt, c_layout) Array1.t.
Some others projects like it exist:
And an issue exist to integrate such API into the standard library: https://github.com/ocaml/ocaml/issues/7772
Then, about the question if we should use
Bigstringaf, I think it depends on your use-case. As explained Hannes Mehnert interview about MirageOS and OCaml by Evrone,
Bigarray has some advantages and some inconveniences.