Linking several *.so libraries produced by dune

Dune makes it very easy to build shared objects (*.so) so that OCaml libraries can be used from C. You can see a minimal example here. To give you an overview of this example, here is the dune file for the shared library:

(library
 (name lib)
 (public_name binding-example)
 (modules lib))

(executable
 (name CAPI)
 (libraries binding-example)
 (foreign_stubs (language c) (names cstub))
 (flags :standard -linkall)
 (modes (native shared_object))
 (modules CAPI))

(install
 (section lib)
 (files
  (CAPI.so as libbinding-example.so)
  (cstub.h as binding-example.h)))

The OCaml functions that we want to make visible from C are registered in CAPI.ml:

let () = Callback.register "hello" Lib.hello

And cstub.c provides a wrapper to call the hello: string -> string function from C:

#include <caml/mlvalues.h>
#include <caml/callback.h>
#include <caml/alloc.h>

char* hello (char* name) {
  static const value* closure = NULL;
  if (closure == NULL)
    closure = caml_named_value("hello");

  value result = caml_callback_exn(*closure, caml_copy_string(name));
  return (char*) result;
}

void initialize_example (char** argv) {
  caml_startup(argv);
}

Dune then produces a *.so file that can be linked with a C client program. To see how this works in more details, you can download the full example and use make.


This worked great for me when linking a single OCaml library produced this way.

However, when I use dune to produce two *.so files for two different ocaml libraries and link them both to a client program, I have a problem. Indeed, the code in CAPI.ml that registers OCaml names is only run for the first library and not for the second one (first being defined by the order in which the libraries are linked). Therefore, functions from the second library segfault as caml_named_value returns a null pointer.

Can anyone explain what is happening and does anyone have a workaround?

1 Like

This is just a guess, but it seems to me that trying to link two shared libraries with many of the same symbols (the full runtime system is included in each one of them) is bound to be dicey. At the very least you will probably need to make the shared symbols “local” to each library to avoid “mixing” them; but even then am not sure it will be easy to make it work…

I think you will be better off putting all your code in a single shared library.

Best wishes,
Nicolás

1 Like

Thanks for your answer. I agree that putting all the code in a single shared library would be easier.
I am wondering if it would be possible to do something more modular though.

Would it be easy to produce a shared object without the runtime for each library and then link the client with every generated *.so and the OCaml runtime (libasmrun.so?)? How would you do it with dune?

Edit: one reason I am instant on using shared objects rather than static libraries is that my goal is to write wrappers of OCaml libraries for the Julia language, whose FFI can only load dynamic libraries.

I don’t see a way to do it with the current compiler: there is an option -output-obj that will produce for you an object file .o which does not link the runtime, but it will still link the stdlib and you will have the same problem.

Perhaps you may want to open an issue at https://github.com/ocaml/ocaml explaining your use case.

Cheers,
Nicolás

Here’s a possible workaround:
(1) first, implement an Ocaml library with C stubs, that will dynamically link specfied .cmo files. Maybe it would use findlib to find prereqs and link those first, keep track of what it linked, etc.
(2) then write some C helper functions to lookup Ocaml entrypoints, and use those in your C wrappers
(3) now, you have a single .SO that contains Ocaml’s runtime and your dynlink harness
and (4) you have a bunch of .SO files you generate that don’t contain ANY Ocaml code at all, and just depend on that first .SO to do the dynamic linking.

1 Like

Thanks everyone for your help.
@nojb Yes, I will find an issue once I get a clearer use case.
@Chet_Murthy I guess this could work, I’ll have a look at this.