Foreign_archives: no static libraries, .dylib on mac

Ecosystem
,
1

I am trying to use “Foreign Build Sandboxing” to include a “vendor” library in an OCaml interface; and also hopefully to better understand dune.

Compiling the vendor library with cmake on macOS gives three files: libortools.9.14.dylib, libortools.9.dylib, libortools.dylib (the last two obviously being symlinks to the first one). So, I added the following to my dune file:

(library ... (foreign_archives ortools.9.14 ortools.9 ortools))

But dune complains that there are no rules for libortools.9.14.a, dllortools.9.14.so, etc.

I guess I could fight with cmake to get it to generate the files expected by dune. But, is there a standard way to get dune to work with the .dylib files?

What do others do in such cases? Should I try to combine “lower-level” options, like library_flags, install stanzas, and use enable_if to have separate rules for macOS?

2

I don’t think dune provide a way to alternate the naming for foreign archives. I believe dune wanted “dllortools.dylib” and libortools.a in mac.

for example, if you declare it like this:

(library
 (name ortools_bindings_c)
 (public_name ortools.bindings.c)
 (preprocess no_preprocessing)
 (foreign_archives ortools))

This definition means you need to provides libortools%{ext_lib} and dllortools%{ext_dll}.

You don’t need to fight it, just use your vendored library own build system, then copy the resulted files to naming convention that dune expected.

1 Like
3

As mentioned by @syaiful6, the simplest is to change the extension from .dylib to .so. Note that the use of .so instead of .dylib is inherited from the OCaml toolchain and is not a choice by Dune (see Build dylibs, not bundles, on macOS by whitequark · Pull Request #988 · ocaml/ocaml · GitHub for some context).

Cheers,
Nicolas

1 Like
4

Thank you both for your suggestions. The context on github is especially helpful. I didn’t realize that the issue is inherited from the underlying compiler. I guess the library stanza abstracts over ocamlmklib, just as executable abstracts over ocamlc and ocamlopt.

As far as I can understand, the renaming approach has two disadvantages.

  1. It would also be necessary to add a symbolic link from libortools.a to dllortools.so.
  2. It makes it more complicated to choose between an existing local installation of the library (libortools.dylib somewhere in LIBRARY_PATH and DYLD_LIBRARY_PATH) and a vendor build (dllortools.so in ~/.opam/<version>/libs/stublibs).

If I can find a convincing alternative with library_flags/install, I’ll post it here.

5

This doesn’t sound right: the .a file is a static library, while the .so/.dylib is a shared library; the two files are incompatible and cannot be used interchangeably.

Cheers,
Nicolas

6

I admit to being a little bit lost with modern development on macOS. I thought the .dylib included both. If not, this only exacerbates the first disadvantage:

  1. It would also be necessary to build static libraries (libortools.a).
7

I am not a macOS user, but I don’t think so.

Yes, this is how the OCaml toolchain works: in native-code, it defaults to static linking, in bytecode to dynamic linking. If you are happy to build only in bytecode, then you won’t need the static library. Otherwise, your only alternative (if you cannot build the .a of your library) is to link with the dynamic library also in native-code, but this you will need to do “by hand”, and, furthermore, it will require that you make the shared library available at runtime which is less convenient than static linking.

If you still want to go that route (dynamic linking in native-code), you should remove the (foreign_archives) field from your stanza and use something like (link_flags -cclib -lortools -L<path to dllortools.so>) instead. “Something like” because there are many factors involved in linking like this, and neither Dune nor the compiler is there to help you, so you will probably need to play around until you find the right incantation (the -verbose flags to both Dune and the compiler are your friends here).

Cheers,
Nicolas

1 Like
8

“Something like” because there are many factors involved in linking like this, and neither Dune nor the compiler is there to help you,

pkg-config makes this less of a pain on many platforms, for example pkg-config –-libs SDL2 prints -L-/opt/homebrew/lib -lSDL2 for me on macOS. Github has current use in pkg-config path:**/dune

2 Likes
9

Thank you @nojb and @jrfondren. You make very good suggestions.

The difficult thing for me is to understand and to adapt to the dune approach.

For a “standard” project, in the sense defined by dune’s functionalities, everything works really well: one writes a minimum of specification and one obtains an automatic and fast build that integrates well with opam. What’s not to like?!

For certain more difficult problems, as I am learning, there are solutions (sandboxes, flags, OCaml scripts, dynamic includes, …) which work increasingly well and whose documentation is improving over time. I thank everyone who is invested in this.

Still, at some point there is probably a trade-off with the configure/Makefile approach, with its own pros and cons. This is another question that interests me.