I have a Dune-project (an application) A that depends on a third library L, this library is located in the workspace of the project, but it is built with a Make script (not using Dune).
The project tree is similar to:
I would like my application to depend on this non-Dune library. I have been looking into the docs and I have tried a number of approaches without much success…
My preferred choice would be to package the library as a Dune-project, while still building it using the existing Make script. I tried following the Foreign build sandboxing idea without success:
(deps (source_tree src))
(chdir src (run make))
(copy src/L.cmxa L.cmxa)
This doesn’t work because the “library” stanza (which is mandatory for declaring a library) must produce “L.cmxa” but it will not know how. I specify a custom rule that produces “L.cmxa”, but this rule conflicts with the “library” stanza.
Another option would be using a pre-compiled “L.cmxa” when building the application A. I could not find the way to specify such dependency. It seems that Dune can only make use of pre-compiled libraries when they are in the “installed world”. I tried copying the .cmxa file with
(copy_files ../L/src/L.cmxa) but one cannot copy files from outside the current directory. Still, if I copy the
L.cm* files of the library into A/src/lib, and then add
(copy_files lib/L.cm*) to A/src/dune, this doesn’t work either (it seems that no file is being copied into _build/default/src).
I would appreciate any recommendations, hints, or pointers to examples.
Thanks in advance!
Hi @iago, the foreign build sandboxing is only meant for writing stubs in foreign languages such as C, not for OCaml libraries. What you are trying to do is currently not supported by Dune. The usual way to mix Dune and non-Dune libraries is to build and install the various libraries separately. Indeed, Dune is able to use globally installed libraries that were not built with Dune, however it is not able to build such libraries itself.
Adding support for this scenario would be possible, however it is a non-negligible amount of work. Have you tried porting the third library to Dune? The process is usually quite straightforward.
Thank your for you answer @jeremiedimino.
It’s not obvious to me how to build my library
L with Dune, although I can believe it can be done. That is why I was trying to just “sandbox” it.
Let me rename L as mylib… the structure of
mylib is something like:
The sources of the native C library are in mylib/c/ and the OCaml bindings in mylib/ocaml/.
I have given it a try at building
mylib with Dune adding the following mylib/dune file:
(c_library_flags -ccopt -fPIC -L. -lmylib)
This almost works but ocamlmklib cannot find mylib. This is expected because it is found in c/libmylib.a… I would need to get the library file copied to the build directory (e.g. _build/default), but I can’t manage to get that to work. I can add a rule like:
(chdir c (run make))
(copy c/libmylib.a libmylib.a))))
But how do I force this rule to run?
The OCaml library containing the stubs can definitely be built with Dune. Regarding the native C library in mylib/c you have two choices:
- if the library is simple enough, i.e. just a bunch of C files compiling in a straightforward way, then you can try to remove the
c_library_flags field and simply add the various C files to
- if not, you should sandbox the build of the the C library only and pack as its own OCaml library, for instance
mylib.c and then make
mylib depend on
To do the former, you can effectively follow the foreign build sandboxing. Technically you need to add something like this in
(deps (source_tree c))
(targets libmylib_c_stubs.a dllmylib_c_stubs.so)
(chdir c (run make)))
(copy libxxx.a libmylib_c_stubs.a)
(copy libxxx.so dllmylib_c_stubs.so)))
Note that the filenames of the
.so files must be of the form
dllXXX_stubs.so. We could extend dune so that the intermediate
mylib_c library is not necessary, however this is not supported at the moment.
Does copy now work? Last time I had tried that I had to shell out to copy the artifact
copy should definitely work, do you remember what didn’t work in your case?
The issue was the one discussed here: Strange error in dune building
Thanks again for the hints @jeremiedimino.
A small thing I noticed is that
(copy c/libmylib.a libmylib_c_stubs.a) introduces a dependency on
mylib/c/libmylib.a. I can see this dependency in the output of
dune rules mylib/libmylib_c_stubs.a:
And indeed Dune complains if I build from a clean workspace:
Error: No rule found for mylib/c/libmylib.a
So I am forced to
make -C mylib/c before invoking Dune although this step is already part of the rule that generates
Is there any way to work around this?
Ah yes, that’s the same issue @mseri pointed. We need to do something about this in dune. In the meantime, the simplest solution is to put the call to make and the various files copy in a shell script rather than use the DSL. For instance, you can write a shell script containing:
make -C c
cp c/libxxx.a libmylib_c_stubs.a
cp c/libxxx.so dllmylib_c_stubs.so
and call this shell script.
I created an issue to track this in GitHub instead: https://github.com/ocaml/dune/issues/2006