Using output of `ocamlfind <params> -only-show` in a dune build

Learning
1

Can it be done?

Context: I’m trying to build LinearML using dune and one remaining stumbling block is this. The compiler limlc build, but when when compiling LinearML stdlib it fails with the following:

../compiler/limlc: Symbol LLVMConstNull’ causes overflow in R_X86_64_PC32 relocation`

Same for LLVMSizeOf

Looking at the output of 2 commands

dune build --verbose

ocamlfind ocamlopt -package llvm,llvm.scalar_opts,llvm.analysis,llvm.bitwriter,unix -predicates llvm.static -linkpkg -only-show

where the important part is -pedicates llvm.static I’ve managed to build an executable which uses the static llvm libs and it works. Now I’m wondering if that can be somehow automated.

Just in case here is the compiler dune file

(executable
 (public_name limlc)
 (name main)
 (libraries 
    llvm 
    llvm.scalar_opts
    llvm.analysis
    llvm.bitwriter
    unix)
 (flags (:standard -w -27-35))
 (link_flags (:standard -linkall)))
 
(ocamllex (modules lexer))
(ocamlyacc (modules parser))
1 Like
2

I had a look at this. Right now, no it can’t be done. If it’s only for local development and you want a quick workaround, you can apply the following patch to dune:

diff --git a/src/findlib.ml b/src/findlib.ml
index c7c1e37d..d1825232 100644
--- a/src/findlib.ml
+++ b/src/findlib.ml
@@ -179,7 +179,7 @@ module Package = struct
   let loc  t = Loc.in_dir t.meta_file
   let name t = t.name
 
-  let preds = Ps.of_list [P.ppx_driver; P.mt; P.mt_posix]
+  let preds = Ps.of_list [P.ppx_driver; P.mt; P.mt_posix; P.make "llvm.static"]
 
   let get_paths t var preds =
     List.map (Vars.get_words t.vars var preds) ~f:(Path.relative t.dir)

For a proper solution, it’s a bit more complicated. We could try to wire up the variant system in dune to the predicate system of findlib, though it might be difficult to do and the result might not be very satisfactory given that the two systems don’t work exactly the same. /cc @bobot who might be interested in this idea.

Another option would be to use dune to build the llvm bindings, and rely on the variant system of dune. /cc @whitequark and @kit-ty-kate about this idea.

3

I don’t think that’s easy (or possibly even feasible). The in-tree LLVM bindings are built with CMake currently, not even ocamlbuild; I wrote that and I don’t really know how is one supposed to use Dune for that. In addition, Dune would need to be in Debian stable for the buildbots to be able to build bindings.

4

I see, that does make it difficult. The next Debian stable will have Dune but only 1.6.2 which doesn’t support variants.

5

Thanks @jeremiedimino for the response and suggestion. And for a bit of contex, I’ve got interested in LinearML some 7 years, and even submitted some patches.

Since the project is still mentioned from time to time, I’ve decided to try to make it easier to check out and build with the latest versions of OCaml and LLVM.

I think I’ll try something like this

 (link_flags (:standard -linkall (:include llvm.static) \ (:include llvm.shared))))

where llvm.static and llvm.shared files will be, obviously, generated.

@whitequark I’m using the official .deb packages from llvm.org. And one difference from the executables is that that one dynamically links to libLLVM.so and the other not. Can it be that the the shared library is not build using the -fPIC option which is required on latest Debian systems?

6

I’m not sure what you mean–can you rephrase? What is “one” and “other” here?

7

Sure. The executable name is limlc. When I build it using the llvm/shared *.cmxa libraries ldd limlc gives me:

$ ldd limlc
	linux-vdso.so.1 (0x00007ffca1de5000)
	libLLVM-8.so.1 => /usr/lib/x86_64-linux-gnu/libLLVM-8.so.1 (0x00007fba76288000)

When I use the llvm/static *.cmxa libraries there is no libLLVM-8.so.1 in ldd output.

Sorry, if this is a silly question.

8

Hi,

Sorry for to reply one year later but I finally I had the time and motivation to work on this. I managed to figure out a solution to build the llvm bindings using dune and to include the statically linked variants.

Here is the repository to test it: https://github.com/kit-ty-kate/llvm-dune

$ cd llvm-dune
$ git submodule update --init
$ ./setup.sh <llvm-config> # Replace with the correct llvm-config command
$ dune build --profile=release
$ dune install

I’m planning to use it in the release of llvm.11.0.0 (I’ll probably need to patch the generated META file first to avoid issues with people using the old findlib predicate feature).
I’ll do proper announces in multiples places first to see if multiple people can test it first to see if there aren’t any issues with it. It works for https://github.com/kit-ty-kate/labrys (both in shared and static mode) but I’m not sure to know who uses the ocaml bindings to ask them directly.

3 Likes
9

Very interesting! I will try to find time to test this. I’m also hoping that following this approach will enable me to unblock D60902.