Hi, I saw recent discussion about Dune’s inclusion in the OCaml platform roadmap, the future of older build systems such as OCamlfind, and whether Dune should be the “blessed” build system. To preface, for most of my OCaml programming needs, Dune has been sufficient and easy to use. However, I’m concerned about the suitability of both OCamlfind and Dune for the LLVM OCaml bindings.
First of all, LLVM is a C++ project. The people developing LLVM are C++ programmers, most of whom don’t know OCaml. This obviously causes friction whenever people need to make changes that require them to touch the OCaml bindings (perhaps because they are changing the C bindings). Therefore, there has been talk on the LLVM forums of moving the OCaml bindings to the “peripheral tier” or out of tree, though I believe that if this happens, the OCaml bindings will eventually fall of sync with new development. Both the Go and Python bindings have been recently removed, leaving the OCaml bindings the only ones left in the repository. Therefore, I am nervous about making any drastic changes that may inconvenience the other LLVM developers or provide motivation for them to move the OCaml bindings out of tree.
As a C++ project, LLVM uses CMake. In the repository, the OCaml bindings use CMake via scripts that invoke OCamlfind. If CMake finds OCaml on the system, it will add the OCaml bindings as a build target and run the OCaml tests as part of the test suite. Therefore, the OCaml bindings are integrated into the rest of LLVM’s build system.
To my understanding, the CMake-OCamlfind build system has caused inconveniences when packaging the LLVM bindings for opam. Up until LLVM 15, the LLVM bindings had patches to their build files kept in opam-repository. The most significant patch modifies the META file to enable linking to either the static or shared LLVM libs, while an accompanying shell script detects if the user has the static and shared LLVM libs installed. This solution makes use of OCamlfind’s “predicate” feature: the predicate
llvm.static activates linking to the static libs. However, Dune does not support OCamlfind predicates, causing issues when using LLVM from Dune.
@kit-ty-kate has worked on a repository, llvm-dune, that has the LLVM repository as a Git submodule and builds the LLVM OCaml bindings with Dune. She has recently transferred its ownership to me. llvm-dune has a shell script that checks for static and shared LLVM libs and dynamically generates
dune files in the source tree. Instead of OCamlfind predicates, it makes use of Dune’s virtual library feature to allow users to select between the static and shared libs. Virtual libraries allow a choice between multiple implementations of a module. So, the script copies each LLVM module’s source code into
shared directories together with an appropriate
dune file with the appropriate linker flags, to create “static” and “shared” implementations.
I’m not sure if the intention was to eventually upstream a Dune-based build solution into LLVM, but my personal feeling is that using a shell script to generate Dune files, and then duplicating the source code in
shared directories, is too hacky to go upstream. I tried to find a better solution with Dune, but I could not find one. Furthermore, I’m not sure how I would integrate Dune into LLVM’s CMake-based build system. So, my concerns with Dune are:
- Dune does not support OCamlfind predicates.
- I cannot find a non-hacky way for Dune to select between static and shared libraries. Virtual libraries involve duplicating source code. For this use case, the source code is exactly the same, but the linker flags are different.
- I am unsure how to integrate Dune into a CMake-based project in a way that won’t inconvenience C++ programmers.
Ideally, I think that the LLVM OCaml bindings should neither need patches and scripts in opam-repository nor a separate repository with the LLVM repository as a Git submodule. However, I do not think that any OCaml build system is currently sufficient. I personally worry that favoring Dune over other build systems is a step backwards because it is too opinionated to integrate into a C++ project that does not revolve around OCaml and does not provide the features that the LLVM package needs.
I want to know what other people think and if a solution can be found.