[ANN] [dk0] CMake, Ninja and Google or-tools packages

Ecosystem
,
1

It is my pleasure to announce the following packages for ([ANN] The dk0 build system):

These packages are my start to a clean replacement for opam depexts. depexts does not work well for me: basic engineering (reproducibility) is gone, system packages often are too far behind their upstream versions, the complete unavailability for Windows users, the occasional symlink break with homebrew, etc.

Here’s the basic idea (macOS only for now):

# There are other ways to install dk0 but for now use this
$ git clone --branch V2_5 https://github.com/diskuv/dk.git dksrc

# Then invoke a fully-qualified target to build `or-tools`
# which will quickly download precompiled artifacts
$ dksrc/dk0 --trial get-object NotInriaParkas_Caml.ORTools@9.15.0 \
  -s Release.Darwin_arm64 -d target/ocaml-ortools-arm64/

# See what is there
$ tree target/ocaml-ortools-arm64 --filelimit 5
target/ocaml-ortools-arm64
├── include  [33 entries exceeds filelimit, not opening dir]
├── shared
│   └── lib  [262 entries exceeds filelimit, not opening dir]
└── static
    └── lib  [131 entries exceeds filelimit, not opening dir]
# ...
$ du -sh target/ocaml-ortools-arm64
464M    target/ocaml-ortools-arm64
# ...
$ find target/ocaml-ortools-arm64 -name 'libortools*'
target/ocaml-ortools-arm64/shared/lib/libortools.9.15.so
target/ocaml-ortools-arm64/shared/lib/libortools_flatzinc.so
target/ocaml-ortools-arm64/shared/lib/libortools_flatzinc.9.15.so
target/ocaml-ortools-arm64/shared/lib/libortools.9.so
target/ocaml-ortools-arm64/shared/lib/libortools.so
target/ocaml-ortools-arm64/shared/lib/libortools_flatzinc.9.so
target/ocaml-ortools-arm64/static/lib/libortools.a
target/ocaml-ortools-arm64/static/lib/libortools_flatzinc.a

or-tools is just a proof of concept to test my CMake package. I know little about or-tools! I just picked it because it was the C library being discussed last week.

Calling a build target looks at first like installing a package:

  • you get a 125MB compressed download of precompiled or-tools libraries. In this case those libraries have been packaged to follow OCaml conventions (both static and shared libraries are present, and .so extensions instead of .dylib on macOS) … the same thing you expect when you use depexts.

But dk0 is a build tool. So if you change to a set of parameters that has not been precompiled like so:

rm t/d/val.1/j* # hack for bug https://github.com/diskuv/dk/issues/98

dksrc/dk0 --verbose -I dksrc/etc/dk/v --trial \
  get-object NotInriaParkas_Caml.ORTools@9.15.0 \ 
  -s Release.Darwin_x86_64 -d target/ocaml-ortools-intel/

the build will happen locally (~1 hour on an M1).

This style of build system may be unfamiliar. So my analogies would be:

  • a nix binary cache with overlays (I haven’t personally used nix); JSON and Lua serve the same purpose as the Nix language, or
  • an Internet-accessible dune cache that everybody can use; JSON and Lua serve the same purpose as dune files and dune (rule ...) expressions.

The CMake package limitations as of 2026-02-12 … but scroll down in this thread to see if I have posted an update:

  • I haven’t had the time to complete dk0 packages for C compilers, so the CMake package is incomplete. CMake + Ninja discovers C compilers on macOS trivially, and last week’s threads were discussing macOS, so I prioritized macOS with xcode.
  • I tested the CMake package on Apple Silicon. Windows is partially tested but don’t use Windows yet.
  • There is no integration with opam or dune yet.

Thanks!

Building your own packages

This is bonus material for those thinking about building dk0 packages

The JSON build file for NotInriaParkas_Caml.ORTools@9.15.0 is https://github.com/diskuv/dk/blob/V2_5/etc/dk/v/NotInriaParkas_Caml/ORTools.values.jsonc

NotInriaParkas_Caml.ORTools@9.15.0 uses the C library package NotGoogleDev_OR.Tools.F_Lib@9.15.0. Its Lua build file is https://github.com/diskuv/dk/blob/V2_5/etc/dk/v/NotGoogleDev_OR/Tools.values.lua.

And NotGoogleDev_OR.Tools.F_Lib@9.15.0 uses the tool package CommonsBase_Build.CMake0.F_Build@3.25.3. You can use a command line variation of it (CommonsBase_Build.CMake0.Build@3.25.3) without any JSON or Lua build files … here is an example:

dksrc/dk0 --trial run CommonsBase_Build.CMake0.Build@3.25.3 installdir=i/llama-cpp \
  >   'mirrors[]=https://github.com/ggml-org/llama.cpp/archive/refs/tags' \
  >   'urlpath=b7974.zip#be9d624603e39cd4edee5fa85e8812eb8e1393537c8e4e4629bc4bd016388053,29881192' \
  >   'nstrip=1' 'gargs[]=-DBUILD_SHARED_LIBS:BOOL=OFF' \
  >   'exe[]=bin/*' \
  >   'out[]=bin/llama-quantize' \
  >   'outrmglob[]=test-*' 'outrmglob[]=*.py' \
  >   'outrmglob[]=llama-[a-p]*' 'outrmglob[]=llama-[r-z]*' \
  >   'outrmexact[]=include' 'outrmexact[]=lib'

With that single command, CommonsBase_Build.CMake0.Build@3.25.3 will download the popular C package llama.cpp, build a static library, remove several of its artifacts, and install what remains (especially bin/llama-quantize) into the i/llama-cpp directory. Then type i/llama-cpp/bin/llama-quantize --help to run what you just built.

If you want to make your own CMake-based package for OCaml, work backwards from that explanation:

  1. use the command line first to build a CMake project. It will do the download, and run the cmake -G, cmake --build, cmake --install on your behalf. The dk0 error messages usually print with helpful recommendations, and look at the top of CommonsBase_Build.CMake0@3.25.3 for documentation.
  2. then make a Lua build file to package it for C. You’ll make a Lua function (“rule”) to set the command line arguments (what you tested in the last step). At minimum, you need your Lua rule to say how to get static vs shared libraries (you need both), and what CMake flags to use for what platforms.
  3. then make a JSON build file to package it for OCaml. Again, rely on the dk0 error messages.
  4. then make a GitHub Actions job to distribute it. Here is the GitHub Actions job for one of today’s announced packages: https://github.com/diskuv/dk/blob/7b19425969be61bed0f1d6bf3015dac26d7966e1/.github/workflows/distribute-2.5.yml#L282-L364. On success, GitHub Actions will print the import commands you can run so all the precompiled artifacts are available on your desktop.
1 Like