Dune problems using Dynlink plugins

Learning
1

Hi,

I’m trying to migrate an OCamlbuild project to Dune, but I’m having trouble building a plugin (for Dynlink).

The idea is that the plugin adds a GTK UI, using lablgtk. This must not be linked into the main program, because it uses a C library that might not be present on the target system.

I’ve put up a minimal repository showing the problem here:

https://github.com/talex5/dune-test

git clone https://github.com/talex5/dune-test.git
cd dune-test

By default, it fails with:

$ make
dune build @all
dune exec ./myprog.exe
Failed to load GTK GUI plugin: error loading shared library:
/home/user/tmp/dune_plugin/_build/default/plugin/plugin.cmxs:
undefined symbol: camlGtkMain__init_inner_102598

That seems to indicate that the lablgtk archive isn’t being linked with the plugin.
The library rule I’m using is:

(library
 (name        plugin)
 (optional)
 (libraries   lablgtk2 lwt_glib))

With ocamlfind, I would use -linkpkg to fix the error.
I’ve added a rule to build real_plugin.cmxs with an external call to ocamlfind:

(rule
  (targets real_plugin.cmxs)
  (deps plugin.cmxa)
  (action (run ocamlfind ocamlopt
                  -shared
                  -linkall
                  -linkpkg
                  -dontlink lwt.unix
                  -thread
                  -package lablgtk2,lwt_glib
                  %{deps}
                  -o %{targets}
  )))

That works (change myprog.ml to load real_plugin.cmxs instead of plugin.cmxs):

$ make
dune build @all
dune exec ./myprog.exe
Plugin init..."LC_CTYPE=en_GB.UTF-8;LC_NUMERIC=C;LC_TIME=en_GB.UTF-8;LC_COLLATE=en_GB.UTF-8;LC_MONETARY=en_GB.UTF-8;LC_MESSAGES=en_GB.UTF-8;LC_PAPER=en_GB.UTF-8;LC_NAME=en_GB.UTF-8;LC_ADDRESS=en_GB.UTF-8;LC_TELEPHONE=en_GB.UTF-8;LC_MEASUREMENT=en_GB.UTF-8;LC_IDENTIFICATION=en_GB.UTF-8"
Plugin loaded OK

However, it’s a bit ugly. Also, adding real_plugin.cmxs to an install stanza forces Dune to try to build the plugin even if lablgtk isn’t available, ignoring the (optional) in the library definition.

I also tried using ocamlfind to get the archives:

(library
 (name        plugin)
 (optional)
 (ocamlopt_flags :standard (:include gtk_archives.sexp))
 (libraries   lablgtk2 lwt_glib))

(rule
  (targets gtk_archives.sexp)
  (action
    (with-stdout-to %{targets}
      (run ocamlfind query -r lablgtk2,lwt_glib -format "\"%+a\"" -predicates native -prefix "(" -suffix ")"
    ))
  ))

However, that fails with:

$ make
dune build @all
    ocamlopt plugin/plugin.{a,cmxa} (exit 2)
(cd _build/default && /home/user/.opam/4.07.0/bin/ocamlopt.opt -w @a-4-29-40-41-42-44-45-48-58-59-60-40 -strict-sequence -strict-formats -short-paths -keep-locs -g /home/user/.opam/4.07.0/lib/lablgtk2/lablgtk.cmxa /home/user/.opam/4.07.0/lib/result/result.cmxa /home/user/.opam/4.07.0/lib/lwt/lwt.cmxa /home/user/.opam/4.07.0/lib/ocaml/unix.cmxa /home/user/.opam/4.07.0/lib/ocaml/bigarray.cmxa /home/user/.opam/4.07.0/lib/lwt/unix/lwt_unix.cmxa /home/user/.opam/4.07.0/lib/lwt_glib/lwt_glib.cmxa -a -o plugin/plugin.cmxa plugin/.plugin.objs/plugin.cmx)
Option -a cannot be used with .cmxa input files.

What’s the correct way to build a plugin with Dune?

3 Likes
2

I believe that usually people use a plugin loader that supports dependencies between plugings such as findlib.dynload. It is then the responsibility of this loader to load the dependencies of plugins.cmxs. In particular, this allows several plugings to use the same dependency.

IIUC, you are trying to link all the dependency of your plugin statically in one cmxs. This is generally not recommended as the same library might end up being linked twice at runtime: either because it is present in both the main executable and the plugin, or because it is present in several plugins.

I would suggest to switch to something like findlib.dynload. Thanks to @bobot dune supports findlib.dynload out of the box, i.e. it makes sure that libraries that are statically linked in the main executable won’t be dynlinked at runtime.

3

Thanks. The target system typically won’t have findlib, so I doubt that will work (this is for 0install). It would certainly be nice if dune could link all the libraries needed by the plugin that aren’t already linked by the main executable (then I wouldn’t need -linkall and could shrink the binaries a bit). But before that, I’d be happy just to get back to where I was with ocamlbuild…

(note that this plugin is shipped alongside the main binary; I don’t need to support third-party plugins in this case)

4

Ok. Something would need to be added to dune to support this properly. If you are happy to spend a bit of time on making dune support this feature, I’m happy to point you in the right directory. Otherwise, I suggest to continue using the workaround you posted, in the end it’s not that bad. The only drawback is that it doesn’t work well if the dependencies are vendored, although this part should be easy to fix.

1 Like
5

For future reference, a similar ticket was just opened: https://github.com/ocaml/dune/issues/1544

2 Likes
6

I’ve got it working now. Here’s the final dune file I ended up with for the plugin:

(* -*- tuareg -*- *)

let have_gtk =
  try
    ignore @@ Jbuild_plugin.V1.run_and_read_lines "ocamlfind query lablgtk2";
    true
  with _ ->
    print_endline "(will skip building the GTK plugin)";
    false

let plugin_ext =
  match List.assoc "natdynlink_supported" Jbuild_plugin.V1.ocamlc_config with
  | "true" -> "cmxs"
  | _ -> "cma"
  | exception Not_found -> "cma"

let dune = {|
  (library
   (name        gui_gtk_lib)
   (libraries   lablgtk2 lwt_glib zeroinstall))

  (rule
    (targets gui_gtk.cmxs)
    (deps gui_gtk_lib.cmxa)
    (action (run ocamlfind ocamlopt -shared -linkall -linkpkg
			  -dontlink lwt,lwt.unix
			  -thread
			  -package lablgtk2,lwt_glib
			  %{deps}
			  -o %{targets}
    )))

  (rule
    (targets gui_gtk.cma)
    (deps gui_gtk_lib.cma)
    (action (run ocamlfind ocamlc -linkall -linkpkg
			  -dontlink lwt,lwt.unix
			  -thread -a
			  -package lablgtk2,lwt_glib
			  %{deps}
			  -o %{targets}
    )))

  (install
    (section lib)
|} ^ Printf.sprintf "(files gui_gtk.%s))" plugin_ext

let () =
  Jbuild_plugin.V1.send (if have_gtk then dune else "")

(from https://github.com/0install/0install/blob/master/ocaml/gui_gtk/dune)

Some things I’m not quite happy with:

  • I use natdynlink_supported to decide whether to build a native or bytecode plugin. I guess I really want to use whatever the main executable is using.

  • I had to use the tuareg hack to skip the (install) section if the plugin isn’t being built. Would be nice if you could do (install (only-if %{lib-available:lablgtk2}) ...) or something like that.

  • There’s some duplication for handing both cma and cmxs, although the invocations aren’t quite the same.

Apart from that it worked quite well. For the unit-tests, I just added a dependency on (alias ../../install), so I didn’t need any conditionals there in the end.

The other (unrelated) source of ugliness was special cases for building on Windows (see https://github.com/0install/0install/blob/master/ocaml/support/dune).

If anyone spots a way to make these dune files cleaner, let me know!

2 Likes
7

Why ocamlfind can’t be installed in 0install? How 0install handles ocaml libraries and shared C libraries?

8

@nojb has opened a PR on dune that supports this very nicely now:

9

I found this issue while hitting a related but different, simpler -linkpkg-looking problem with Dune: getting a C library (that is part of the current project) linked into an executable that uses it (from C stubs, obviously). I ended up opening an upstream issue: dune #4409: confused by Dune foreign_libraries doc: how to define a C library used by C stubs?

1 Like