Dune, ocamldebug, foreign_stubs

Learning
, ,
1

I stumbled upon a bug in a compiler written in ocaml, namely the haxe compiler.
I figured I could try to fix it, or at least use ocamldebug to see how I could work around it.

I compiled from source and found an annotation suggesting to uncomment (modes byte) to be able to use ocamldebug.

I did just that, but ran into the following error:

Loading program… Fatal error: cannot load shared library dllextc_stubs
Reason: dllextc_stubs.so: cannot open shared object file: No such file or directory

Which I managed to address by running :

CAML_LD_LIBRARY_PATH=path/to/file1.so:path/to/file2.so:$CAML_LD_LIBRARY_PATH ocamldebug ...

for all the .so files built by dune in the project.

I am now facing the following:

Fatal error: cannot load shared library dllextc_stubs
Reason: /home/qlambert/Hacking/haxe/_build/default/libs/extc/dllextc_stubs.so: undefined symbol: inflate

which I tracked down to a symbol created by another library in the project, namely swflib.
dune does create a .cma for this library so I would have expected it to be included in the executable it produces.

On the other hand I found the following in dune’s documentation

Lastly, note that .bc executables cannot contain C stubs. If your executable contains C stubs you may want to use (modes exe).

So now, I am wondering if what I am trying to do is at all possible. Can someone give me pointers on how to run a program containing C stubs with ocamldebug, if it is possible?

2

run ocamldebug in the dune context (dune exec -- ocamldebug ...)?

3

This has the same behaviour as a regular execution of ocamldebug.
I get “No such file or directory” and “undefined symbol: inflate” if I properly set CAML_LD_LIBRARY_PATH.

4

The paragraph you quoted is just saying that pure bytecode executables cannot be statically linked with C code. But there is no obstacle to linking dynamically with native code. Linking dynamically requires the runtime and dynamic linker to be able to find the needed libraries though, which is what you are having trouble with, from the look of it.

From a distance, it sounds like the dynamic linker cannot find a native shared library on which the stubs depend depend. If you are able to find the shared library in question you can try adding its path to LD_LIBRARY_PATH and see if it works.

As mentioned, pure bytecode cannot be linked statically with shared libraries, and dynamic linking must be used instead. Dune does not know much about native libraries. Often all that is recorded in the .cma archive is the linking flag for the native library -lfoo and the runtime depends on the load path for the dynamic linker to be set up correctly so that the flag can be resolved at runtime.

Unfortunately, dynamic linking has a fair share of tricky corners and it often takes some twiddling to get it working…

Cheers,
Nicolas

5

We have 1 stub, we invoke ocamldebug through a script which declares deps for dune on a bunch of .cmas and on the stub rocq/dev/dune at master · rocq-prover/rocq · GitHub
(ie we do dune exec -- dev/dune-dbg ...)
So I guess you need the cmas and stubs in the install layout, and then dune exec may work.

6

@nojb’s hints are correct:

The following fixes your symbols:


  diff --git a/libs/extc/dune b/libs/extc/dune
  @@ -6,6 +6,7 @@
        (foreign_stubs
                (language c)
                (names extc_stubs))
  +     (c_library_flags -lz)
        (modules extc)
        (wrapped false)
   )

  diff --git a/libs/mbedtls/dune b/libs/mbedtls/dune
  @@ -5,5 +5,6 @@
        (foreign_stubs
                (language c)
                (names mbedtls_stubs))
  +     (c_library_flags -lmbedtls -lmbedx509 -lmbedcrypto)
        (wrapped false)
   )

  diff --git a/libs/pcre2/dune b/libs/pcre2/dune
  @@ -5,5 +5,6 @@
        (foreign_stubs
                (language c)
                (names pcre2_stubs))
  +     (c_library_flags -lpcre2-8)
        (wrapped false)
   )

src/dune claims it is using the threads library and ocamldebug will outright refuse to work if it encounters any threads, but in my own testing this didn’t seem to occur very often.

After the patch:

$ ldd _build/default/libs/extc/dllextc_stubs.so
        linux-vdso.so.1 (0x00007ffff7fc2000)
        libz.so.1 => /nix/store/ixhlv41i2wpl84xgjcks061dz4yssbg3-zlib-1.3.2/lib/libz.so.1 (0x00007ffff7f95000)
        libc.so.6 => /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib/libc.so.6 (0x00007ffff7c00000)
        /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc4000)

$ ldd _build/default/libs/mbedtls/dllmbedtls_stubs.so
        linux-vdso.so.1 (0x00007ffff7fc2000)
        libmbedtls.so.21 => /nix/store/wkjkvkjiq3xccimja7a1nn2p78hjyav3-mbedtls-3.6.5/lib/libmbedtls.so.21 (0x00007ffff7f61000)
        libmbedx509.so.7 => /nix/store/wkjkvkjiq3xccimja7a1nn2p78hjyav3-mbedtls-3.6.5/lib/libmbedx509.so.7 (0x00007ffff7f4b000)
        libmbedcrypto.so.16 => /nix/store/wkjkvkjiq3xccimja7a1nn2p78hjyav3-mbedtls-3.6.5/lib/libmbedcrypto.so.16 (0x00007ffff7ead000)
        libc.so.6 => /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib/libc.so.6 (0x00007ffff7c00000)
        /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc4000)

$ ldd _build/default/libs/pcre2/dllpcre2_stubs.so
        linux-vdso.so.1 (0x00007ffff7fc2000)
        libpcre2-8.so.0 => /nix/store/khhzkpj9169ydnyg7jjrjx7s5ygdkcas-pcre2-10.46/lib/libpcre2-8.so.0 (0x00007ffff7f04000)
        libc.so.6 => /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib/libc.so.6 (0x00007ffff7c00000)
        /nix/store/fjkx1l5cnskzrqacf08z7i8z17256w0j-glibc-2.42-61/lib64/ld-linux-x86-64.so.2 (0x00007ffff7fc4000)

Hope that helps !

7

Sorry I forgot to include something important in my earlier reply and that’s how I actually invoked ocamldebug. Unfortunately dune doesn’t know anything about ocamldebug at the moment, and it could be doing a lot to help potentially. See the tracking issue: ocamldebug support · Issue #4229 · ocaml/dune · GitHub

In the spirit of ignoring that reality, the following seems to work for me:

$ export CAML_LD_LIBRARY_PATH=$(find _build -name "dll*_stubs.so" -exec dirname {} \; | sort -u | tr '\n' :)
$ export LD_PRELOAD=$(pkg-config --variable=libdir libuv)/libuv.so                                                                      
$ ocamldebug -I _build/default/src/.haxe.eobjs/byte _build/default/src/haxe.bc                                                          
 
        OCaml Debugger version 5.4.1

(ocd) set arguments --version
(ocd) break @ Dune__exe__Haxe 82
Loading program... done.
Breakpoint 1 at 3:6063788: file src/compiler/haxe.ml, line 82, characters 20-44
(ocd) run
Time: 493256 - pc: 3:6063788 - module Dune__exe__Haxe
Breakpoint: 1
82 let args = List.tl (Array.to_list Sys.argv)<|a|> in
(ocd) backtrace
Backtrace:
#0 Dune__exe__Haxe src/compiler/haxe.ml:82:44
(ocd) quit
Removed breakpoint 1 at 3:6063788: file src/compiler/haxe.ml, line 82, characters 20-44

You will have to fight the name mangling/module aliasing that dune is doing with the modules, but it should be workable.

8

Thank you it seems to work!

Can you explain how you found which flags to add after c_library_flags for the different libraries?
What about LD_PRELOAD, I was looking at libuv between your two messages, why is it treated so differently that the other shared object. That is, why can’t I preload all the other libraries, or why doesn’t ocaml_luv’s c_library_flags suffice?

9

These were taken straight from line 450 onwards of src/prebuild.ml which Haxe was already providing when getting Dune to link the executable. The patch just moves them into the library that owns the stubs. The includes in the stubs and ldd were further hints. Dune will propogate c_library_flags to ocamlmklib -ldopt when creating dynamic archives. On the dune side we could probably do a much better job at hiding this complexity.

This was something I had to hack in due to how I set up my packages. I didn’t have too much time to get to the bottom of it:

Without it we have

Loading program... Fatal error: cannot load shared library dllluv_c_stubs
Reason: _build/_private/default/.pkg/luv.0.5.14-ac6277019adfb8eacb4a4de6a9768bb1/target/lib/stublibs/dllluv_c_stubs.so: undefined symbol
: uv_wtf8_to_utf16

so something is going wrong with the dynamic archive that ocaml_luv is producing.

$ nm _build/_private/default/.pkg/luv.0.5.14-ac6277019adfb8eacb4a4de6a9768bb1/target/lib/stublibs/dlluv.so | g
rep uv_wtf8_to_utf16
000000000000b430 t uv_wtf8_to_utf16

I would expect this to be a T rather than a t. However this is all on me because I’m using nix to do depexts here and dune pkg on top of that, so I’m not certain if I’m causing the problem. I suspect you are using opam, and if it works without this hack then you can simply ignore it. If on ther other hand you also run into this issue, it might be worth poking around in ocaml_luv’s vendored luv library and how its being built.