How to properly define a library with C bindings?

Learning
, ,
1

I’m using ctypes and writing custom very small bindings to libsecp256k1, and I can get it to work, but only if I add (flags :standard -cclib -lsecp256k1) on my executable.

But that feels wrong, I think it should be able to work by just declaring these things on the library side. Here’s everything I have (minimal reproducible example):

2024-04-02-120134_681x835_scrot
2024-04-02-120134_681x835_scrot681×835 62.2 KB

The same code is here: https://github.com/andunieee/dune-ffi-repro

If you run it with dune exec ./bin/main.exe it will give this message:

Fatal error: exception Dl.DL_error("_build/default/bin/main.exe: undefined symbol: secp256k1_context_create")

Also calling dune build ./bin/main.exe and then ldd ./_build/default/bin/main.exe will give

linux-vdso.so.1 (0x00007ffdaa7f5000)
libffi.so.8 => /usr/lib/libffi.so.8 (0x00007b726ea12000)
libzstd.so.1 => /usr/lib/libzstd.so.1 (0x00007b726e72d000)
libm.so.6 => /usr/lib/libm.so.6 (0x00007b726e640000)
libc.so.6 => /usr/lib/libc.so.6 (0x00007b726e45e000)
/lib64/ld-linux-x86-64.so.2 => /usr/lib64/ld-linux-x86-64.so.2 (0x00007b726ea4b000)

Which means the reference to libsecp256k1 is not being acknowledged somewhere in the compilation or linking step (I don’t understand these things).

But if you edit bin/dune with this:

 (executable
  (name main)
+ (flags :standard -cclib -lsecp256k1)
  (libraries dune_ffi_repro))

then everything will work (well, you must have libsecp256k1 installed locally).

What am I doing wrong?

2

It should work if instead you put the flags stanza in your library instead of the c_library_flags part?

4

I have tried that too, doesn’t make any difference (just tried again now to check, same undefined symbol error).

But what is the purpose of c_library_flags anyway? Everywhere I read it says that’s what should be used.

5

Ah, having actually tried it, Dune is doing the right thing, but ld is not on your side! The ctypes-foreign library adds -Wl,--no-as-needed to the list of linker options, but in your example -lsecp256k1 appears before that. Ultimately, the linking command is:

gcc -O2 -fno-strict-aliasing -fwrapv -pthread -Wall -Wdeclaration-after-statement \
    -fno-common -fexcess-precision=standard -fno-tree-vrp -ffunction-sections \
    -Wl,-E -o bin/main.exe \
    -L/home/dra/.opam/dev-4.14/lib/integers \
    -L/home/dra/.opam/dev-4.14/lib/ctypes \
    -L/home/dra/.opam/dev-4.14/lib/ocaml \
    -L/home/dra/.opam/dev-4.14/lib/ocaml \
    -L/home/dra/.opam/dev-4.14/lib/ctypes-foreign \
    -L/home/dra/.opam/dev-4.14/lib/ocaml \
    /tmp/build_26587b_dune/camlstartupc0e6c4.o \
    /home/dra/.opam/dev-4.14/lib/ocaml/std_exit.o \
    bin/.main.eobjs/native/dune__exe__Main.o \
    lib/dune_ffi_repro.a \
    /home/dra/.opam/dev-4.14/lib/ctypes-foreign/ctypes_foreign.a \
    /home/dra/.opam/dev-4.14/lib/ocaml/threads/threads.a \
    /home/dra/.opam/dev-4.14/lib/ocaml/unix.a \
    /home/dra/.opam/dev-4.14/lib/ctypes/ctypes.a \
    /home/dra/.opam/dev-4.14/lib/bigarray-compat/bigarray_compat.a \
    /home/dra/.opam/dev-4.14/lib/integers/integers.a \
    /home/dra/.opam/dev-4.14/lib/ocaml/stdlib.a \
    -lsecp256k1 \
    -lctypes_foreign_stubs \
    -lffi \
    -Wl,--no-as-needed \
    -lthreadsnat \
    -lpthread \
    -lunix \
    -lctypes_stubs \
    -lintegers_stubs \
    /home/dra/.opam/dev-4.14/lib/ocaml/libasmrun.a \
    -lm

When you added the -cclib to flags in bin/dune, the -lsecp256k1 then also appears after the -Wl,--no-as-needed and so it works. This fixes lib/dune:

--- a/lib/dune
+++ b/lib/dune
@@ -1,4 +1,4 @@
 (library
  (name dune_ffi_repro)
- (c_library_flags -lsecp256k1)
+ (c_library_flags -Wl,--no-as-needed -lsecp256k1)
  (libraries ctypes ctypes-foreign))

although it’s not terribly satisfying…

2 Likes
6

Wow, this is amazing. I just tried and it worked!

But I don’t get it, is it just my ld that is behaving weirdly? In your case it worked without the -Wl,--no-as-needed part?

Mine is GNU ld (GNU Binutils) 2.41.0.

7

From pg_query sources:

(* Hack needed to make symbols available, see constfun's comment here
 * https://github.com/ocamllabs/ocaml-ctypes/issues/541 *)
external _force_link_ : unit -> unit = "pg_query_free_parse_result"

The idea is to force a static link, then the linker link the library. The declared symbol is a dummy one (fake type). Foreign is used to link dynamically the needed symbol with accurate types.