Libraries in an archive (.a file) are not available with Ctypes-foreign

Ecosystem
1

I try to access a function from an archive file (.a) with Ctypes-foreign but the linker doesn’t include it in the exe file because the function is not statically linked.

A tiny example of a program that fails to execute is here: Frédéric Loyer / ctype_error · GitLab

How shall I indicate that the function dynamically called by Ctypes-foreign must be included in the executable.

My issue appears on Linux and Windows (opam.2.2.0~beta1/MinGW).

2

If you want to make an object visible to the dynamic loader (so that dlsym that ctypes-foreign uses), you can link it with --export-dynamic. Try passing -Wl,--export-dynamic in your C flags.

3

I have tried compiling my C function with -Wl,--export-dynamic. This doesn’t help. Even adding them to my dune file. ccopt or cclib.

However, I have found:

Then, I have get it works with the following dune stanza:

 (link_flags -cclib -Wl,--whole-archive -cclib libtest.a -cclib -Wl,--no-whole-archive)

It is not very practical, and require the main program to be compiled like this (the link_flags is not supported in a library scope).

1 Like
4

I have found how pg_query proceeds:

(* Hack needed to make symbols available, see constfun's comment here
 * https://github.com/ocamllabs/ocaml-ctypes/issues/541 *)
external _force_link_ : unit -> unit = "pg_query_free_parse_result"

However, this works on Linux, not on Windows, even if the symbols are exported (I mean listed by nm).

A (link_flags -ccopt -Wl,--export-dynamic) stanza fails on Windows.

# x86_64-w64-mingw32-gcc: error: unrecognized command-line option ‘--export-dynamic’; did you mean ‘-export-dynamic’?

EDIT: I progress… objdump _path_of_pg_check gives:

/cygdrive/c/Users/frede/AppData/Local/opam/default/bin/pg_check.exe:     file format pei-x86-64

SYMBOL TABLE:
***plenty of symbols***

DYNAMIC SYMBOL TABLE:
no symbols

Then, I have to find how to export dynamic symbols.

1 Like
5

Finally, the problem is solved, but not in a satisfactory way.

I have to compile my main file (executable stanza) with the following dune stanza:

 (link_flags -cclib -Wl,-Wl,--export-all-symbols)

-cclib is to tell Ocamlopt to pass the following to the linker. Gcc is used as a linker, then -Wl, must be used to indicate something has to be passed to the linker. On opam2.2.0~beta1/Windows/MinGW, there is something which deals with flexlink and gcc is called once again… a second -Wl is required. Then the --export-all-symbols which is the way to export all symbols in the dynamic table. (--export-dynamic doesn’t work with PE executables, i.e. Windows EXE).

I guess that a Dune/Ocaml should have make things simpler. Typically a stanza which would avoid the need of -Wl,-Wl. Shouldn’t -cclib escape its argument and pass it really to the linker. (The documentation is only about -cclib -l... then the use without a -l is unspecified.) If one want to pass something to gcc, there is -ccopt.

Is there a way to make this stanza condionnal ? (Only in a MinGW environment)

1 Like
6

I have finally manage to make the link_flags optional with something like this:

(executable
 (name pg_check)
 (public_name pg_check)
 (link_flags (:include linkflags.sexp))
 (libraries ctypes ctypes.foreign cmdliner pg_query))

(rule
 (targets linkflags.sexp)
 (action (with-outputs-to linkflags.sexp
   (run sh -c "[ %{os_type} == Win32 ] && echo \"(-cclib -Wl,-Wl,--export-all-symbols)\" || echo \"()\""))))

I find this to be quite cumbersome. I would dream of something like this:

 (link_flags (:if (= %{os_type] Win32) (-cclib -Wl,-Wl,--export-all-symbols)))

Shouldn’t a conditional stanza be included in dune for my need… and surely many others ? Or is it already done in an other way ?

7

It’s a bit more verbose but you can split the rule into two enabled_if OS-specific rules.

8

I have tried:

(executable
 (name pg_check)
 (public_name pg_check)
 (link_flags -cclib -Wl,-Wl,--export-all-symbols)
 (enabled_if (= %{os_type} Win32))
 (libraries ctypes ctypes.foreign cmdliner pg_query))

(executable
 (name pg_check)
 (public_name pg_check)
 (enabled_if (<> %{os_type} Win32))
 (libraries ctypes ctypes.foreign cmdliner pg_query)

But I have the error

Error: Executable "pg_check" appears for the second time in this directory
9

Yes, sorry for not being specific, I meant splitting the rule that generates the flags file.
(ocaml-gccjit/lib/dune is an admittedly ugly example.)