`pthread` is missing on OS X

Learning
,
1

Hi everybody! :wave:

Can anybody help me to figure out why this can happen on OS X (arm64)?

I have a very simple code

let hello () = Lwt_main.run (Lwt_io.printl "Hello, world!")

which compiles, but doesn’t link

% ocamlfind opt -output-complete-obj -thread -package lwt.unix -linkpkg -o lib.o lib.ml
ld: library not found for -lpthread
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

I can read man 3 pthread, but maybe I miss something on my system :man_shrugging:

2

I get the same with linux with x86_64, so it is not to do with OS X specifically. Nor is it anything specific to lwt.unix: I seem to get the error with any attempt to combine the --output-complete-obj flag with ocamlfind’s -linkpkg. I also seem to get the same problem if I use dune instead of ocamlfind and include a line (modes native object) in the relevant executable/library stanza.

So it seems to be a problem with the implementation of the compiler’s --output-complete-obj flag. From a very limited amount of experimenting it appears I can work around the problem on a toy example by using the --output-obj flag instead and linking libunix.a, libthreadsnat.a, libasmrun.a and liblwt_unix_stubs.a into the final executable by hand. You could try giving that a go - I haven’t tested it further to say more than that.

3

Thank you @cvine for experimenting! What you’ve proposed works fine for me. I just put a snippet for history.

LIB=`ocamlc -where`

libtool -static -o libexport.a \
	${LIB}/libasmrun.a \
	${LIB}/libunix.a \
	${LIB}/libthreadsnat.a \
	${LIB}/../lwt/unix/liblwt_unix_stubs.a \
	lib.o
4

Two things: first you can find the location of liblwt_unix_stubs.a with `ocamlfind query lwt.unix`, which might simplify your Makefile.

Secondly further experimentation shows that using the -output-complete-obj compiler flag with ocamlfind’s -thread and -linkpkg options will fail at link time where the use of the -thread option induces ocamlfind (through the -linkpkg option) to attempt to link to threads.cmxa. In consequence, this seems to prevent any ocaml code compiled with the -output-complete-obj flag from using the Thread, Mutex or Cond modules, which may not be too awful. However more seriously, it prevents any library which requires such linking from being used, which stops use of lwt.unix and no doubt some others, which is a nuisance. The same applies to use of dune’s (modes native object) field, which fails with lwt.unix in the same way.

It appears to be a bug with the -output-complete-obj implementation. However I am not clear about the circumstances in which ocamlfind’s -thread option is still required. If you omit that option when linking to lwt.unix it will link, but with a warning "Linking problems may arise because of the missing -thread or -vmthread switch". Clearly dune also thinks that such linking is necessary.

5

I am running into the same issue. Is this a bug in ocamlfind or ocaml? Do you know if a bug report has been filed?

6

It seems that in the failing case ocamlopt calls ld with -lpthread, whereas by default (when linking an executable or shared object) it calls gcc with -lpthread and -pthread.
In fact it looks like the problem seems to come from the use of ld -r vs regular ld, perhaps pthread is handled specially.

-output-complete-obj seems like a fairly specific use case though (I never needed to use it), are you using this in combination with some C libraries where the main program is C?
Would it be better to produce a .so file that can be linked with the needed files, and then link the .so in your C program? (might be better than using output-complete-obj which relies on the incremental mode of the linker, which is quite obscure, even for C programs)

7

I found the bug report (closed due to inactivity): ld: warning: /usr/lib/libpthread.dylib, ignoring unexpected dylib file · Issue #7933 · ocaml/ocaml · GitHub

The issue is relevant to me in the context of an iOS project using opam-cross-ios where the main program is in C. I am building the OCaml lib and C stubs with dune with (modes (native object)) which uses -output-complete-obj. iOS does not support dynamic linking for user libraries, so an .so is not an option.

8

@cvine Do you have a working dune file for your workaround that you can share? When I add (ocamlopt_flags (-output-obj)) to my dune library stanza, I get an error:

Please specify at most one of -pack, -a, -shared, -c, -output-obj
9

No, I couldn’t get dune to work with ocaml’s -output-obj flag but I didn’t spend much time trying. Since the C part of the program used Makefiles (generated by autoconf) I just used a Makefile for the ocaml code also, which called up ocamlopt via ocamlfind to generate a library archive. The linking in of libunix.a, libthreadsnat.a, libasmrun.a and liblwt_unix_stubs.a occured when finally linking up the C executable.

If you want to use dune, the only way I could get it to work (with my not particularly extensive testing) was with a field (modes native shared_object), so generating a dynamically loadable library. The disadvantage of this is of course that you have to distribute the library with your executable.

10

For the record, this works for me without error:

(executable
 (name lib)
 (libraries threads)
 (flags (:standard -noautolink -cclib -lunix -cclib -lthreadsnat))
 (foreign_stubs (language c) (names libwrap))
 (modes (native object)))
2 Likes