Ocamlfind can't find threads.cmxa after adding `-package threads`

I am trying to compile a single file, minimized below, with ocamlfind ocamlopt. It succeeds as expected with -I +threads threads.cmxa, but fails when I add -package threads or a package which depends on threads (in my case -package coq-core.kernel). What am I doing wrong?

let x = Thread.self ()
$ ocamlfind ocamlopt -I +unix -I +threads -linkpkg unix.cmxa threads.cmxa a.ml
# OK

$ ocamlfind ocamlopt -I +unix -I +threads -package threads -linkpkg unix.cmxa threads.cmxa a.ml
File "a.ml", line 1:
Error: Cannot find file /home/sam/.opam/myswitch/lib64/threads.cmxa
ocamlfind 1.9.6
ocaml 5.1.1

I think you need to remove unix.cmxa and threads.cmxa from the command-line; -linkpkg will take care of linking those archives into your final artifact.

Cheers,
Nicolas

If I remove those, both commands fail.

$ ocamlfind ocamlopt -I +unix -I +threads -linkpkg a.ml         
File "a.ml", line 1:
Error: No implementations provided for the following modules:
         Thread referenced from a.cmx

$ ocamlfind ocamlopt -I +unix -I +threads -package threads -linkpkg a.ml                       
File "a.ml", line 1:
Error: Cannot find file /home/sam/.opam/myswitch/lib64/threads.cmxa

unix.cmxa and threads.cmxa are prescribed by the OCaml manual: OCaml - The threads library

If you are using ocamlfind then you don’t normally need to pass -I flags nor .cmxa archives by hand in the command-line, all of these is taken care of by the -package and -linkpkg flags. The OCaml manual does not use ocamlfind which is why they pass -I flags and .cmxa archives on the command-line.

The first failure is expected because you are not passing -package flags. The second failure is less clear. Maybe your installation is not correctly configured? What is the output of ocamlfind printconf?

Cheers,
Nicolas

I see. Then it’s extra strange because it’s fine linking with other packages.

$ ocamlfind printconf
Effective configuration:
Configuration file:
    /home/sam/.opam/myswitch/lib/findlib.conf
Search path:
    /home/sam/.opam/myswitch/lib64
    /home/sam/.opam/myswitch/lib
Packages will be installed in/removed from:
    /home/sam/.opam/myswitch/lib
META files will be installed in/removed from:
    the corresponding package directories
The standard library is assumed to reside in:
    /home/sam/.opam/myswitch/lib64
The ld.conf file can be found here:
    /home/sam/.opam/myswitch/lib64/ld.conf

The ocamlfind option -thread seems to fix this. It’s confusing because when I first saw it, with ocamlfind ocamlopt --help, it only showed ocamlopt’s -thread which says it’s deprecated:

-thread  (deprecated) same as -I +threads

and I then found ocamlfind’s actual -thread flag documented on man ocamlfind. Or am I still misunderstanding it?

-thread

This standard option causes that the predicate “mt” is added to the set of actual predicates. If POSIX threads are available, the predicate “mt_posix” is selected, too. If only VM threads are available, the predicate “mt_vm” is included into the set, and the compiler switch is changed into -vmthread.

Your ocamlfind printconf looks a bit strange normally these lib64 should be the directory ocaml.

The -thread option should also no longer be needed in ocamlfind in OCaml >= 5.0.0 since the libraries are no longer predicated by mt and mt_posix.

Could you output the result of cat $(ocamlfind query -format "%m" threads). If all is well (but this lib64 is suspicious) you should see something like:

# otherlibs/systhreads/META.  Generated from META.in by configure.

version = "5.1.0"
description = "Multi-threading"
requires = "unix"
archive(byte) = "threads.cma"
archive(native) = "threads.cmxa"
type_of_threads = "posix"

package "posix" (
  requires = "threads"
  version = "[internal]"
)

I’m getting the same thing.

# otherlibs/systhreads/META.  Generated from META.in by configure.

version = "5.1.1"
description = "Multi-threading"
requires = "unix"
archive(byte) = "threads.cma"
archive(native) = "threads.cmxa"
type_of_threads = "posix"

package "posix" (
  requires = "threads"
  version = "[internal]"
)

Well in fact I can perfectly reproduce your problem.

Eco-system tools always needs special casing around upstream provided libraries because they can’t be bothered.

What you see here is likely the result of some form of special casing in ocamlfind hitting back (a git grep on threads in ocamlfind’s sources will show the many potential offending locations).

The following command should actually compile the source you provided but it does not.

  1. It’s missing the -I to the ocaml/threads directory
  2. It locates the library in the ocaml libdir instead of ocaml/threads.

All this despite the META file being well aligned on ocamlfind’s convention (i.e. present in ocaml/threads and as provided above).

> ocamlfind ocamlopt -only-show -package threads -linkpkg a.ml
ocamlopt.opt -I /Users/dbuenzli/.opam/5.1.0/lib/ocaml/unix \
 /Users/dbuenzli/.opam/5.1.0/lib/ocaml/unix/unix.cmxa \
 /Users/dbuenzli/.opam/5.1.0/lib/ocaml/threads.cmxa \
 a.ml

Just to nail that down a bit further, it must be somewhere in the tool wrapping logic as ocamlfind query (which I use in my builds) reports the right bits, e.g.:

ocamlfind query threads -predicates 'native' '-format' '%+A' 
/Users/dbuenzli/.opam/5.1.0/lib/ocaml/threads/threads.cmxa

I suspect this should be adapted to a new reality in OCaml >= 5.0.0. and the “MAGIC” here looks wrong.

That, in addition to stopping the passing of -thread to ocamlopt at all, is in Disable various parts of -thread handling for 5.x by dra27 · Pull Request #76 · ocaml/ocamlfind · GitHub, with the compliments of an un-bothered upstream maintainer.

3 Likes