A tutorial on parallel programming in OCaml 5

I’m unable to recreate this error on a fresh installation of ocaml 5.0.0~rc1 on an M2 MacBook Pro. Let me see if I can recreate this error on a different platform. Also, #thread is not relevant to this tutorial.

I used a fresh OCaml 5.0.0~rc1 switch on aarch64 Linux fedora and was able to get the error…

$ mkdir abcxyz
$ cd abcxyz
$ opam switch create 5.0.0~rc1 .
$ opam install domainslib down
$ ocaml
# #require "domainslib" ;;
Error: Reference to undefined global `Virtual_atomic'

On x86-64 ubuntu, I recreated the problem following what @sid wrote above.

I used a fresh installation of ocaml 5.0.0~rc1 on an M2 MacBook Air.

My output after #require “domainslib”:

/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic/containers_monomorphic.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/either: added to search path
/home/tom/.opam/5.0.0~rc1/lib/either/either.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/seq: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers/containers.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/oseq: added to search path
/home/tom/.opam/5.0.0~rc1/lib/oseq/oseq.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/dscheck: added to search path
/home/tom/.opam/5.0.0~rc1/lib/dscheck/dscheck.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/lockfree/virtual_atomic: added to search path
/home/tom/.opam/5.0.0~rc1/lib/lockfree: added to search path
/home/tom/.opam/5.0.0~rc1/lib/lockfree/lockfree.cma: loaded
Line 1:
Error: Reference to undefined global `Virtual_atomic'

One thing is that virtual_atomic is added to the search path but no “.cma” is loaded. Not sure if this is relevant.

I guess this is something to do with virtual libraries, and the ocaml toplevel not knowing what implementation to load for virtual_atomic. I know very little about this unfortunately.

By pointing the toplevel to the atomic_stdlib implementation, and loading it explicitly, I was able to “require domainslib”:

# #directory "/home/tom/.opam/5.0.0~rc1/lib/lockfree/atomic_stdlib";;
# #load "atomic_stdlib.cma";;
# #require "domainslib";;
/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic/containers_monomorphic.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/either: added to search path
/home/tom/.opam/5.0.0~rc1/lib/either/either.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/seq: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers: added to search path
/home/tom/.opam/5.0.0~rc1/lib/containers/containers.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/oseq: added to search path
/home/tom/.opam/5.0.0~rc1/lib/oseq/oseq.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/dscheck: added to search path
/home/tom/.opam/5.0.0~rc1/lib/dscheck/dscheck.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/lockfree/virtual_atomic: added to search path
/home/tom/.opam/5.0.0~rc1/lib/lockfree: added to search path
/home/tom/.opam/5.0.0~rc1/lib/lockfree/lockfree.cma: loaded
/home/tom/.opam/5.0.0~rc1/lib/domainslib: added to search path
/home/tom/.opam/5.0.0~rc1/lib/domainslib/domainslib.cma: loaded

So this seems to confirm that the toplevel does not know what implementation to use for virtual_atomic. I don’t know what the correct fix is - perhaps lockfree should specify a default implementation?

Could it be that the problem is in ocamlfind?
I could build the projects with dune, but if i try to directly use ocamlfind it fails:

% ocamlfind ocamlc -o pmsort -linkpkg -package domainslib pmsort.ml
File “pmsort.ml”, line 1:
Error: Module Virtual_atomic' is unavailable (required by Lockfree__Ws_deque’)

Erm, I’m not sure. I guess dune has some cleverness around virtual modules. Reading the docs Toplevel Integration — Dune documentation perhaps the correct approach is to type the following in the ocaml toplevel:

#use_output "dune ocaml top";;
#require "domainslib";;

This works in my switch (whereas the plain “require domainslib” fails). You can inspect the output of “dune ocaml top” by just typing it in on the command line:

 $ dune ocaml top
#directory "/home/tom/.opam/5.0.0~rc1/lib/containers";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/domainslib";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/dscheck";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/either";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/lockfree";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/lockfree/atomic_stdlib";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/oseq";;
#directory "/home/tom/.opam/5.0.0~rc1/lib/seq";;
#directory "/tmp/l/github/other/ocaml5-tutorial/_build/default/lib/.ocaml5_tutorial.objs/byte";;
#load "/tmp/l/github/other/ocaml5-tutorial/_build/default/lib/ocaml5_tutorial.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/either/either.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/containers/monomorphic/containers_monomorphic.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/containers/containers.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/oseq/oseq.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/dscheck/dscheck.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/lockfree/atomic_stdlib/atomic_stdlib.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/lockfree/lockfree.cma";;
#load "/home/tom/.opam/5.0.0~rc1/lib/domainslib/domainslib.cma";;

I think that with #use_output “dune ocaml top”
#require “domainslib” would no longer be necessary.

I spoke to @rgrinberg and they think it is likely an issue with dune. I will file a github issue and link here. There are some workarounds above.

Could it not be that findlib does not know the dependency between domainslib and lockfree.atomic_stdlib?

I am not sure. The lockfree library seems to correctly specify the default implementation. And the default implementation should be loaded automatically when a user opens the toplevel and types “#require …”. I don’t know where exactly the problem arises. I have filed the following issue against dune: OCaml toplevel sometimes fails to load default implementation for a virtual library · Issue #6680 · ocaml/dune · GitHub

I made an experiment:

I changed the META file in .opam/5.0.0~rc1/lib/lockfree, replacing the 3rd line:
requires = “dscheck lockfree.virtual_atomic”
by
requires = “dscheck lockfree.atomic_stdlib”

Then #require “domainslib” seems to work properly in the toplevel, as it does compiling with ocamlfind (-package domainslib)
But i am not aware of any possible side effects. (?)

By the way, compiling with
ocamlfind ocamlopt -c -package domainslib test.ml
throws the warning:
findlib: [WARNING] Interface virtual_atomic.cmi occurs in several directories: /Users/molineli/.opam/5.0.0~rc1/lib/lockfree/atomic_stdlib, /Users/molineli/.opam/5.0.0~rc1/lib/lockfree/virtual_atomic

Does this shed any light?

I think it is solved. Final version 5.0.0 does not seem to show this behavior.

1 Like