Dynlink with static binary

hi all. I want to use dynlink with a statically linked binary.

my dune file looks like

(env  (static (flags (:standard -ccopt -static )))) 

(executable  (name main)  
(libraries dynlink)  
;(link_flags -linkall) I can add this, but the problem is the same
 (modules main)  ) 

(executable   (name dyn)   
(modules dyn)  
 (link_flags -linkall) 
  (modes (native plugin)) 
  (embed_in_plugin_libraries stdlib) )

and this is the error I get :

dune build main.exe --profile static ; \
	dune build dyn.cmxs ; \
	cd _build/default ; \
	./main.exe ; \
	ldd main.exe || true ; \
	ocamlobjinfo dyn.cmxs ; \
	cd -
    ocamlopt main.exe
/usr/bin/ld: /home/abate/Projects/repos/tezos/tezos_master/_opam/lib/ocaml/libasmrun.a(unix.n.o): in function `caml_dlopen':
/home/abate/Projects/repos/tezos/tezos_master/_opam/.opam-switch/build/ocaml-base-compiler.4.12.0/runtime/unix.c:271: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
error loading shared library: Dynlink.Error (Dynlink.Cannot_open_dll "Failure(\"/home/abate/Projects/repos/tezos/test-dynlink/_build/default/dyn.cmxs: undefined symbol: camlStdlib__format__printf_1230\")")
	not a dynamic executable
File dyn.cmxs
Name: Dune__exe__Dyn
CRC of implementation: 3f141250177d7b60b987e7f5d99446f3
Globals defined:
	Dune__exe__Dyn
Interfaces imported:
	82dadcd908dde621b8842c96ba644dc6	Stdlib__uchar
	185075aa99b059ad4cae15cb411fcc0c	Stdlib__seq
	e8696010fc66d4ce9a3bc3f311f6ea20	Stdlib__format
	9eaf2b1cd70e33601c152d1ce1bb8be9	Stdlib__buffer
	4b04b4eda19aa722df365141895fb347	Stdlib
	9fc7c355dbbc8f9dae052e4c2ee0e13d	Dune__exe__Dyn
	b6c6694955e10001aed267571104a961	CamlinternalFormatBasics
Implementations imported:
	41296ac99a85560c95c2f9d56ed27a7d	Stdlib__format

there are two problems.

  • first the warning. Does this mean if I compile the statically linked library on alpine (musl ) i cannot use any dyn linked modules compiled on glibc : this is just a confirmation, then answer seems clear.

  • second: how do I solve this linking problem with format ?

this is a small repo to explain the problem : Files · master · pietro / test-dynlink · GitLab

make dyn compiles the dyn linked library ( no errors )
make compiles the statically linked library (errors above)

What Am I doing wrong ?

many thanks,
pietro

I’m not sure I follow, but off the bat it seems strange to try to embed the stdlib into the plugin. I would try removing (embed_in_plugin_libraries ...) and (link_flags -linkall) in the rule for the plugin and add back (link_flags -linkall) to the rule for main.exe.

Cheers,
Nicolas

I tried with this dune file as suggested ( that was also my first try ):

(env
 (static (flags (:standard -ccopt -static ))))

(executable
 (name main)
 (libraries dynlink)
 (link_flags -linkall)
 (modules main)
 )

(executable
  (name dyn)
  (modules dyn)
  ;(link_flags -linkall)
  (modes (native plugin))
  ;(embed_in_plugin_libraries stdlib)
)

but get exactly the same problem.

I think the env stanza will also apply to dyn, so it might create an unusable cmxs file (not sure about the details).

  • do you see -static on the command line of the command that builds dyn.cmxs? (in _build/log)
  • What happens if you set (flags) on just main?

I don’t think the static profile is used in the dynmodules
I get the same problem if I specify explicitely two different profiles
dune build main.exe --profile static and `dune build dyn.cmxs --profile dev" .

Here in dev is the default profile.

this is the log:

- _build/default/dyn.cmxs
Running[2]: (cd _build/default && /home/abate/_opam/bin/ocamlc.opt -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -bin-annot -I .dyn.eobjs/byte -no-alias-deps -opaque -o .dyn.eobjs/byte/dune__exe__Dyn.cmo -c -impl dyn.ml)
Running[3]: (cd _build/default && /home/abate/_opam/bin/ocamlopt.opt -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -I .dyn.eobjs/byte -I .dyn.eobjs/native -intf-suffix .ml -no-alias-deps -opaque -o .dyn.eobjs/native/dune__exe__Dyn.cmx -c -impl dyn.ml)
Running[4]: (cd _build/default && /home/abate/_opam/bin/ocamlopt.opt -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -o dyn.cmxs -shared .dyn.eobjs/native/dune__exe__Dyn.cmx)
error loading shared library: Dynlink.Error (Dynlink.Cannot_open_dll "Failure(\"/home/test-dynlink/_build/default/dyn.cmxs: undefined symbol: camlStdlib__format__printf_1230\")")

For main.exe I get the --ccopt -static as expected :

Running[3]: (cd _build/default && /home/abate/_opam/bin/ocamlopt.opt -w -40 -ccopt -static -g -I .main.eobjs/byte -I .main.eobjs/native -intf-suffix .ml -no-alias-deps -o .main.eobjs/native/dune__exe__Main.cmx -c -impl main.ml)

and it looks ok to me

I’ve added (flags :standard) and same error …

Building static binaries is very system-specific and I wouldn’t be surprised if by doing so the final executable main.exe ended up stripped (no symbol table) which would make using Dynlink impossible.

You can check this by running nm -g main.exe (NB this is from memory) and check if the symbol camlStdlib__format__printf_1230 is listed.

Cheers,
Nicolas

$ nm -g _build/default/main.exe  | grep camlStdlib__format__printf_1230
00000000004b1f00 T camlStdlib__format__printf_1230

so it’s seems the symbol is there …

I just tried this in a standard Ubuntu install and it works. The issue you are having must be related to your use of musl. Linking objects built with different libc’s sure sounds like a long shot.

Cheers,
Nicolas

you mean you tried my example and it worked (thank you for trying !) ?

I’m doing most of my experiments on debian. On my box I expect everything to work fine… the musl problem should not be related.

I’ll be baffled if this is a setup problem :flushed:

Sorry, I had tried the wrong thing. I think that the problem is what I mentioned earlier, namely that by building a static executable the binary no longer has a “dynamic symbol table” (this is the table of symbols visible to shared objects which want to link against the binary). To check this, try readelf --dyn-syms main.exe. I think you will find it empty.

Cheers,
Nicolas

This resets the flags because :standard is not there - does (link_flags :standard -linkall) change what’s happening?

@emilion : no changes …

it’s indeed empty. So if I manage to populate this table, then symbol problem should disappear ? At the same time I’m not sure if -ccopt -static is compatible with the idea of having dynamic symbols…

That’s my uneducated guess as well.

Cheers,
Nicolas