Statically link an OCaml project with Docker and Alpine

[pdonadeo]

Hi all, we have a project for which we will provide support for 20+ years and a large and important part of it is written in OCaml, of course. The production server is CentOS 7 and the recent commercial issues about CentOS and RedHat are suggesting that we should switch to Docker to keep a freezed building environment.

We easily built a Docker image based on CentOS 7, thanks to the excellent job the community have done making base images available on Docker Hub.

Of course the artifacts built inside the Docker image run there, but not outside because the linker cannot find the libraries. I searched for a solution and I found this thread pointing to this blog post of @rgrinberg.

I started from ocaml/opam:alpine-3.13-ocaml-4.08 and switched opam to 4.08.1+musl+static+flambda and a simple “hello world” using -ccopt -static works as expected but if I add to hello.ml PostgreSQL:

dune:

(executable
  (name hello)
  (flags (-ccopt -static))
  (libraries postgresql))

hello.ml:

let c = new Postgresql.connection ~host:"localhost" ~user:"user" ~password:"pwd" ()

let main () =
  Printf.printf "Hello world...?\n%!"

let () = main ()

it doesn’t even compile, complaining with a long list of missing references:

$ dune build hello/hello.exe
    ocamlopt hello/hello.exe (exit 2)
(cd _build/default && /home/opam/.opam/4.08.1+musl+static+flambda/bin/ocamlopt.opt -ccopt -static -g -o hello/hello.exe -I /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/threads -I /home/opam/.opam/4.08.1+musl+static+flambda/lib/postgresql /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/unix.cmxa /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/threads/threads.cmxa /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/bigarray.cmxa /home/opam/.opam/4.08.1+musl+static+flambda/lib/postgresql/postgresql.cmxa hello/.hello.eobjs/native/hello.cmx)
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/lib/libpq.a(fe-connect.o): in function `parseServiceFile':
fe-connect.c:(.text+0xbee): undefined reference to `pg_strncasecmp'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xda8): undefined reference to `pg_strcasecmp'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xdbf): undefined reference to `pg_strcasecmp'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xddb): undefined reference to `ldap_init'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xdff): undefined reference to `ldap_set_option'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xe13): undefined reference to `ldap_simple_bind'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xe41): undefined reference to `ldap_result'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xe65): undefined reference to `ldap_msgfree'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xe7e): undefined reference to `ldap_set_option'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xeb4): undefined reference to `ldap_search_st'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xed1): undefined reference to `ldap_msgfree'
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: fe-connect.c:(.text+0xed9): undefined reference to `ldap_err2string'

... ... ... for more than 300 lines ... ... ...

Any ideas about how to solve this problem? Is it solvable, in the first place?

[mseri]

I incurred into similar issues in the past, I don’t have a direct solution for production use but my issues were resolved using -ccopt -static -ccopt -Wl,--no-export-dynamic. I am not 100% sure it is still relevant, or if that is what I did, because since then I relied on [GitHub - EduardoRFS/reason-mobile] for cross compilation and for statically linked executables. Maybe @EduardoRFS knows better what is going on.

[VPhantom]

Similar situation here (20+ year forward support). I’m not sure how much this applies to your own requirements, but in our case I discovered that I need musl but not a +static build to build static executables. I also ended up not needing Docker nor Alpine:

• Installed switch 4.xx+musl_flambda;
• Installed musl, musl-dev and musl-tools on my host (Debian Buster);
• Told Dune to use (ocamlopt_flags (:standard -ccopt -static)).

The +musl OCaml compiler build makes it use musl-gcc and link to musl instead of glibc, and the build flags instruct the compiler to link statically.

In your specific case, seeing as the unresolved symbols start with pg_ and ldap_ I suspect you might just be missing a library to link against in addition to postgresql. I suspect you might have to do something specific in order to link to libpq statically.

[pdonadeo]

Thanks for all the replies. The clear problem here is my ignorance in understanding the linking phase of an OCaml library, and how to “pilot” the process through dune.

The missing symbols are actually defined in /usr/lib/libpq.a (inside the container) and even invoking the compiler (without dune) adding /usr/lib/libpq.a on the command line:

/home/opam/.opam/4.08.1+musl+static+flambda/bin/ocamlopt.opt \
  -ccopt -static -ccopt -Wl,--no-export-dynamic -g \
  -o hello/hello.exe \
  -I /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/threads \
  -I /home/opam/.opam/4.08.1+musl+static+flambda/lib/postgresql \
  /usr/lib/libpq.a \
  /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/unix.cmxa \
  /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/threads/threads.cmxa \
  /home/opam/.opam/4.08.1+musl+static+flambda/lib/ocaml/bigarray.cmxa \
  /home/opam/.opam/4.08.1+musl+static+flambda/lib/postgresql/postgresql.cmxa \
  hello/.hello.eobjs/native/hello.cmx

produces the very same output.

[VPhantom]

I hadn’t noticed initially but your first error line hints that libpq.a is indeed found by the linker:

/usr/.../ld: /usr/lib/libpq.a(fe-connect.o): in function `parseServiceFile':

So a function in libpq.a called parseServiceFile() refers to pg_strncasecmp() which the linker cannot find, even though presumably it’s defined in that same file (although probably not in the fe-connect.o portion).

I don’t know enough about the process to suggest something, but it looks like the linker might need an extra argument.

[VPhantom]

ocamlopt -I tells the compiler directories in which to look for .cm* files. It doesn’t look like the right place to mention libpq.

I don’t see -cclib -lpq in that command line, but I bet it’d be needed:

-cclib -llibname
Pass the -llibname option to the linker. This causes the given C library to be linked with the program.

[mjambon]

You don’t need to use a special opam switch if you’re on alpine. See my recent message.

[pdonadeo]

Thanks @VPhantom, you put me on the right track: -lpq was missing. This means I should refresh my C experience, too many years have passed since I was proficient with C/C++ and I have completely forgotten the “joy” of the linker.

(side note about the OCaml toolchains as a whole: do someone really think it’s not adequate? Really?)

So yesterday I spent a lot of hours in this loop:

1. "undefined bla_bla_bla"
2. Google search for bla_bla_bla
3. Google search for an Alpine package containing bla_bla_bla
4. add -lbla

and I could finally compile almost all the project.

The last problem is more about dune: the project has 3 (internal) libraries and many executables. Those executables which do not depend upon any library compile without problems. Dune wants to generate plugins for those libraries and in this setup the link fails, this is the log:

ocamlopt common/common.cmxs (exit 2)

(cd _build/default && /home/opam/.opam/4.08.1+musl+static+flambda/bin/ocamlopt.opt -w -40 -ccopt -static -ccopt -Wl,--no-export-dynamic -cclib -lldap -cclib -lsasl2 -cclib -lgdbm -cclib -lssl -cclib -lcrypto -cclib -lgssapi -cclib -lkrb5 -cclib -lheimntlm -cclib -lheimbase -cclib -lhx509 -cclib -lcom_err -cclib -lhcrypto -cclib -lwind -cclib -lsqlite3 -cclib -lroken -cclib -lasn1 -cclib -lpgcommon -cclib -llber -cclib -lpgport -cclib -lpq -g -shared -linkall -I common -o common/common.cmxs common/common.cmxa)

/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
/usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/../../../../x86_64-alpine-linux-musl/bin/ld: /usr/lib/gcc/x86_64-alpine-linux-musl/10.2.1/crtend.o: relocation R_X86_64_32 against `.ctors' can
not be used when making a shared object; recompile with -fPIC
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking

I don’t think I need cmxs in this context but I’m not sure how to tell dune not to compile plugins.

[VPhantom]

The “recompile with -fPIC” suggestion might be key here. Have you tried -ccopt -fPIC? I haven’t used it personally but since “Position Independent Code” and “relocation” seem related, I bet that warning puts you on the right track.

[pdonadeo]

Yes, I tried -ccopt -fPIC but nothing changes. Since I will never use the cmxss I only need a way to express this fact in the dune file.

[EduardoRFS]

libpq.a is not equivalent to libpq.so you will need a couple more libpq_ to compile it statically, -lpq is not enough, I think @ulrikstrid knows the list.

Also if you want to compile statically give it a try to GitHub - EduardoRFS/reason-mobile so that you don’t need to use docker with GitHub - esy-packages/postgresql