Compiling 100% static executables

Ecosystem
1

TL;DR: I cannot get OCaml 4.13.1 to compile static executables with native Musl nor with Alpine Linux. The only partial success I had was with musl-tools in Debian and a +musl switch, but musl-tools does not include g++, which package re2 requires to install.

We’ve had a few threads about this before, and a few blog posts are out there, but I’ve been going around in circles for two days now so it’s time to summarize where I’m at and hope that someone out there shares how they can actually compile static executables with OCaml, because I sure can’t. :wink:

1. Full Musl, local

One environment I have is the full native gcc/g++ suite from Musl libc locally from musl.cc, without interfering with the system’s compilers and libraries. Just use the $PATH and $LD_LIBRARY_PATH exports when you want to use Musl instead of Glibc. This does not require an OCaml +musl switch because Musl’s gcc is native, just like in Alpine’s.

Building 
cd /tmp ; wget https://musl.cc/x86_64-linux-musl-native.tgz
cd /usr/local ; tar -zxf /tmp/x86_64-linux-musl-native.tgz
ln -s /usr/local/x86_64-linux-musl-native/lib/libc.so /usr/local/x86_64-linux-musl-native/bin/ldd
ln -s /usr/local/x86_64-linux-musl-native/lib/libc.so /lib/ld-musl-x86_64.so.1
export PATH="/usr/local/x86_64-linux-musl-native/bin:$PATH" ; hash -r
export LD_LIBRARY_PATH="/usr/local/x86_64-linux-musl-native/lib"
opam switch create 4.13.1+muslnative+flambda ocaml-variants.4.13.1+options ocaml-option-flambda
eval $(opam env --switch=4.13.1+muslnative+flambda)

2. Full Musl, Alpine

I also have an official OCaml release under Alpine, to get a 100% bona fide native musl in case my local environment is broken.

Building and usage 
# Dockerfile
FROM ocaml/opam:alpine-ocaml-4.13-flambda
RUN sudo apk add --no-cache m4 linux-headers
RUN opam install dune
WORKDIR /work/

#!/bin/bash
# Get a shell inside the Alpine container.
exec docker run --rm -u `id -u`:`id -g` \
  -e LINES=`tput lines` -e COLUMNS=`tput cols` -it \
  -v `pwd`:/work my-docker-image:latest bash

Dynamic linking with Musl

Both full Musl setups above behave absolutely identically, so they are interchangeable from this point on. In both I can compile things dynamically (no ocamlopt_flags) and the result just depends on the dynamic loader and Musl’s libc.

file & ldd output 
$ file test.exe
test.exe: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV),
 dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
$ ldd test.exe
        linux-vdso.so.1 (0x00007fff3c38e000)
        libc.so => /usr/local/x86_64-linux-musl-native/lib/libc.so (0x00007f7e576eb000)

Static linking with Musl

Attempt 1

What used to work before:

(env (_ (ocamlopt_flags (:standard -ccopt -static))))

…now under a native Musl and Alpine fails to build (here shown with paths, etc. removed):

(ocamlopt.opt -w @1..3@5..28@30..39@43@46..47@49..57@61..62-40 -strict-sequence -strict-formats -short-paths -keep-locs -g -ccopt -static -shared -linkall -I lib -o lib/gxd.cmxs lib/gxd.cmxa)
ld: crtbeginT.o: relocation R_X86_64_32 against hidden symbol `__TMC_END__' can not be used when making a shared object
ld: crtend.o: relocation R_X86_64_32 against `.ctors' can not be used when making a shared object; recompile with -fPIC
collect2: error: ld returned 1 exit status
File "caml_startup", line 1:
Error: Error during linking (exit code 1)

If I use (ocamlopt_flags (:standard -fPIC -ccopt -fPIC -ccopt -pie -ccopt -static)), OCaml can’t even find its own symbols at link time like caml_call_gc. With one combination of arguments I even got _start_c from crt1.o to fail to find main!

Attempt 2

Despite having had some fatal failures in the past with a +static switch (I think Bin_prot in particular failed to build without dynamic library support), I tried one just in case:

# NOTE: --assume-depexts prevents opam from installing musl-tools.
cd /usr/local/x86_64-linux-musl-native/bin/ ; ln -s gcc musl-gcc
opam switch create --assume-depexts 4.13.1+muslnative+static+flambda ocaml-variants.4.13.1+options ocaml-option-static ocaml-option-flambda
file & ldd output 
$ file test.exe
test.exe: ELF 64-bit LSB pie executable, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib/ld-musl-x86_64.so.1, with debug_info, not stripped
$ ldd test.exe
        /lib/ld-musl-x86_64.so.1 (0x7f7b85d59000)
        libc.so => /lib/ld-musl-x86_64.so.1 (0x7f7b85d59000)

Still dynamic by default. If I add -ccopt -static, just like in attempt 1 I get the R_X86_64_32 against .ctors error, and if I play with PIC/PIE I get the same subsequent linking failure.

I’m Stumped :stuck_out_tongue:

If anyone builds static executables with OCaml under Alpine or with a native Musl (not Debian’s incomplete musl-tools), I’d love to hear how the heck you worked around these errors. I don’t even know what else I could possibly try.

5 Likes
2

I followed the blog post from this Discuss thread: Generating static and portable executables with OCaml to put together the solution in this repo: GitHub - rbjorklin/throttle-fstrim

3 Likes
3

I also use musl.cc packages for compiling static executables, but with a much simpler setup that doesn’t require PATH changes:

# unpack the tarball to /opt/x86_64-linux-musl-native (can be anywhere)
$ CC=/opt/x86_64-linux-musl-native/bin/gcc CXX=/opt/x86_64-linux-musl-native/bin/g++ opam switch create 4.13.0
…
$ opam install dune
…
$ echo '(executable (name test) (ocamlopt_flags :standard -ccopt -static))' > dune
$ touch test.ml      
$ dune build test.exe                                                             
$ ldd _build/default/test.exe
	not a dynamic executable

As far as I know, only two opam packages need workarounds:

  1. z3 needs the CC and CXX flags repeated: CC=/opt/x86_64-linux-musl-native/bin/gcc CXX=/opt/x86_64-linux-musl-native/bin/g++ opam install z3
  2. ctypes needs: PKG_CONFIG_PATH=/opt/x86_64-linux-musl-native/lib/pkgconfig opam install ctypes

If you need C libraries (e.g. openssl, gmp, libev, etc.), you’ll need to compile them yourself in the musl sysroot. Usually something like CC=/opt/x86_64-linux-musl-native/bin/gcc ./configure --prefix=/opt/x86_64-linux-musl-native && make && make install

5 Likes
4

Thank you for sharing your solutions.

Despite those scary PIC/PIE related linker errors, my problem was of a completely different nature and I was way, way off-track. My global (env (_ (ocamlopt_flags (:standard -ccopt -static)))) was applied to (executable) and (library) items by Dune. Building the libraries in my project is what failed; the executables were fine the whole time as it turns out, and my musl.cc installation was fine as well.

The solution was thus to remove my global (env) and to copy the -ccopt -static boilerplate into each individual (executable). And thanks to the full Musl GCC installation, my final switch is merely a “vanilla” +flambda, no need to edit compiler build flags or any notion of PIC/PIE at all.

Talk about searching in the wrong direction… Now I can finally start the work I wanted to tackle Monday morning. :sweat_smile:

6 Likes
5

I’ve got to the same problem and I’ve set up a Dockerfile for doing that, mainly because I needed OpenSSL support, which I couldn’t get with Fedora’s.

3 Likes