Is 16 minutes normal to install opam deps in a Docker build?

Ecosystem
,
1

I have a fairly run-of-the-mill Dream project. When I do incremental compiles with dune, it’s super-fast. When I do a Docker build and the previous build’s cache is invalidated, it takes about 16 minutes to download and build my dependency cone:

 => [build-deps 6/7] RUN opam exec -- opam update && opam exec -- opam install --deps-only --with-test -y .                     998.4s

This is on macOS 12.6 M1. The Dockerfile is nothing crazy, just a few layers:

# https://hub.docker.com/layers/ocaml/opam/ubuntu-22.04-ocaml-4.14/images/sha256-73f7c5dd28f7b757de325d915562a3f89903a64e763c0e334914fb56e082f5cb?context=explore

FROM ocaml/opam:ubuntu-22.04-ocaml-4.14@sha256:73f7c5dd28f7b757de325d915562a3f89903a64e763c0e334914fb56e082f5cb as build-deps

RUN sudo apt-get update && sudo apt-get install -y libev-dev libssl-dev libsqlite3-dev pkg-config nodejs
WORKDIR zk
ADD zk.opam ./
RUN opam init
RUN opam exec -- opam update && opam exec -- opam install --deps-only --with-test -y .

Does this match up with people’s experiences? Any suggestions on how to cache the opam deps in a more granular way so that changing a single dependency doesn’t blow away the entire cache?

2

How many cores is your Linux VM configured to use? How much memory have you given it also?

3

Docker Desktop says:

  • CPUs: 4
  • Memory: 7.9 GB
  • Swap: 1 GB

EDIT: wait, I just remembered, I get a message in the logs about upgrading to the latest opam which apparently has many performance improvements. I’m happy to upgrade but kinda puzzled that I would need to in the first place, because I’m using an image that was uploaded 5 days ago with pretty recent versions of Ubuntu and OCaml: Docker

4

Are you volume sharing between the Mac filesystem and the Linux VM included in Docker for Desktop, or just executing a docker build?

5

I am just doing docker build -t zk:latest .

Tweaked the build stage slightly while I was working on this, but no substantial change from before:

# https://hub.docker.com/layers/ocaml/opam/ubuntu-22.04-ocaml-4.14/images/sha256-73f7c5dd28f7b757de325d915562a3f89903a64e763c0e334914fb56e082f5cb?context=explore

FROM ocaml/opam:ubuntu-22.04-ocaml-4.14@sha256:73f7c5dd28f7b757de325d915562a3f89903a64e763c0e334914fb56e082f5cb as build

WORKDIR zk
RUN sudo apt-get update && sudo apt-get install -y libev-dev libssl-dev libsqlite3-dev pkg-config
RUN curl -L -O https://github.com/tailwindlabs/tailwindcss/releases/download/v3.3.1/tailwindcss-linux-x64 && chmod 755 tailwindcss-linux-x64
ADD zk.opam ./
RUN opam init
RUN opam exec -- opam update && opam exec -- opam install --deps-only --with-test -y .
ADD . .
RUN sudo chown -R opam .
RUN opam exec -- dune build && opam exec -- dune test
RUN ./tailwindcss-linux-x64 -i ./static/input.css -o ./static/output.css --minify && rm static/input.css
6

You should try this with podman on M1. It will allow you to install a Linux VM (be sure to set similar CPU/memory).

7

I’m using Github Actions and was able to shave ~18 minutes off my build by separating it into two separate pipelines one of which is multi-stage.

I start by building my “builder” image from Dockerfile.builder:

ARG ALPINE_VERSION=3.17
FROM alpine:${ALPINE_VERSION} AS builder

RUN apk add \
    bash\
    bubblewrap\
    ca-certificates\
    coreutils\
    gcc\
    git\
    gmp-dev \
    libc-dev \
    libev-dev \
    libpq-dev \
    linux-headers \
    m4\
    make\
    musl-dev\
    opam \
    openssl-dev \
    postgresql14-dev\
    rsync

RUN opam init \
    --disable-sandboxing \
    --auto-setup \
    --compiler ocaml-base-compiler.5.0.0 \
    && opam clean

RUN opam repo add alpha git+https://github.com/kit-ty-kate/opam-alpha-repository.git

RUN opam install --yes\
  alcotest \
  atd \
  atdgen \
  atdgen-runtime \
  caqti \
  caqti-driver-postgresql \
  caqti-lwt \
  cmdliner \
  cohttp-lwt-unix \
  conduit-lwt-unix \
  conf-libev \
  conf-libssl \
  conf-postgresql \
  core \
  dune \
  logs\
  lwt \
  lwt_ppx \
  lwt_ssl \
  odoc \
  prometheus-app \
  websocket-lwt-unix \
  yaml \
  yojson \
  && opam clean

next I have my multi-stage Dockerfile that actually builds the image I end up deploying:

ARG TAG=5.0.0
ARG ALPINE_VERSION=3.17
FROM rbjorklin/ocaml-build:${TAG} AS builder

ARG PROFILE=dev

WORKDIR /deps
ADD Makefile /deps/
ADD *.opam /deps/

RUN make deps

WORKDIR /build
ADD . /build/

RUN make build PROFILE=${PROFILE}

FROM alpine:${ALPINE_VERSION}
RUN apk --no-cache add\
    ca-certificates\
    libev\
    libpq\
    python3\
    py3-pip

RUN pip install selenium
COPY --from=builder /build/_build/default/src/bin/app.exe ./
ENTRYPOINT ["./app.exe"]

The relevant Makefile targets I use are:

deps:
    opam install --yes --deps-only .

build:
    opam exec -- dune build --profile $(PROFILE) @install

The Dockerfile.builder image rarely needs to be updated which allows me to save quite a bit on every build. I hope any of this ends up being useful :slight_smile:

1 Like
8

Thanks! I think the key here is running opam install dep1 dep2 ... depN directly in the Dockerfile instead of allowing it to install the deps listed in my_project.opam file. I took a middle ground, I figured since Dream pulls in the largest dependency cone out of everything installed, so doing RUN opam exec -- opam install dream in an earlier layer should cache almost all the dependencies.

And that does work, although in a later layer when I install the rest of the deps, annoyingly some of the earlier-installed ones are forced to recompile.

But at least now the build time is down to about 6 minutes even if I change anything in my opam file.

Cheers!