I spent a long time today speeding up my build, and I wanted to share my findings. The two biggest changes:
We use docker and share the application directory between the host and container. Docker for Mac has an extremely bad performance problem here which is widely discussed, as docker wants to make sure a shared directory is entirely consistent between host and container. This slows all the file reads and writes that occurs in Dune. I observed somewhere between a 3x - 5x speed improvement by solving this. A lot of complicated approaches exist such as docker-sync, or building in a separate directory and syncing it with unison, but I found the simplest solution was mounting a volume to replace my _build directory so that it was no longer shared.
For an unknown reason, when I gave Docker on Mac 8 CPUs, it was much slower than 4 CPUs (unaffected by dune’s
-jflag). I dropped from 8 CPUs to 4 CPUs (I tried all other ranges) for a 40% speed improvement. I also found it did better with 2GB of RAM than with 8GB of RAM, though I wasn’t scientific enough to be sure about that.
Hoep that’s useful for other docker/ocaml/dune users!