Opam local switches seem pretty heavy-weight

I’m following the tips here: https://opam.ocaml.org/blog/opam-20-tips/ and trying to figure out a way to set up a simple opam project with a reproducible build. I went through the following steps:

  • Started with a system switch
  • Wrote dune-project and dune files
  • Wrote a my-pkg.opam file
  • Ran opam lock . to create a my-pkg.opam.locked file
  • Ran opam switch create . --deps-only ocaml-base-compiler.4.07.1 to create a local switch and completely seal off the project’s build environment from the system state.

I then ran du -sh . in my project root and it reports a size of 454MB. This is quite massive for a tiny project. Is there any way for opam to share packages with identical versions across switches?

Maybe it’s possible to change the script used for compilation cache in opam. And replace the copies with symlinks. https://github.com/ocaml/opam/blob/4756ca1/shell/opam-bin-cache.sh

AFAIK the main issue is that the compiler and compiler artifacts are not relocatable; they contain absolute paths (e.g. the path to ocamlrun for bytecode objects, or, in the case of the compiler, the path to its stdlib).

That’s why moving a local switch will not work properly, and there’s no easy way to setup a cache. Eventually this is something that we would like to improve on the compiler side, so that opam can provide by default a binary cache that works well…

Thanks. Following up on this–I saw that esy can import and build opam projects: https://esy.sh/docs/en/opam-workflow.html

I gave it a try and it seems to be working so far. Esy takes care of the compilation cache so it seems to be the best of both worlds: I keep an opam package and get esy to manage the actual installation and compilation. Disk usage is now 29MB.

1 Like

Second that. I have had much better experience with esy as a package manager than the opam when I want a reproducible build environment.

1 Like

One thing to be aware of with a global cache is that the model works better for an interpreted language than for a compiled one, since each compilation of the same version package with newer dependencies requires creating new binaries. If opam is implementing something like this, it should also implement a proper garbage collection method for preventing the cache from growing endlessly.

Yeah I guess it’s either that or moving to dynamic linking. Having an immutable cache with a graph of compilation artifacts, and doing garbage collection on them, is starting to sound a lot like Nix.

2 Likes