Opam local switches seem pretty heavy-weight

Learning
1

I’m following the tips here: https://opam.ocaml.org/blog/opam-20-tips/ and trying to figure out a way to set up a simple opam project with a reproducible build. I went through the following steps:

  • Started with a system switch
  • Wrote dune-project and dune files
  • Wrote a my-pkg.opam file
  • Ran opam lock . to create a my-pkg.opam.locked file
  • Ran opam switch create . --deps-only ocaml-base-compiler.4.07.1 to create a local switch and completely seal off the project’s build environment from the system state.

I then ran du -sh . in my project root and it reports a size of 454MB. This is quite massive for a tiny project. Is there any way for opam to share packages with identical versions across switches?

2

Maybe it’s possible to change the script used for compilation cache in opam. And replace the copies with symlinks. https://github.com/ocaml/opam/blob/4756ca1/shell/opam-bin-cache.sh

3

AFAIK the main issue is that the compiler and compiler artifacts are not relocatable; they contain absolute paths (e.g. the path to ocamlrun for bytecode objects, or, in the case of the compiler, the path to its stdlib).

That’s why moving a local switch will not work properly, and there’s no easy way to setup a cache. Eventually this is something that we would like to improve on the compiler side, so that opam can provide by default a binary cache that works well…

4

Thanks. Following up on this–I saw that esy can import and build opam projects: https://esy.sh/docs/en/opam-workflow.html

I gave it a try and it seems to be working so far. Esy takes care of the compilation cache so it seems to be the best of both worlds: I keep an opam package and get esy to manage the actual installation and compilation. Disk usage is now 29MB.

1 Like
5

Second that. I have had much better experience with esy as a package manager than the opam when I want a reproducible build environment.

1 Like
6

One thing to be aware of with a global cache is that the model works better for an interpreted language than for a compiled one, since each compilation of the same version package with newer dependencies requires creating new binaries. If opam is implementing something like this, it should also implement a proper garbage collection method for preventing the cache from growing endlessly.

7

Yeah I guess it’s either that or moving to dynamic linking. Having an immutable cache with a graph of compilation artifacts, and doing garbage collection on them, is starting to sound a lot like Nix.

2 Likes