How are opam switches supposed to be used?

opam switch seems to encapsulate a specific compiler version + a set of installed packages.

I had assumed they were intended to be used like a Python virtualenv, i.e. you’d create one for each project in order to keep their packages separate.

But I’ve just noticed that opam switch set <some switch> seems to change the active switch globally on my system.

I’d assumed it would be possible to work on multiple projects simultaneously, in different terminal windows with different switches set active. But that doesn’t seem to be the case - when I set the switch in one terminal it affects which switch is active in all the others.

Do I have the wrong idea about what switches are for? Or there’s something wrong with my opam setup?

2 Likes

They are indeed like Python’s virtualenv. If you have enabled the shell hooks during opam init and are using local switches you basically will never use opam switch. Even if you decide to have some global switches you can use different ones in different terminals by using eval $(opam env --switch=XXX) (eventually with the set-switch option)

2 Likes

I see, so I generally shouldn’t be using opam switch set XXX

and instead should use eval $(opam env --switch=XXX)

(seems like it would have been better if the more useful command was the more obvious one…)

$ eval $(opam env --switch=XXX)
[NOTE] To make opam select the switch XXX in the current shell, add --set-switch or set OPAMSWITCH

So it seems that the full command needed is:

$ eval $(opam env --switch=XXX --set-switch)

How about just using eval $(opam env) in the directory where the local switch was created? Wouldn’t that work?

I’ve no idea, I had assumed there was no link between the dir and the switch

opam switch --help (or man opam-switch) includes this text:

opam switch set sets the default switch globally, but it is also possible to select a switch in a given shell session, using the environment. For that, use ‘eval $(opam env --switch=SWITCH --set-switch)’.

Yes. The opam command respects the presence of a local _opam subdirectory (e.g., created by opam switch link). In particular, eval $(opam env) will suffice to update the environment in that case.

2 Likes

Note that, as @Khady said, in the case of local switches, you don’t even to run eval $(opam env).

$ opam switch
#  switch                                     compiler                          description
→  4.13.1                                     ocaml-base-compiler.4.13.1        4.13.1
   5.0.0                                      ocaml-base-compiler.5.0.0         5.0.0
   5.0.0+tsan                                 ocaml-variants.5.0.0+tsan         5.0.0+tsan

$ opam switch create ./myproject 5.0.0
$ cd myproject
$ opam switch
#  switch                                     compiler                          description
→  /home/olivier/myproject                    ocaml-base-compiler.5.0.0                             /home/olivier/compiler/ocaml-multicore
   4.13.1                                     ocaml-base-compiler.4.13.1        4.13.1
   5.0.0                                      ocaml-base-compiler.5.0.0         5.0.0
   5.0.0+tsan                                 ocaml-variants.5.0.0+tsan         5.0.0+tsan

[NOTE] Current switch has been selected based on the current directory.
       The current global system switch is 4.13.1.
2 Likes

That is only if you have enabled the shell hooks. Otherwise, only the opam command itself knows about the current switch; the PATH environment variable will not magically change depending on the current directory. Note that, if you do not trust the shell hooks shipped with Opam, you can achieve a similar effect with more mainstream solutions such as direnv (i.e., opam env --switch=foo > .envrc).

4 Likes

The only problem with local switches is that they install everything. So a local switch with a couple of libraries gets easily a couple of Gb on disk.

direnv is great!

It will be a little bit more robust if you use:

echo 'eval $(opam env --switch=foo --set-switch)' > .envrc

That way the opam switch is added to your existing PATH (etc.) whenever you enter the directory, rather than your entire PATH replaced with whatever PATH existed at the time you created .envrc.

--switch=<something> is only needed if you use global opam switches.

1 Like

thanks for your help!

it sounds like local switches are closest to what I was looking for

so if I understood…

  • opam switch create myproject 4.14.1 (what I’d been doing) will create a switch under my home dir that needs eval $(opam env --switch=myproject --set-switch) to activate it in current shell
  • opam switch create ./myproject 4.14.1 will create a local switch that opam will use by default as long as I am in ./myproject dir

and opam switch set XXX to set a switch globally is something I probably never want to use

I cannot speak for you but for my own part I use opam switch set XXX a lot. I don’t generally use local switches as it causes each project with a local switch to keep its own compiler and all the libraries that it uses, which is a lot of infrastructure for a small project (if it is a small project). If using opam switch set offends you or is inconvenient, local links can be used as a lower overhead alternative to the local switch.

Say you have two global switches with the names 5.0 and 4.14 containing the ocaml 5.0.0 and 4.14.1 compilers respectively, and you have a project which uses the 4.14.1 compiler. You can set up a local link to the 4.14 switch by going into the top-level directory of your project and entering the command opam switch link 4.14 "." . This causes a symbolic link ./_opam to be created to the directory $OPAMROOT/4.14 containing your global switch. This in turn will cause the 4.14 compiler switch to be selected automatically when the project directory is entered, without changing the global default (assuming you have opam’s shell hooks, or direnv, set up correctly).

3 Likes

That should be less of a problem when the compiler is more relocatable I believe. The relocatable compiler would make copying the compiler directory tree around work, but I wonder if symlinks will also be possible by extension… that way a new switch can just symlink the set from the old switch.
currently that’s not the case, and you’re right, having half a dozen switches or so hanging around starts to bloat up disk usage. I know opam is THE version mananger for OCaml, but I think it’s worth investing in guix/nix more on the side, they’re today’s solutions to separate deduplicated environments.

2 Likes

I have used Deno (JavaScript/TypeScript runtime) in a few projects and it does not create a directory like node_modules with all your dependencies. You don’t even need a package.json file declaring your dependencies (though you can use an import map).

Every time there’s an import (and the import is from an external URL), the runtime checks if the imported file is in the global cache, and if it’s not, the runtime downloads it and registers it in the cache. So projects that use the same dependencies don’t need to duplicate them locally.

So even without a relocatable compiler, opam could use a cache to avoid having to replicate everything for every local switch. Another option could be having a switch being derived from another, e.g. you have a global 5.0 switch with the dependencies you tend to use in most projects, and then you create a local switch for a project that has the 5.0 global as a parent. The local switch can add new dependencies, opam looks in the local switch first, then goes to the parent if it can’t find a dependency.

BTW Strong warning, but local switches have bugs on Mac (Intel and M1) and I’ve had to move away from them back to global switch on the project I’m currently working on. (For example: Can't install on MAC · Issue #31 · janestreet/re2 · GitHub)

(I believe Mac people on my project are either all using a global switch or nix to circumvent all these issues)

The issue you link to is not related to local switches. As pointed out in the re2 bug tracker the error was related to an update to clang as part of a macOS update, and this was resolved with a patch to the OCaml compiler that was published as part of ocaml 4.12.0. There are no issues installing re2 in a local switch.

Mac OS #include <version> vs $(ocamlc -where)/lib/ocaml/version · Issue #10410 · ocaml/ocaml · GitHub and Do not install VERSION file. by bschommer · Pull Request #9895 · ocaml/ocaml · GitHub have details about the fix.