Dune lockfiles are not cross-platform. How should we approach checking them into version control?

Ecosystem
21

I think it’s true that it can be unclear what to do when you have true OS or arch-specific dependencies. But it is also true that, for whatever reason, this has been a non-problem in the Go ecosystem as far as I can tell. So @yawaramin’s comments ring true to me, even if I am not certain what technical proposal we should advance.

One issue is that even the simplest OCaml project results in a lockfile that is extremely architecture dependent — whereas I never had this experience in other ecosystems.

(EDIT: I just realised, my problems in the past were from opam lockfiles, not Dune lockfiles. Perhaps the same discussion applies, but I wanted to be transparent about that.)

22

I’m curious. Do you have an example of the architecture dependent packages you get ?

My bet would rather be that most OCaml project do not need architecture or operating system dependent packages. But I’m happy to lose my bet.

1 Like
23

In general yes, however the PR adding Windows support to opam-repository recently added architecture-dependent meta packages (e.g. arch-x86_32) so opam lockfiles are currently almost always architecture dependent currently sadly. dune lockfiles probably do suffer from this too if they use opam-repository.

Solutions to fix this are welcome. This is tracked in ocaml/opam#6042

3 Likes
24

Ah very interesting!

I always had the gut feeling that trying to solve configuration problems by encoding configuration options in package names was a very bad idea®.

Now at least I get a confirmation that it’s not just a feeling :–)

Now I’m going to make a totally uninformed comment but in another life I wrote this proposal, I suppose these input variables could help with the problem that is being solved by encoding configuration into package names. With the idea that input variables will only be part of the lock file if they are being set explicitely.

5 Likes
25

I’ve made a repo to test the behaviour of checked in lockdirs on different machines. The lockdir was generated on an x86_64 machine running nixos. To my surprise I was able to build the project from the lockdir on an M1 mac. The lockdir contains a lockfile for the package host-arch-x86_64 but this seems to have no effect on the build.

3 Likes