A weird error when upgrading OPAM from 2.1.6 to 2.2.0

Ecosystem
1

Just FYI, I recently upgraded from OPAM 2.1.6 to 2.2.0 on both Debian and Ubuntu. On Ubuntu, I started seeing weird OPAM code stack-traces, and installing packages that worked before no longer did so. After a bit of debugging to try to repro cleanly, I just got bored and blew away my entire OPAM instance, recreated with opam init and since then, everything appears back to normal.

Just thought I should report, so that if anybody else sees a similar problem, they’ll know they’re not alone, and that it’s easy enough to remedy.

Fatal error:
/home/chet/Hack/Opam-2.1.2/bin/opam-2.2.0: "open" failed on /home/chet/Hack/Opam-2.1.2/GENERIC/log/log-372807-eb85c4.out: Read-only file system
Backtrace:
  Raised by primitive operation at OpamProcess.create.tee in file "src/core/opamProcess.ml", line 336, characters 13-40
  Called from OpamProcess.create.(fun) in file "src/core/opamProcess.ml", line 357, characters 16-21
  Called from OpamProcess.run in file "src/core/opamProcess.ml", line 757, characters 10-32
  Called from OpamSystem.run_process in file "src/core/opamSystem.ml", lines 534-537, characters 8-27
  Called from OpamSystem.read_command_output in file "src/core/opamSystem.ml", lines 578-580, characters 4-9
  Called from OpamSysPoll.command_output in file "src/state/opamSysPoll.ml", line 15, characters 10-44
  Called from OpamSysPoll.poll_os_version.(fun) in file "src/state/opamSysPoll.ml", line 134, characters 4-46
  Called from CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 49, characters 17-27
  Re-raised at CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 56, characters 4-11
  Called from CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml" (inlined), line 78, characters 27-67
  Called from Stdlib__Lazy.map in file "lazy.ml", line 75, characters 10-19
  Called from CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 49, characters 17-27
  Re-raised at CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 56, characters 4-11
  Called from CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml" (inlined), line 78, characters 27-67
  Called from OpamGlobalState.load.(fun) in file "src/state/opamGlobalState.ml", line 97, characters 60-78
  Called from CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 49, characters 17-27
  Re-raised at CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 56, characters 4-11
  Called from CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml" (inlined), line 78, characters 27-67
  Called from OpamPackageVar.resolve_global in file "src/state/opamPackageVar.ml", line 65, characters 12-33
  Called from OpamPackageVar.resolve_switch_raw in file "src/state/opamPackageVar.ml", line 107, characters 12-38
  Called from OpamFilter.resolve_ident_raw in file "src/format/opamFilter.ml", line 212, characters 10-44
  Called from OpamFilter.resolve_ident in file "src/format/opamFilter.ml", line 236, characters 8-53
  Called from OpamFilter.reduce in file "src/format/opamFilter.ml", line 390, characters 8-54
  Called from OpamFilter.reduce_aux in file "src/format/opamFilter.ml" (inlined), line 364, characters 15-55
  Called from OpamFilter.reduce_aux.(fun) in file "src/format/opamFilter.ml", line 371, characters 11-19
  Called from OpamFilter.reduce in file "src/format/opamFilter.ml", line 390, characters 8-54
  Called from OpamFilter.reduce_aux in file "src/format/opamFilter.ml" (inlined), line 364, characters 15-55
  Called from OpamFilter.reduce_aux.(fun) in file "src/format/opamFilter.ml", line 381, characters 55-65
  Called from OpamFilter.reduce in file "src/format/opamFilter.ml", line 390, characters 8-54
  Called from OpamFilter.reduce_aux in file "src/format/opamFilter.ml" (inlined), line 364, characters 15-55
  Called from OpamFilter.reduce_aux.(fun) in file "src/format/opamFilter.ml", line 379, characters 57-67
  Called from OpamFilter.reduce in file "src/format/opamFilter.ml", line 390, characters 8-54
  Called from OpamFilter.eval_to_bool in file "src/format/opamFilter.ml" (inlined), line 398, characters 54-68
  Called from OpamSwitchState.filter_available_packages.(fun) in file "src/state/opamSwitchState.ml", lines 40-42, characters 8-40
  Called from Stdlib__Map.Make.filter in file "map.ml", line 433, characters 20-25
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 434, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from Stdlib__Map.Make.filter in file "map.ml", line 432, characters 19-29
  Called from OpamSwitchState.filter_available_packages in file "src/state/opamSwitchState.ml", lines 39-43, characters 4-11
  Called from OpamSwitchState.compute_available_and_pinned_packages in file "src/state/opamSwitchState.ml", line 55, characters 3-59
  Called from CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 49, characters 17-27
  Re-raised at CamlinternalLazy.do_force_block in file "camlinternalLazy.ml", line 56, characters 4-11
  Called from CamlinternalLazy.force_lazy_block in file "camlinternalLazy.ml" (inlined), line 78, characters 27-67
  Called from OpamSwitchState.load in file "src/state/opamSwitchState.ml", line 392, characters 12-45
  Called from OpamSwitchState.with_.(fun) in file "src/state/opamSwitchState.ml", line 1305, characters 11-33
  Called from OpamStd.Exn.finally in file "src/core/opamStd.ml", line 1761, characters 10-14
  Re-raised at OpamStd.Exn.finalise in file "src/core/opamStd.ml", line 1758, characters 4-38
  Called from OpamConfigCommand.var_show in file "src/client/opamConfigCommand.ml", lines 1294-1297, characters 8-81
  Called from OpamCommands.Var_Option_Common.var_option.(fun) in file "src/client/opamCommands.ml", line 1153, characters 9-40
  Called from OpamGlobalState.with_ in file "src/state/opamGlobalState.ml", line 207, characters 14-18
  Re-raised at OpamStd.Exn.finalise in file "src/core/opamStd.ml", line 1758, characters 4-38
  Called from Cmdliner_term.app.(fun) in file "cmdliner_term.ml", line 24, characters 19-24
  Called from Cmdliner_term.ret.(fun) in file "cmdliner_term.ml", line 41, characters 25-32
  Called from Cmdliner_eval.run_parser in file "cmdliner_eval.ml", line 35, characters 37-44
  Called from Cmdliner_eval.eval_value in file "cmdliner_eval.ml", line 203, characters 14-39
  Called from OpamCliMain.run.(fun) in file "src/client/opamCliMain.ml", line 459, characters 8-84
  Called from OpamCliMain.main_catch_all in file "src/client/opamCliMain.ml", line 339, characters 6-10
2

Could this be related to the Ubuntu specific kernel patches about user namespaces, as discussed e.g. here?

https://www.openwall.com/lists/oss-security/2024/04/14/1

–
Ian

3

i’ve never seen this issue before and i have no idea where it could come from but i those kernel patches don’t seem related to me.

The backtrace indicates that it fails on a Unix.openfile. What is the output of ls -l /home/chet/Hack/Opam-2.1.2/GENERIC/log/ ?

4

one way to validate that is to retry with the opam sandbox disabled.

5

The complement of files in that directory were as expected – indeed I got the backtrace from a “.out” file in that directory. As I mentioned, the actual problem was that packages that had installed fine before (and were released to OPAM) were no longer installing. “remove”/“create” of the switch didn’t help. “update”/“upgrade” didn’t help. But blowing away the entire OPAM installation made the problem go away.

I didn’t mean for this to be something anybody worried about debugging: I consider it a Heisenbug, and the best way to deal with a Heisenbug is to reset the offending system (reinit OPAM) and just keep a’goin’.

I haven’t had any such problems since the reinit.