Installing in-progress opam project

Learning
1

I’ve been coding in OCaml and using opam for quite a long time now. I’ve still not managed to install a local directory without first creating a git commit. It’s a papercut I’ve worked my way around for a long time now.

Specifically, running e.g.

opam install . --switch=myswitch

will not install the current folder if I have not commited my last changes. Which is annoying, sometimes I just want to install whatever my current work is.

I think the most annoying part is the fact that opam will tell you:

[NOTE] Ignoring uncommitted changes in /current/folder
       (`--working-dir' not specified or specified with no argument).

But then running

opam install . --switch=myswitch --working-dir

just says “already installed”.

Is there a simple work around? I’ve read the documentation of opam install but it doesn’t say much more.

Cheers

2

You could try to rather opam pin add -kpath . but then you subject yourself to picking unwanted change when you opam upgrade. Be careful when you juggle with git branches – that’s the reason why -kgit is the (right) default.

(Also it’s likely a bug that you should report)

2 Likes
3

Um …. maybe I’m not understanding. You have a switch, myswitch. You want to install the current directory’s package. Yes? So you set the switch thus:


$ opam switch 5.4.0~beta1
# To update the current shell environment, run: eval $(opam env)
$ refresh_switch

[refresh_switch is just an alias for eval $(opam env)]

then you run


$ opam install --working-dir .

Am I missing something? Sometimes the package is already pinned to some revision, so you have to do opam pin list and opam pin remove <package-name> or whatever.

Is the –-switch=myswitch doing something untoward?

4

Hmm, pinning with the kpath raises a bunch of warnings with rsync, but seems to work

5

The issue is:

opam switch myswitch
eval $(opam env)
opam install --working-dir .

echo "let () = failwith \"Test\"" >> src/main.ml

dune build @all
opam install --working-dir .
#^^^^^^^^^^^^^^^^^^^^^^^^^^^
# Doesn't do anything, while it should install the updated main executable
6

The documentation says:

This only affects packages explicitly listed on the command-line

Maybe it doesn’t do anything because the package is not explicitly named?

7

Oh, ok, I … think that makes sense. Let me try to explain back to you:

(1) opam install doesn’t merely -install-: it runs the “build” and “install” stanzas of the opam file.

(2) So your first “opam install’ built and install the package from sources. But it doesn’t build in-place: it copies to a build-tree over in opam-managed space.

(3) so you updated the source and built using dune, in the local source-tree.

(4) then you tried opam install again. But that doesn’t work, b/c it is already installed. The -package- is already installed in step #2. So you need to uninstall the package, make sure it’s unpinned, and then opam install again.

Does that make sense? [I’m expressing what I believe to be true; I could be very wrong, but this is the working model I use of how opam works] I think you’re thinking of opam install as kind of like “make install”, but really it’s more “apt-get install” (combined a build).

Maybe what you want to do is a “dune install” ?

So often when I’m developing, I want to stay in the actual source directory: I want to build, then install, then maybe modify something, rebuild, reinstall, etc. I use Makefiles, but I suspect the same workflow cycle works with dune. I do:

(a) “make all install”

(b) [modify some files]

(c) “make all install”

(d) maybe I realized I need to do very pervasive fixup/change, so I do “make uninstall”, then “make clean”, then “make all install”

etc. None of this touches the opam-managed files, and eventually when I want to do an opam install I’m careful to do a make uninstall first. A few times, I’ve gotten into a situation where there both a make install and an opam install version of a package, and that can get confusing.

Does this help?

8

The issue comes with your step (3)

But it doesn’t build in-place: it copies to a build-tree over in opam-managed space.

What opam copies is the latest committed version, not the working directory. So if you have uncommited changes, they wont get transfered to the opam-managed space. So even if I uninstall and re-install, it doesn’t change anything until I git commit my current changes.

9

I had hope and tried

opam install ./my-package.opam --switch=myswitch

But the result is the same

10

What opam copies is the latest committed version, not the working directory.

Mmmm …. no, if I do opam install –working-dir ., it does indeed install the current work-dir sources. I just tested this out with opam 2.4.1, but it’s worked that way since I can remember.

(1) I modified the sources of camlp5 so the version was 8.05.00 (the committed source is at version 8.04.00)

(2) I DID NOT commit this change

(3) I did an opam install –working-dir .

(4) then camlp5 -version reports 8.05.00.

11

I feel like reiterating that opam install isn’t a sort of “make install”. It is a “install a package with this name”. If you tell it opam install . you’re saying “get the package (name & sources) from the current directory’s opam file.” If tell it opam uninstall, that’s the same for uninstalling. And this is -packages-, not source-trees. So once you’ve installed a package named “foo”, then you’re done: if you ask to install that package again, it’ll tell you you’ve already done it, right?

I think you’re looking for a lifecycle that a build-tool gives you, and that’s why I’m suggesting you use “dune build” then “dune install” etc etc. That matches what you’re trying to do.

ETA: and “opam pin add” adds a little bookmark to opam, so that if you tell it to install the package again, it knows where to get the sources (place and revision). So if you want it to “forget” the sources for a package you have installed, you need both uninstall it, and “pin remove” it. So that it both forgets the sources (from the opam-managed directory) and forgets the particular version, and where to get that version. so that the next time you tell it to opam install –working-dir, it’ll use the version of the sources right in the working-dir right there.

Now, it might qualify as a bug, that if you have a pin for a package, and you try to opam install –working-dir ., it doesn’t complain saying “you already told me where to find the sources for this package, why are you now telling me to use the ones in this directory? make up your mind already?” grin

12

I never use these install workflows but yes usually if PKG package is already installed and you opam install PKG nothing happens. So it may all make sense.

Perhaps @giltho’s desire is simply:

opam reinstall --working-dir .
2 Likes
13

Huzzah! That works. I confess I never use opam reinstall, but maybe I should start.

To verify, I did

(1) opam install –working-dir .

(2) modify sources

(3) opam reinstall –working-dir .

(3) modify sources AGAIN

(4) opam reinstall –working-dir . (AGAIN)

each time I installed, I ran a command to check that the version that got installed was what I expected.

14

That being said if opam install . follows VCS changes. I think that opam install --working-dir . should follow working directory changes aswell.

The current behaviour seems suprising and somehow inconsistent with a mental model that tries to map install . and install --working-dir . on the behaviour of -kgit and -kpath pins.

15

Note that you can likely consistently use opam reinstall it will propose to install if not installed.

1 Like
16

I feel like it should be possible to bypass this confusing workflow altogether and just vendor the dependency package (maybe as a git clone with instructions to gitignore its directory) and just let dune take care of building it as part of the project.

1 Like
17

Not sure what dune has to do here, we are talking about opam. The worfklow per se is not confusing, it just looks buggy as far as user expectations are.

18

I have a lot of hope for this but:

┬─[sacha@SachaMBP:~/P/soteria]─[09:16:26]─[G:cbmc =]
╰─>$ opam reinstall soteria-c --switch=bfa-install --working-dir

<><> Synchronising pinned packages ><><><><><><><><><><><><><><><><><><><><>  🐫 
Processing  1/1: [soteria-c.dev: git]fish: Job 1, 'opam reinstall soteria-c --swit…' terminated by signal SIGSEGV (Address boundary error)

Looks like something is segfaulting when I try on my project

19

Well invoke with -v -v and make a nice bug report :–)

20

@giltho what version of opam do you have? Versions before 2.4 were known to have problems like this (ocaml/opam#5567).
As for the segfault, i’ve never seen anything like that, so yes please, do open a ticket in our bug-tracker