for learning ocaml, I’m writing a program that uses GitHub - vbmithr/ocaml-websocket: Websocket library for OCaml and Lwt. My program is a websocket client. I’m investigating an issue where if the websocket server crashes, my client program will not report an exception until a few writes afterwards (even though I can see using strace that the OS is returning EPIPE for the write calls).
I took this problem as an opportunity to practice my debugging skills. For debugging this problem, I’d like to insert some printfs here and there in my dependencies’ code. I did that succesfully for ocaml-websocket, where I cloned the repo into my own source repo and dune was able to pick up my local copy. I tried to do the same for lwt, but that seems to be harder:
$ dune exec sharer-server --release
Error: Conflict between the following libraries:
- "lwt" in _build/default/vendor/lwt/src/core
- "lwt" in /home/braulio/.opam/4.14.0/lib/lwt
-> required by library "lwt_log" in /home/braulio/.opam/4.14.0/lib/lwt_log
-> required by library "websocket-lwt-unix" in
_build/default/vendor/ocaml-websocket/lwt
-> required by executable server in bin/dune:10
-> required by _build/default/bin/server.exe
-> required by _build/install/default/bin/sharer-server
I’m guessing some of my transitive dependencies (lwt_log in this case) were built against the version of lwt that I originally got from opam, and now that I’m trying to build my code against a local copy of lwt, there are conflicts between the two lwts?
Can you think of any ways to work around this problem? Will I have to build “everything” from source if I want to use a local copy of a somewhat low-level library like lwt?
I’m an ocaml beginner, btw, so I don’t have a very good mental model of how ocaml builds work.
In this case, it looks like dune doesn’t know whether to use your modified Lwt or the one that is installed. You should start by uninstalling it (opam remove lwt). You probably also need to remove all the lwt_* packages and let dune figure out what it needs.
Also, when working on package sources, I advise that from the repo’s root, you do:
opam install ./lwt.opam --deps-only --with-test
opam should let you know if some of your dependencies at not at the right version for the development version of the package that you’re working on, including the dependencies of the test suite (--with-test) and propose to solve the conflicts.
I’m guessing some of my transitive dependencies (lwt_log in this case) were built against the version of lwt that I originally got from opam, and now that I’m trying to build my code against a local copy of lwt, there are conflicts between the two lwts?
Can you think of any ways to work around this problem? Will I have to build “everything” from source if I want to use a local copy of a somewhat low-level library like lwt?
Yes, I believe that is what is happening. One way I was able to solve this issue to have a local opam switch targeting the OCaml version your are developing against. And then not install the packages that you want to vendor in dune. So in this particular case don’t install lwt in the opam switch since you want dune to vendor the lib.
Thank you very much for your suggestions. You were very quick at answering. My bad that I couldn’t get to reply earlier.
I’ve tried deleting lwt from the switch. However, then the problem I have is that opam also uninstalls things that unconditionally depend on lwt, like lwt_logs or cohttp-lwt-unix. However, websocket-lwt-unix (which I have vendored in my project and depend on) is declared as optional as a dune library and depends on those two libraries. So if I want to use websocket-lwt-unix (and I do) I have to make lwt_logs and cohttp-lwt-unix available to my build. I tried to opam install those libraries (using the command that @otini suggested; thanks! that’s a good one) but then that will also install “transitively” lwt.
My guess is that I will have to vendor all of the dependencies that my project transitively depends on which also depend on lwt. So following my example above, I will have to vendor lwt_logs, cohttp-lwt-unix (and other libraries in this situation, there are a bunch because lwt is a foundational library). Does what I’m saying ring true? Or am I missing something?
This is awesome, now I can see my changes in lwt when I build my program. Also I see opam rebuilding everything that depends on lwt, which makes a lot of sense to me.
During my debugging, I guess my workflow will be:
Make changes to lwt (mostly adding prints)
opam pin lwt <path>: opam rebuilds my changes and all packages in my switch that depend on lwt.
dune build my application
(I had to take lwt outside my repository so that dune wouldn’t try to build it - otherwise I’d get the same conflict as in the OP).
It would be pretty cool if I didn’t have to rebuild, “from scratch”, all lwt rdepends when making any change to lwt, and that instead builds were incremental (as much as they could). But I’m guessing opam wouldn’t be able to do this?