I’ve created a new dune project (with dune init project <proj_name>) for the sake of solving all learning exercises with the hoped-for comfort of automated build system. So far I had no need to tinker with initial dune-project settings, only manual interventions in respective dune files for executable, lib and test.
Now I’d like to play with some 3rd party libraries, so I
ve added manually a new dependency like tsdl in dune-project file at project’s root, expecting particular <package_name>.opam will be re-generated to reflect the changes, while meeting criteria by opam Integration — Dune documentation . Unfortunately this is not the case, .opam remains untouched, ie. out of sync with dune-project changes, after dune build. After an hour reading man-pages for dune and readthedocs documentation, I’ve found no answer how to keep opam in sync. Out of despair I’ve deleted the .opam and dune build finally generated an updated version. Like, really?
Now straight to questions:
what is the correct way to keep in sync dune-project and project.opam files?
what is the correct/idiomatic way to add/remove project’s dependencies?
is there a more idiomatic way to install dependencies then googled-out opam install . --deps-only trick?
is there or is planned an option to install dependencies not globally but per-project?
what are motives to not keep and manage all declarations, recipes and options in a single place like dune-project file but scatter them around the whole project tree and in different formats?
Not sure what happened with your project, but it seems to me you describe the recommanded way of doing it. That’s what I do, and I don’t have to delete the .opam file. If it happens again, maybe posting a repo with the problematic dune-project/opam file would help us to undertand?
is there or is planned an option to install dependencies not globally but per-project?
what is the correct/idiomatic way to add/remove project’s dependencies?
is there a more idiomatic way to install dependencies then googled-out opam install . --deps-only trick?
I think you already found something close to the current optimal solution.
what are motives to not keep and manage all declarations, recipes and options in a single place like dune-project file but scatter them around the whole project tree and in different formats?
Mostly historical, I guess if ocaml tooling was created today it would have a single tool for packaging and building ocaml code.
But OCaml had (and still has) a lot of different build tools used by different people, opam took the view of being build tool agnostic so the whole community was able to switch to it.
Dune generating the opam file automatically is an (good in my opinion) attempt to bridge that gap, but it doesn’t remove the need of an .opam file of the usage of the opam command.
Tooling in ocaml is still evolving, maybe you would be interested in Drom which tries to unify dune and opam a bit more and give a cargo like experience.
I don’t understand what this means. Can you explain further what you expected to happen? What specific package are you referring to when you say <package_name>.opam? What is the contents of your dune-project file?
By the way, I agree with you that these build configs are needlessly complicated and should be simplified.
(lang dune 3.7)
(name playground)
(generate_opam_files true)
(package
(name playground)
(depends ocaml dune tsdl)
^^^ here added a new dependency
From playground.opam
# This file is generated by dune, edit dune-project instead
opam-version: "2.0"
depends: [
"ocaml"
"dune" {>= "3.7"}
"odoc" {with-doc}
]
build: [
["dune" "subst"] {dev}
[
"dune"
"build"
"-p"
name
"-j"
jobs
"@install"
"@runtest" {with-test}
"@doc" {with-doc}
]
]
See no tsdl between .opam file dependencies.
Attempt to invoke dune build or dune build @install to update .opam file, as a side-effect, but it was not changed. No errors. Not found in documentation other dune’s command how to achieve this synchronization.
Why it should matter? I’ve placed open Tsdl in bin/main.ml and got Unbound module error.
Now when I try to replay the case, playground.opam got suddenly updated without its removal. I’ve uninstalled tsdl and other dependencies, removed tsdl from playground.opam, bin/dune and bin/main.ml and dune build now surprisingly put the dependency back in .opam file. Weird.
I’m following along here because I went through this exact same thing a few weeks ago, and I still don’t really get it. What I ended up doing was:
dune install <new_pkg>
opam install <new_pkg>
opam lock
[add new pkg to lib/dune, write code using pkg, commit and push]
[pull updates on different machine]
opam install . --locked
This seems dumb, but it works. I am highly motivated to not have merlin yell at me about unbound modules.
I understand that dune install can add a new package to my top-level project opam file, but then I’m not sure what the “right” opam command to run after that is; meanwhile “opam install <the_same_package_over_again>” is easy to remember and seems to do exactly what I need.
It was very surprising not to be able to find clear guidance on this anywhere. A simple six-bullet-point list like the one above is all I would have needed. I don’t even care what the details are, just tell me what to do!
Yeah, the problem is that opam and dune try to claim they are covering different parts of the developer toolchain to provide flexibility, but in reality package management and build management have quite a lot of overlap, which makes for many sharp edges at the interaction points between the two.
There’s technically not an overlap from the perspective of the two tools doing different tasks, but there is an overlap from the perspective of the user having to give similar information to both the tools separately. As you said, one should drive the other. To that end, I think it would be helpful for dune to infer library dependencies and get them through opam: Automatically add opam dependencies to dune-project file · Issue #7449 · ocaml/dune · GitHub
Basically, dune should treat opam as a service (OpaaS?) to get the build dependencies it needs.