I have two dune projects and I would like one to include the other as a git submodule. Thus, both projects have their own dune-project file, and (after updating the submodule) one project will be a subdirectory of the other. Is there a way to convince dune for the outer project to be willing to load libraries from the inner project, even though the inner project has its own dune-project file?
I’ve come across several posts online suggesting that this should just work. But when I try it, dune run in the outer project always tells me “library not found” for the inner project. It works as soon as I delete the dune-project file from the inner project, but I don’t want to have to do that; I want that directory to stay clean so it can be updated from the inner project’s git as needed.
You want vendored_dirs (something like that), look it up in the manual. It allows the toplevel project to use other dune projects in the specified sub-directories.
it should work without vendored_dirs stanza though. But you might need to make your library public and reference it using its public name in the outer project.
Supporting this situation is one of Dune’s basic motivations and has been supported pretty much from the beginning. However you must keep in mind that only “public items” are visible across projects. Concretely, only public libraries and executables of a given project will be visible to other projects in the same workspace. The idea is that dropping a Dune project within an existing one works semantically the same as if the project in question had been installed (eg by OPAM) in the current context.
Thanks, it works if I make all the libraries from the subproject public. But I’m disappointed if there’s no other way. These projects are co-developed and tightly coupled, so I think it makes sense for one of them to use private libraries of the other one, and I don’t want to have to make those libraries public.
I guess making the libraries public is not so bad. After thinking about it some more I can see the perspective that if these really are two separate projects, then the libraries from one that I want to use in the other do have to be “public”. Alternatively I could make a third project containing only the libraries and have both other projects depend on that.
The name under which the library can be referred as a dependency when it’s not part of the current workspace, i.e., when it’s installed. Without a (public_name ...) field, the library won’t be installed by Dune.
This is the build-level equivalent of separate compilation. If module A needs something from module B, the latter must publish it in its interface. If lib A needs something from lib B, the latter must publish it.
That doesn’t say anything about what happens when one dune project is included in another as a subdirectory, since in this case neither of them is “installed”.
You are right, it doesn’t say that explicitly. The implicit assumption here is that when you do a dune build, dune makes sure that (public) library dependencies are ‘installed’. So in the case of your outer project, it lists the inner project’s library as a dependency library. So dune would make sure it was ‘installed’, but only if it has a public_name field.
I use git subtree (and may be migrating to git subrepo) rather than git submodule. But I share the original problem.
I have a monorepo with subprojects that are open-source which need independent git histories. Since developing within the parent monorepo project is my primary mode, I want dune to run the tests of the subprojects, and more generally run aliases and private executables of the subprojects … but only the public targets of the subprojects are available. So, I remove the dune-project of my subprojects, manually keep copies of the .opam files in sync with the monorepo, and resort to post-processing when I release to opam (example: DkML / Build Tools / MlFront · GitLab and my cryptic comment in my recent announcements that docs won’t be available for it on ocaml.org).
Anyway, I find the auto-vendoring of subprojects too inflexible. The practical consequences for me are that it is difficult to open-source selective parts of my monorepo code base … so I do open-source releases less frequently. Not the end of the world though.