How to avoid creating a gazillion opam files in a multi-library project?

Ecosystem
, , ,
1

We are happy with the way dune/jbuilder magically wraps the modules of a library in one containing module… except that it seems to require creating one opam file per library, as shown in the file tree below:

.
├── Makefile
└── src
    ├── Makefile
    ├── sub1
    │   ├── lib
    │   │   ├── a.ml
    │   │   ├── a.mli
    │   │   ├── b.ml
    │   │   └── jbuild
    │   └── test
    │       ├── a.ml
    │       ├── b.ml
    │       └── jbuild
    ├── sub1.opam
    ├── sub2
    │   ├── bin
    │   │   ├── bar_main.ml
    │   │   ├── foo_main.ml
    │   │   └── jbuild
    │   ├── lib
    │   │   ├── a.ml
    │   │   └── jbuild
    │   └── test
    │       ├── a.ml
    │       └── jbuild
    ├── sub2.opam
    ├── test
    │   ├── jbuild
    │   └── run_tests.ml
    ├── test-sub1.opam
    └── test-sub2.opam

(the corresponding source code is at https://github.com/mjambon/jbuilder-starter/tree/wrapped/sample-application/src for now)

In this setup, sub1 and sub2 are sample libraries. There would be many more in a real project. Assuming we really wanted to keep them as separate OCaml libraries, is there a way to get rid of all the opam files or perhaps merge them into one? If not, is there a way to put them in their own folder to they don’t clutter up the dune project root?

Note that we’d like our tests to be located not too far from the source code they’re testing. This is how we ended up with test-sub1.opam and test-sub2.opam. They are libraries that export test suites, which are run by a single executable elsewhere (src/test/run_tests.ml).

2

You can use sub-libraries (like (public_name foo.bar)), and then have only foo.opam. See for example jbuild files under src/ in https://github.com/c-cube/ocaml-containers

4 Likes
3

What c-cube said is correct, but note that for the case of test only libraries. That is, libraries that are only used to share code between test executables, you should simply not give them a public_name.

2 Likes
4

I am not an expert of how jbuilder dune works, but if you don’t want each library to correspond to an opam package, I believe you can avoid public_name as mentionned, or gather them in a single package.

Note that recent versions of opam allow you to gather all opam files into a single opam/ directory at the root of your source tree. Not sure if jbuilder dune supports it yet, but it should make it possible to avoid too much clutter.

1 Like
5

Thank you all. I implemented these solutions, and this template is now the only one offered by dune-starter (permalink).

The file tree is now:

proj
├── .gitignore
├── Makefile
├── .ocp-indent
├── README.md
└── src
    ├── Makefile
    ├── .ocamlinit
    ├── proj.opam
    ├── sub1
    │   ├── lib
    │   │   ├── a.ml
    │   │   ├── a.mli
    │   │   ├── b.ml
    │   │   └── jbuild
    │   └── test
    │       ├── a.ml
    │       ├── b.ml
    │       └── jbuild
    ├── sub2
    │   ├── bin
    │   │   ├── bar_main.ml
    │   │   ├── foo_main.ml
    │   │   └── jbuild
    │   ├── lib
    │   │   ├── a.ml
    │   │   └── jbuild
    │   └── test
    │       ├── a.ml
    │       └── jbuild
    └── test
        ├── jbuild
        └── run_tests.ml
6

Hm, opam doesn’t do recursive search at the moment, so with the layout above, since the .opam file isn’t at the root of the git tree, nor in an opam/ subdir, you won’t be able to git-pin proj and get the package(s) autodetected (or use opam-publish, etc.). Of course, you could use proj/src directly as your package source, but then opam will use rsync instead of git (there is work-in-progress to allow it, though: https://github.com/ocaml/opam/pull/3174)

1 Like
7

Thanks for the heads up.

This explains why opam was complaining that we weren’t in a git repo (when my build was doing something like searching for the git root).