Tool to compute dependencies between OCaml modules in a dune project?

Hi, I’m looking for a tool that automatically computes the fine-grained dependencies between the .ml files of a dune project. Does such a tool exist already ?

More precisely: given a dune-managed source tree, the tool I’m looking for would give me a dependency graph between the modules (along with actual paths of the .ml or .cmt files) that belong to the dune project, and also dependencies to the modules of the standard library, and to the modules of external libraries as well. In other words : a complete, self-contained dependency graph.

The purpose of this dependency graph is to provide a structured description of a source tree, that serves as a starting point to perform analyses on a complete source tree.

To avoid reinventing the wheel, I have skimmed related projects, that solve similar problems (but not quite the problem I’m looking for). Here is what I’ve found so far:

  • dune describe: this command provided by dune gives a description of a dune project, that includes dependencies at the level of dune libraries or executables. This is already a very good start. Unfortunately, some info are missing (dependencies to external libraries, or to modules of the standard library) and the dependencies are not fine-grained enough (e.g., within a library, the dependencies between .ml files are not provided).
  • dune-deps: this tool produces a graph that describes the structure of a dune project (at the level of dune libraries and executables). It does so by transforming the output of dune describe into a .dot graph. It has the same limitations as dune describe.
  • dune describe + ocamldep: dune produces .ml.d files, that contain the output of ocamldep, and give the modules a given .ml file depends on. Unfortunately, not all .ml have their .ml.d companion file in the _build directory (this is the case of files generated by dune, that have the .ml-gen extension). Also, the dependency info does not include the paths of the modules (though I could probably retrieve the paths, based on the module names and on the include dirs).
  • Use info recorded in .cmt (or .cmo/.cmi) files: here I must admit I do not really understand what is recorded in the list of “imported interfaces” or “required globals”. I am surprised that there are more modules in the “imported interfaces” than what is detected by ocamldep, for instance, and also that a module always seems to belong to the list of its own “imported interfaces”. Is there any extensive documentation on that part somewhere, that I might have missed ?
  • cmgraph: this tool exploits the info recorded in compiled files (.cmo/.cmi/.cmx) to produce a fine-grained dependency graph. Looking at the source code, it seems some heuristic is used to determine which dependencies are actually relevant. I have not been able to compile this tool, unfortunately (it depends on ppx_meta_conv, that requires caml < 4.03.0, for which the installation of omake fails… again, I might have missed an obvious solution).
  • codept: this tool aims at computing dependency information, that is more precise than what ocamldep could provide. While it could serve as a replacement for ocamldep, it does not seem to fit my needs : for example, like ocamldep, it was not designed to understand the structure of a dune project.

Have I missed a tool or technique that is already available or could be easily adapted ? Or have I drawn any incorrect conclusions for some items in the list ? Any feedback and advice is welcome.

I feel that, ideally, the build system (dune) could provide more specific info about the dependency graph : after all, it is dune’s job to discover the dependencies, and an external tool could only do its best to mimic dune’s behavior (and keeping such tool consistent with dune could be painful). Do you think dune describe or the new dune RPC system could be extended to support this ?

3 Likes

I would think “yes”; indeed, it seems dune is the best place for this functionality to live. However, as far as I know no-one is working on this right now. But if you want to give it a try it would be a nice addition. If you go down this road, make sure to open an issue in Dune’s issue tracker where some of Dune’s developers may be able to give you some hints to get started.

Cheers,
Nicolas

Thanks for your answer.
There was already an open issue on that topic for dune, to which I responded a few days ago.
My hope is that the current thread will draw the attention of more dune developers :wink:

Another possibility would be to adapt codept to read dune files.

One advantage of this approach is that it would be possible to extract the dependency graph even when the project cannot be built.

1 Like

This is now implemented in the dune describe command in the latest release of dune 3.1.0.
In particular, you will get the direct dependencies between modules within an executable or library with the command dune describe workspace --with-deps.

3 Likes

I tried it in a project a get some errors like this. Any idea what they actually mean?

Error: No rule found for ahrefskit/.ahrefskit.objs/aggregator.pp.mli.d
Error: No rule found for ahrefskit/.ahrefskit.objs/aggregator.pp.ml.d

also is there some doc on what arguments dune describe accepts in addition to workspace?

That’s interesting… could you please submit a bug report with the steps to reproduce the errors you observe?
There is no documentation indeed for the arguments of dune describe. This must be fixed in the man page and in the “readthedocs” manual as well.
So far, the two available sub-commands are workspace and opam-files.

I randomly tried it on a project and got a similar error:

Error: No rule found for gobview/src/.App.eobjs/App.pp.re.d

Mine involves a js_of_ocaml Reason component, so I thought it might be something exotic related to that, but maybe not. Considering yours, seems more like some issue with preprocessed file dependencies.

We have successfully tested some projects that involve PPXes. More tests are needed, it seems.
Can you please submit a bug report with a minimal reproduction procedure, so that we can make progress on fixing this issue?

This should be fixed in the 3.2.0 release of dune. Please keep me informed if the bug persists.