(Unit) testing module internals (aka ignoring .mli files)

[cemerick]

Newly faced with the prospect of testing what will be a public library with a defined interface, I struggled for a bit with how to test the library internals (i.e. those bits not included in what would be defined in an .mli file) without resorting to putting those internals in a separate library. In the interim, I simply avoided having an .mli to delay having to deal with the question.

Thankfully, I eventually came across this thread/post:

Unit testing with jbuilder

Daniel, Do you have an example of a project for us to consider? One idea is that if you’re using the wrapped mode for library, you can use the private names for testing (e.g. Lib__MyModule), while not adding them as aliases in your lib.ml file. Otherwise, I agree with the other poster that getting inline tests to work in jbuilder is the way to go in the future.

To spell things out explicitly, this works like so (example file paths in the preceding comments):

(* src/dune *)
(library (name lib)) ; (wrapped true) is the default

(* src/impl.ml *)
(* your library implementation goes here in full *)

(* src/lib.ml *)
include Impl

(* test/test.ml *)
module Lib = Lib__Impl
(* your tests here, which can now use everything in Impl, regardless
   of the contents of any src/lib.mli that might exist *)

What’s going on is that the default “wrapping” of the library mangles the public names of all its modules, by default aliasing them into a generated module of the same name as the library. However, you can provide that “main” module (as above), and refer to the mangled names as needed to match any defined interface, or to access hidden modules and definitions in test contexts.

(IMO, this topic deserves proper treatment in either documentation for wrapping or the more narrative sections on testing, as I suspect few are aware! Hopefully it not being particularly documented doesn’t mean this technique is subject to breakage in the future…)

Note that the internal modules’ mangled names do not show up in code completion suggestions in e.g. vscode, I suspect because they are marked as “private” via an advisory mechanism as suggested in Properly wrap a package’s modules with dune

[rgrinberg]

I do not recommend this workaround anymore. It relies on an implementation detail of dune that is subject to change. What about just introducing a Private module in your library that contains the parts that you’d like to test?

[cemerick]

Hah, I didn’t expect to get whiplash re: “Hopefully it not being particularly documented doesn’t mean this technique is subject to breakage in the future…” quite that fast!

I don’t think you mean simply referring to Impl directly as described above; that doesn’t seem to work?

Or, are you suggesting that library interfaces include a Private module that is wildly subject to change?

[dbuenzli]

@cemerick

Or, are you suggesting that library interfaces include a Private module that is wildly subject to change?

That’s a not a bad suggestion. You can also hide it from the API docs by putting it in a stop comment.

Another solution, but I don’t know if it’s workable in dune is to have your internals in a dedicated module whose interface is hidden at install time by not installing its .cmi.

[cemerick]

If this approach is actually confluent with community expectations, then it seems ideal…but I have the vague impression that even well-versioned churn is viewed as gauche?

(As I have previously spent a ton of time with technologies that allowed if not encouraged using implementation-private bits, I’ve often wished that OCaml libraries or the language itself were somewhat more lax in what they expose and how. Having the escape hatch available of reaching behind the curtain if absolutely necessary is quite reassuring.)

[rgrinberg]

As I have previously spent a ton of time with technologies that allowed if not encouraged using implementation-private bits, I’ve often wished that OCaml libraries or the language itself were somewhat more lax in what they expose and how. Having the escape hatch available of reaching behind the curtain if absolutely necessary is quite reassuring.

I agree entirely. I’m not against escape hatches for testing. Just this particular escape hatch.

First, let’s clarify that there’s actually two problems at play here. The first one is that one would like to get around the barrier that dune allows the users to put up with a “library interface”. E.g.

(library
 (name lib)
 (modules lib foo bar))

The library interface here is Lib and it may not allow accessing the module Foo. For tests, one would still like to access Foo as if Lib didn’t exist. That can be done if the alias module (the module which undoes the mangling - Lib__ in this example) is available. So the problem is reduced to just creating an entry point for this alias module. That can be done in multiple ways, and I can list two possibilities right here:

1. Custom stanza:

(alias_module_of_lib
 (library lib)
 (name lib_private))

This would create a library that just exposes the innards of lib if lib_private is used instead.

2. Do it as use site:

(executable
 (libraries lib)
 (expose_internals_of (lib as lib_private))

Dune will generate Lib_private for you to access the innards of Lib.

The second, and more general problem is to allow access to arbitrary modules as if the corresponding mli didn’t exist. I think that also can be achieved but it’s a lot trickier. You need to compile every module against its inferred cmi and maintain yet another generated module that will reseal everything with the actual interface:

(* This Private__ module will be opened to peek into the internals of the library.

any name will work as long as it doesn't collide with user code. *)
module Private__ = struct
  module Foo = Lib__Foo
  module type Foo = module type of Lib__Foo_intf
end
module Foo = (Private__Foo : Private__Foo) (* to compile the rest of the lib*)

I’m no sure if this is guaranteed to work, and I’m fairly certain you lose out on some optimizations as well. All of this means is that dune needs to work hard and build the entire library again to be used for your tests. What a nightmare.

[cemerick]

Are these options (alias_module_of_$foo and expose_internals_of) actual options, or only prospective solutions to the general problem? Reasonably-translated stanzas/fields seem to be unrecognized by dune, and a quick spelunk in its sources doesn’t turn up such keywords.

Honestly, unless a churn-prone Private module is viewed as truly abhorrent, I’d much rather take that route than try to wedge dune into doing even more backflips (as impressive as they might be).

[rgrinberg]

Yes, they’re only proposal from the dozens of discussions I’ve had about this issue. None of them are implemented because we don’t know which way to go and none of them are particularly easy to implement. The Private module is a good workaround that everyone is familiar with. I’d stick to that whenever possible.

[cemerick]

@rgrinberg

The Private module is a good workaround that everyone is familiar with.

Fabulous. Seems like “workaround” is faint praise, since the practice carries all sorts of other benefits.

Thanks for the 10¢; I promise I wasn’t trying to nerd-snipe!