# (Unit) testing module internals (aka ignoring .mli files)

**URL:** <https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884>\
**Category:** Learning\
**Tags:** testing, build, dune\
**Created:** [November 24, 2021, 8:31pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884 "2021-11-24T20:31:44Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [November 24, 2021, 8:31pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/1 "2021-11-24T20:31:44Z")

</div>

Newly faced with the prospect of testing what will be a public library with a defined interface, I struggled for a bit with how to test the library internals (i.e. those bits not included in what would be defined in an `.mli` file) without resorting to putting those internals in a separate library. In the interim, I simply avoided having an `.mli` to delay having to deal with the question.

Thankfully, I eventually came across this thread/post:

> [@Unit testing with jbuilder](https://discuss.ocaml.org/t/unit-testing-with-jbuilder/563/3):
>
> Daniel, Do you have an example of a project for us to consider? One idea is that if you’re using the wrapped mode for library, you can use the private names for testing (e.g. Lib\_\_MyModule), while not adding them as aliases in your lib.ml file. Otherwise, I agree with the other poster that getting inline tests to work in jbuilder is the way to go in the future.

To spell things out explicitly, this works like so (example file paths in the preceding comments):

```ocaml
(* src/dune *)
(library (name lib)) ; (wrapped true) is the default

(* src/impl.ml *)
(* your library implementation goes here in full *)

(* src/lib.ml *)
include Impl

(* test/test.ml *)
module Lib = Lib__Impl
(* your tests here, which can now use everything in Impl, regardless
   of the contents of any src/lib.mli that might exist *)

```

What’s going on is that the default “wrapping” of the library mangles the public names of all its modules, by default aliasing them into a generated module of the same name as the library. However, you can provide that “main” module (as above), and refer to the mangled names as needed to match any defined interface, _or to access hidden modules and definitions in test contexts_. 🚀 👍

(IMO, this topic deserves proper treatment in either [documentation for `wrapping`](https://dune.readthedocs.io/en/stable/dune-files.html?highlight=wrapped#library) or the more narrative sections on [testing](https://dune.readthedocs.io/en/stable/tests.html), as I suspect few are aware! 🙂 Hopefully it not being particularly documented doesn’t mean this technique is subject to breakage in the future…)

Note that the internal modules’ mangled names _do not_ show up in code completion suggestions in e.g. vscode, I suspect because they are marked as “private” via an advisory mechanism as suggested in [Properly wrap a package’s modules with dune](https://discuss.ocaml.org/t/properly-wrap-a-package-s-modules-with-dune/4404)

---

<div class="post-metadata">

**Author:** ![rgrinberg](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/rgrinberg/32/40_2.png) [@rgrinberg](https://discuss.ocaml.org/u/rgrinberg)\
**Post date:** [November 24, 2021, 8:47pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/2 "2021-11-24T20:47:47Z")

</div>

I do not recommend this workaround anymore. It relies on an implementation detail of dune that is subject to change. What about just introducing a `Private` module in your library that contains the parts that you’d like to test?

---

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [November 24, 2021, 9:02pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/3 "2021-11-24T21:02:20Z")

</div>

Hah, I didn’t expect to get whiplash re: “Hopefully it not being particularly documented doesn’t mean this technique is subject to breakage in the future…” quite _that_ fast!

I don’t think you mean simply referring to `Impl` directly as described above; that doesn’t seem to work?

Or, are you suggesting that library interfaces include a `Private` module that is wildly subject to change?

---

<div class="post-metadata">

**Author:** ![dbuenzli](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/dbuenzli/32/18_2.png) [@dbuenzli](https://discuss.ocaml.org/u/dbuenzli)\
**Post date:** [November 24, 2021, 9:20pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/4 "2021-11-24T21:20:58Z")

</div>

> [@cemerick](#):
>
> Or, are you suggesting that library interfaces include a `Private` module that is wildly subject to change?

That’s a not a bad suggestion. You can also hide it from the API docs by putting it in a [stop comment](https://ocaml.org/manual/ocamldoc.html#ss:ocamldoc-stop).

Another solution, but I don’t know if it’s workable in `dune` is to have your internals in a dedicated module whose interface is hidden at _install_ time by not installing its `.cmi`.

---

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [November 24, 2021, 9:36pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/5 "2021-11-24T21:36:32Z")

</div>

If this approach is actually confluent with community expectations, then it seems ideal…but I have the vague impression that even well-versioned churn is viewed as gauche?

(As I have previously spent a ton of time with technologies that allowed if not encouraged using implementation-private bits, I’ve often wished that OCaml libraries or the language itself were somewhat more lax in what they expose and how. Having the escape hatch available of reaching behind the curtain if absolutely necessary is quite reassuring.)

---

<div class="post-metadata">

**Author:** ![rgrinberg](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/rgrinberg/32/40_2.png) [@rgrinberg](https://discuss.ocaml.org/u/rgrinberg)\
**Post date:** [November 24, 2021, 11:00pm UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/6 "2021-11-24T23:00:34Z")

</div>

> As I have previously spent a ton of time with technologies that allowed if not encouraged using implementation-private bits, I’ve often wished that OCaml libraries or the language itself were somewhat more lax in what they expose and how. Having the escape hatch available of reaching behind the curtain if absolutely necessary is quite reassuring.

I agree entirely. I’m not against escape hatches for testing. Just this particular escape hatch.

First, let’s clarify that there’s actually two problems at play here. The first one is that one would like to get around the barrier that dune allows the users to put up with a “library interface”. E.g.

```auto
(library
 (name lib)
 (modules lib foo bar))

```

The library interface here is `Lib` and it may not allow accessing the module `Foo`. For tests, one would still like to access `Foo` as if `Lib` didn’t exist. That can be done if the alias module (the module which undoes the mangling - `Lib__` in this example) is available. So the problem is reduced to just creating an entry point for this alias module. That can be done in multiple ways, and I can list two possibilities right here:

1. Custom stanza:

```auto
(alias_module_of_lib
 (library lib)
 (name lib_private))

```

This would create a library that just exposes the innards of `lib` if `lib_private` is used instead.

1. Do it as use site:

```auto
(executable
 (libraries lib)
 (expose_internals_of (lib as lib_private))

```

Dune will generate `Lib_private` for you to access the innards of `Lib`.

The second, and more general problem is to allow access to arbitrary modules as if the corresponding mli didn’t exist. I think that also can be achieved but it’s a lot trickier. You need to compile every module against its inferred cmi and maintain yet another generated module that will reseal everything with the actual interface:

```auto
(* This Private__ module will be opened to peek into the internals of the library.

any name will work as long as it doesn't collide with user code. *)
module Private__ = struct
  module Foo = Lib__Foo
  module type Foo = module type of Lib__Foo_intf
end
module Foo = (Private __Foo : Private__ Foo) (* to compile the rest of the lib*)

```

I’m no sure if this is guaranteed to work, and I’m fairly certain you lose out on some optimizations as well. All of this means is that dune needs to work hard and build the entire library again to be used for your tests. What a nightmare.

---

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [November 25, 2021, 1:05am UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/7 "2021-11-25T01:05:44Z")

</div>

Are these options (`alias_module_of_$foo` and `expose_internals_of`) actual options, or only prospective solutions to the general problem? Reasonably-translated stanzas/fields seem to be unrecognized by dune, and a quick spelunk in its sources doesn’t turn up such keywords.

Honestly, unless a churn-prone `Private` module is viewed as truly abhorrent, I’d much rather take that route than try to wedge dune into doing even more backflips (as impressive as they might be).

---

<div class="post-metadata">

**Author:** ![rgrinberg](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/rgrinberg/32/40_2.png) [@rgrinberg](https://discuss.ocaml.org/u/rgrinberg)\
**Post date:** [November 25, 2021, 1:18am UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/8 "2021-11-25T01:18:14Z")

</div>

Yes, they’re only proposal from the dozens of discussions I’ve had about this issue. None of them are implemented because we don’t know which way to go and none of them are particularly easy to implement. The `Private` module is a good workaround that everyone is familiar with. I’d stick to that whenever possible.

---

<div class="post-metadata">

**Author:** ![cemerick](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/cemerick/32/1383_2.png) [@cemerick](https://discuss.ocaml.org/u/cemerick)\
**Post date:** [November 25, 2021, 1:41am UTC](https://discuss.ocaml.org/t/unit-testing-module-internals-aka-ignoring-mli-files/8884/9 "2021-11-25T01:41:49Z")

</div>

> [@rgrinberg](#):
>
> The `Private` module is a good workaround that everyone is familiar with.

Fabulous. Seems like “workaround” is faint praise, since the practice carries all sorts of other benefits.

Thanks for the 10¢; I promise I wasn’t trying to nerd-snipe! 😃