Replacing library wrapper modules with single large .ml files. Bad idea?

Learning
1

Hi, I’m an Ocaml beginner and I would appreciate feedback about this idea related to information hiding :smiley:

Ocaml .mli files offer a great way to declare some definitions as private. However, in a bit larger programs we may want to additionally declare visibility at a higher level, so that a module is explicitly available to a subset of our program. I found that there are multiple ways to achieve this:

  • Approach 1: The ocaml website presents Library Wrapper Modules, which works fine. However, in a way this adds (with my intuition…) unnecessary indirection, making the program more complex.

  • Approach 2 (is there a term for this?): Structuring the project into relatively large .ml “library” files, each containing large number of modules. The mli files would thus expose only some modules from this “library”, similar to the library wrapper module explained above. We can declare inline module signatures to achieve information hiding at the individual module level too.

I think the main advantage of approach 2 over approach 1 is that it’s simpler, but it still achieves same degree of abstraction. One nuisance of approach 2 is that the files would be quite large. But is that inherently negative? Text editors usually have code folding, etc. And I feel that the Ocaml language features support well this style of programming.

3

Depending on the size of the resulting modules, you may run into build-time slowdowns due to the limits this style places on separate compilation. A recent post also suggested some typechecking slowdowns when files contain many modules directly. Whether or not either of these would matter in practice would depend on the specifics of your project, but they’re worth keeping in mind

(I initially misread your post as suggesting the entire project be in a single file, which would make these issues much worse)

4

However, in a way this adds (with my intuition…) unnecessary indirection, making the program more complex

With IDE based go-to-definition, there is no indirection visible to the user. I tested this in VSCode to confirm that if you define f in Cumulus.ml (referring to the example in the docs Libraries With Dune · OCaml Documentation) and then refer to Wmo.Cumulus.f in a client module (e.g., the main executable) then “go to definition” on Wmo.Cumulus.f takes you directly to Cumulus.ml, not Wmo.ml

maybe you could talk about what you mean by “explicitly available.”

If library A is listed in the (libraries ...) clause of the dune file for the library/executable B you’re currently writing, then all modules in A are “explicitly” available in B without any need for an import statement as in other languages. You can refer directly to A.f, you can open A in B or any submodule of B, and so on.

If you have many different modules A1, ... An and you want their contents - or some of their contents - to be available in all modules in your current folder, you could create a helper module A_scope

module A_scope = struct
  let f  = A1.f
  let g  = A2.g  (* Just rebind some of the values you use most often *)
  (* .... *)

  include Ak (* "Smart copy-paste" of Ak into A_scope *)
end

then do open A_scope at the top of every submodule in B to refer to A1.f, A2.g, and so on.

Or, a similar design, which will help to avoid name conflicts - introduce some very short module names

module O = struct
  let f = A1.TooLongToTypeOutEveryTime.f
  let g = A2.VerboseModuleName.g
  module A = Long.Chain.Of.Dot.Projections
end

and then simply refer to O.f, O.g, and do let f () = let open O.A in (...)

5

Approach 1 benefits from this optimization: https://ocaml.org/manual/5.5/modulealias.html.
That is, the module aliases in the library wrapper module allow more granular dependency and linking structure. This means faster rebuilds and smaller binaries when using only some parts of the library, not all.

6

Thanks all. To add more background, I don’t yet have a real project which would really need this, as this is just for learning Ocaml. I’m more familiar with other programming languages, but this is quite interesting topic.

maybe you could talk about what you mean by “explicitly available.” If library A is listed in the (libraries …) clause of the dune file for the library/executable B you’re currently writing, then all modules in A are “explicitly” available in B without any need for an import statement as in other languages. You can refer directly to A.f, you can open A in B or any submodule of B, and so on.

By explicitly available I mean to somehow make it clear for other progammers (and future me) how some parts of a large project are implementation details and some parts are public interfaces. Now when I think about it, the (library …) clause indeed can be informative for similar purposes, with libraries. And I was surprised about that VSCode thing, quite useful.

In a way it feels intuitive if I view the wrapper module is some sort of wrapper interface file. I tried to rename it from .ml to .mli but unfortunately dune doesn’t seem to agree with my mental model, and it errors with due to missing implementation. I think for now I keep using the wrapper modules, i.e. Approach 1.

7

By explicitly available I mean to somehow make it clear for other progammers (and future me) how some parts of a large project are implementation details and some parts are public interfaces.

Maybe this example will be useful.

I create a library, (library (name Abc_test)).

The library has a few files:

  • Abc_test.ml, the main entry point
  • A.ml, A.mli
  • B.ml

A.ml

let x = 1
let y = 2
let z = 3

A.mli

val x : int
val y : int

B.ml

let bx = A.x
let by = A.y+1

Abc_test.ml

module A : sig
  val x : int
end = A

module B : sig
  val by : int
end = B

What is the relationship here?

  • A.z is totally private to A.
  • A.x is totally public to all clients of the library Abc_test
  • A.y is public to other modules within Abc_test such as B (because it’s listed in the A.mli file) but not to clients of Abc_test (because it doesn’t appear in the signature that Abc_test associates to A)
  • B.bx and B.by are public to other modules in Abc_test, but only B.by is public to clients of Abc_test

So in a main executable that imports Abc_test,

let x = Abc_test.A.x
let y = Abc_test.A.y
let bx = Abc_test.B.bx
let by = Abc_test.B.by

the first and fourth lines are legal, the second and third lines will fail.

And modules can be nested, right? Within A.ml I could define

module AA : sig
  val xx : int
end = struct
  let xx = 4
  let yy = 5
end

and then yy is private to AA, xx is public within A.ml but not visible to B.ml, and so on. So you can create a deeply nested hierarchy of modules. The (include_subdirs qualified) dune command is helpful to create a module hierarchy that follows your folder/file hierarchy, where it’s easy for files in the same folder to share with each other, and cousins have slightly more restricted access.

Using functors, more sophisticated patterns of sharing are possible, because A can depend on B (in a sense) even when B is not defined yet. We can do

A.ml

module type Stack = sig
 type 'a t
 val push : 'a -> 'a t -> 'a t
 exception Empty
 val peek : 'a t -> 'a (* raises Empty *)
 val pop : 'a t -> 'a * ('a t) (* raises Empty *)
end

module UsesStack(S : Stack) = struct
  (* some concrete code depending on a hypothetical stack implementation S *)
end

B.ml

module ConcreteStack = sig
  type 'a t = 'a list
  let push = List.cons
  exception Empty
  let peek x = try (List.hd x) with _ -> raise Empty
  let pop x = try (List.hd x, List.tl x) with _ -> raise Empty
end

module Combined = A.UsesStack(ConcreteStack)

So, even though UsesStack depends on a stack, you’re able to invert the ordering here so that UsesStack can be defined independently of ConcreteStack, and you can leave the implementation details of ConcreteStack out of the Stack interface you use when defining UsesStack.