While playing with dependency injection concept for sihl, I discovered that you cannot have nested modules mixed with recursive modules. What I tried to achieve is the following:
module rec Ctx : sig
include Core.Context
module Authn : Authn.S with type ctx = t
module Users : Users.S with type ctx = t
module Authz : Authz.S with type ctx = t
val empty : unit -> t
end = struct
type t = { authenticated_user: UsersImpl.t option }
module Users = UsersImpl
module Authn = AuthnImpl
module Authz = AuthzImpl
let empty () = { authenticated_user = None }
end
and UsersImpl : (Users.S with type ctx = Ctx.t) = Users.Make (Ctx)
and AuthnImpl : (Authn.S with type ctx = Ctx.t and type user = UsersImpl.t) = Authn.Make (Ctx) (UsersImpl)
and AuthzImpl : (Authz.S with type ctx = Ctx.t) = Authz.Make (Ctx)
Full code you can find here. When running it, it fails:
$ dune test
...
[exception] File "test/test.ml", line 134, characters 50-56: Undefined recursive module
Raised at file "camlinternalMod.ml", line 38, characters 27-65
Called from file "test/test.ml", line 144, characters 16-58
Called from file "src/alcotest-engine/core.ml", line 269, characters 17-23
Called from file "src/alcotest-engine/monad.ml", line 34, characters 31-35
...
Am I hitting some unexpected corner case or is this simply something I should not do, because of a reason (I would appreciate if someone explains me what is the reason)? Or am I simply doing something wrong?
Your main problem is that one of your recursive modules (Ctx) contains pointers to other recursive modules as subfields.
What happens is the following:
All four modules are initialised with dummy values according to their types. We get something like this:
module Ctx0 = struct
module Authn = struct
let authenticate _ = raise Undefined_recursive_module
end
module Users = struct
let get_by_id _ = raise Undefined_recursive_module
let get_by_username _ = raise Undefined_recursive_module
let password_match _ = raise Undefined_recursive_module
let get_username _ = raise Undefined_recursive_module
end
module Authz = struct
let authorize _ = raise Undefined_recursive_module
end
end
module UsersImpl0 = struct
let get_by_id _ = raise Undefined_recursive_module
let get_by_username _ = raise Undefined_recursive_module
let password_match _ = raise Undefined_recursive_module
let get_username _ = raise Undefined_recursive_module
end
module AuthnImpl0 = struct
let authenticate _ = raise Undefined_recursive_module
end
module AuthzImmpl0 = struct
let authorize _ = raise Undefined_recursive_module
end
The main thing you should notice is that Ctx0.Users and UsersImpl0 are two distinct modules (same for the others).
Then patching is done, function by function:
module Ctx1 = struct
module Users = UsersImpl0
module Authn = AuthnImpl0
module Authz = AuthzImpl0
let empty () = { authenticated_user = None }
end
Ctx0.Authn.authenticate <- Ctx1.Authn.authenticate (* problem here *)
Ctx0.Users.get_by_id <- Ctx1.Users.get_by_id (* problem here *)
Ctx0.Users.get_by_username <- Ctx1.Users.get_by_username (* problem here *)
Ctx0.Users.password_match <- Ctx1.Users.password_match (* problem here *)
Ctx0.Users.get_username <- Ctx1.Users.get_username (* problem here *)
Ctx0.Authz.authorize <- Ctx1.Authz.authorize (* problem here *)
Ctx0.empty <- Ctx1.empty
module UsersImpl1 = Users.Make (Ctx0)
UsersImpl0.get_by_id <- UsersImpl1.get_by_id (* Ok *)
UsersImpl0.get_by_username <- UsersImpl1.get_by_username (* Ok *)
UsersImpl0.password_matches <- UsersImpl1.password_matches (* Ok *)
UsersImpl0.get_username <- UsersImpl1.get_username (* Ok *)
module AuthnImpl1 = Authn.Make (Ctx0) (UsersImpl0)
AuthnImpl0.authenticate <- AuthnImpl1.authenticate (* Ok *)
module AuthzImpl1 = Authz.Make (Ctx0)
AuthzImpl0.authorize <- AuthzImpl1.authorize (* Ok *)
Because of the order in which the initialisations are done, the dummy implementation of Ctx.Authn.authenticate gets overwritten by the dummy implementation of AuthnImpl.authenticate instead of the correct one.
It looks like simply changing the order in which your modules are defined would fix the problem here, but I would advise against it (the order of evaluation of recursive modules is not specified). Instead, you should avoid putting recursive modules inside each other, and build the correct hierarchy afterwards:
module rec Ctx0 : sig
include Core.Context
val empty : unit -> t
end = struct
type t = { authenticated_user: UsersImpl.t option }
let empty () = { authenticated_user = None }
end
and UsersImpl : (Users.S with type ctx = Ctx0.t) = Users.Make (Ctx0)
and AuthnImpl : (Authn.S with type ctx = Ctx0.t and type user = UsersImpl.t) = Authn.Make (Ctx0) (UsersImpl)
and AuthzImpl : (Authz.S with type ctx = Ctx0.t) = Authz.Make (Ctx0)
module Ctx = struct
include Ctx0
module Authn = AuthnImpl
module Users = UsersImpl
module Authz = AuthzImpl
end
It makes perfect sense, but unfortunately this simply means that I cannot use this mechanism for what I wanted to solve. My goal was to make functors more simple and do the dependency injection through this context. For example in code for AuthnImpl one could access UsersImpl simply calling through Ctx.Users. This would allow me to reduce all injected dependencies in different functors to be only Ctx instead of listing all of them individually and potentially making functors quite long while with Ctx approach, everything would still be statically checked.
The suggested approach does not help, because composition of different modules is done in the level of final application while different libraries could still need access to features introduced by signatures without knowing the exact implementation. Think of separation of implementation for some domain libraries where they expect interface for Users and Authz without really knowing which implementation for each is picked (Authz could be for example AuthzAclImpl or AuhtzRbacImpl or something similar).
Anyway, thanks for helping me to understand the problem.
One thing I don’t understand with your example (looking at the full code): the various Make functors take as arguments a module of type Core.Context, so they can’t access the submodules of Ctx anyway.
Are the functors just an artifact of the way you constructed a minimal example ?
In any case, if you need something that works, closer to your initial example, I can suggest alternatives based on either first-class modules or functors; both cases will force the “submodules” to be treated as regular values which can then alias correctly with the other recursive modules. But regular submodules in recursive definitions will likely not work as intended.
It’s an experimentation that I wrote quickly based on one failed attempt in the past. It’s not real enough example and I didn’t realize that mistake. Basically each module should introduce their own Context type that includes services they need, but now that I’m thinking of it, it gets even more messy than simply having functors with expected dependencies.
