Dependency Injection alternatives for web development

Learning
1

Hey, I’m used to OOP, but am interested in learning more about FP and in particular using OCaml as a web server.

I am at the very start, but I am already facing some barriers and can’t find good resources on how to solve these very basic and fundamental problems.

I went through the thread about functions as DI and this reply really describes my issue: Functors for dependency injection?

I am used to structuring code through DI like this:

interface Logger {
 info: (message: string) => void;
 error: (message: string) => void;
}
interface Service {
  run: (req: Request) => Response;
}
class MyService implements Service {
  constructor(private readonly Logger logger) {
  }
  run(req: Request): Response {
    this.logger.info("starting run");
    // ...
    return res;
  }
}

What this allows me is to have one logger implementation in development (for example logging to the console), but entirely different logger implementation in production (like sending messages to a remote API);
It allows easy testing as well since we can test MyService by passing fake db/repository implementations, so we don’t need a DB to test our service

Using modules and passing them around is the closest thing that comes to mind, but in the linked article @rgrinberg points out that this comes with its own set of problems and there are better ways.

I have also tried passing functions, for example

  const run = (req: Request, info: (message: string) => void, ...): Response {
    info("starting run");
    // ...
    return res;
  }

but this only creates new problems - now static dependencies need to be wired through every single function, which is very tedious once the dependencies increase.

I have also tried using a service locator where a singleton object is created that can be “queried” for services, similar to: https://github.com/giraffe-fsharp/samples/blob/57128d2d295c06aac2791d01a22fa2291139927d/demo-apps/IdentityApp/IdentityApp/Program.fs#L126

but this also looks like a step in the wrong direction since now instead of listing dependencies in constructors, they become hidden, making testing;

I am starting to feel like classes are so popular, because they are the right solution for this use case,
or I am missing some very fundamental information about how to solve this with FP concepts.

Can anyone please make a concrete suggestion on how this problem can be solved?

2

I borrowed the conceptual solution for this from Clojure and built archi.

It can be used for what you’re describing.

5 Likes
3

It’s very common to use functors for DI. Try it out with a small example and go from there. You may end up never hitting any issues.

3 Likes
4

I use functors for this. It’s pretty easy. A quick example - define a module signature:

module type HTTPClient = sig  
  val request : ?body: string -> Uri.t -> (response, string) result Lwt.t
end

Using optional parameters means you can easily override when testing, but you don’t have to pass it in when using the function normally.

let my_fun_that_makes_http_calls
    ?(client: (module HTTPClient) = (module DefaultClient)) 
    () =
  (* Get a module we can access from the `client` parameter *)
  let module HTTPClient = (val client) in
  (* Use it like a normal module *)
  HTTPClient.request (...)

You could wrap up multiple modules into a record to pass around to ease the burden of passing multiple modules, or perhaps decide at runtime with some sort of flag the set of modules you need to use.

4 Likes
5

We use functors to inject service dependencies in one setup service.ml file. This is the “static” dependency injection. It looks a bit heavy but the it is quite explicit which service depends on what.

(* Kernel Services *)
module Random = Sihl.Utils.Random.Service.Make ()

module Log = Sihl.Log.Service.Make ()

module Config = Sihl.Config.Service.Make (Log)
module Db = Sihl.Data.Db.Service.Make (Config) (Log)

module Repo = Sihl.Data.Repo.Service.Make ()

module MigrationRepo = Sihl.Data.Migration.Service.Repo.MakeMariaDb (Db)

module Cmd = Sihl.Cmd.Service.Make ()

module Migration =
  Sihl.Data.Migration.Service.Make (Log) (Cmd) (Db) (MigrationRepo)
module WebServer = Sihl.Web.Server.Service.MakeOpium (Log) (Cmd)

(* Repositories *)
module TokenRepo = Sihl.Token.Service.Repo.MakeMariaDb (Db) (Repo) (Migration)
module SessionRepo =
  Sihl.Session.Service.Repo.MakeMariaDb (Db) (Repo) (Migration)
module UserRepo = Sihl.User.Service.Repo.MakeMariaDb (Db) (Repo) (Migration)
module StorageRepo =
  Sihl.Storage.Service.Repo.MakeMariaDb (Db) (Repo) (Migration)
module EmailTemplateRepo =
  Sihl.Email.Service.Template.Repo.MakeMariaDb (Db) (Repo) (Migration)
module QueueRepo = Sihl.Queue.Service.Repo.MakeMariaDb (Db) (Repo) (Migration)

(* Configurations providers *)
module EmailConfigProvider = Sihl.Email.Service.EnvConfigProvider (Config)

(* Services *)
module Token = Sihl.Token.Service.Make (Log) (Random) (TokenRepo)
module Session = Sihl.Session.Service.Make (Log) (Random) (SessionRepo)
module User = Sihl.User.Service.Make (Log) (Cmd) (Db) (UserRepo)
module Storage = Sihl.Storage.Service.Make (Log) (StorageRepo) (Db)
module EmailTemplate =
  Sihl.Email.Service.Template.Make (Log) (EmailTemplateRepo)
module PasswordReset = Sihl.User.PasswordReset.Service.Make (Log) (Token) (User)
module Schedule = Sihl.Schedule.Service.Make (Log)
module Queue = Sihl.Queue.Service.MakePolling (Log) (Schedule) (QueueRepo)
module BlockingEmail =
  Sihl.Email.Service.Make.Smtp (Log) (EmailTemplate) (EmailConfigProvider)
module Message = Sihl.Message.Service.Make (Log) (Session)
module Authn = Sihl.Authn.Service.Make (Log) (Session) (User)
module Email = Sihl.Email.Service.MakeDelayed (Log) (BlockingEmail) (Db) (Queue)

Usage would be:

let* () = Service.Email.send ctx email in 
let* user = Service.User.find ctx ~id:... in
...

This looks amazing, we have been working on something similar for “runtime” dependencies but I like archi’s API more, thanks for posting it.

A service defines a lifecycle, which is used to start services in the right order:

  let lifecycle =
    Core.Container.Lifecycle.make "user"
      ~dependencies:[ Log.lifecycle; CmdService.lifecycle; DbService.lifecycle ]
      (fun ctx ->
        Repo.register_migration ();
        Repo.register_cleaner ();
        CmdService.register_command create_admin_cmd;
        Lwt.return ctx)
      (fun _ -> Lwt.return ())

It is a bit annoying that these two are similar and they have to be kept in sync.

3 Likes
6

This is what I do also, and is the “classic” way of using functors to inject DI. Thank you for sharing this. As an aside, this “first class module” stuff has been around a long time, but for me, it’s still “new-fangled” and I’ve literally never used it (maybe I should grin). But your example (to me) has the (also) eminent positive that it’s all static/compile-time configuration.

7

Do note that in Archi your dependencies are type safe. This is probably not highlighted in the current (WIP) Readme, but it should.

The simplest example is:

  let component =
    Component.using ~start ~stop ~dependencies:[ Database.component; Logger.component ]

if your component depends on a database and a logger, they can have different types and the inputs to the start function will be of those types, in the order you depend on them (the dependencies “list” isn’t the stdlib list).

While it “looks” dynamic, it’s still very much statically checked at compile time that you’re depending on the right type of components.

8 Likes
8

Indeed i didn’t see that while quickly checking the readme. This is really useful!

9

Definitely follow the other responses for ocaml, but if you are just trying to imagine alternatives to OO for a typical web database stack, you might find ZIO ecosystem (scala) as an interesting gateway between OO and FP. A coworker and I were recently debating whether to use it to ease the transition for some java developers we worked with.

10

I am thinking about how to replace a service.ml with functor applications with archi, so we have a single source of truth for dependencies. Do you think archi can be used to inject module implementations in an idiomatic way?
So in your example to swap out the Logger without changing Database. We ditched an approach where we pass modules as values in the past, mainly because we didn’t have static typing like archi provides but also because of the unwrap overhead.
And is it possible to use the dependency in some other function than start without using mutable state?

11
  1. You can use modules as components too as long as they conform to the component interface.
  2. Please help clarify if I’m wrong, but it sounds like what you want is “systems” depending on other “systems”. This is possible (see e.g. this test)
12
  1. Looking at your test.ml, I would like to be able to do something like this in the WebServer module:
  ...
  let component =
    Component.using ~start ~stop ~dependencies:[ MariaDb.component ]
 ...

and

  ...
  let component =
    Component.using ~start ~stop ~dependencies:[ PostgreSql.component ]
  ...

without touching the WebServer module. In my thinking, I have to functorize WebServer to take some module of type DATABASE and by doing so duplicating what I express with archi already. Please let me know if that approach doesn’t make sense.

13

When learning OCaml and designing Sihl with @jerben, I experimented using mutually recursive functors for DI. Idea was to have request/service context that allows accessing services that are needed by other services and that would also support cyclic dependencies. Of course, everything checked at the compile time:

module type ServiceContextType = sig
  type t
  module Services : sig
    module Users : Users.S with type ctx = t
    module Authn = Authn.S with type ctx = t
  end
end

module rec ServiceContext : ServiceContextType = struct
  type t = { authenticated_user: Users.User.t option }
  module Services = struct
    module Users = UsersImpl
    module Authn = AuthnImpl
  end
end
and UsersImpl : (Users.S with type ctx = ServiceContext.t) = Users.Make (ServiceContext)
and AuthnImpl : (Authn.S with type ctx = ServiceContext.t) = Authn.Make (ServiceContext)

Here my idea was to make Context application specific thing where each Service simply define what they expect from it and additionally dependency injection is done through Context.Services. This approach would also allow cyclic dependencies between Services that is often design flaw, but in some rare cases really needed.

Unfortunately such structure is not supported, I cannot bind the concrete type of a nested module when using mutually recursive modules. I got a bit cryptic compiler error (@jerben might remember what it was) that lead us reading OCaml compiler code to find out that there is a comment about such structures saying that they are not supported. That snippet of code might have some other mistakes too since, but the main take here is that using mutually recursive functors for DI this way does not work.

Just bringing this up, if someone is planning to try something similar or maybe someone with better knowledge can tell me what I was doing wrong.

1 Like
14

You can either use a functor, or more simply a function that creates a server component from whatever database component gets passed in.

15

Okay then, to confirm my understanding (without libraries for now, and then I’ll check out archi)

The equivalent code of this OOP code https://sketch.sh/s/V2lpLFWV5JKdpwZJRp28EJ/

open Lwt.Syntax

class type http_client = object
  method request : Uri.t -> string -> (int, string) result Lwt.t
end

class type logger = object
  method info: string -> unit
  method error: string -> unit
end

type logger_level = Trace | Err
type logger_config = {
 level: logger_level
}

class console_logger (config: logger_config) : logger = object
  method info m = match config.level with
    | Trace -> Format.printf "[INFO] %s" m
    | Err -> ()
  method error m = Format.printf "[ERROR] %s" m
end

class remote_logger (config : logger_config) (http_client : http_client) : logger = object
  method info m = match config.level with
    | Trace -> ignore (http_client#request (Uri.of_string "https://my-analytics") m)
    | Err -> ()
  method error m = Format.printf "[ERROR] %s" m
end

class type some_service = object
  method request: unit -> unit Lwt.t
end

class my_service (logger: logger) : some_service = object
  method request () =
   logger#info "starting";
   (* .. *)
   Lwt.return_unit
end

type services = {
  logger: logger;
  some_service: some_service;
}

let level = Trace
let createFakeServices () : services =
  let logger = new console_logger { level } in
  {
   logger = new console_logger { level };
   some_service = new my_service logger
  }

let main { logger; some_service } =
  let+ () = some_service#request () in 
  logger#info "done"

let () = (main (createFakeServices ())) |> ignore

with modules would be this: https://sketch.sh/s/M356XsGC05FUYrdsUVo3wF/

open Lwt.Syntax

module type HttpClient = sig  
  val request : Uri.t -> string -> (int, string) result Lwt.t
end

module type Logger = sig
 val info: string -> unit
 val error: string -> unit
end

module type LoggerConfig = sig
  type level = Trace | Err
  val level: level
end

module ConsoleLogger (Config : LoggerConfig) : Logger = struct
  let info m = match Config.level with
    | Trace -> Format.printf "[INFO] %s" m
    | Err -> ()
  let error m = Format.printf "[ERROR] %s" m
end

module RemoteLogger (Config : LoggerConfig) (HttpClient : HttpClient) : Logger = struct
  let info m = match Config.level with
    | Config.Trace -> ignore (HttpClient.request (Uri.of_string "https://my-analytics") m)
    | Config.Err -> ()
  let error m = Format.printf "[ERROR] %s" m
end

module type SomeService = sig
  val request: unit -> unit Lwt.t
end

module MyService (Logger: Logger) : SomeService = struct
 let request () =
   Logger.info "starting";
   (* .. *)
   Lwt.return_unit
end

module type Services = sig
  module Logger: Logger
  module SomeService: SomeService
end

module FakeServices : Services = struct 
  module Config : LoggerConfig = struct
    (* how to avoid defining this twice: *)
    type level = Trace | Err
    let level = Trace
  end
  module Logger = ConsoleLogger(Config)
  module SomeService = MyService(Logger)
end

module Main (Services: Services) = struct
  let+ () = Services.SomeService.request () in 
  Services.Logger.info "done";
end

module App = Main(FakeServices)

what is the advantage of using modules as opposed to classes?

16

@lepoetemaudit what do you mean wrap up multiple modules into a record?

with this passing of modules though:

let my_fun_that_makes_http_calls
    ?(client: (module HTTPClient) = (module DefaultClient)) 
    () =
  (* Get a module we can access from the `client` parameter *)
  let module HTTPClient = (val client) in
  (* Use it like a normal module *)
  HTTPClient.request (...)

is is somehow possible to provide different implementations when running this code in different environments? (see the logger example above) Aren’t we limited to 1 implementation while running (otherwise the module needs to be wired through all functions) ? Perhaps I’m missing something though, I wasn’t aware of this pattern :slight_smile:

18

@mudrz indeed - when I said ‘functors’, I should really have said ‘first class modules’ - I typically use functors to produce these, though.

You are right that the module needs to be wired through all functions, but the default argument means that you only have to provide if you want to ‘override’ a default and passing it around is syntactically not that painful. I find this works well in circumstances where I want the possibility to override during testing, but mostly just have a single ‘main’ implementation. I like how explicit it ends up being, personally.

By ‘passing around modules in a record’, you can bundle all the modules you want to pass around in one go so it’s easier to pass multiple first-class modules around together. A minimal example:

module type Word = sig
  val word: string
end

module type Number = sig
  val number: int
end

type modules = {
  word: (module Word);
  number: (module Number);
}

module Hello = struct
  let word = "hello"
end

module Answer = struct
  let number = 42
end

let default = {
  word = (module Hello);
  number = (module Answer);
}

let use_mymodules ?(modules=default) () =
  let module W = (val modules.word) in
  let module N = (val modules.number) in
  print_endline W.word;
  print_endline @@ string_of_int N.number
    
let () = 
  (* By default, we don't have to pass anything in *)
  use_mymodules ();
    
  (* We can explicitly set the definition of the modules *)
  use_mymodules 
    ~modules: {
      word = (module struct let word = "bye" end);
      number = (module struct let number = 24 end);
    } ()

This does mean that you’ll be dynamically dispatching on the modules at runtime, but I find for simple cases it’s a little easier than functorising all of your code to inject dependencies. You can of course do the same by putting the modules you want into a module as per your example.

If you’re envisaging situations where the dependencies are likely to change radically based on various runtime factors, the proper functor approach is likely the best.

1 Like
19

Thanks @lepoetemaudit this is pretty interesting, I didn’t realize that it’s possible to include modules in records and variants, I’ll look into that;

I’m still trying to understand the advantages of using modules as opposed to objects/classes (above example).
Is it because the wiring is static? (this can be both a pro and a con)

20

I suspect that dependency injection is a bit heavier and clunkier in OCaml with modules compared to typical OOP approaches. In turn I get to use OCaml everywhere else, including my business logic. Seems to be a good trade to me.

I think it could be interesting to do dependency injection with meta programming, maybe with a ppx. We will look into it at some point for Sihl to improve ergonomics. It would be cool to get the runtime information of who depends on who so that things can get started in the right order by looking at the “static wiring”.

Edit: There is https://github.com/mirage/functoria/, a DSL to organize functor applications. Maybe that could be used to generate the dependency graph as well as managing functor applications?

21

Thanks @jerben I’ll check out functoria and Sihl, just trying to understand the general patterns at use in OCaml as well

I really liked this:

On a more serious note, originally we wanted to collect a set of services, libraries, best practices and architecture to quickly and sustainably spin-off our own tools and product.

since as someone evaluating OCaml at the moment - I find it difficult compared to other ecosystems to dive in and be productive, patterns and workflows people are using seem to be a bit hidden (for example another basic problem I couldn’t find an existing answer to: OCaml web server run multiple processes - #2 by roddy )

I suspect that dependency injection is a bit heavier and clunkier in OCaml with modules compared to typical OOP approaches. In turn I get to use OCaml everywhere else, including my business logic. Seems to be a good trade to me.

fair point, what I mean is - what is the disadvantage of using objects and classes in OCaml, vs modules

for example the post above shows the same DI implemented with both classes Sketch.sh - Interactive ReasonML/OCaml sketchbook and modules Sketch.sh - Interactive ReasonML/OCaml sketchbook , both in OCaml.
What I’m trying to figure out is what is the distinct advantage of using the module system as opposed to the objects one?
The module system is the one that is generally preferred in the community, so there must be a reason, but my question is - what is it (don’t want to follow a trend blindly without understanding the pros and cons)?