Functors for dependency injection?

The basic idea is that functors get an argument when instantiated which stipulates a precise interface of what functions and types they can access. Classes, by default, have no such argument. They could optionally be built to receive such an argument, but it then needs to be carried around by anything that constructs them and things get messy.

First class modules can also serve as a complement/alternative to functors.

One thing that apparently Java DI supports which OCaml does not is runtime resolution of dependencies. This is not a great idea IMO, and is anathema to OCaml’s notion of compile-time resolution.

It’s at runtime, but executed by “framework code” (like Spring) so in fact, it happens before any real user code executes. Might as well be static. The “dependency” in DI pretty much corresponds to a functor-argument, and the “injection” to functor-application.

It’s also worth noting, at least as a side point, that in my experience (in C, C++, python, and some OCaml), interfaces that can mix and match with multiple services are mostly a myth. You end up with terrible hacks to make those interfaces work, and you get stuck with nasty hierarchies that eventually lead to full rewrites. Interfaces as separation of concerns are generally a good idea. As an ability to change components, I rarely see them work out well unless they were deliberately designed with those multiple components in mind, and you’re generally better off refactoring code than trying to get to the mythical place of not needing to modify code.

The nice thing in OCaml is that with modules, you’re not limited to where you place those separations of concerns, wheres with some OOP languages, you’re mostly confined to the class level, which is restrictive (but may not feel that way unless you’ve used modules).

1 Like

Lots of platitudes are being thrown around in this thread about how DI is bad and modules are good. My experience with java leads me to believe that it isn’t so simple.

If you use functors to implement DI (for the purpose of separating interfaces from implementations) you will quickly run into a combinatorial explosion of sharing constraints. It’s bad enough that the mirage team had to create a code generation tool to write them - see functoria.

Classes and objects seem to be more appropriate to solve this problem because they are structurally typed. However, you will still need to wire the dependency graph completely manually as OCaml lacks reflection.

My suggestion would be to learn more about other decoupling techniques, in particular, those that are successful in other FP languages. Free monads and tagless final come to mind. If we ever get typed effects, we’ll have another tool in our arsenal.

4 Likes

Three Four thoughts:
(0) ML sharing constraints express (statically checkable) constraints that are simply inexpressible in Java. The equivalent constraints are only dynamically-checkable in Java, and typically aren’t checked. Programmers can and will make mistakes, and there’s not much there to save them.

(1) I feel that you’re implying that people dissing DI have no (or insufficient) experience with Java, and while this may be true of some, it’s certainly not true of all.

(2) I think that perhaps you’re judging the most complex ML systems, written by rather competent people, and finding them wanting. But most Java systems deployed in the wilderness aren’t written by the caliber of programmer that works on ML [and on this, I have personal knowledge] and furthermore the (ahem) “corporate/career/resume/political constraints” of Java systems force all sorts of awful stuff to get pushed into standards and standard implementations [again, I have personal knowledge; also think about the fact that Apache is -funded- by major corps, hence a resume-enhancement channel for senior architects].

(3) I was talking with a friend who works for a “major Internet streaming video company, all of whose major IT systems are written in Java” and he was lamenting that basically everyone around him is in the mode of “just repeat the incantation that somebody told you and hope it works” and nobody actually knows how any of the enterprise Java infrastructure they’re using is wired-together. He is dinged on his performance reviews for trying to get to the bottom of problems, instead of just applying somebody’s “incantation” and hoping it works.
He specifically cited Spring in this regard.

Look: I get that you’re trying to say “let’s not pat ourselves on the back too much”. But having spent 1998-2008 debugging Java internals, app-servers, and deployed systems, both inside IBM and at probably every major IBM customer in the US and many around the world, having seen how the sausage gets made, I think it’s completely accurate to say that the ways the Java world has invented to cope with complexity are much worse than the ways the ML world has invented.

It could have been different: Java could have adopted a system of static, checkable modularity, and on that basis could have had something not unlike ML functors by now. After all, MSFT’s assemblies are nearly all static. But they didn’t choose that path, and I can -assure- you that it happened for PURELY career-driven reasons: technical correctness didn’t enter into it.

1 Like

It’s not that I disagree with that is said, it’s just that all this information doesn’t seem to be directly relevant to what is being asked in this thread. Put it this way, if I knew Java and was curious about OCaml, I would be just as confused as to whether I can - or should, use functors for DI in OCaml.

Naively restated, the OP is curious about the consequences of porting standard Java code like this:

interface Service {
  Response run(Request request);
}

class App {
  Service service;
  App(Service service) {
    this.service = service;
  }
}

into OCaml written in this style:

module type Service = sig
  type t
  val run : t -> Request.t -> Response.t
end

module App (Service : Service) = struct
  ..
end

I’m telling him that while it’s possible, there are some serious practical disadvantages that explain why the majority of OCaml code isn’t written this way. Discussions about immutability vs mutability, frameworks, whether DI is good or bad, the virtue of ML modules, do not help the OP to understand the trade-offs of this technique.

I probably came off a little too strong than I’ve liked in my first reply, but it just seemed like the OP asked a very concrete question, and received responses containing a lot of fascinating, but largely inapplicable theory crafting about OO and FP.

6 Likes

Well, except that this sort of separation is actually quite tractable in ML. I’ve done a ton of it (as I’m sure others have). Right off the top of my head, I can remember:
(1) a tran-log for a blockchain system, that I wrote unit-tests for, using a mock-ed Filesystem in order to inject data corruption and test recovery (think “the tran-log at the bottom of leveldb/rocksdb”).
(2) for the same, a layer between the tran-log and its client code, that allowed for having multiple instances of the client, but not flushing the tran-log each time the client requested (so I could run multiple client instances in a single process (for testing) without saturating the laptop SSD)
(3) similar tricks to allow the clients (which communicated over Thrift) to have multiple instances run in a single address-space, communicating -not- using Thrift (so faster, but also no network).
That’s just off the top of my head.
The use of functors and modularity to allow this sort of “wiring together in the main program, and not before” is something many systems-builders find quite appealing about ML.

I also use this style when it seems like the weight of the abstraction is justified. It’s just that it’s never my first choice because how heavy it is and the fact that there are good alternatives. In Java on the other hand, DI is definitely my first choice.

1 Like

I’m surprised you think it’s heavy. There’s the code to write for the actual mock, and there’s the interface (signature) to interpose, modifyin the client code to use the operations in the interface, and then writing tests on top of that. But only writing the actual mock is nontrivial, and that’s gonna depend on how much interesting behaviour it displays.

OTOH, for sure in Java, there’s no other choice, so I get why you use DI.

Is there really no other choice? I’m asking because I really don’t know. In C++, you don’t normally write entirely to interfaces unless you really need to for the sake of something like an enterprise codebase where you only own a specific part. What is it about Java that makes this an essential style? Or was it just popularized that much in Java that people over-engineer right away?

I’m not the only one who thinks this way. After all, functoria is a thing and bigfunctors are one of the oldest features on the wishlist.

For a small library that does a lot of wiring, have a look at libmonda. Quite a bit of boilerplate for a fairly small library. I can’t imagine building a large application this way.

Isn’t a big part of the reason for functoria the fact that Mirage has multiple backends and such, all of which cannot be compiled together? So it’s a compilation resolution mechanism at the same time as stringing together the functors.

I agree that there is syntactic overhead for functor application, but for serious codebases concerned with interfacing and especially with being able to substitute large pieces of code for each other (as DI is), I think it’s a good solution. And here is where you can question the merits of DI in the first place, at least as it pertains to most code found in the open source community. In reality, if you’re going to be switching the dependencies around, it probably isn’t a big deal in most codebases to just make small modifications as needed, including constructing other inner classes. FP style functions can be tested in isolation as well. For giant codebases maintained by separate groups within an org, it’s probably a different story.

TL;DR this is gonna have some old war stories, and maybe those uninterested oughta just skip past it.

Ah, therein lies a tale. And I’ll start with the funny part, b/c it’s the part I like best. In the story below, there’s a point where Jim Colson (if I accidentally ran over him, I’d back up and run over him again, the fuckin’ plagiarist) took my code, broke it, open-sourced it, and forbade me to publish the original version (owned by IBM). I was so angry, that when Paul Horn, the Director of IBM Research (and Colson’s “executive mentor”) (by this time, it’s 2005) called me up and told me I had to go clean up a gynormous Websphere nightmare at a big custodial bank (turned out it was AIX, JVM, Websphere, Portal Server, and customer code bugs, took six months to one-by-one hunt them down) I got so goddamn angry I threw my IBM-issued laptop across my kitchen and destroyed it. But I’d already told him I’d go (b/c the alternative was quitting, and stupid me, I didn’t realize that that was the intelligent thing to do) so I went and cleaned up the mess. They never charged me for that laptop, and for all I know, my manager still has the busted hulk in his office.

(1) In C++, lots is possible using templates (which as I’m sure you know, were inspired by SML modules, “back in the day”).

(2) even in C (hence in C++ also) you can use conditional-compilation and linking to write a piece of code that uses some subsystem, and then run it in environments with different versions of that subsystem. It’s very commonly done. I have a (former) friend who wrote a complete S(torage)A(rea)N(etwork)-on-IP system, and at every level from the bottom-to-the-top, he used mocking of various kinds to allow him to test both subsystems and entire deployed systems in single processes with pseudo-random event-clocks for reproducibility.

(3) But in Java, first you don’t have conditional compilation, and second you really don’t have “linking” in any real sense. And this was by design (you can look at the original writeups for Java from SUN, and see them boasting about the lack of #defines, linking, etc. Hell, they didn’t support #line directives, so even IF you wanted to write a preprocessor, you had to jump thru INSANE hoops to make it work (I know: I did it twice).

(4) But even worse, because Java was so damn BIG (and remember, we’re talking a time when a Gig was a LOT of memory on a server) you were forced to use “app-servers” which were meant to stay up for a long time. Even today, I’d bet that it’s rare to see toplevel programs written in Java: almost all Java code is “loaded/managed” by runtimes of some sort. Those runtimes enforce the linking, so you get what they give you.

All of this means that you really can’t use any of the tricks we’re used-to from C to do mocking, or swap-out implementations of some interface “at the last moment”. And since you don’t actually have a static modularity mechanism, you can’t use CLR/C++/ML’s tricks.

[I do take your point that C++ doesn’t have “interfaces”. So you don’t get the typechecking. But you -do- get the “visibility of names” and “access control” that you get with ML functors.]

So the runtime implementors started building DI as a way to get the value of those above techniques, for Java code.

OK, now here’s where I get on my soapbox. IT DIDN’T HAVE TO BE THIS WAY. AT THE TIME, it was clear that MSFT’s assemblies were a fine, fine approach. AT THE TIME (in fricken’ 2003), I wrote a paper[1] (with Corwin, Bacon, and Gross) and an implementation, applying it to Tomcat as a proof, that you could modularize Java servers in a way that allowed you to then apply the tricks above, without some dynamic insanity. But you see, technical correctness doesn’t matter in the Java world: at the time, Jim Colson and the “pervasive computing” idiots at IBM had built this thing called OSGI (R2) that was deeply broken, and they needed a solution. So they did two things: (a) took the implementation and modified it (breaking critical bits) to turn it into OSGI R3, which they open-sourced; (b) forbade me to open-source my implementation, since it was (of course) owned by IBM,and as senior architects, they could do that. It’s a feudal kingdom after all, not a well-run tech company.

I distinctly remember an email and conversation with Glyn Normington (then of IBM Hursley) who assured me that he’d done everything he could to convince them to keep static modularity, but well, them’s the breaks. Glyn went on to SpringSource, of course.

The upshot of this long tale, is that Java still has no static modularity, not even of the kind that the CLR has enjoyed since its inception. So everybody uses DI, and DI is driven by runtimes that take various config-files (last I checked, it was XML) and build graphs of objects (“beans” bleccch) in the runtime. It’s all dynamic, it all depends on the runtime getting things right, nothing is truly statically checked, and (haha) no wonder, to most programmers it’s a black-box (as my friend at that streaming company described to me).

[1] https://dblp.org/rec/conf/oopsla/CorwinBGM03
I don’t claim that this was revolutionary. C#'s assemblies were basically this. Rather, this was something that could be -done- in Java at the time. When you work in “the business” you try to propose solutions that can be implemented now, and don’t require boiling the ocean. That’s all this was.

2 Likes

Just did a quick read of libmonda. How would this be implemented without functors? first class modules?

There’s another (historical) bit that might help explain. Java originally came out of “microservers” (intended to run in light-switches) but it rapidly became clear that its real niche was in transaction-processing servers. There had been a long-standing attempt to re-implement these, based on CORBA and C/C++ … which had failed miserably, and I mean -miserably-. One of the tenets of the entire school, was the idea that you couldn’t extend the programming language: everything was done by IDL compilers and other add-ons. Interpreted config-files were necessary, since they could bridge the gap. And “frameworks” were all the rage.

An aside on frameworks: when architects [spit] propose frameworks, they’re telling you “here’s a set of nouns and verbs, and here are the rules for using them; OBTW, we provide no static checking that you’re obeying the rules: screw up and die a horrible death”. A framework is a substitute for a language, by a weakling who isn’t (wo)man enough to actually design a language (or extension, at least).

In large enterprises, this has a particular appeal, b/c it means that the armies of programmers you spent a lot of money to train, can be repurposed to the next new technology, without having to learn a new language. [ignore that a sufficiently different framework -is- a new language … just ignore that.] So vendors learned “don’t design or offer new languages; instead, just offer new frameworks”.

Frameworks and config-file-driven initialization are almost-necessarily going to force you to runtime configuration. In the context of Java, which purposely (and bravely!) eschews any kind of preprocessing, linking, link-time control, etc, voila voila voila, you end up with something like DI. Because if “a new language” or even “a new extension to the current language” is off-the-table, and you can’t control the environment in which code executes statically at build-time, then runtime configuration is all you have left.

BTW, something else to think about: NOT ONE of the major commercial J2EE app-servers was ever open-sourced. WebLogic? Nope. Websphere? Nope. There’s a good reason for that, even though at this point (and heck, 10yr ago) only suckers use these suckers. The code inside is horrendous. Just. Horrendous. Every nightmare you can imagine, and then some. And yet, the Java standards were and are driven to a great extent by “expert groups” convened from and funded by the big enterprise vendors. So the same people who built these monstrosities, are the ones who come up with these various standards.

1 Like

It’s not really like that anymore. In android, all of this runtime configuration crap had to be thrown out of the window because reflection is far too slow and kills dead code elimination. Both of these are quite important for resource constrained environments. The end result is dagger2: the initialization of your object graph is now generated at compile time via annotation processing. As usual for java, aesthetically, it’s a monster. But I’d like to point out that the java world has moved a little forward since your last stint has an enterprise programmer a decade ago :wink:

Entertaining tales though.

I don’t know how first class modules would help, but bigfunctors would solve this problem trivially.

Well, yes. Dalvik started off with “take a collection of classfiles and grind them down to a single executable”, right? IIRC the original Dalvik didn’t have classloaders, either. And Dalvik was the creation of a single company, a company known for (at the time) technical execution quality. Unlike Java and the Java standards. Is Dalvik today a fully J2SE-compliant platform? I’d guess “no”. [just to be clear: I don’t think this is an error on Dalvik’s part: quite to the contrary.]

This is precisely what I was wondering. Thank you for making it clearer!

I’ve just read up on these two (including your “Free Monads in the Wild” post) and it’s really interesting. Haven’t wrapped my head around tagless final yet, but free monads definitely looks like a nice tool to have on your belt. In the context of decoupling, is it correct that with free monads we would write different interpreters for different implementations (e.g. actual vs. mocks) operating on the same operations?

I feel like this should be an obvious issue but I’m still unsure, would you be able to point me to an example of how or where this might be a problem? And how quickly is “quickly” (i.e. is the actionable “avoid it at all cost” or “we can use it, but don’t go overboard”)?

1 Like

This twenty years old ICFP presentation by Benjamin Pierce makes the case quite well that a fully functorized style does not scale in practice (slides 45-48) and gives you a key insight on when to actually use a functor (slide 27-b).

5 Likes