Hello !
OCaml compiler is justifiably praised for being fast, but fast is never fast enough. Is there any distcc equivalent for ocaml I could entertain the many machines that stay dormant around the house with?
1 Like
If you can find a distributed version of “make”, you can combine that with NFS to and a uniform namespace (e.g. via automount) to achieve this. I used to do this with “pmake + customs” back in the day (1992). But long ago I lost track of pmake, and doubt that the distributed version of it exists. Even so, I would be surprised if there were no distributed make today.
I have a vague memory of some version of gmake that had been modified to use “customs”, but again, it’s only a vague memory.
Are you familiar with Bazel? It is a mature, multi-language build system pinoneered by Google that was designed from the start with distributed builds and distributed/shared build caching in mind.
There has been some work on supporting Ocaml in Bazel: GitHub - obazl/rules_ocaml: A Bazel Language Support Package for OCaml . Not sure how production ready it is.
Isn’t it perhaps possible to share the dune cache among machines ?
I cannot find a mention of a pmake that promise to run jobs on multiple machines, only that pmake which is about local parallelism only (like make -j). I don’t know what “customs” is in that context, and that name from pre-google era does not help. Was it a job “exporter” of some sort? I did a quick search for any current such job exporters and came back empty handed.
That would be one piece of the puzzle but you would still need some synchronizaton between the jobs.
In any case I would favor a build tools that can build more things than just ml→cmx, but that’s arguably a separate concern.
I used to be. The fact that one has to code support for every build chains have always made me think of it as somewhat inferior, but one has to admit that this is precisely the use case that this approach makes possible. Basically, what I’d lke to have is a distributed shell, which is several orders of magnitude harder.
I will have another look at OCaml support, thank you for the link!
My initial idea was to abuse distcc (or icecream) to support OCaml, but what makes things complicated is the dependency on external files. I wish there was a way to have ocamlopt output a single file with both the ml to be compiled and all the information from all needed external cmx/cmi files. The beauty of distcc is that you don’t even have to setup a shared file system.
it’s old stuff: I used it over 30 years ago. It’s possible it no longer exists on the internet. But here’s a reference.
1 Like
Only two things are infinite: the universe and how long nerds cling on to old tech: ftp://linux.s390.org/pub/ThinkBlue/SRPMS/pmake-2.1.33-5.src.rpm 
Too funny! Thank you for that chuckle!
I wonder if that version contains customs support (kinda doubt it does). As I spool thru backup tapes of my brain-state, I have a vague memory that maybe Platform Computing had a “distributed make” once upon a time. Googling, it seems like there have been a ton of efforts to either bolt-on distributed builds to make/gmake, or to build commercial work-alikes.
The only one that seems to have survived is bazel, and sure, it’s perfectly cromulent (I used blaze, and it was lovely). But it’s a -pity- that there’s no distributed version of gmake these days: it’s an obvious thing to want.
ETA: the reason I think it’s a pity is that used-to-be, you wrote your Makefile for running on a single machine, but with parallelism (so, your dependencies had to be correct). And then at the price of arranging a cross-mountable uniform NFS namespace (which you got with autofs and /local→/home mounts), you got a distributed make to soak up all the sweet, sweet cycles on other machines in your LAN. So no change to your Makefiles.