With the coming of domains and CSP inspired channels, are teams out there experimenting and applying them to any of the available distributed systems libs available? Concurrency, Parallelism, and Distributed Systems | OCamlverse. Is it worth rebuilding a new supervisor-actor system?
This is just an exploratory thought to see how OCaml compares with others out there.
For context, when it comes to the “let it crash” philosophy, Erlang/Elixir and OTP reigns supreme. If an Elixir process crashes due to some faulty system error (maybe a random database connection loss), instead of requiring copious rescue code, a parent process restarts the crashed process to a new, healthy state. Many ecosystems have attempted their own version of the supervisor-actor model.
AFAIK upcoming channels are for inter domain (i.e. inter os thread) communication in OCaml multicore. However we need channels that are less “heavy”. In golang you can have channels between arbitrary goroutines (green threads). In erlang, we can send messages between arbitrary actors (which are a bit like green threads, except fully isolated and pre-emptible).
I’m sure that the abstractions that we’re getting in multicore domains can be used to build the primitives for an actor style concurrency system like erlang and Akka. There will have be to compromises in design – Akka is operating on the JVM which was not really designed to be an actors platform. Similarly for OCaml - there will have to be compromises. (Erlang/OTP is custom built for an actors style system but the problem is that it does not have types which is its main weakness according to me)
However, this space is quite crowded already – Akka is a well funded and professional effort lasting many years. So is Erlang/OTP.
Generally speaking OCaml is a more principled system than the JVM and I can see an actor system implemented in it being a compelling option. But it will require many human years of sustained effort – otherwise it is likely to remain just an interesting experiment.
Just to add one often overlooked actor implementation:
F# - Akka.NET
In my opinion there is still an opportunity for a well-designed OCaml equivalent due to its ‘closer-to-the-metal’ implementation, rich and practical typing, compact but readable syntax as well as a REPL.
Of all the actor systems my personal view is that there really are only 2 - erlang and akka for the JVM that really matter. The rest are experimental or really niche. I don’t know much about akka .NET but its probably a really really small number 3.
Yes there is there is an opportunity for OCaml to have a good actor system here. But:
OCaml ecosystem is still in a huge flux or rather will be in a flux. Multicore will take some time to get merged and libraries will take a year or two after that to get settled. Even though a few people would argue that multicore is not necessary for a Actor like system in OCaml (as most of the time Actors tend to be IO bound and not CPU bound mostly) but in 2021 if you’re building a brand new actor system your chosen language runtime should be capable of true parallelism (a personal opinion). Also monadic concurrency is not a very user friendly. Here OCaml effects should help but that again will come with an unknown delay after Multicore is finally merged
I don’t know the history of Akka but today it is a professional effort run by a company (lightbend). An OCaml effort is unlikely to get corporate backing due to the smaller size of the OCaml ecosystem. A passionate band of contributors could come up with something in a couple of years that becomes a phenomenon (corporate backing be dammed). That is always possible.
OCaml’s small contributor pool currently is another limiting factor. However, though I may sound pessimistic I think that OCaml actually has an extremely bright future ahead of it. Its technical foundations are actually quite strong and in some respects better than its often compared cousin Haskell (strict by default, simpler runtime, simpler language, no bottom in every type, generally a more nimble runtime, less type jugglery, fast compile times and the list goes on) but it is still remains very principled. It hits the right spot for me at least.
TL; DR - OCaml is the right language for all kinds of systems of the future but first the language needs to get to the future – fast! (multicore and effects specifically speaking).
Also Gleam and Caramel are fundamentally flawed options in my opinion. If you’re running on the Erlang platform then you’re mainly there for the OTP/Supervisors ecosystem which is sadly all untyped. So if you’re using Gleam and Caramel you’re doing type safe stuff only in a very limited space – as soon as you get into OTP/supervision/message passing etc. things rapidly become untyped. So might as well just use Erlang / Elixir is my point of view and not bother with Gleam and Caramel… If anybody has a counter viewpoint on this – I’d love to hear about it…
Its a bit like random Javascript library <==> Elm interop or random Javascript library <==> Reason interop. One end is untyped and the other end is typed. So there is this transition zone between the typed world and untyped world which is always full of compromise in terms of type safety within the typed world.
Thanks for pointing out the Gleam OTP library, I should explore it and the Gleam ecosystem more to form a more informed opinion. I don’t know much about it currently but my guess was that since the underlying OTP platform is un-typed there is only that much you can do. I could be wrong.
I was focused on akka for a little while. At that time, I concluded that I would rather use stateless services that use a distributed cache for most use cases. I would consider actor systems for clustered stateful service for a chat system or online event ticket system, where state is mostly held in memory.
To that end, I would focus ecosystem energy on any ideas for improving how much time it takes to implement new stateless services, interfacing with distributed caching systems, and utilities to implement cqrs and related patterns.
Yes, Akka (2.6.x) now uses typed actors by default. The old akka or akka classic had untyped actors which sort of defeated the point of using scala i.e. strong types!
Cool! It’s awesome that you’ve used Akka. I agree that actors make sense only when its important to have state in memory. Most of the time we should avoid keeping state in memory and can just let the db (or similar) worry about it.
For me the main benefit to Akka is static typing and benefit to Erlang/OTP is the low tail latency. Both are important too me so its quite a quandary!
Yes, I believe you may be referring to dialyzer . You can use it in Elixir also.
It is supposed to help a bit but its not super effective like a typical type system is. There are a lot of resources available across the internet that go into detail about dialyzer and the kind of bugs it catches / does not catch.
I’ve spent a fair amount of time writing Erlang and Elixir code, and have a love hate relationship with dialyzer.
In Erlang I’ve found it to works very well in practice; but I can’t say the same about Elixir. That largely (I believe) comes from Elixir’s polymorphism and Enumerable.t type def.
RE: the effort to build a reasonable, stable and usable actor system with domain’s and effects. I don’t believe requires the years worth of effort to get right at this point. The problem space is reasonably well understood and many of the ideas transfer well into oCaml. That said, I’m not sure who would build it or adopt it. I think most of the community is working in problems spaces where it’s actors are not generally the right solution.
@pmonson711 BTW did the lack of true static typing in Elixir make it difficult to build your system and refactor with confidence? Or were you able to get by more or less fine?