Multiple domains design question in Eio/etc

Learning
, ,
5

OK, I’ve suggested a multi-core runtime strategy, basically:

let () = Par.run @@ env ->
  ...run on multiple cores...

  (* Distribute slices of the array across all worker domains *)
  let result = Par.sum env large_float_array in
  ...

So this takes care of starting up the recommended number of domains and running the app across all of them, while also setting up a way to submit parallelized (ie CPU-intensive) tasks and getting a promise of the result.

This is a POC right now (linked above) but I believe this is a good direction: users don’t need to worry about setting up domains, they don’t need to hand over all of the domains to a specific subsystem like the HTTP server, they don’t need to figure out how many domains to allocate for what.

Of course, this is not thoroughly tested or benchmarked right now; more to come. But happy to discuss more.

1 Like