On concurrency models

Thanks @polytypic for your comments, very much appreciated!

Some are likely spot on. For example I’m not super attached and convinced on the current blocking mecanism (as is written in my todo) – in fact I would like eventually to get to a compositional synchronisation mecanism like CML provides. But I’m not sure your objections about the structure I want to enforce are as problematic as you make it sound (or that I’m not willing to pay a reasonable cost for it in exchange of a good mental model), but we likely have different agendas here and, more importantly, the proof is in the pudding, and there is absolutely no pudding yet[1] :–)

For the rest of the larger discussion, I feel myself a bit off. I’m mostly interested in ergonomics and design of usable systems and good mental models for programming, none of which I find either in this discussion or the systems that are being talked about.

But I just want to emphasis that:

The day your application needs to pull two libraries each relying on another concurrency model, then you don’t really get a choice. And that is the problem.


  1. I usually do not pre-announce work. Affect got out just to provide a design counter point in the long discussion that followed eio’s first release. ↩︎

6 Likes

The Picos sample libraries actually contain an implementation of the Event API. What is missing is an implementation CML style channels with rendezvous. The Computation abstraction of Picos has a Tx module allowing one to transactionally complete multiple computations. (This actually gives more expressive power than is required for CML style 2-way rendezvous.) With those it is not super difficult to implement a CML style channel, but it is difficult to make one that is fast and space efficient. (Somewhat related, there is also a PR that changes Kcas to use Picos.)

Ah… But that is something I and many other people also care a lot about. The thing is that this is where things tend to get highly opinionated.

The Picos core library is not primarily designed for ease of use. It is designed to have low overheads and to allow efficient implementation of higher level models. And one of the reasons for this is indeed that it should allow one to implement various different models efficiently.

Also, as the author of Picos, I don’t necessarily want to critique the higher-level models proposed by others. I definitely have opinions on the merits of various models. For example, Miou has a similar strict parent-child constraint as Affect, but I don’t recall publicly critiquing Miou’s model as such in that respect (previously). Ideally I’d like people to be informed about the various programming models so that they could choose wisely. As the author of Picos I don’t actually want to prevent or talk people out of implementing whatever they like — the main thing I care about is that I want any such implementations to be correct and use Picos correctly. The reason I chose to critique the Affect model in this case and in this respect is that, while it might be Ok in many cases, the strict parent-child restriction is really not a model I’d want to see to become the one lowest-level model available. The performance issues should be quantifiable. The convenience aspects are more opinionated.

BTW, for parallelism one actually wants an even more restricted fork-join model, but there the goal is to limit expressiveness sufficiently that a more efficient implementation becomes possible.

4 Likes

Aaah but this is where things finally get interesting! Now we talk about design which is about saying no and making choices – in my view there is no “opinionated” system, there are those systems that are designed and those that are not :–)

So I’m interested on the why of:

(In affect I did not rule out allowing to detach parallel asynchronous function calls from their parents. I can see it as being useful, albeit it think should be the exception not rule. In affect I just want to introduce one concept that I feel is natural for OCaml: parallel asynchronous function calls; the strict parent-child relationship (a.k.a “structured concurrency” nowadays) fits then naturally with synchronous function calls)

1 Like

For those interested, we’ve spent some time writing a book on how to use Miou and asynchronous programming with Miou — basically, it introduces Miou’s design. In addition, resources that may be of interest are:

  • a retrospective of a handheld scheduler implementation compared to what Miou offers
  • a manifesto that says the same thing as what I said above

A next release of Miou is in preparation and additions to this ‘little book’ will be made. In particular, there will be an explanation of how we implemented happy-eyeballs, which remains a good example.

5 Likes

Sorry but I have not been following this discussion too deeply, but I wanted to bring up something here about our our library, Abb, works, which is derived in a good chunk from your earlier work with Fut. We have a parent-child relationship when executing work. However, as pointed out, sometimes it is necessary to adjust that relationship, so we have a fork function:

val fork : 'a t -> 'a t t

That has worked pretty well (but really battle tested with anyone other than me).

1 Like