On concurrency models

Community
,
1

Indeed it’s the only way to get to a real compositional, multi-core friendly eco-system.

It’s quite clear that the current breed of multi-core abstractions and their various pools, is totally ill suited. And picos, despite all the excellent work by @polytypic that goes in it, by enabling them to live along each other is making things even worse: there’s nothing you can do when you have multiple, very different, concurrency models in your program (e.g. IIRC miou checks stuff about your ressources dynamically). Basically it’s not solving the right problem, the problem is not to enable concurrency model to co-exist, the problem is that there are multiple concurrency models :–)

Effect based, user defined concurrency, sounds like a good and cool thing, but I think they are a real disaster for the language and eco-system. The concurrency model should be mandated by the language and be a property of the runtime system. That in turn allows programmers to develop proper compositional mental models for it and dedicated tooling for dealing with the curse of concurrent programs.

With respect to this while I’m curious about Eio’s capability system (though I acknowledge the fact that it seems users are finding it painful) I’m definitively not convinced about its concurrency model (which is a separate thing).

I recently made a new design round on affect and tried to pin down a concurrency model that can fit on a page and hopefully in the user’s head. It tries to minimize the number of concepts (it seems programmers are extreemely keen on inventing new terminology and structures when they have to deal with concurrency) and fit into OCaml’s exisiting paradigms. Namely we introduce just one thing: parallel asynchronous function calls (in fact things like “structured concurrency” become self-evident when you think in term of function calls, a function always returns after all its functions have executed those being synchronous or asynchronous). See the design notes. There’s nothing ready to use yet but I think it’s the kind of fixed concurrency model I would like to eventually program with in OCaml and being mandated by the runtime system.

8 Likes
2

I don’t particularly have the motivation to write a long post again, but I’m a bit vexed by this specific part and the trivial dismissal of existing solutions. Dedicating entire domains to each library is indeed terrible, but I’ve been proposing to use Thread.t pools on top of a single domain pool for a year and a half now. We have the one good abstraction and it’s called threads. All we need is to entirely forget about the bad abstraction named Domain.t, treat it (like Go does) as an implementation detail, just like a CPU core, and provide libraries with:

val pool_size : unit -> int

val spawn_on : int -> (unit -> 'a) -> 'a Thread.t

and that’s it. That’s all you need. I have an implementation of that in moonpool, but it really should belong in the stdlib.

5 Likes
3

It’s not a trivial dismissal, I’m just not convinced by what you propose. I don’t think that making explicit scheduling choices on pools (Fut.spawn ~on) is a good idea and compositional.

4

I’m not even proposing to make moonpool the standard (it’s likely too big for that), so Fut.spawn ~on is out.

I’m more specifically talking about using the domain pool underlying all thread pools to start threads. Threads are compositional, domains are not, that’s all there is to it.

1 Like
5

It’s roughly the direction taken by Java and the new virtual threads abstraction. By reusing threads, they allow the programmers to keep a abstraction they are already familiar with, so no need for a new terminology :slight_smile:

6

One advantage of a library like Picos is that it enables interoperation for users of a concurrency library, e.g. you can have multiple data-structures or libraries that want to expose a concurrency-friendly abstraction.
With monads you “only” had about 2 major concurrency libraries to support (and that already was a maintenance burden), but there are already 7 or more effect based concurrency libraries. Unless your entire application can be implemented by just using what the concurrency library/framework provides, you’ll likely want to use multiple libraries written by different people. And if each of them picked a different concurrency library then you’d be a bit stuck if you want to write an application, you’d have to port at least one of them to another framework.

I agree that you should probably only have exactly one effect scheduler (or “engine” as Lwt would call it) in an application (except for transition or testing purposes).

No disagreements there, the interoperability of multiple effect based concurrency models belongs in the standard library (and I think one of the longer term goals of Picos is to explore what that interface should be). But it is probably better to explore the design and interoperability for such a library externally first, and once it has proven to have been adopted/ported by multiple frameworks then adopt it into the standard library. That way we know it is not missing some fundamental abstraction that would require an incompatible rewrite.

I think it is useful to make a distinction between a concurrency programming model/semantics (where clearly no single design can satisfy everyone’s requirements or we wouldn’t have 7 alternatives already), and a concurrency runtime (or interop interface).

E.g. you don’t need to make a choice in the runtime on what cancellation semantics you want, or whether it is supported at all, or how to deal with resources, etc.
What you need is a cooperative way to block, to spawn new fibers, and if you implement cancellation then a way to cooperatively handle that.

i.e. there needs to be an agreement on the interface between users of concurrency abstractions.
Looking at Picos this amounts to 5 effects GitHub - ocaml-multicore/picos: Interoperable effects based concurrency

I’ve been thinking whether this is indeed the minimal set, or whether it could be made even more minimal, but I don’t have good idea to propose a change here. Every time I think that one effect could be removed, I can come up with a counterexample of an application that is not implementable if that effect is removed.

For example, although Yield is very useful for testing, is it really required for an application? OTOH if you can’t deterministically test your applications then you won’t have a (well behaving) application. Can you express the semantics of Yield by an artificial trigger that self-triggers after one scheduler cycle?

Similarly does Fiber.Current really need to be there? It is useful for fiber-local storage and cancelation, so I don’t see a good way to remove it, although it feels like it should be more implicit.

3 Likes
7

Big disagreement here :–) There should be no multiple effect based concurrency models.

I don’t think that trying to make a distinction between the language, the concurrency model, and the runtime is productive.

The details of these primitive effects you are talking about shouldn’t even matter, it should be abstracted by the standard library and only perhaps exposed to those who want to write new schedulers (and which you can perfectly indicate you are going to break them if you are unclear at what is the best way to implement your concurrency model so far).

Programming concurrent/parallel programs is a mess. You want your language to mandate one way of describing these programs, that is one model. At least that’s what one would expect from a helpful programming language. You don’t want seven models, i.e. seven time the mess. You need to make a choice, you need your language to be designed, not to be defined by piling up features.

Once your programming language has defined a fixed concurrency model, you can actually optimize your runtime system for it and develop tools for it to help programmers deal with the mess. It will also allow them to shape mental models for it and work with it in a compositional manner.

When I wanted to clarify the semantics for cancellation in affect this autumn, my searches lead me look to look into Swift’s concurrency model and trying it a bit I was quite surprised by all the hints the compiler would give you about data race safety, there’s a reason for that: a single, mandated, concurrency model.

I don’t think that allowing seven different concurrency models and the standard library providing tools to enable them to co-exist makes OCaml attractive in any sense. On the contrary, it rather shows that it keeps on repeating the same errors.

6 Likes
8

I don’t even agree with that :slight_smile: . I think an IO oriented scheduler, and batch processing scheduler, have different design tradeoffs, and I want to freely mix them in an application.

It’s also useful for fairness, if you’re running a costly task on a shared scheduler it’s good to yield regularly to give other fibers a chance?

That’s how I think of picos. Picos takes the mantle from the dusty floor where the core OCaml team had left it, and defines a reasonable concurrency model using fibers, computations, and triggers. You can build more exotic stuff on top of it (structured concurrency for example, or fork-join) but it gives everyone a shared meaning of what it means to complete/cancel/await a computation.

For sure, fragmentation sucks. But it’s going to keep happening unless there is actually a language-defined concurrency paradigm (my forecast for this is circa 2047) or unless we agree to build a common foundation, ie picos. :person_shrugging:

4 Likes
9

And even beyond, given the fact that OCaml ships with a standard library and metaprogramming system, and there are third-party alternatives even to those :wink:

1 Like
10

I think we all agree with @dbuenzli that it would have been much better for OCaml 5 to have determined a single concurrency model. Given the fact that it didn’t, we need to try and do the best we can to whatever degree is possible.

I’d like to highlight some additional issues/thoughts:

  1. Figuring out which model is best via competition outside the runtime sounds nice in theory. In reality, there are very few clients for this stuff. Most OCamlers (and probably most of the core team) aren’t going to be touching much of this anytime soon. Among the applications within the domain, lwt still dominates. So instead of a good, solid evolutionary process, we’re getting a process of genetic drift, where the decisions of a few designers and a tiny crowd of users is causing effects that will have major consequences for the ecosystem later on down the line.

  2. @c-cube and I were discussing this earlier, and we have 2 main competitors right now: eio and miou (as well as moonpool, which chooses to do things very differently). eio has tacked on controversial design decisions such as capabilities - as if we actually needed more complexity right now - and also doesn’t really have a coherent multicore story. miou has a better multicore story but has its own controversial decisions, such as supporting only select. Does miou support work stealing? Is it really optimized for multicore? Not sure. The idiosyncrasies in these 2 systems are precisely the kind of thing that creates ‘genetic drift’ – odd decisions that determine our future and can hurt the community long term.

  3. AFAICT, the handling of the limited resource called domains ultimately means that no matter what you do in the concurrency space with effects, you want to have one resource manager for domains, which ultimately means one scheduler. You also want to have work stealing if you can have it, which again means tight integration between effect-based scheduling and domain management. It also means a scheduler has to have a solid multicore-management story.

  4. We don’t have to/want to reinvent the wheel. OCaml lacks systems/multicore people (both producers and consumers), and we’re late arrivals to this space. The most logical thing for us to do would be to copy the things that worked well in other ecosystems. The most prominent example I know of is tokio in Rust. Looking at the 2 major alternatives, we lack an alternative that is as good as tokio (I’m not very experienced with it myself so I’ll let @c-cube elaborate). Again, moonpool is neat, but I don’t think we want threads to remain our main concurrency primitive, even if it’s often convenient.

4 Likes
11

First of all, thanks for the compliment!

Looking at the API that Affect proposes, it basically gives you 'a Fiber.t that you can start and await on (and a blocking interface).

With Picos you can easily have that same interface to start a fiber and await on it. That fiber can then internally use an implementation of some desired concurrency model. This model does not need to be visible outside of the fiber.

This ability can be rather easily demonstrated. IOW, it is easy to demonstrate that you can have a program that uses multiple different concurrent programming models in a single program that does something. And, if one keeps the interfaces of concurrent abstrations “conservative”, it can even be entirely reasonable and practical (i.e. individually developed concurrent abstractions may internally use different models and the details of those models do not need to leak to clients).

What makes you believe there is nothing you can do when you have multiple different concurrency models in your program?

It is not difficult to implement a concurrent programming model essentially like the one provided by Miou using the Picos interface.

In fact, I wrote one such draft months ago already when I was thinking about making certain extensions to the Picos interface. The conclusion I came to with the draft was that some of the extensions, which I initially thought would be needed, weren’t actually needed for the particular use case (of providing a Miou style model) and so that draft has been discarded.

Picos basically specifies an abstraction of a scheduler. The way I put it in my OCaml workshop talk is that the Picos interface “trivializes” the scheduler. I’ve noticed that scheduler authors haven’t necessarily thought about having such a minimal and unopinionated interface to (or abstraction of) just the scheduler/multiplexer and have a much more monolithic design and implementation where various features have been implemented at any convenient point without necessarily thinking about how and whether things could and should be modularized.

As a particular example, the Picos interface specifies that the scheduler, or the multiplexer, as I tend to call it, must give you unstructured fibers (i.e. there are no built-in parent-child relationships or other lifetime restrictions). Any desired structuring constraints must then be implemented outside of the scheduler and the Picos interface is specifically designed to give the tools for that. This is a key idea that allows different programming models to coexist. Picos allows one to essentially define “scopes” that delineate different structuring models and gives boundaries to propagation of cancelation.

Now, if you build a concurrent programming library, like Affect or Miou, and e.g. absolutely insist that only structured fibers are provided (by the multiplexer), then that will make your library pretty much incompatible with Picos. That is one of the reasons why Miou cannot be Picos compatible. It is not because a Miou style model could not be implemented in Picos or made Picos compatible.

Indeed, that can be considered a problem. How do you plan to arrive at the one winning model?

An idea with Picos is to allow multiple models to coexist and allow the winning model(s) to develop over time.

Evolution instead of revolution. Cooperation instead of competition.

6 Likes
12

Since you appear to be putting out Affect as a candidate to be the one model to rule them all, then allow me to give some feedback on it.

Note that I haven’t spent a lot of time looking at Affect so maybe I’ve misinterpreted some things. I’ve been busy with travel and other things in the past couple of days.

You seem to be making several design choices that seem problematic to me.

First of all, I believe that making the parent always responsible for awaiting direct children is a problematic idea, because it is often cumbersome and unnecessarily restrictive, and it also increases the number of synchronization points, which are expensive. It is actually rarely necessary to have a promise to hold a result from a fiber and promises are actually relatively speaking rather expensive (requiring allocations and synchronization that could be avoided). Scoping constructs like Switches, Bundles, Flocks, or whatever you want to call them, and other constructs that start and await multiple fibers, allow for less cumbersome programming, can avoid allocating promises, and can amortize the synchronization costs. For example, the Bundle construct provided as a sample with Picos (internally) uses one atomic counter to track the number of live fibers — it doesn’t require each fiber to have its own promise to return a result (of course, every fiber does still have a bit of data associated with it (at minimum a (fiber) record of three elements, possibly an entry in a queue or other data structure of some kind, and the runtime stack with an effect handler for the fiber)).

Your design seems to tie cancelation to the fiber identity, which seems to mean that every time you want some operation to be individually cancelable, e.g. you want a timeout for an individual operation, you will either need to spawn a fiber (which is expensive) or have a custom mechanism (which is expensive to implement). If you would separate cancelation from the fiber identity, like is done in Picos, individual (scoped) operations could then be canceled, e.g. due to a timeout, without canceling the underlying fiber and you would not need to spawn fibers nor write custom mechanisms for everything cancelable.

As a specific minor point, since you mentioned it, Fun.protect is problematic, because it does not know about cancelation. If the cleanup action given to Fun.protect may require suspending the fiber, then that could be interrupted by cancelation, which is usually not what you want. You can, of course, always explicitly forbid cancelation within the action given to Fun.protect, but it is easy to forget.

The blocking interface you propose seems problematic in a number of ways:

  • First of all it is a higher-order interface. Even a quick glance at your specification shows that, with such higher-order state, you need to carefully consider all kinds of annoying cases (exceptions, effects performed, what if the actions block (e.g. lock a mutex), from which fiber, thread, or domain are the higher-order actions being called, can actions be called multiple times or concurrently, …).

  • Your interface seems to tie blocking to the fiber identity with no other data. This probably means you will be needing additional data structures (to transmit things to unblock functions) making things slower and more cumbersome.

  • The design of the callbacks specifies that the block function is called by the scheduler immediately and one should then register the operation. This is problematic, because it reduces concurrency and increases latency as the blocking operation can only be published after the scheduler calls block with the fiber handle or you will need an additional mutation (and logic) to atomically add the handle to a previously registered blocking entry.

  • Your blocking interface also allows for a value to be transmitted and that is another potential problem spot, because it then means you need to be very careful about making sure that such values are not accidentally dropped e.g. in case of cancelation. IOW, you really generally want to make sure things can always be implemented so that nothing gets lost under special circumstances or race conditions like cancelation.

If you look at the Trigger mechanism of Picos, then it allows the trigger to be allocated without performing any effects and allows the trigger to be inserted / registered where desired, and allows the trigger to be signaled before any effects are performed. The Trigger design in Picos is intentionally first-order (from POV of user) and requires value transmission (if needed) to be done separately. This avoids all of the nasty issues with higher-order state, allows the Picos library and multiplexers to implement the suspending mechanism correctly, and makes it clear that client code using triggers is responsible for making sure that any information transmission is done correctly. In Picos unblocking is a single step: a call of Trigger.signal calls the callback set by the scheduler to directly enqueue the fiber to whatever internal queue is used by the scheduler (no intermediate data structure or trip via unblock is needed). I would expect the Picos trigger mechanism to generally outperform your blocking API and be easier and simpler to use and implement safely and correctly.

The unblocking interface you propose is also higher-order (and so problematic similar to what I described above for blocking) and seems problematic:

  • Unblocking functions do not really compose as such — you cannot call two unblocking functions with ~poll:false except if you call them from two different threads. How do you interrupt your unblocking calls in some thread/domain in case you want to tear things down due to an error (exception) in another thread/domain?

  • Your scheduler implementation (which seems to be an incomplete draft at this point) calls unblock ~poll:true every time it switches between fibers. This is expensive and becomes more expensive as more unblock mechanisms are added.

In Picos, the way the trigger mechanism works, adding more things that might call Trigger.signal does not add overhead to scheduling.

I hope you’ll find something useful (e.g. things to think or ask about) from the above feedback!

8 Likes
13

we have 2 main competitors right now: eio and miou

I’d like to emphasise this sentence. As far as Robur and Miou are concerned, we don’t see ourselves as a competitor to Eio. In fact, we don’t consider Eio to be bad but, as it says in Miou’s genesis, our aim has been to separate ourselves (by creating a new scheduler) from Eio because we don’t want to fit in socially with what Eio represents (in its proposal and in its method of adoption).

In this respect, and I’ve been able to say this publicly several times, we’ve had various differences of opinion with the Eio team that aren’t technical. Being a small team at Robur, we decided, precisely 2 and a half years ago, to save our brain time because we were not satisfied with the interactions we were able to have with the Eio team.

the problem is not to enable concurrency model to co-exist, the problem is that there are multiple concurrency models

I’d also like to reiterate what we’ve been saying for quite a few years now. The fragmentation of the community across several schedulers didn’t pose any particular problem for us at the time of Lwt/Async. It still doesn’t pose a problem for us at this stage, despite the saturation of schedulers for OCaml 5. We think of our libraries independently of schedulers and our approach has worked for over 10 years. I’ve already said that we’ll continue this approach even if we develop Miou ourselves. We could be more authoritarian and consider that all our interactions with the system in our libraries should go through Miou but our experience shows us day after day that the time spent making libraries independent from schedulers is a good approach for the community in general.

A home-made composition of these libraries with any scheduler is often quite satisfying (especially when it comes to grasping the specific advantages of these schedulers).

As far as I can remember, the history of compatibility between schedulers does not date back to OCaml 5 but attempts have been made in the past. My opinion (and I may be wrong but I tend to think so) is that it is indeed impossible to reconcile several models of cooperation even if partial efforts can be made in this direction at the community level. In this case, the trials between Picos & Miou have confirmed this observation more than the other way round. The past also confirms it.

I also think that Picos goes too far in what it proposes and already has a certain opinion of what a scheduler should be. Here too, like Eio, an attempt has been made to develop our libraries in a way that would satisfy the community. Unfortunately this attempt failed because of trivial issues. And here again, as a small team unable to manage several fronts, we decided not to make any more effort than was necessary.

More generally, I think that two trends are emerging on the question of schedulers. There is one approach, which I would call authoritarian, where the idea is to propose a single model of task cooperation that will satisfy as many people as possible. It seems accepted that the latter should be integrated into OCaml, a point on which I agree. One could imagine Picos as a candidate but it has been confirmed to me that the Picos developers do not wish to interact with the OCaml core-team. Eio also took this approach at the beginning, presenting itself as the scheduler for OCaml 5 and having a fairly aggressive communication compared to the other proposals.

Personally, and from my experience with Lwt and Async and my opinions too, I’m not too keen on this approach preferring diversity of solutions and the possibility of choice.

The other approach is to accept the situation as it is. Again, as far as Robur is concerned, this situation is not unsatisfactory. Eio supports co-exist with those of Miou and Lwt in our libraries. Switching from one scheduler to another is not a problem for us. It may be unsatisfactory for new users, but I think that from this particular point of view, even a new user can identify and understand the difficulty of homogenising an entire community on a single scheduler. They can also understand the frameworks within which the various schedulers have been developed.

As it happens, some articles and discussions I’ve had with others about the choice between Lwt and Async are essentially based on social and concrete realities (maintenance of the scheduler by a large company or a small cooperative, documentation, examples of use, interactions with maintainers, etc.). And if we had to specify the subtle and technical differences that there may be between schedulers in order to make a choice, a third possibility exists (just as legitimate): you could make your own scheduler! And this is perhaps the big difference between OCaml 4 and OCaml 5, with the effects, it’s easy to make a new scheduler.

its own controversial decisions, such as supporting only select.

Miou was designed (as was Affect, I think) for strict separation between the system polling method and the scheduler. The very basic idea behind such a separation (which also concerns Lwt by the way) is that we don’t have Unix.select on our unikernels.

Extending Miou with epoll or io_uring is possible. Unix.select was chosen because it is the easiest and quickest to develop.

The most prominent example I know of is tokio in Rust.

I don’t think the grass is any greener elsewhere. Rust also has a fragmented community when it comes to schedulers (radeon, smol, embassy, glommio).

This is where I’d like to re-clarify Miou. Miou is based on objectives linked to our cooperative: developing services and unikernels. That’s the case with all the other proposals for that matter - but I’d be careful not to say what their objectives are. Miou’s ambition is not to satisfy as many people as possible or to compete with anyone else. Its aim is to satisfy, in this case, 4 people first and foremost, the members of our Robur cooperative - which is not bad at all! And that’s all… Then we can try to communicate about Miou and show that it’s a solution that at least works for us. But we won’t pretend that it’s the solution.

Finally, if people want to participate in Miou, they can, as is the case with all our other projects. Our interactions with users over the years are proof of our openness to contribution and compromise. But if people want to use something else, that’s their fundamental choice.

13 Likes
14

What I assume you mean by this is that you separate the core logic of libraries from the concurrency model and schedulers. This is a noble idea and can certainly work nicely in many cases. In general, however, it can substantially increase complexity and the cost of developing libraries and also increase the overheads at points where concurrency is needed.

Picos proposes an interface to request services from schedulers or multiplexers and indeed it does constraint the multiplexer. I disagree that it goes too far. I believe the difference in opinion here comes from viewpoint/goals and also due to the history in the sense that Miou was started before Picos existed.

Early on there was work on “composing schedulers” that mainly was about the ability to block (i.e. a Suspend effect proposal). However, that is insufficient to properly deal with things like cancelation.

Picos primarily extends the scope to allow cancelation to also be dealt with properly and secondarily also adds a couple more features that I feel are important enough to be included (FLS and timeouts). This then allows not only e.g. awaiting on someone elses promises, but also actually developing concurrent programming models that have specific ways to manage concurrency and propagate errors and cancelation. The idea is that almost all code needing concurrency could be implemented in Picos and run on a Picos compatible scheduler.

Note that when I say “implemented in Picos” I do not mean written directly against the Picos core interface. I mean implemented using libraries and frameworks and whatnot that ultimately, through dependencies, are written against the Picos core interface.

The alternative is that interoperability would be more shallow or more limited (i.e. just ability to await on things). This would then mean that pretty much all code would be intimately tied to the programming model and the whole scheduler that comes with that model. Programs using concurrent abstraction implemented using different schedulers would always need to literally run all of those schedulers (in different systhreads / domains). I simply do not think that this would be practical.

With Picos you can have a program using multiple multiplexers, however, anything that is “implemented in Picos” can run on any “Picos compatible” multiplexer. So, independently developed concurrent abstractions, possibly internally using different programming models, can run on any scheduler, which means that an application can be lean and just pick one (or more when intentionally needing/using different ways to schedule different fibers) instead of requiring a whole bunch of schedulers running concurrently and consuming resources (for no reason other than that one cannot have the banana without the monkey and the whole jungle with it).

On the first two occasions that I interacted on ocaml/ocaml I feel was insulted by one of the regulars there (for no good reason). I have no interest to listen to insults. I blocked that person on GitHub and then later he actually raised a OCaml CoC complaint against me for having blocked him (apparently interfering his work) and for having mentioned the incident (not mentioning names, but it is still, of course, possible to figure things out). So, I’ve been intentionally avoiding ocaml/ocaml and, TBH, I’ve been plenty busy with other OCaml related programming projects.

I would not recommend at this point to add Picos to Stdlib. I believe it makes sense to really have a couple more “schedulers” (or programming models implemented in terms of Picos) and libraries written by others and to also make sure that commercial users are also onboard.

I’m confident that if and when the time comes that it makes sense to add Picos to Stdlib, there will either be other people willing to interact on ocaml/ocaml or I will do it myself. I do know a bunch of the core maintainers.

4 Likes
15

Intent doesn’t really affect the experience of the consumer of the library. Miou is an alternative product that competes with Eio and Moonpool.

Does this generalize to multicore? Can an high quality multicore management scheme be attained - particularly one with resource constraints such as the ones we have in domains - when different schedulers are cooperating?

That’s awesome to hear.

A quick search suggests that tokio has near total dominance at this point. To me, this suggests a pretty good library that we should strive to copy from if we can. A tokio-like alternative would at least give us a comparison point to the other existing libraries.

As an aside, I believe this level of communication, criticism and honesty is critical for OCaml’s success. OCaml has many alternatives out there, and in order to make sure OCaml succeeds as much as possible, we have to maximize our communication and offer as much constructive criticism as possible. At the same time, I appreciate the huge amounts of effort that have been put into every library.

8 Likes
16

Just to elaborate on the situation in rust, tokio is the most popular scheduler indeed. Embassy is popular in embedded, and there’s other niche schedulers. All schedulers use the standard Future trait (which would be the equivalent of a good chunk of picos, and provides compatibility on the .await front).

What’s missing in rust is a compatible async IO abstraction (async read, async write, etc) and afaik it was waiting on async traits/HKTs. It’s now possible to represent these things so it’s possible that in the next few years there’ll be a scheduler agnostic way to read and write.

3 Likes
17

I think the issues for newcomers arise when there is no dominant solution.
If we take the build systems as an example, dune is the de facto default build system for OCaml. It’s far from the only, and people are free to create new ones. But most code out there use dune, most newcomer documentation/articles use dune, the editor plugins treat dune integration as a priority. A newcomer can focus on learning OCaml, even if they have to copy and paste some mysterious sexp expressions.
Contrast that with Lwt/Async, one of our major book uses Async, but most blog post out there use Lwt. Some libraries are compatible with one, but not the other. So a newcomer is forced to pick a concurrency library when they barely know how to write hello world…

The problem with multicore, the only library that has officially tried to be this “strong default, but alternative exists” is Eio. But its design hasn’t been well received by the community, and the Eio devs have been unwilling to work with the community to address some of the concerns. (that’s based on what I can read here, I haven’t directly interacted with Eio dev process)

I’d rather change the blessed multicore approach once a decade than try to pretend we can have 5 major multicore schedulers with top-notch support across libraries, tools and documentation.

5 Likes
18

I don’t think that’s the case. Miou is developed by the Robur team and imho their intent and opinion are the primary thing that matters for its future direction and development. Eg, if some people want to add a new feature to Miou, but it doesn’t make sense for Robur, then likely that feature won’t get added.

I think we should respect what the folks developing these systems are saying:

  • Miou is primarily for Robur but it may work for others
  • Picos is a building block for a concurrency library but is not ready to become a standard yet
  • Affect is in a design phase right now but is intended to become a community standard
  • Moonpool is also intended to become a standard and promote using the existing abstraction of lightweight threads running on top of domains
  • Eio is also intended to be the community standard and is the most mature of all of the options, but of course its issues have been widely discussed already.
9 Likes
19

Yes. It is important to understand that what, I believe, OCaml developers traditionally think when you use the word “scheduler” and what Picos calls a “scheduler” are very different things.

In Picos, a “scheduler” is pretty much just the multiplexer that can (typically) run large numbers of cooperative fibers on a smaller number of threads/domains and is free to decide which (ready) fiber to run next on which thread/domain. Picos gives an interface against which many kinds of concurrent abstractions can be implemented without depending on a specific implementation, i.e. on a specific way to pick the next ready fiber to run.

The traditional understanding is that a “scheduler” includes a whole application level concurrent programming model, IO system, collections of communication and synchronization primitives, higher level libraries like libraries for making HTTP requests or HTTP servers, and so on. With the traditional understanding of “schedulers”, the multiplexer is actually often a detail that people don’t care too much about. Yet, with the traditional understanding, everything is written to depend on the specific multiplexer of the “scheduler”. Occasionally the scheduling details do leak out and people intentionally or unintentionally write code that depends intimately on the way that the multiplexer picks things to run.

Picos specifically says that one should avoid depending on such scheduling details, much like with traditional preemptive threads, and instead write code that should work independent of the scheduling, e.g. by carefully using synchronization abstractions. There are, of course, cases where the scheduling details matter, and in those cases one should then document such expectations and applications should then pick schedulers appropriately. For example, if you want to do fine grained parallel stuff, then you should really pick a work-stealing scheduler that mostly uses LIFO scheduling.

So, coming back to what you were saying, Picos basically aims to solve that problem. It separates (a large subset of) concurrent abstractions from specific multiplexers and in doing so allows such abstractions to interoperate at a deep level. So, it should be entirely practical to have many different concurrent programming models (structured, structured in a different way, unstructured, actors, PI calculus based, …) and it should be entirely practical to share various things like top notch IO libraries, robust synchronization and communication abstractions, and so on.

And, of course, this is in no way a suggestion that people should just try to mix all the possible models in a single application. That is not the intention at all. The intention is to make it practical to build stuff from independently developed parts when the authors don’t always agree on what might be the best concurrent programming model. And, to simplify things a bit, a lot of things can actually just be functions 'a -> 'b that happen to do something concurrent or parallel internally and as long as such functions are written correctly to do what they are supposed to and not to unintentionally leak out their implementation details, then people shouldn’t really care about which concurrent programming model is being used.

So, to put it another way, Picos abstracts a detail that most people don’t really care about and by doing so allows a whole bunch of things to be written in an interoperable way. What is the problem?

5 Likes
20

From the point of view of a user of schedulers, not implementor, absolutely nothing. I think a world with 5 schedulers based on Picos is a bit better than one with 5 schedulers with their own abstraction for a fiber.
I’d go even further, and agree with @dinosaure, a world with 5 schedulers, even not compatible, isn’t that bad.

My position is more social than technical.
As @dbuenzli aid, having a clear model for concurrency is important to implement applications that are correct and efficient. We even touched on some of these points in this thread, e.g. what’s the cost of creating a fiber, or does my scheduler uses work stealing or not. This a not just implementation details, they will matter for the performance of the code we write and shape our solutions.
So ideally that model should be shared not only in the concurrency library documentation, but in blogs and tutorials to be easily accessible for all. We should have a large enough pool of experts here and discord/IRC to answer questions.
For a community the size of the of the ocaml community to do it well once is a difficult task. To do it well 5 or 7 times is impossible.
In that world with a largely agreed upon concurrency model, if unikernels need to be different, the same way rust embedded needs to be different, then so be it. Niche things sometimes requires different approaches and it’s ok.

2 Likes