I think the programming interface we should all want comprises more than just the Domain and Atomic modules. You really need algebraic effects and a scheduler that uses a domain pool, and you’re not going to get that with just a shim over the Thread library in the current compiler. I think the best path forward will probably be to provide existing concurrency libraries like Lwt and Async a compatibility layer on the multicore compiler branch for the Thread library.
An open question I’d like to see answered is whether my Thread library implemented in terms of Domain and Atomic needs to simulate the global interlock in the unicore to be adequately compatible with the needs of applications using the legacy concurrency libraries.
An open question I’d like to see answered is whether my Thread library implemented in terms of Domain and Atomic needs to simulate the global interlock in the unicore to be adequately compatible with the needs of applications using the legacy concurrency libraries.
It does. Plenty of code relies on this assumption for correctness. People know which program points the threads may context switch and rely on code between those points being atomic.
@lpw25, I know about things like Nano_mutex in core, but is there much more that this? I don’t think such modules were ever intended to survive in a proper multicore world.
Yeah, I think they’re are quite a few, and they are hard to find. For example, I know there is one in jenga, but I don’t think anyone currently working on jenga even knows it is there. A Thread library without a GIL is just not backwards compatible.
I don’t think such modules were ever intended to survive in a proper multicore world.
To be clear, Thread itself should not survive in a proper multicore world. We just need a backwards compatible implementation so that we don’t need to force everyone to update their code all at once.
Okay. I’ll add the global interlock to my Thread compatibility library before I push.
Do we believe it would help or hinder multicore adoption if the compatibility layer offered a function to disable the interlock as an option? I can make it mandatory if we think that it would hinder adoption of algebraic effects and proper domain pools. If so, then I think that would entail some slightly different design choices in my implementation, so it might be worth discussing here before I bring a branch forward for review.
I think it would be better without an option to disable it, to help push people onto the APIs that are designed with true parallelism in mind. Another issue is that if the option is global state then you risk one library turning it off whilst another library is relying on it being on. If it is not global state then its not so bad.
I think it’s perfectly reasonable for Thread to continue having a global lock, and be there for purely retrocompatible purpose, without a way of opting out.
That being said, would there be a reasonably simple way of porting code from Thread to domains, without a total rewrite, and with similar synchronization primitives (mutex and condition), to benefit from multiple cores? There would be no guarantee of compatibility anymore (eg. no implicit critical sections between allocation points) but also no fibers, just heavy threads. Does it make sense?
An additional risk is that the library has turned off the global lock but actually relies on it in certain edge cases, so it might introduce a subtle potentially hard to trigger bug.
Indeed I’d dare to say that Threads should remain a complete drop-in replacement of the non-multicore version, so even the lock should not be able to be disabled.