I want to implement a silly filesystem using ocamlfuse. In the silly filesystem implementation, everything is in the Lwt monad.
ocamlfuse (and FUSE itself) allow working in two different modes: single-threaded and multithreaded.
ocamlfuse has an “event loop” I believe. If we stick to single-threaded FUSE/ocamlfuse usage, the main thread repeatedly takes a request and services it.
Now, if the implementation is using Lwt, there seem to be two problems: 1) we can’t run the top-level Lwt_main.run loop in the main thread, because that is being used for the ocamlfuse event loop, and 2) after servicing a request using Lwt code, it is not immediately obvious how to extract a result from the Lwt monad and return it to the ocamlfuse event loop (?).
Regarding 1), can I just create a new Thread.t and run the Lwt_main.run loop there, whilst ocamlfuse executes in the initial thead?
Regarding 2), in single-threaded mode (for the time being), can I just create eg an ('a ref * semaphore) spawn the lwt thread (and block the ocamlfuse thread by waiting on the semaphore), and at the end of the lwt thread, make sure I place the result in the ref and signal the semaphore to indicate that the result is available ?
For 1), Lwt should work that way. As I remember, Lwt does not really know which thread is the “main” thread. There is a pipe that Lwt’s worker threads use to communicate Lwt I/O completions back to the Lwt main loop, and Lwt_main.run monitors that pipe. So, you should, in principle, be able to monitor it from a spawned thread.
I’m not sure what the issue in 2) is. What kind of semaphore are you creating? If it is something based on Mutex and Condition from the standard library, then this question seems to be about ordinary inter-thread communication, and yes, you can do it. What does single-threaded mode refer to here? It seems clear you want two threads to communicate with each other. If the ocamlfuse thread has a main loop to run, can it afford to block on the semaphore?
ocamlfuse can service filesystem operations in either a single-threaded mode one-at-a-time, or using multiple threads (which is potentially faster of course). At the moment I am just trying to get the single-threaded mode to work.
As you say, I want to communicate a value back from the lwt thread to the ocamlfuse thread which is presumably doing something like waiting on a Stdlib.Mutex.t. Indeed, the current implementation has the ocamlfuse thread waiting on a mutex, and the lwt code just signals a condition variable in the usual way before unlocking the mutex.
I guess my question then is: when the lwt thread initially acquires the mutex, at the moment it does a plain Mutex.lock; since this is potentially long-running (although in this scenario, not much) does this need to use something like Lwt_preemptive.detach to avoid blocking the main Lwt loop? The issue is that there may be other lwt promises doing all sorts of things, which I don’t want to block while one lwt thread waits on a Stdlib.Mutex. Does this make sense?
Maybe the question is: what are the best practices for lwt when working with Stdlib.Mutex and Stdlib.Condition (i.e., when trying to co-exist with other threads)?
You could use Lwt_preemptive.detach — just make sure to use one call for the whole lock-signal-unlock sequence, not multiple, and interact with the rest of Lwt by calling Lwt_preemptive.run_in_main in between. This “should” convert blocking on a mutex into blocking on the internal pipe that is inside Lwt_main.run. It does seem a little inconvenient to need a third thread for the two threads to interact. I am not sure if there is some better way to make Lwt and ocamlfuse interact, but I would try to find one.
As for best practices, I think these situations are rare enough to be unique, so it’s not clear what the best practices are.