From an LWT documentation I got an impression that the library always tried try to do as much work as possible. Essentially scheduling only happens when things block. As on Linux reading from a file never blocks, I assumed that when processing a large file I would need to add explicit Lwt.pause calls between reads from the file. This is to ensure progress of other promises. However, a test revealed that this is not the case. For example, consider the following code:
let test_file_read path = Lwt.( let read_fd fd = let buffer_size = 4096 in let buffer = Bytes.create buffer_size in let rec read_chunks sum = Lwt_unix.read fd buffer 0 buffer_size >>= fun nread -> if nread == 0 then return sum else read_chunks (sum +. (float_of_int nread)) in read_chunks 0.0 >>= (fun total -> Printf.sprintf "Total read: %.3f MB" (total /. (1024.0 *. 1024.0)) |> Lwt_io.printl ) in Lwt_unix.(openfile path [ O_RDONLY; O_NONBLOCK; O_CLOEXEC ] 0) >>= fun fd -> finalize (fun () -> read_fd fd) (fun () -> Lwt_unix.close fd) ) let timer n = Lwt.( let rec tick n () = if n <= 0 then return_unit else Lwt_io.printl (string_of_int n) >>= fun() -> Lwt_unix.sleep 10.0 >>= (tick (n - 10)) in tick n () )
test_file_read path read the file from the given path and prints the total number of bytes read when done.
timer function prints each 10 seconds a timer ticks until the timer expires.
My expectation was that when I run
Lwt.join [test_file_read "/var/tmp/zeros"; timer 60]
I would always see first a message about the number of read bytes and only then the timer and its ticks would show up. However in utop the output was:
utop # Lwt.join [test_file_read "/var/tmp/zeros"; timer 60];; 60 50 40 30 Total read: 11718.750 MB 20 10 - : unit = ()
So the file read code processing 11MB file and the timer did run in parallel without any efforts from my part. Why it is so?