# Ignoring Lwt values **URL:** \ **Category:** Learning\ **Created:** [August 29, 2019, 9:59pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301 "2019-08-29T21:59:30Z")\ **Posts on this page:** 14\ **Page:** 1
**Author:** ![sapristi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sapristi/32/1488_2.png) [@sapristi](https://discuss.ocaml.org/u/sapristi)\ **Post date:** [August 29, 2019, 9:59pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/1 "2019-08-29T21:59:30Z")
Basically my question is: what happens when one does ```auto ignore (x : 'a Lwt.t) ``` I am writing a logging module, and I want it to be able to send messages through a Lwt\_pipe and to stdout at the same time, e.g. ```auto let pipe = Lwt_pipe.create ~max_size:1000 () in let log s = Lwt_io.write Lwt_io.stdout s <&> Lwt_pipe.write_exn pipe s ``` but most of the time I call the log function, I am not inside of a Lwt context, so that the returned `unit Lwt.t` is ignored. This seems to works (surprisingly) quite well, but I don’t understand why. Is there anything I am missing here ? (in the context where I have tested is, the program’s main loop is Lwt). ---
**Author:** ![Leonidas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/leonidas/32/4039_2.png) [@Leonidas](https://discuss.ocaml.org/u/Leonidas)\ **Post date:** [August 30, 2019, 2:22pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/2 "2019-08-30T14:22:54Z")
It might work since the Lwt thread is possibly registered in the scheduler upon creation and since Lwt is eager it will try to execute it at the time the computer reaches the line, but I don’t think it is considered good style and has a likelihood of breaking. ---
**Author:** ![sapristi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sapristi/32/1488_2.png) [@sapristi](https://discuss.ocaml.org/u/sapristi)\ **Post date:** [August 30, 2019, 6:34pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/3 "2019-08-30T18:34:22Z")
> [@Leonidas](#): > > I don’t think it is considered good style I can understand that, but it still seems much better to me than (re)writing a somewhat big code base to be Lwt compatible. > [@Leonidas](#): > > has a likelihood of breaking. Yes, but how so exactly ? ---
**Author:** ![Leonidas](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/leonidas/32/4039_2.png) [@Leonidas](https://discuss.ocaml.org/u/Leonidas)\ **Post date:** [August 30, 2019, 7:42pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/4 "2019-08-30T19:42:19Z")
> [@sapristi](#): > > I can understand that, but it still seems much better to me than (re)writing a somewhat big code base to be Lwt compatible. I would disagree to some point, since you write something fragile that only accidentally works. I think rewriting a codebase so it actually works for sure is probably a better way to not be surprised by odd behaviour from your code in the long run. I understand that it is painful that’s why I would advise against introducing Lwt or Async late into a programs life. We have some blocking code at work and we know that adding async to it be a major effort that is not worth it. You can of course just try to carefully write parts using Lwt with proper blocking and then encapsulate that by running the Lwt main thread in a blocking way. > [@sapristi](#): > > Yes, but how so exactly ? You’re sort of depend on the implementation of the scheduler. It could change how things are implemented and also change in which order your promises are executed if at all. If you never wait on them they might get executed whenever so you don’t really have ordering guarantees that thread 1 executes before thread 2, thus you might run into odd out-of-order behaviour that is difficult to debug because it might also just work correctly in some or most of the time. To really understand what exact issues you might run into you probably have to ask @antron or @jeremiedimino. ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [August 30, 2019, 8:02pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/5 "2019-08-30T20:02:05Z")
It is indeed ok to ignore an `'a Lwt.t`. However, we have a function [`Lwt.async`](https://github.com/ocsigen/lwt/blob/1223535e59b7d219882bc642e914a000961b5ebb/src/core/lwt.mli#L792) for this purpose. The difference between `ignore p` and `Lwt.async (fun () -> p)` has to do with rejection handling: `p` can be rejected with an exception, in which case the exception is stored in `p`. `ignore p` silently forgets that stored exception. `Lwt.async (fun () -> p)` will pass the exception to `!Lwt.async_exception_handler` (which kills your program by default, but prints a stack trace). In any case, it is desirable to either (1) make sure that the computation that resolves `p` doesn’t raise any exceptions, or (2) you handle rejection of `p` with `Lwt.catch`. ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [August 30, 2019, 8:13pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/6 "2019-08-30T20:13:26Z")
As for why the code works: computation in Lwt eager, as @Leonidas mentioned. So, just calling the Lwt I/O functions is enough to start I/O operations. As long as you call `Lwt_main.run` somewhere, those computations can eventually _complete_. Note that `Lwt_main.run` does not _start_ operations — it’s only necessary to _complete_ them. When each operation completes, `Lwt_main.run` resolves its promise (`'a Lwt.t`), which calls more callbacks, which eagerly start more I/O operations, and so on. It’s not necessary to depend on any of the promises returned. Each I/O operation is internally registered with `Lwt_main.run` by the function that starts the operation. Somewhat more precisely, each function registers the promise that should be resolved when the operation completes. So, you can ignore these promises completely in your own code, as long as you call `Lwt_main.run` somewhere. There is, of course, a lot more detail to this in the implementation, but this is a fairly accurate mental model of the API 🙂 ---
**Author:** ![andyman](https://avatars.discourse-cdn.com/v4/letter/a/8491ac/32.png) [@andyman](https://discuss.ocaml.org/u/andyman)\ **Post date:** [August 30, 2019, 10:59pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/7 "2019-08-30T22:59:42Z")
> [@antron](#): > > As for why the code works: computation in Lwt eager, as @Leonidas mentioned. So, just calling the Lwt I/O functions is enough to start I/O operations. As long as you call `Lwt_main.run` somewhere, those computations can eventually _complete_. Note that `Lwt_main.run` does not _start_ operations — it’s only necessary to _complete_ them. Awesome explanation. I found futures(/threads) most confusing until I got this into my head. ---
**Author:** ![sapristi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sapristi/32/1488_2.png) [@sapristi](https://discuss.ocaml.org/u/sapristi)\ **Post date:** [August 31, 2019, 4:20pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/8 "2019-08-31T16:20:22Z")
Ok thank you both for your input, I’m getting a much better picture now. So, as i understand it, the promise dependency is here mainly to manage the scheduling of the I/O operations, and a promise can be ignored (or rather “executed asynchronously”) if nothing depends on it. I didn’t notice at first, but in my usecase there is one (big) downside at ignoring promises, which is that the ignored promise will be ran only after the “main” promise completes, so that - logs display would be delayed - and, more problematic, that the log promises would fill up the memory. So I think I’ll just give up with that for now (the initial aim was to send log messages through a websocket, but I can live without that). ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [August 31, 2019, 5:55pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/9 "2019-08-31T17:55:56Z")
> [@sapristi](#): > > the promise dependency is here mainly to manage the scheduling of the I/O operations Yes, the promises are for you to sequence operations one are after another, so scheduling in the sense of how you can compose your code, rather than scheduling in terms of the internals of Lwt. > [@sapristi](#): > > which is that the ignored promise will be ran only after the “main” promise completes, so that > > - logs display would be delayed > - and, more problematic, that the log promises would fill up the memory. That shouldn’t, in general, be the case. The reasons I can immediately think of why this is happening would be: - Your “main” promise is resolved immediately (synchronously), so it is “fast.” - Your logging operations are buffering or queuing, so they are “slow.” But, yes, with the way `Lwt_main.run` works, if there are I/O operations whose promises you are not asking `Lwt_main.run` to wait on (by calling `Lwt_main.run` on a promise that depends on them), `Lwt_main.run` may return before those operations complete, because it returns when just the promised passed to it resolves. ---
**Author:** ![sapristi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sapristi/32/1488_2.png) [@sapristi](https://discuss.ocaml.org/u/sapristi)\ **Post date:** [September 1, 2019, 2:50pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/10 "2019-09-01T14:50:21Z")
An approximation of my program would be this (run by opium) : ```auto for i = 0 to 100 do Unix.sleepf 0.01; Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 1; message %i \n" i); Unix.sleepf 0.01; Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 2; message %i \n" i); Unix.sleepf 0.01; done; |> Lwt.return ``` > [@antron](#): > > But, yes, with the way `Lwt_main.run` works, if there are I/O operations whose promises you are not asking `Lwt_main.run` to wait on (by calling `Lwt_main.run` on a promise that depends on them), `Lwt_main.run` may return before those operations complete, because it returns when just the promised passed to it resolves. So a solution might be pass promises returned by log events to Lwt.main\_run ? Like writing `unit` for each log to a Lwt\_pipe, that Lwt.main\_run would read ? ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [September 1, 2019, 3:08pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/11 "2019-09-01T15:08:04Z")
> [@sapristi](#): > > An approximation of my program would be this (run by opium) : I’m assuming that at the end of the program, it’s ```ocaml |> Lwt.return |> Lwt_main.run ``` since the code block doesn’t show where `Lwt_main.run` is called 🙂 If that’s so, then your main promise is indeed "synchronous with regard to Lwt I/O (i.e. it won’t ever try to fully do Lwt I/O), because `Lwt_main.run` is handed an already-resolved promise (since you create it with `Lwt.return`). That means `Lwt_main.run` won’t do anything, i.e. it won’t try to resolve any of the other promises, that are associated with I/O, that may have completed. If you either change `Unix.sleepf` to `Lwt_unix.sleep` and depend on those promises in `Lwt_main.run`, or introduce any other Lwt I/O into the flow of your program (even a `Lwt_unix.sleep` right before `Lwt_main.run may do), some or all of your `Lwt\_io.write`promises may actually have a chance to be resolved by`Lwt\_main.run`, thus `Lwt\_main.run` will drop its references to them. > [@sapristi](#): > > So a solution might be pass promises returned by log events to Lwt.main\_run ? Like writing `unit` for each log to a Lwt\_pipe, that Lwt.main\_run would read ? For that to work, you would need to know when the last write is in the pipe. In that case, you may as well use a single promise to signal the last write, and have `Lwt_main.run` wait on that. ---
**Author:** ![sapristi](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/sapristi/32/1488_2.png) [@sapristi](https://discuss.ocaml.org/u/sapristi)\ **Post date:** [September 1, 2019, 5:36pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/12 "2019-09-01T17:36:39Z")
Ok thanks a lot, so e.g. this seems to achieve what I wanted ! ```auto for%lwt i = 0 to 100 do ( Unix.sleepf 0.05; Lwt.async (fun () ->Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 1; message %i \n" i ) ); Unix.sleepf 0.05; Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 2; message %i \n" i) ; Lwt_io.flush_all () ) done |> Lwt_main.run ``` > [@antron](#): > > If that’s so, then your main promise is indeed "synchronous with regard to Lwt I/O (i.e. it won’t ever try to fully do Lwt I/O) Ok, so this is why the following, which I had previously tried, did not work ? ```auto for%lwt i = 0 to 100 do ( Unix.sleepf 0.05; Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 1; message %i \n" i ); Unix.sleepf 0.05; Lwt_io.write Lwt_io.stdout (Printf.sprintf "This is 2; message %i \n" i); Lwt.return () ) done |> Lwt_main.run ``` So to resolve asynchrounous pending I/O promises, Lwt\_main.run needs to depend on a I/O promise (EDIT: or Lwt\_unix.sleep ?), but how does this generalize to other kind of operations ? E.g if I declare asynchronous write promises to a Lwt\_pipe or Lwt\_stream instead of I/O, how would I force these promises to be resolved ? ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [September 1, 2019, 6:17pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/13 "2019-09-01T18:17:49Z")
> [@sapristi](#): > > Ok, so this is why the following, which I had previously tried, did not work ? if you look at that code, every iteration, it synchronously does some blocking `Unix` sleeps, synchronously spawns some I/O computation, synchronously (immediately) returns a resolved promise. So, you end up with 101 iterations of executing synchronous code, and then pass an already-resolved promise (roughly, the last `Lwt.return`) to `Lwt_main.run`, so `Lwt_main.run` doesn’t try to finish any of the I/O, and returns right away. `for%lwt` doesn’t drive any Lwt I/O, only `Lwt_main.run` does that. In the first code block in your comment, you have the `for%lwt` waiting on a non-resolved promise in each iteration. This means that on the first iteration, you construct a non-resolved promise, and ~~that first promise is what~~ `for%lwt` passes _an non-resolved promise_ into `Lwt_main.run`. That promise from the first iteration doesn’t resolve until the last iteration completes (that’s what `for%lwt` does), so `Lwt_main.run` ends up waiting for the `flush_all` from iteration 101. > [@sapristi](#): > > but how does this generalize to other kind of operations ? To completely guarantee that a promise is resolved before `Lwt_main.run` returns, you have to pass the promise, or a promise that depends on it, as the argument to `Lwt_main.run`. Note that you aren’t forcing _the promises_ to become resolved. You are just forcing _your program_ to wait until they become resolved. ---
**Author:** ![antron](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/antron/32/62_2.png) [@antron](https://discuss.ocaml.org/u/antron)\ **Post date:** [September 1, 2019, 6:20pm UTC](https://discuss.ocaml.org/t/ignoring-lwt-values/4301/14 "2019-09-01T18:20:10Z")
Another way of thinking about `for%lwt` is that it tries to do as much work as it can up front, and then evaluates to either the final resolved promise (if it did all the work), or to the first pending promise it hits along the way. In that second case, it evaluates to a pending promise that represents the rest of the computation inside the `for%lwt`.