Lwt.finalize and failure in the cleanup code

Learning
1

The documentation for Lwt.finalize states:

finalize f g returns the same result as f () whether it fails or not. In both cases, g () is executed after f.

Yet the following example demonstrates that this is not the case:

Lwt.finalize (fun() -> Lwt.fail (Failure "1")) (fun () -> Lwt.fail (Failure "2")) |> state;;

This gives a state with Failure "2", not the Failure "1" so the failure comes from the execution of g. Is it a bug in the documentation?

In any case, what is a good way to deal with a failed finalize when main promise also failed ? I can create own version of finalize that wraps both failures into a new exception and fail with that but it feels heavy.

1 Like
2

This is indeed a bug in the documentation, among many – the documentation will be rewritten. For comparison, we expect the behavior you are observing in the test cases. I’m not sure if that’s how it should be, but that’s what the code did when the tests were written. Basically, failures in g overrule what happens with f.

I think to handle this, you can use Lwt.catch or Lwt.try_bind. This will allow you to run different code depending on whether f () fails or not, so you can decide whether to pair exceptions if g exn also fails, or do something else. I guess what to do when both f and g fail tends to be application-specific. Maybe we should add to the docs that it is best if g can’t fail; if it can, users should write custom code.

I’ve opened an issue in the Lwt repo for clearing up the docs. Thanks for asking/reporting.

3

There is a related issue that a failure in finally overrides even the cancel status and

let t = Lwt.finalize (fun () -> Lwt_unix.sleep 1000.0) (fun () -> Lwt.fail (Failure "hello from finalize")) in (Lwt.cancel t; Lwt.state t);;

gives the Fail state, not Canceled. I think what would be ideal is to have a hook similar to Lwt.​async_exception_hook that will be called whenever the finalize code fails irrespectively if it was a normal cleanup or cleanup on failure. This way an application can at least log the failure and stop it from propagating farther.

4

Also I do not see a specific test that covers the interaction of cancel and fail from the finalizer. Have I missed it?

5

There isn’t such a test. That is the expected behavior, because cancelation causes an ordinary failure of the inner promise returned by Lwt_unix.sleep, and the (weird?) semantics of finalize take over after that. We will add a test for the sake of thoroughness. Again, I don’t know if this is how it should be, but that’s how it is…

For comparison, Lwt.catch is also weird with cancel, in that the cancelation “failure” can be captured by the exception handler:

(* In [p' = Lwt.catch (fun () -> p) f], if [p] is cancelable, canceling
   [p'] propagates to [p], and then the cancelation exception can be
   "intercepted" by [f], which can complete [p'] in an arbitrary way. *)
test "catch: task, pending, canceled" begin fun () ->
  let p, _ = Lwt.task () in
  let p' =
    Lwt.catch
      (fun () -> p)
      (fun exn -> Lwt.return "foo")
  in
  Lwt.cancel p';
  Lwt.return
    (Lwt.state p = Lwt.Fail Lwt.Canceled &&
     Lwt.state p' = Lwt.Return "foo")       (* <-------- !!!! *)
end;

Regarding the hook, it does seem like handling “double faults” in some way is desirable.

I think this is a much-discussed problem in all languages that have finally blocks, i.e. Python, Java, etc., so the first thing to do is probably to mine them for ideas. There are probably strong arguments for one behavior or another, and we should find them, save links, and document it all.

6

Those “double faults” from finalize is a simpler instance of the problem of proper handling faults when joining multiple promises. The current Lwt.join is unsatisfactory as it encourages to ignore the case of multiple failing promises by returning with fail immediately as one of the promises fails. It also encourages to leave waiting promises around with unclear consequences.

In my application instead I use a version of join that on the first failure cancels other promises, wait until they really terminates after the cancel attempt, check for any promises that failed and merge their faults into a common one which is reported from the join. But that means that the application already have a code to merge multiple faults into one. I can use that merging facility to also deal with the double fault in finalize. So it would be nice to have a hook that allows to do such fault merging with Lwn.finalize without the need to insist on using a custom safe_finalize in all the code.

2 Likes