Mapping over Writer.t

Learning
,
1

Hi,

I have a function that takes a Writer.t (can’t change that) and I would like to map somehow over the bytes. Turns out it is harder than I expected since I have to create a Pipe that points to the original Writer.t (stdout), then create a transfer function (in this case transfer_id but it doesn’t really matter, it also does not work with transfer or transfer') and then I can attempt to create a writer out of that Pipe and then use that where I would originally have the Writer.t.

Now the problem is that it has a ~30% chance of working, because sometimes the data does not make it though the pipe. I don’t understand why.

Here’s a sort-of minimal version of the code that has this behaviour:

let () =
  let open Command.Let_syntax in
  Command.async
    ~summary:"Failure"
    [%map_open
      let () = return () in
      fun () ->
        let open Deferred.Let_syntax in
        let stdout = Writer.stdout |> force in

        let sp = Writer.pipe stdout in
        let w = Pipe.create_writer (fun reader ->
          Pipe.transfer_id reader sp)
        in
        let i = Info.of_string "" in
        let%bind stdout, _ = Writer.of_pipe i w in

        Writer.write_line stdout "Hest";
        let%bind () = Writer.close stdout in
        return ()
      ]
  |> Command.run

I thought it might be due to some flushing so I tried adding flush to all kinds of locations but it never improved. I don’t seem to understand the Writer/Pipe interaction at all. Any ideas?

2

I think you need to wait for the deferred returned by the Writer.of_pipe:

open! Core
open! Async

let () =
  let open Command.Let_syntax in
  Command.async
    ~summary:"Failure"
    [%map_open
      let () = return () in
      fun () ->
        let open Deferred.Let_syntax in
        let stdout = Writer.stdout |> force in

        let sp = Writer.pipe stdout in
        let w = Pipe.create_writer (fun reader ->
          Pipe.transfer_id reader sp)
        in
        let i = Info.of_string "" in
        let%bind stdout, `Closed_and_flushed_downstream deferred = Writer.of_pipe i w in

        Writer.write_line stdout "Hest";
        let%bind () = Writer.close stdout in
        deferred
      ]
  |> Command.run

I’m not 100% if this is the bug you’re experiencing, since I get about a 3% failure rate with your original code, but waiting for that deferred seems to make the bug disappear.

3

Yes, you’re right, this seems to be it, thanks! Serves me right for just discarding a deferred and then wondering about races.