Unable to print to stdout from inside lwt bound function

Learning
1

Hello,

I have been trying to use the csv module with lwt and am going around a bit because i haven’t been able to make sense of the documentation.

In my experiments to get an in_channel Lwt.t out of a string in memory, i’ve noticed that after binding with lwt, i’m not able to print and flush to stdout anymore.

I’ve tried using the print_endline function, as well as Lwt_io specific ones, but i’m not able to print to stdout.

I have more experience with other languages and i’m wondering if there might be some issue that gets masked, so i don’t get an error back? The code compiles fine, so i’m wondering where the issue might be.

The libs i’m using here are just csv csv-lwt lwt lwt-unix

open Lwt
let embedded_csv =
  "\"Clickin\",\"Number\",\"Percentage\",\n\
   \"brand.adwords\",\"4,878\",\"14.4\"\n\
   \"promo.home.topX.SSH.366\",\"310\",\"0.9\""

let () =
  let open Syntax in
  print_endline "\n";
  let p =
    let csv_bytes = Lwt_bytes.of_string embedded_csv in
    let ic, oc = Lwt_io.pipe ~in_buffer:csv_bytes () in
    print_endline "this line prints";
    let* ss = Csv_lwt.of_channel ~has_header:true ic in
    print_endline "this one doesn't print";
    let* () = Lwt_io.printf "will not print %s" "this message" in
    let* () = Lwt_io.(flush stdout) in
    let () = Printf.printf "still not printing %s \n!" "the other message" in
    let* () = Lwt_io.write Lwt_io.stdout "chat gpt suggestion, fml lol" in
    let headers = Csv_lwt.Rows.header ss in
    List.iter (fun h -> print_endline h) headers;
    (* nothing gets printed *)
    return ()
  in
  Lwt_main.run p

Thanks in advance for your help

2

I’ve figured out that the issue lies with the Lwt_io.pipe function. If i use a different one (Lwt_io.of_bytes), the code works as expected.

I’m still not sure what is happening, though. My guess is that i’m writing to some out_channel other than stdout, but don’t understand why it starts doing this.

3

Based on which lines print in

    print_endline "this line prints";
    let* ss = Csv_lwt.of_channel ~has_header:true ic in
    print_endline "this one doesn't print";

I’m gessing that the ss promise doesn’t resolve. I don’t know what Csv_lwt.of_channel does so I can only make a guess. Still there it is:

Does Csv_lwt.of_channel use Lwt_io.read internally? If it does, note that Lwt_io.read resolves to the whole content of the channel which requires to not resolve until the channel is closed (otherwise it can’t know that the whole content has been read).

Does Csv_lwt.of_channel use some Lwt_io.read_line, Lwt_io.read_lines, Lwt_io.read_line_opt? Maybe try adding a \n on the last line of your embedded_csv to make it resolve?

4

My assumption would be that ic is connected to one end of the pipe, but when Csv_lwt.of_channel tries to reads the header line this blocks because nothing is writing to the other end.

5

Re-reading this I think I get the problem better:

The parameter in_buffer doesn’t mean “put this into the buffer”, it means "use this bytes value as the buffer for the “in” part of the pipe”. It’s in case you don’t want to allocate a new buffer because you have one lying around that’s available.

So there are no writes onto the pipe. The buffer it uses happens to contain bytes that represent the data you want to use, but those bytes have not been written onto the pipe, they are just considered left-over dirty memory.

6

Thank you all for your replies. In the end, i’m actually not sure what the issue was, and didn’t have the opportunity to dig further. I’ve had limited time lately and my ocaml study has taken me in other directions.

I have tested adding the “\n” at the end of the csv string, but it didn’t make any difference.

But it does make sense to me that the issue lies with the in_buffer not being used to write into.