Hello,
I have been trying to use the csv
module with lwt
and am going around a bit because i haven’t been able to make sense of the documentation.
In my experiments to get an in_channel Lwt.t
out of a string in memory, i’ve noticed that after binding with lwt, i’m not able to print and flush to stdout anymore.
I’ve tried using the print_endline
function, as well as Lwt_io
specific ones, but i’m not able to print to stdout.
I have more experience with other languages and i’m wondering if there might be some issue that gets masked, so i don’t get an error back? The code compiles fine, so i’m wondering where the issue might be.
The libs i’m using here are just csv csv-lwt lwt lwt-unix
open Lwt
let embedded_csv =
"\"Clickin\",\"Number\",\"Percentage\",\n\
\"brand.adwords\",\"4,878\",\"14.4\"\n\
\"promo.home.topX.SSH.366\",\"310\",\"0.9\""
let () =
let open Syntax in
print_endline "\n";
let p =
let csv_bytes = Lwt_bytes.of_string embedded_csv in
let ic, oc = Lwt_io.pipe ~in_buffer:csv_bytes () in
print_endline "this line prints";
let* ss = Csv_lwt.of_channel ~has_header:true ic in
print_endline "this one doesn't print";
let* () = Lwt_io.printf "will not print %s" "this message" in
let* () = Lwt_io.(flush stdout) in
let () = Printf.printf "still not printing %s \n!" "the other message" in
let* () = Lwt_io.write Lwt_io.stdout "chat gpt suggestion, fml lol" in
let headers = Csv_lwt.Rows.header ss in
List.iter (fun h -> print_endline h) headers;
(* nothing gets printed *)
return ()
in
Lwt_main.run p
Thanks in advance for your help
I’ve figured out that the issue lies with the Lwt_io.pipe
function. If i use a different one (Lwt_io.of_bytes
), the code works as expected.
I’m still not sure what is happening, though. My guess is that i’m writing to some out_channel
other than stdout
, but don’t understand why it starts doing this.
Based on which lines print in
print_endline "this line prints";
let* ss = Csv_lwt.of_channel ~has_header:true ic in
print_endline "this one doesn't print";
I’m gessing that the ss
promise doesn’t resolve. I don’t know what Csv_lwt.of_channel
does so I can only make a guess. Still there it is:
Does Csv_lwt.of_channel
use Lwt_io.read
internally? If it does, note that Lwt_io.read
resolves to the whole content of the channel which requires to not resolve until the channel is closed (otherwise it can’t know that the whole content has been read).
Does Csv_lwt.of_channel
use some Lwt_io.read_line
, Lwt_io.read_lines
, Lwt_io.read_line_opt
? Maybe try adding a \n
on the last line of your embedded_csv
to make it resolve?
My assumption would be that ic
is connected to one end of the pipe, but when Csv_lwt.of_channel
tries to reads the header line this blocks because nothing is writing to the other end.
Re-reading this I think I get the problem better:
The parameter in_buffer
doesn’t mean “put this into the buffer”, it means "use this bytes
value as the buffer for the “in” part of the pipe”. It’s in case you don’t want to allocate a new buffer because you have one lying around that’s available.
So there are no writes onto the pipe. The buffer it uses happens to contain bytes that represent the data you want to use, but those bytes have not been written onto the pipe, they are just considered left-over dirty memory.
Thank you all for your replies. In the end, i’m actually not sure what the issue was, and didn’t have the opportunity to dig further. I’ve had limited time lately and my ocaml study has taken me in other directions.
I have tested adding the “\n” at the end of the csv string, but it didn’t make any difference.
But it does make sense to me that the issue lies with the in_buffer
not being used to write into.