Need help on using Piaf & Lwt

Learning
,
1

I am trying to index multiple documents one by one to Elasticsearch using Piaf lib.
In the code below, only the first document is indexed. I noticed that only one http call is fired.
Could anyone help me to understand the problem? Thanks.

let main() =
  let headers =
    [
      ("Content-Type", "application/json");
      ("Accept", "*/*");
      ("Accept-Encoding", "gzip,deflate,br");
    ]
  in
Lwt_main.run
begin
    docs |> Lwt_list.map_s(fun doc ->
        let* resp = Piaf.Client.Oneshot.post ~headers ~body:doc "http://localhost:9200/books/_doc" in
        if Piaf.Status.is_successful resp.status then begin
          Piaf.Body.to_string resp.body
        end
        else begin
          let msg = Piaf.Status.to_string resp.status in
          Lwt_result.fail (`Msg msg)
        end
    )
end

let () = main() |> ignore
2 Likes
2

Hi @Jazz,

I assume the let* refers to Lwt_result.Syntax.(let*)? (At least with my version of piaf, Oneshot.post returns a Lwt_result.t.) Have you tried unpacking the results from the map_s and printing them? Perhaps you’re getting rate-limited, or the subsequent POSTs are failing for some other reproducible reason.

Something like this should do it, provided you have fmt installed:

let () =
  main ()
  |> Fmt.Dump.(list (result ~ok:string ~error:Piaf.Error.pp_hum)) Fmt.stdout

Cheers,
Craig.

2 Likes
3

I am not familiar with Piaf so this answer is mostly a question.

Can Piaf.Client.Oneshot.post raise exceptions? If so, it might be that the second call (i.e., the iteration on the second element of the list) fails and interrupts the map.

Alternatively, can Piaf.Body.to_string resp.body or Piaf.Status.to_string raise exceptions? If so it might be possible that during the first iteration (but after the side-effect is done on the server), an exception interrupts the map.

2 Likes
4

Hi @CraigFe

Dump the result like you mentioned and it works!.
Thanks.

1 Like