Using Lwt locally for a HTTPS GET request?

Learning
1

I’d like to implement a synchronous HTTPS GET request in a project that does not use Lwt yet. It seems the best way is to use cohttp but it only provides GET requests in an Lwt context. Is it possible to run the Lwt main loop locally in a function until it returns?

2

I am not sure I got the point, but you can (at least in principle) force the promise to run with Lwt_main.run your_lwt_thing. But if you want the GET calls to be sequential instead of synchronous, you could still do all of them in the lwt context with map_s

3

If you want to avoid lwt or async, you can make your own synchronous IO module and conduit implementation and use those to establish the connection and deal with requests and responses (I think something like that is done in xapi-idl), but it’s a lot of work

4

Probably it could be enough to have the IO module and use something like https://github.com/xapi-project/xen-api-client/blob/0f085e31a22a4b61304e1cb73b8868df89472818/lib/xen_api.ml#L51 to deal with the connection

5

Sorry for being too vague. I don’t want to convert the existing code to use Lwt globally. Instead, I wonder whether it is possible to use Lwt just inside a function that implements the GET request. This function would call the Lwt main loop and wait for it to return.

6

If I understand correctly, yes this should be possible via Lwt_main.run.

The type of run is:

val run : 'a Lwt.t -> 'a

If you see the example on cohttp README, you will need to just modify the let () to be a local function:

let your_own_function () =
  (* This will block until the promise is fulfilled *)
  let body = Lwt_main.run body in
  (* do anything you want with body synchronously,
     we're no longer in Lwt world. *)
  body (* for example returning the body string as is *)

The type of your_own_function would be unit -> string in this case, and no Lwt whatsoever.

7

I understand that this possible from, say, a types perspective. But could something bad happen under the hood when the main application is using threads and its own I/O?

8

I may be wrong, but looking at the documentation it does not seem such a good idea to hide it in a function: http://ocsigen.org/lwt/4.1.0/api/Lwt_main

9

Depending on other requirements, it may be reasonable to just wrap the curl command. I’ve done it for a test suite before.

10

Perhaps @antron could shed some light on the behavior for this use case?

11

Lwt_main.run p will use select/epoll/kevent to repeatedly block the main thread until p resolves. AFAIK, select, etc., don’t interfere with other I/O or other threads, unless you are sharing file descriptors between threads. So, Lwt_main.run in the main thread should be safe for converting some Lwt call into a call that looks synchronous.

3 Likes