# Lwt alternative to async.pipe

**URL:** <https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265>\
**Category:** Ecosystem\
**Tags:** lwt, cohttp, async\
**Created:** [February 12, 2021, 1:53pm UTC](https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265 "2021-02-12T13:53:40Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![mattpallissard](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/mattpallissard/32/2645_2.png) [@mattpallissard](https://discuss.ocaml.org/u/mattpallissard)\
**Post date:** [February 12, 2021, 1:53pm UTC](https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265/1 "2021-02-12T13:53:40Z")

</div>

Hey all, I’m new to the ocaml ecosystem and I’m trying to get my bearings on libraries and conventions around common operations.

I have some code that downloads a lot of large files. Currently I’m doing this with Cohttp and Async using `Cohttp_async.Response.to_pipe`. Coming from a systems programming world, this is a _really_ straight forward API.

Using it in combination with `Async.Pipe.iter`, does does exactly what one would assume. The writer blocks intermittently, keeping memory use low while the reader picks data off the fifo. Using it in combination with output channels lets you write it out to a file, or open up the standard input of another program (super handy). And when I say blocking the writer, I mean the literal thread handling that work. This is all “hidden” by the async libraries.

I haven’t figured out how to approach this in Lwt yet; enqueue data into a fifo, blocking the writer so memory usage stays low, and then have the ability do dequeue that data into an arbitrary file descriptor.

If anyone has input, I’d greatly appreciate it. In the meantime, I’ll keep looking through the docs.

---

<div class="post-metadata">

**Author:** ![mbacarella](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/mbacarella/32/6765_2.png) [@mbacarella](https://discuss.ocaml.org/u/mbacarella)\
**Post date:** [February 12, 2021, 2:56pm UTC](https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265/2 "2021-02-12T14:56:05Z")

</div>

Hi Matt. Welcome. I have noticed Lwt is missing an equivalent to Async Pipe as well. Discussed previously, very briefly, here: [Migrating an Async project to Lwt, a short primer](https://discuss.ocaml.org/t/migrating-an-async-project-to-lwt-a-short-primer/5908)

I haven’t dived into any of those suggestions yet but if you do please report back on how it went! 🙂

---

<div class="post-metadata">

**Author:** ![pdonadeo](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/pdonadeo/32/6309_2.png) [@pdonadeo](https://discuss.ocaml.org/u/pdonadeo)\
**Post date:** [February 12, 2021, 9:08pm UTC](https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265/3 "2021-02-12T21:08:22Z")

</div>

Did you checked out [lwt-pipe](https://github.com/c-cube/lwt-pipe)?

---

<div class="post-metadata">

**Author:** ![mattpallissard](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/mattpallissard/32/2645_2.png) [@mattpallissard](https://discuss.ocaml.org/u/mattpallissard)\
**Post date:** [February 13, 2021, 1:53am UTC](https://discuss.ocaml.org/t/lwt-alternative-to-async-pipe/7265/4 "2021-02-13T01:53:12Z")

</div>

Thanks for the suggestions. I was able to figure out lwt\_stream. I came up with something similar to the following.

```
aux_http header uri
>>= function
  | l ->
      Lwt_stream.iter_s
        (fun b ->
          let buf = Bytes.of_string b in
          let len = Bytes.length buf in
          Lwt_io.write_from process#stdin buf 0 len
          |> fun _ -> return ())
        l

```

which is similar to the original async version I was working with, though not quite as terse.

```
aux_http header uri
>>= function
  | l -> (
    Async.Pipe.iter l ~f:(fun b ->
        return
          (Async_unix.Writer.write_bytes process_oc
             (Bytes.of_string b)))
```