Tutorial for Cohttp-lwt as API client

Hi everyone,

I was trying to get started with practical OCaml via writing API clients and decided to experiment with https://deckofcardsapi.com/ as a starting point to discover the libraries.

However, the only pieces of documentation I’ve come across for these are the following

But unfortunately, I haven’t quite been able to make my way through these. So, I decided to ask for help from the OCaml community regarding the same.

Could you guide me a bit here ?

There is also the example in Real world ocaml, althouh that uses cohttp-async. It should be relatively straightforward to translate.

Having a nice tutorial to add to the repo and the community documentation would be good. I am not a power user, but I can try to help. What you would loke to know?

1 Like

Thanks @mseri for the link!

However, I’d really love to get started with lwt as I’m more inclined towards the mirageOS that relies on lwt .

For now, what I’d like to do is to just wrap the deckofcards API in a utop session as the starting point. If possible, could you please share the snippets that could help me accomplish these tasks ?

Here are the libraries I’ve identified so far

  • yojson
  • lwt
  • cohttp-lwt

Are there any good resources to learn lwt apart from the main ocsigen docs ?

I wish to use OCaml as my own standard way to wrap and join the various online APIs which I often find using in other language clients.

@abhi18av I hope the following will help, it’s just a rough example. You can actually copy and paste it in utop. You will need cohttp-lwt-unix and yojson installed. For yojson refer to RWO, it is fine even if you don’t use core.

#require "cohttp-lwt-unix";;
#require "yojson";;

(* A bit overkill to open everything *)
open Cohttp
open Cohttp_lwt_unix
open Lwt.Infix
open Yojson
;;

(* Make a get call and return the body parsed as json.
 * You could use ppx_deriving_yojson or atdgen to
 * reduce the json parsing boilerplate, but for such small
 * example is overkill.
 *
 * val json_body : string -> Basic.json Lwt.t
 *)
let json_body uri =
  Client.get (Uri.of_string uri)  >>= fun (_resp, body) ->
  (* here you could check the headers or the response status
   * and deal with errors, see the example on cohttp repo README *)
  body |> Cohttp_lwt.Body.to_string >|= Yojson.Basic.from_string
;;

(* In utop the promise is realised immediately, so if you run the
 * function above, you'll get the result immediately *)
json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1"
;;
(*Output:
- : Basic.json =
`Assoc
  [("shuffled", `Bool true); ("success", `Bool true); ("remaining", `Int 52);
   ("deck_id", `String "2svxenasd9qj")]
*)

(* In the code you will have more likely something like the following. *)
let shuffled_deck =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1"
;;
(* val shuffled_deck : Basic.json Lwt.t *)

(* To use it you just have to get use to the monadic bind, and once you
 * are ready, call `Lwt_main.run`. e.g. the kind of useless example below *)

(* val get_cards : Basic.json -> int -> Basic.json Lwt.t *)
let get_cards sdeck n =
  (* The use of Yojson is well explained in RWO *)
  let module YBU = Yojson.Basic.Util in
  let success = sdeck |> YBU.member "success" |> YBU.to_bool in
  if not success then Lwt.fail_with "Error while shuffling" else
    let deck_id = sdeck |> YBU.member "deck_id" |> YBU.to_string in
    Printf.sprintf "http://deckofcardsapi.com/api/deck/%s/draw/?count=%d" deck_id n
    |> json_body >>= fun cards ->
    Lwt.return cards

(* val do_something : unit -> string list Lwt.t *)
let do_something () =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1" >>= fun sdeck ->
  get_cards sdeck 3 >>= fun jcards ->
  let module YBU = Yojson.Basic.Util in
  let cards = jcards |> YBU.member "cards" |> YBU.to_list in
  List.map (fun c -> c |> YBU.member "code" |> YBU.to_string) cards
  |> Lwt.return

(* val do_something_else : unit -> unit Lwt.t *)
let do_something_else () =
  do_something () >>= fun cards ->
  List.iter (Printf.printf "%s\n") cards |> Lwt.return;;
;;

Lwt_main.run (do_something_else ()); print_endline "Done!"
;;

(* Be careful with the signatures, the example above can be re-run multiple
 * times, while the one below does nothing after the first run *)

(* val do_something_1 : string list Lwt.t *)
let do_something_1 =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1" >>= fun sdeck ->
  get_cards sdeck 3 >>= fun jcards ->
  let module YBU = Yojson.Basic.Util in
  let cards = jcards |> YBU.member "cards" |> YBU.to_list in
  List.map (fun c -> c |> YBU.member "code" |> YBU.to_string) cards
  |> Lwt.return

(* val do_something_else_1 : unit Lwt.t *)
let do_something_else_1 =
  do_something_1  >>= fun cards ->
  List.iter (Printf.printf "%s\n") cards |> Lwt.return;;
;;

Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run do_something_else_1; print_endline "Done_1!"
;;
Lwt_main.run (do_something_else ()); print_endline "Done!"
;;
2 Likes

Hi,

I recently published zeit which is client library for a JSON REST API using cohttp-lwt, yojson and ppx_deriving_yojson. You might be interested by the Client module.

2 Likes

Managing json APIs is what atdgen was designed for. You write your types and atdgen derives the OCaml code to convert your data from/to json safely. It also provides ways to deal with APIs that don’t fit the OCaml type system perfectly (json adapters being the most flexible but last-resort method).

These are the type definitions that we used at my previous company for using the Google Calendar API: https://github.com/esperco/esper-gcal-api

Thanks for the reverts everone!

I have since started looking into atdgen as a way to use typed-json interaction also zeit has been of help for sure :slight_smile:

While trying out this example, I did come across the following error


 (* val do_something : unit -> string list Lwt.t *)
let do_something () =
  json_body "http://deckofcardsapi.com/api/deck/new/shuffle/?deck_count=1" >>= fun sdeck ->
  get_cards sdeck 3 >>= fun jcards ->
  let module YBU = Yojson.Basic.Util in
  let cards = jcards |> YBU.member "cards" |> YBU.to_list in
  List.map (fun c -> c |> YBU.member "code" |> YBU.to_string) cards
  |> Lwt.return

;;
Error: This expression should not be a function, the expected type is
'a list

I think, perhaps there’s a typo I’m not able to resolve.

@abhi18av it is working fine in my utop. I can copy and paste the whole block, or each bit sequentially, and it is executed without errors. Are you sure get_cards was copied and pasted correctly?

Yup, I am quite sure that it has worked previously in my utop however, it’s only this time that it’s throwing up this error.

I have tried on a different machine and it’s the same type error.

Well, anyhow your code has been really helpful in helping me understand how to approach API clients in OCaml. A good next step, I feel is to package this up as a library for understanding how the package distribution ecosystem really works. Could you guide me here a bit ?

Sure. You can proceed as follows.

Create a new git repository for your project, put it e.g. on github. Let’s assume you just want to publish a CLI that runs the code that I had posted above. First remove the #require lines, all the double semicolons, and any other useless line. For readability you usually have the main entrypoint at the end of the file as let () = ....

Put the code in src/mything.ml and create democli.opam and src/dune, the latter with the following content

(executable
 (name mything)
 (public_name democli)
 (libraries cohttp-lwt-unix lwt yojson))

You should now be able to build the executable with dune build. You now need to populate the opam file. Pick another file as a reference. Once that is written and you have committed all the changes, install dune-release with opam. Releasing the package is now a matter of dune-release tag and dune-release, see also https://github.com/samoht/dune-release

It’s two years later, so I thought I’d drop a note with great detailer for the next passer by for what worked well for me.

  • atdgen is neat, but by default in their demo/instructions serializes your JSON to a binary format, requires Biniou (not optional, even though the instructions don’t suggest as much), and Biniou is not maintained.
  • yojson seems to be active and the the hip thing. i installed yojson and ppx_deriving_yojson

Draft your type:

(* MySweetData.ml *)
type my_sweet_data = {
  a: int;
  b: string;
} [@@deriving yojson];;
(* im actually in an ocaml/reason hybrid project, wasn't sure how to do this in reason syntax *)

Configure your build to support the ppx:

(library
 (name lib)
 ...
 (preprocess (pps ppx_deriving_yojson))

Then some nice serialization/deserialization functions just show up conveniently:

MySweetData.my_sweet_data_of_yojson (* deserialize *)
MySweetData.my_sweet_data_to_yojson (* serialize *)

(* ... *)

MySweetData.my_sweet_data_of_yojson (Yojson.Safe.from_string some_json)

Donzo.