How to write a simple socket-based web client (for Docker)


#1

I’m trying to create a simple client for Docker that uses the unix:///var/run/docker.sock

Conduit_lwt_unix.init ~src:"unix:///var/run/docker.sock" () >>= fun ctx ->
  let ctx = Cohttp_lwt_unix.Client.custom_ctx ~ctx () in
  Cohttp_lwt_unix.Client.get ~ctx (Uri.of_string "http:/version/") >>= fun (resp, body) ->
  let code = resp |> Response.status |> Code.code_of_status in
  Printf.printf "Response code: %d\n" code;
  Printf.printf "Headers: %s\n" (resp |> Response.headers |> Header.to_string);
  body |> Cohttp_lwt.Body.to_string >|= fun body ->
  Printf.printf "Body of length: %d\n" (String.length body);
  print_endline ("Received body\n" ^ body)

let () =
  Lwt_main.run (Lwt_unix.handle_unix_error client_test ())

The error is ‘Failure “Invalid conduit source address specified”’ - I guess the Conduit_lwt_unix.init doesn’t like the socket address. Can anyone suggest a simple way forward here?
Thanks


#2

According to the sources conduit uses Lwt_unix.getaddrinfo and I think this does no longer work with unix domain sockets. Please correct me if I am wrong. I have tried with "/var/run/docker.sock" but I get an empty list as well


#3

You almost had it – just need to create a custom resolver in order to map a hostname into a Unix domain socket. There is no standard way to do this in HTTP (something has to be present in the Host field in HTTP/1.1), so we just make up a name like docker.

open Lwt.Infix

let t =
  let resolver =
    let h = Hashtbl.create 1 in
    Hashtbl.add h "docker" (`Unix_domain_socket "/var/run/docker.sock");
    Resolver_lwt_unix.static h in
  let ctx = Cohttp_lwt_unix.Client.custom_ctx ~resolver () in
  Cohttp_lwt_unix.Client.get ~ctx (Uri.of_string "http://docker/version") >>= fun (resp, body) ->
  let open Cohttp in
  let code = resp |> Response.status |> Code.code_of_status in
  Printf.printf "Response code: %d\n" code;
  Printf.printf "Headers: %s\n" (resp |> Response.headers |> Header.to_string);
  body |> Cohttp_lwt.Body.to_string >|= fun body ->
  Printf.printf "Body of length: %d\n" (String.length body);
  print_endline ("Received body\n" ^ body)

let _ = Lwt_main.run t

Run this with CONDUIT_DEBUG=1 in the environment and I get:

Resolver static: http://docker/version ((name http) (port 80) (tls false)) -> (Unix_domain_socket /var/run/docker.sock)
Response code: 200
Headers: api-version: 1.37
content-length: 574
content-type: application/json
date: Fri, 23 Mar 2018 17:27:32 GMT
docker-experimental: true
ostype: linux
server: Docker/18.03.0-ce-rc4 (linux)


Body of length: 574
Received body
{"Platform":{"Name":""},"Components":[{"Name":"Engine","Version":"18.03.0-ce-rc4","Details":{"ApiVersion":"1.37","Arch":"amd64","BuildTime":"2018-03-15T07:42:29.000000000+00:00","Experimental":"true","GitCommit":"fbedb97","GoVersion":"go1.9.4","KernelVersion":"4.9.87-linuxkit-aufs","MinAPIVersion":"1.12","Os":"linux"}}],"Version":"18.03.0-ce-rc4","ApiVersion":"1.37","MinAPIVersion":"1.12","GitCommit":"fbedb97","GoVersion":"go1.9.4","Os":"linux","Arch":"amd64","KernelVersion":"4.9.87-linuxkit-aufs","Experimental":true,"BuildTime":"2018-03-15T07:42:29.000000000+00:00"}

#4

Nice stuff. This kind of examples should go straight to the documentation imho!


#5

Excellent - thanks - those crafty Resolvers!


#6

‘get’ works a treat, but with ‘post’ I’m still having problems. The following curl works, but the ocaml trips up with a “resolution failed: name resolution failed” -

curl --unix-socket /var/run/docker.sock -H "Content-Type: application/json" \
  -d '{"Image": "alpine", "Cmd": ["echo", "hello world"]}' \
  -X POST http:/containers/create

with ocaml -

let resolver =
    let h = Hashtbl.create 1 in
    Hashtbl.add h "docker" (`Unix_domain_socket "/var/run/docker.sock");
    Resolver_lwt_unix.static h in
  let ctx = Cohttp_lwt_unix.Client.custom_ctx ~resolver () in
  let headers = Header.init () in
  let headers = Header.add headers "Content-Type" "application/json" in
  let body = `O ["Image", `String "alpine"; "Cmd", `A [`String "echo"; `String "hello world"]] in
  let body = Ezjsonm.to_string body in
  let body = Cohttp_lwt.Body.of_string body in
  Cohttp_lwt_unix.Client.post ~ctx ~body ~headers (Uri.of_string "http:/containers/create") >>= fun (resp, body) ->
  let code = resp |> Response.status |> Code.code_of_status in
  Printf.printf "Response code: %d\n" code;
  Printf.printf "Headers: %s\n" (resp |> Response.headers |> Header.to_string);
  body |> Cohttp_lwt.Body.to_string >|= fun body ->
  Printf.printf "Body of length: %d\n" (String.length body);
  print_endline ("Received body\n" ^ body)

giving an error of -

Fatal error: exception (Failure "resolution failed: name resolution failed")
Raised at file "src/core/lwt.ml", line 3008, characters 20-29
Called from file "src/unix/lwt_main.ml", line 42, characters 8-18
Called from file "test/test.ml", line 87, characters 8-22

#7

For me neither works as it is, but both are working if I replace http:/containers/create with http://docker/containers/create.


#8

Thanks - works for me too - should have seen that!


#9

What the custom resolver is doing here is replacing the docker part of a URI with a domain socket, so it’s important that all your URIs that connect to the Docker socket contain that particular hostname. To explain further…

In a conventional name resolver such as getaddrinfo(3), only internet address (AF_INET or AF_INET6) are usually returned. What we want to do is to resolve some hostnames into non-Internet things, such as Unix domain sockets or (in the case of MirageOS) custom shared memory endpoints such as vchan.

You might wonder at this point why we can’t just have a URI like unix://foo.bar instead. The reason is that the schema we want to use is still the http:// protocol, but connecting over a non-TCP transport. If we maintain the use of the http:// schema, then it also becomes easy to figure out what the Host: header included in the request should be as well.

Resolvers in Conduit are powerful but very undocumented, and the API could use a lot of cleanup. The suggested documentation patches would be most welcome, and I would love to see Docker API bindings in opam as well :slight_smile:


#10

You may be interested by the docker-api package (a new version should be released soon). You are welcome to submit PRs to add functions you care about.


#11

Excellent news - this is a great addition - thanks Chris