Receiving/sending http requests in an Ocaml program

Learning
1

Hi,
I have an Ocaml program that does its job.
Now I would like to make it deliver services over the internet as soon as possible.
I’m not experienced in the web side of an Ocaml program. I’ve just studied some tutorials.
Could you please indicate me how to setup that in a straigth manner?

I understand that I need two things:
1/ receiving http request:

  • get incoming data flow from listened port
  • transform (json/xml) data in OCaml values

2/ sending http request:

  • transform Ocaml values in json data
  • send data over http (http request targetting IP:port)

That should be pretty simple for people doing that everyday.

I see that Yojson can read a json data flow (Yojson.Basic.from_channel) and print a json data (Yojson.Basic.pretty_to_string).
So it should answer one requirement. Am I right?

I intended to wrap curl command to send request, but there should be more elegant methods.

My main question seems to be: how can I receive and send http requests using a json (or xml) object?

Thanks

2

Hi Luc,

yojson is a good choice for the JSON part. Here are two simple options for the web server:

  • either cohttp [0], to have total control on your server
  • or opium [1] which seems well-suited for your task and is more high-level (opium runs on top of cohttp)

HTH,
Philippe.

[0] https://github.com/mirage/ocaml-cohttp
[1] https://github.com/rgrinberg/opium

1 Like
3

Thanks Philippe,
it’s very helpful.

I could run the opium examples instantly, once I fixed the ocamlbuild command (‘ocamlbuild -pkg opium hello_world.native’ instead of ‘ocamlbuild -pkg opium.unix hello_world.native’). Maybe it’s related to one of the packages required to compile the cohttp examples: async_extra, async_kernel, async_unix, conduit-async (see hereafter).
Anyway the command ‘dune build @examples’ works.

The ocamlbuild cohttp command works fine on server_example.ml. I just had to find which port the server is listening to (port 8000). But in fact it’s written in the code.
The ‘dune build @examples’ command requests to install a bunch of core and async packages!
In fact, using server_example.ml only requires lwt, cohttp, cohttp_lwt_unix.

I imagined using a curl ocaml package to make http requests, but I see that cohttp also allows to make http requests. Great.

I read a warning regarding how to compile the cohttp package when handling https (iinstall ssl and lwt_ssl before cohttp).
I also remember trouble using 1.1/1.2 TLS.
can you pls. tell me the key points I need to look after to get a reliable web server?
Thanks

4

Thanks for catching that. I’ve updated the README to fix the installation instructions & the instructions to compile via ocamlbuild. → opium/README.md at 2c07589bc6e9694229e0ecde0d01ae167afe4fe0 · rgrinberg/opium · GitHub

1 Like
5

Already updated? Great. It’s a good thing for newcomers to OCaml because when I started to learn OCaml, I’ve often been stuck in the middle by small mistakes in documentation that I couldn’t see because everything was new to me (OCaml, packages, etc.).

6

BTW, how do you monitor what’s going on between a server (OCaml) and a client (js) or (another server) through http requests? What tools are good to see what’s really happening?

Also, I’m used to learn and fine tune my OCaml program in the Toplevel.
I think I can send http requests, but I can’t receive requests in the Toplevel.

For example, server_example.native works fine but if I try to evaluate/execute server_example.ml (the one on page https://github.com/mirage/ocaml-cohttp with Lwt), I get that kind of error:

# let () = ignore (Lwt_main.run server)

Exception: Unix.Unix_error (Unix.EADDRINUSE, "bind", "").
Raised at file "src/core/lwt.ml", line 2998, characters 28-29
Called from file "src/unix/lwt_main.ml", line 26, characters 8-18
Called from file "//toplevel//", line 11, characters -404--383
Called from file "toplevel/toploop.ml", line 180, characters 17-56

Is it normal? (unix, thread…), or Is there a chance to make this work without compilation?

7

Hmm i just tried to run the example under utop and i’m able to run the server. Your error seems like the port you are trying to bind to is already under use.

8

Please open issues against opium if there is something missing/unclear in the examples or documentation :slight_smile:

9

You’re right (hum, Unix.EADDRINUSE was clear).
Does your program running in utop stops listening to the port when you stop it? (the port is still listened to, in my case)

I have to figure out how which incoming requests the server sees and what he sends back to the caller or sends to a another server. Especially, I have to accurately understand things like url redirection and url cancellation and errors handling, while I’m not used to work with networks at this level.
In order to do that, I want to insert a “pause and read request” sequence in the server code (like in a debugger tool). I think it’s simple to do.
However, is there any tool that can be useful to validate and fine tune the server?
I’m thinking about wiresharck that I know a little bit, and I heard about fiddler I’ve never used.
Is there any Ocaml tool that can help doing that?

10

It stops listening when i exit utop.

I’m not sure i know of any OCaml tool that has the “pause and read request” functionality.
What you might be able to do is to create a request and then run it through your “handler” function without going through the server. Since its all just a request -> response function anyways. Doing that should allow you to use any regular tool you use to debug OCaml.