Excited about Dream web framework

how strange - I would have expected the 5 pre-defined xml entities here - Extensible Markup Language (XML) 1.0 (Fifth Edition) - but &apos: is different. How that?

Edit: Got it! hex!

1 Like

I used the hex entity because of this in the OWASP recommendations:

' not recommended because its not in the HTML spec (See: section 24.4.1) ' is in the XML and XHTML specs.

This probably refers to the HTML4 spec, because it is in the HTML5 spec, and has always been, in the years I’ve worked with it. Nevertheless, I decided to be paranoid and prefer the hex entity.

2 Likes

I’ve tested it yesterday, and it is awesome. Can’t wait to see it available on opam to use it on a new project

4 Likes

Do we have a comparison table between Ocsigen, Opium and this new one?
And where could we create one?

I can’t think of a good home for this content. Maybe host on ocamlverse, and have each framework link to it from their readme.

I don’t have a table ready, but I can share a list of comparisons with Opium from my notes. I can’t compare with Ocsigen, because I never tried it — it always seemed more complicated than what I needed.

The list below is some of what I consider to be advantages of Dream. Some of these could be addressed in Opium with work, but others, I think, are API-level design limitations.

Compared with Opium, Dream…

  • Supports HTTP/2, HTTPS and WebSockets, i.e. all modern Web transport protocols. Dream will experiment with HTTP3. Thanks to @anmonteiro here.

  • Includes a templater. Opium strongly favors TyXML, which also comes with a nice syntax. In both frameworks, the templater is easily interchangeable, and I already plan to add a TyXML example to Dream today.

  • Is one flat module. I didn’t think that Opium’s submodules are necessary for either framework, and it is confusing to jump between them while reading the documentation. They make code more verbose, e.g. Opium.Request.header vs. Dream.header, which by a small trick, also works for responses.

  • Has a much simpler programming model. For example, Opium middlewares are wrapped in records, whereas Dream middlewares are bare functions which you can trivially define anywhere like any others. I also simplified everything else to clear concepts wherever possible, and tried to stick to base OCaml types.

  • Dream is even simpler than Rock, the simple under-layer of Opium.

  • Opium’s more verbose middleware API interacts somewhat pathologically with its lack of a full set of predefined middlewares. When you are then forced to define your own, you almost inevitably have to deal with Hmaps, S-expressions, repeatedly wrapping and unwrapping records and application builders, etc.

  • As implied by the previous point, Dream has much more built-in middleware and helpers for getting things done (yet all of which remains loosely coupled). See the API docs for a survey.

  • Dream’s helpers “bake in” readings of RFCs, security recommendations, etc. For example, calling Dream.set_cookie with only the basic arguments automatically generates the most secure cookie possible for your site, and Dream.cookie “knows” how to parse it. You can still undo all that automation with optional arguments and/or lower-level helpers. Likewise, the form helpers automatically both generate CSRF-safe forms, and parse them with CSRF checking. Current Opium doesn’t offer this, leading to a paranoid experience, where you spend the first several days to a week learning or re-learning security topics, and/or are afraid of writing too much unfixably insecure code if you choose not to learn right away. You should still learn the topics with Dream, eventually, of course :slight_smile: To help, Dream’s docs and examples include security discussions and links throughout. There is even an example of creating a vulnerable “app” and exploiting it.

  • Current Opium routing is shoehorned into a middleware-like model (app builders). If you like algebra, Opium looks like (X, *), though it uses a “fat” * operation to recover +. Dream has first-class routers from the start, and looks like (X, *, +) — that is, handlers, handler sequencing (middlewares), and handler choice (routing). Dream.scope implements a distributive law, correctly applying a chain of middlewares to a subset of your routes. All of this is more loosely coupled in Dream than in Opium, because middlewares and routing are fully independent concerns in Dream, and, apart from Dream.scope, middlewares and routes don’t “know” anything about each other.

  • Again on simplicity, in Dream, out of (X, *, +), handlers are just plain OCaml functions, handler sequencing (middleware) is just plain OCaml function composition, and the main reason handler choice (routing) is not just pattern matching, is the need to implement the distributive law in Dream.scope concisely.

  • Dream has unified error handling, where even errors occuring in the HTTP stack below your application are passed to your error handler for decoration with a response. Dream grew out of an internationalized project, and its goal is to not surprise users with any untranslated and undecorated low-level messages directly generated by, say, http/af, or a low-level adapter to it.

  • I kept Dream’s request and response types (its only “real” data types) abstract, so that they can be internally optimized. For example, if we find people tend to read and re-read the same header a lot, we might add some kind of mutable cache for its parsed value.

Opium’s “fat middleware” and “app” model does have the advantage of easily supporting introspection, which is not immediately available on Dream’s bare function middlewares. However, in Dream, because of first-class routing, site structure is given by routes, not middlewares (or app builders), and Dream routes are “fat” objects and will eventually support introspection. And, it’s still possible to define an introspection-friendly composition that decorates even middlewares with names as they are composed.

In any case, I didn’t think that introspection by default is worth the considerable syntactic overhead that it imposes on everyone, whether they need introspection or not.

This is just what I was able to quickly recover from the notes and recall, but there are more differences, as I did many detailed passes through the API, and probably will still, during the Dream alphas :slight_smile:

21 Likes

I activated the wiki in the Dream repo and created a page for this, in case people would like to get started.

I won’t have time to create a neat table immediately, as I am still writing some examples!

If we end up with anything on that page, we can move it to ocamlverse or elsewhere, later.

7 Likes

Probably, you should. I don’t find Ocsigen complicated for basic stuff (see Writing a basic Web site in OCaml ). And it is by far the most powerful Web framework in the world …

3 Likes

Even I defintely like Ocisgen, the fact that it is not yet compatible with Dune (I’m dune-biased) makes me cold when I want to start a web project.

3 Likes

What is the story with client-side code? Does Dream have plans to make it easy to have client and server-side code? Or do you assume people will just use js_of_ocaml and communicate with the server via JSON/graphql? Would be nice to have a chat applications like the one in Ocsigen but written using Dream.

Ok just saw dream/example/w-fullstack-jsoo at master · aantron/dream · GitHub which answers partially my answer. Still, would be great to have a more complex full-stack-jsoo example like a Chat application.

BTW incredible work, incredible documentation and examples, and incredible clean/simple API.

4 Likes

I don’t have any Dream-specific client plans at the moment. However, I have added three examples showing full-stack client-server usage:

  • r-fullstack-jsoo uses Js_of_ocaml. Both client and server are written in OCaml.

  • r-fullstack-rescript uses ReScript. The server is written in OCaml, and the client in ReScript. The common code is written in OCaml, because ReScript can compile it.

  • r-fullstack-melange uses Melange, a fork of BuckleScript/ReScript. Both server and client are written in Reason. They could also be written in OCaml, because Melange can compile it.

Personally, I am leaning towards using GraphQL and basically just JS with a couple of files compiled from OCaml for shared strings and the like.

I want to look into improving server-side JSON handling, to make it more similar to how it is in JavaScript (but maybe still with some types!). Not sure how much is possible, though.

I wasn’t planning something like Ocsigen. But I am not sure where things will go. I lean towards making Dream immediately familiar to people who have, for example, done Web development in any other language, so the “basic” layer of Dream has to be clear and somehow well-done, as the highest priority. Anything inspired by the more advanced features of Ocsigen would be optional over that.

4 Likes

We cross-posted :slight_smile: I’ll add chat to the examples roadmap.

For now, chat-related examples are:

  • k-websocket for the obvious reason that WebSockets are good for chat.
  • h-sql stores comments from a comment form in a database, and serves them.

There are also examples with two obsolete ways of achieving async communication, that I just needed to test to make sure Dream is behaving correctly. I’m including them because they generate fake chat messages for clients to retrieve, and maybe somebody knows when you might want to use one of these methods over WebSockets at this point.

2 Likes

Nice. I’m very interested also in the w-react-spa in your roadmap.

Also plans to explain how to deploy a dream server to cloud platforms would be great (e.g., how to deploy an ocaml server to AWS), but maybe there’s already good documentation about that in other ocaml web frameworks?

2 Likes

I think main difference is that opium is an unopinionated webserver so you would probably choose your dependencies to build your stack while dream opinionated some choices for you.
This way dream should probably rather be compared to Sihl, an opiniated webserver built on the top of opium + catqi although Dream is built on the top of itself (its core is not a 3rd party dependency).

Whatever dream API is quite neet and it’s really enjoying to see emulation about webserver in OCaml.
@antron did already an amazing job and a “not yet alpha” with as much examples and middlewares is terrific :slight_smile:

I used opium and enjoyed it too

2 Likes

Quite the opposite :slight_smile: The basic “Dream” core is less opinionated than Opium. It looks like we will eventually factor that core out to port Dream to Mirage, maybe to Node, etc. You can see the core separated out in the directory structure here — that core directory is src/pure.

The “opinionated” impression of Dream probably comes from that, I carefully considered what

  1. everyone typically needs anyway,
  2. can still be included without adding any large or system dependencies, and
  3. can be added in a way that it is still strictly optional.

…and added that to Dream under a neat interface. Caqti is a good example here, because the Caqti base package does not depend on anything that Dream doesn’t already pull in on its own, so the only cost to including and recommending Caqti is only compilation time during dependency install. And:

  1. because Dream is otherwise unopinionated, you can simply ignore the Caqti support, and use any other database, no database, or any other means of accessing SQL databases;
  2. Dream does not depend on caqti-driver-sqlite3, etc. — you choose those as a user, if you need them, and these are the parts that actually have system dependencies. Dream does not pull those system dependencies in on its own.

So this is not truly a matter of being opinionated, but strictly some convenience provided by an otherwise unopinionated library.

Dream is absolutely not like Sihl in this regard. By contrast, Sihl is a much higher-level framework, adds a lot of features, but at the price of e.g. threading SQL statements into its codebase. Dream simply has nothing like that, on that scale.

So, even though Dream is lower-level than Opium, Dream’s convenience features do end up extending Dream’s reach to slightly higher levels than Opium, encroaching slightly on Sihl — but only on the very lowest layers of Sihl. If I could offer an ugly diagram (keeping in mind, for other readers, than Sihl is based on Opium):

           Higher level ->
+-------------+--------------------+
|    Opium    |        Sihl        |
+-------------+--------------------+
+-----------------+
|      Dream      |
+-----------------+

EDIT: Not to scale!

I really want to emphasize that Dream is about doing things like what Opium does, in a way that I considered to be better-factored and easier to use. It’s not in any way equivalent or similar to Sihl.

However, I do see the temptation to compare Dream to Sihl, because I do think that a substantial proportion of the true value of Sihl can be taken out and added in an unopinionated way to an Opium-like framework as mere convenience functions, which is, of course, as I said above, what I did with Dream :slight_smile:

(EDIT for clarification: that’s not to say that Dream’s convenience features that partially overlap with Sihl’s are based on Sihl’s — they just occupy equivalent niches).

I also believe, based on recent Opium PRs, that there are efforts between Sihl and Opium to do just that, upstream some things from Sihl to Opium. So, speculating, but I don’t think I’m alone in making this observation.

8 Likes

Just to illustrate the comparison, in Dream, you always trigger any of the convenience stuff explicitly. For example, if you don’t want to use SQL, or Dream’s SQL or Caqti helpers, you simply never call Dream.sql, etc, and none of Dream’s SQL-related code runs at all.

The only thing you will have paid is compilation time of caqti during opam install dream, but not any of its dependencies — they are already required by Dream or its other dependencies. So Dream’s SQL code becomes just some dead code in your final binary. I don’t know if current OCaml will eliminate it.

By comparison, Sihl, at least as of two months ago when I tried it out, appeared to require you to use an SQL database hosted by a separate PostgreSQL or MariaDB process — at least the starter examples did (and this is fine for a different use case).

So this is completely different. Dream “degenerates” neatly into a bare framework the less its convenience helpers are used.

Everything “extra” in the API is built in this opt-in way.

7 Likes

I would be very happy to link, from Dream, to any middleware libraries, database access libraries, etc.

This is again on the subject of being unopinionated — Dream just offers a programming model and some basic default choices that are easy to bypass. It is my …dream… that people would experiment with options for Web apps. I would be happy to create a section in the README for compatible projects and/or insert links from the API docs and examples in the right places.

Some of Dream’s examples already rely on external projects. w-fullstack-jsoo is one obvious case, as it shows js_of_ocaml.

However, Dream also has w-tyxml, showing how to use TyXML instead of Dream’s default templater. I think TyXML is a good and perfectly valid choice (indeed, I’ve made some non-trivial contributions to it in the past), and I am happy if someone gets started with Dream’s simple templates, gets used to OCaml, finds out about TyXML and wants to switch to it, and is helped by an example.

I’d also be happy to link to project templates and starter projects showing how Dream can be composed with various client-side libraries, build systems, etc. Dream itself tries to be minimal — you should be able to start with a single source file, grow and factor it gradually, and understand always exactly what you added and what is going on.

However, in many cases, it is practical to start with a large project skeleton that already has a lot of the “outside” set up. Dream’s own full-stack examples offer that to a limited extent (and a few more will be added). But I welcome other “opinions” about how to do it, and will link to them.

6 Likes

This post is pretty accurate in my opinion.

Sihl aims to be a set of composable service interfaces for common infrastructure components in web apps like user handling, job queues, authorization, block storage, caching and so on. The goal is to cover those infrastructure concerns that are in between the “lower level” concerns (such as HTTP API, routing, middlewares, persistence API and others) and the application.

The less time we spend implementing CSRF middlewares, the more time we can spend developing nice dashboards for the job queue. So I am exited to see that those (from Sihl’s point of view) lower level concerns are taken care of, by Opium and now Dream.

2 Likes

Just want to clarify, there is no CSRF middleware (or helper) in Opium right now AFAIK. Part of the motivation for the convenience parts of Dream, including CSRF checkers, was to read all the OWASP best practices stuff once, encode it into an implementation in Dream, and forget about it forever (save for security updates) — but link to it and offer a nice tutorial treatment for new readers (or when a refresh is needed), and in such a way that it can be immediately acted on by calling nice, ready-made helpers. Rather than the current way of spending weeks choosing ciphers, comparing protocol obscurities, and other such details.

7 Likes