Tutorial: Full-Stack Web Dev in OCaml w/ Dream, Bonsai, and GraphQL

Learning
, , , ,
1

Hi everyone! I’ve written a tutorial blog series about full-stack web development in OCaml, and wanted to share it here.

Last semester, I took Penn State’s CMPSC 431W, where our final project was to build a database-driven web application. Since I’m fairly familiar with web programming through my work on Flarum and past internships/side projects, I decided to use this opportunity to explore the OCaml web development ecosystem. I used Dream for the backend, and Bonsai for the frontend.

While working on this project, I realized two things:

  • OCaml is very underrated for web development. In addition to all the language’s great features and safety guarantees, the ecosystem is pretty good! Dream near-perfectly coincides with my vision of backend webdev, and Bonsai has a great balance of flexibility/elegance and safety.
  • I couldn’t find realistic but accessible full-stack web projects in OCaml available for reference. I found tutorials for bits and pieces, but nothing that connected all the dots.

I really enjoyed writing an article series on hardware design with OCaml, so I decided to do so for web development as well. In total, I wrote 7 articles that walk through my project’s:

  1. Full-Stack WebDev in OCaml Intro. This includes some background on the project, and instructions for accessing the live demo (Chrome only for now).
  2. Backend WebDev w/ Dream and Caqti.
  3. Building GraphQL APIs with Dream
  4. Setting up Bonsai.
  5. Understanding Bonsai. I actually wrote the first draft of this before I decided to do a blog, while trying to, well, understand Bonsai. It goes over some underlying concepts (SPAs, Frontend State Management, Algebraic Effects, Monads), as well as Bonsai’s core design.
  6. Using GraphQL in Bonsai.
  7. Routing in Bonsai and Project Conclusion.

Additionally, the project’s README has a comprehensive overview of the tech stack, folder structure, and usage instructions. It also includes some reflections on design decisions and my experience working with these libraries.

I had a lot of fun writing these, and I hope they’re useful to anyone considering OCaml for web development. Would be happy to answer any questions or comments.

57 Likes
2

Hello, thanks for the write-up, looking forward to reading it. Quick note, the live demo does not load in Safari nor Firefox (macOS). Error in the FF console:

InstallTrigger is deprecated and will be removed in the future. utils.js:30:22
Uncaught TypeError: Illegal constructor.
    install_in_dom https://cmpsc431.ceramichacker.com/static/main.bc.js:667204
    <anonymous> https://cmpsc431.ceramichacker.com/static/main.bc.js:667240
    <anonymous> https://cmpsc431.ceramichacker.com/static/main.bc.js:667244
3

Ah I noticed that at some point but forgot to investigate, thanks for the heads up! The offending code is in a block annotated with "../.js/ppx_css.inline_css/inline_css.cma.js", and my project depends transitively on ppx_css through the Bonsai forms component library. Seems like an unfortunate collision between Js_of_ocaml, ppx_css, and Firefox, where Js_of_ocaml thinks that joo_global_object.jsoo_runtime.caml_get_global_data().Js_of_ocaml__Js[52][1].CSSStyleSheet is a valid constructor, and for some reason it isn’t in Firefox. Guessing the error is in this line, although I’ve never worked with this codebase so I can’t be sure. I’ve opened an issue in the Js_of_ocaml repo, and updated my instructions in the post to mention that it only works on Chrome for now.

4

Interesting project! Works in Chrome as you mentioned.

Does any category list products? I suspect that the graphQL requests of the ones which do, return a 500 error.

e.g., Wellness

5

Fixed, sorry about that. I set up a cron job to reset the database on an interval, but uploaded an outdated sqlite file with an incompatible schema. Also, forgot to mention this originally, but I recommend accessing the demo with one of the emails from this file or this file (all passwords are still here), as those users can also demo create/update functionalities.

6

Cool. Works now!

In your analysis, you mentioned:

Js_of_ocaml output bundles for Bonsai are absolutely huge (>30MB for this project!).

Chrome loads a main.bc.js of 4.9MB. Not that its a trivial size, but it falls quite short of 30MB.

This is quite an analysis. I see myself coming back to this later.

7

Great news ! Though the release of dream is pretty recent, so that’s probably why ocaml does not yet have a good reputation for web dev : people are not aware of dream.

2 Likes
8

This is a major difference when building with the ‘dev’ dune profile (the default) and the ‘release’ profile.
The ‘dev’ profile will perform separate compilation and will not do global deadcode elimination.

3 Likes
9

That makes more sense.

10

That’s actually the gzip-ed 30MB file. That being said, I recompiled the frontend under the release profile as suggested by @hhugo, and it came out to 3.5MB ungzip-ed, and 950KB gzip-ed. Still decently large, but no longer a deal breaker. This is great to know, thanks!

I agree that Dream is a game-changer here. Even though it’s fairly new, still in alpha, and has many features under active development, it’s already had an impact. I think that Bonsai is also very significant: its current API is very new and it’s a bit challenging to learn, but the central idea is extremely flexible and powerful.

I hope that as Dream and Bonsai grow and stabilize, we’ll see new libraries, community examples, and guides that will propel the OCaml web ecosystem forward. One particularly significant tool could be more extensive GraphQL tooling. The setup I described in my blog already offers a completely type-safe bridge between backend and frontend with minimal additional work needed, which is frankly incredible. More performance features (e.g. automated caching or a “smart” frontend client like Relay) would make this even better.

5 Likes
11

Just to get a sense, what is the file size of a React-based website or something equivalent?

12

Most frontend frameworks are a lot smaller: A RealWorld Comparison of Front-End Frameworks 2020 | by Jacek Schae | DailyJS | Medium

Of course file size is not everything, but bonzai + js_of_ocaml feels huge. From an Signals and Threads podcast I got the impression that filesize doesn’t really matter for Janestreet because of their internal network infrastructure.

I guess if you want to build frontend apps you should consider if the bundlesize is an issue for you or not. For an app it might not matter that much?

1 Like
13

11 posts were split to a new topic: Htmx/hc web development approach

23

I am a bit confused about Bonsai. This is what I’ve understood about Bonsai:

  • Its deep – it has a lot of abstractions. This makes it powerful but then also complex.
  • Bonsai yields huge amounts of JS

When I see bonsai code my head hurts. Many of the bonsai examples seem quite complicated.

I don’t know whether it would be:

(a) Worthwhile to persist trying to learn it (for example my head hurt with Rust but I persisted and was rewarded) or
(b) Conclude that Bonsai is Enterprise Java for the browser and abandon it :slight_smile:

My gut feel is that something like Bonsai would be better in a pure functional language like Haskell. For “Haskell like” code, OCaml can quickly become verbose and unreadable. For that kind of programming if you want to target the browser it is better to use just just something like Purescript or ghcjs (which is quite unwieldy too).

Part of the reason why I’m confused is that janestreet has produced a lot of community appreciated libraries. So if janestreet has made Bonsai it cannot casually be eliminated.

Can someone with some Bonsai experience give their frank review? The large JS bundles also seem worrying in general.

1 Like
24

Looking at it, I don’t see any issue with the design or verbosity per se. It’s similar to ReactJS or VueJS but translated to the OCaml world. It’s natural in this case to use monads to allow for computation to happen “behind the scenes” while keeping a functional and relatively simple style.

The main (and perhaps only) issue from my POV is the bundle size. It’s basically only appropriate for company intranets or as a desktop GUI framework. Since Jane Street is the developer, and this is their use case, they have little incentive to pour money into minimizing the bundle sizes.

1 Like
25

Apart from this blog post, I’m struggling to find reasonably sized Bonsai applications. Indeed many of the examples in the Bonsai repository seem to be missing dependencies. Does anyone have an example they can point to? I’m trying to understand how I structure consuming a stream of web socket updates and pushing them into an Incremental computation.

26

Eh, I think Bonsai is quite a lot more complicated than React (since React has multiple OCaml bindings you can do a fair comparison). My model of the design is that incr_dom = Elm architecture + incrementality and Bonsai = incr_dom + composability. It’s very powerful, but there are costs in terms of how easy it is to understand both the types and the underlying computation model (given a good vdom implementation it’s pretty easy to write an Elm-like framework but I think very few people could invent Bonsai). I think a very underexplored part of the design space is Elm architecture + composability without incrementality. I tried making something along those lines but didn’t get very far.

1 Like
27

I have seen this mentioned a few times (in the author’s post as well [4/x] Setting up Bonsai - Blog - Ceramic Hacker)

What I don’t get is what makes it ‘powerful’?

How is it different than the React reactive state libraries like https://recoiljs.org About MobX · MobX 🇺🇦 ?
They allow you to create computations, trigger them only when they are observed, automatically re-render only changed components etc.

I find the biggest issue with Bonsai to be the documentation, which does not provide a step-by-step introduction to the concepts, what Bonsai really is and what problems it solves

28
  • Its deep – it has a lot of abstractions. This makes it powerful but then also complex.

First and foremost, I think it’s worth noting that Bonsai was built to solve a specific problem. That problem is complex UIs displaying large amounts of data that need to update performantly in real time, and can’t crash during runtime. Bonsai is definitely overkill for informational sites, and honestly for mass-internet-facing web apps. But if you need to build some complex UIs that you need to be confident in, Bonsai is a very good choice.

I think there’s more applications than just that; there’s plenty of management dashboards or data visualization tools behind a login where bundle size doesn’t really matter all that much. In those cases, a few MB isn’t a dealbreaker.

Since you’ll need to update your models at arbitrary times, you should use the Var module to create mutable data containers that your websocket subscription should update. Then, incrementally consume that data where needed in your webapp. It would be relatively easy to write a simple actions / action dispatcher system around a Var to provide a more rigorous interface than just mutable updates. Honestly, that might be a good feature for Bonsai itself to add.

I really like Recoil: it’s my favorite JS state management system by far, and it fits in really well with React’s functional style. I’ve referenced it a few times throughout the blog series. That being said, JS (or even TS/Flow) as a language is limited, and can guarantee a lot less than OCaml. I think of Bonsai as somewhat of a generalization of Recoil + React, where instead of having separate, orthogonal trees for incremental state and the virtual DOM, Bonsai allows you to have a single incremental DAG through which both state and vdom can be cleanly represented.

Also, both Flow and especially TS really struggle with higher order types, and types for classes, which significantly restricts the components you can safely implement. Even in Flow, where there’s some React HOC support, you’re restricted by one specific interface, whereas in Bonsai, you can create whatever types you want, and manipulate them much more freely.

I also found Bonsai’s documentation challenging, but to be fair, the API has radically changed over the past few years. Hopefully we’ll see documentation and example improvements as Bonsai continues to stabilize. For now though, hopefully my blog series can somewhat help in this regard. If there’s any parts I didn’t explain well, or that you would like more detail on, I’d love to hear feedback!

10 Likes