Is there a drop-in solution for serving responses from cache in Dream?

Context:

  • Serving a website built with Dream.
  • static assets are served either compressed or uncompressed, these can, and would ideally, be cached indefinitely.
  • certain HTML responses should be cached for a limited amount of time.

Common approaches how to do this are to integrate an existing key-value store that provides expiration functionality and integrate this, or to use a package that implements an in-memory cache with expiration.

How have you solved this and which package(s) do you recommend?

Do you think it would be worthwhile to have a package that provides a configurable Dream middleware that caches the response?

1 Like

I’m going to be really boring here and suggest that we should get in touch with Cloudflare and Fastly and see if they would be interested in sponsoring the OCaml website with their CDN service. This would easily solve the caching issue and provide DDOS protection, nice metrics dashboards, TLS certificates, IPv6 support, all on top. [EDIT: AWS CloudFront CDN’s Free Tier may also work!]

If we are going to roll our own cache, my next boring answer :slightly_smiling_face: is to use Varnish which is a specialized tool for exactly this use case: Introduction to Varnish — Varnish HTTP Cache

Why not code something up in Dream? Well, people can do all sorts of funky things with HTTP. It can become a full-time job just keeping up with all the different ways that people can try to mess around with your infrastructure. My recommendation is to let the specialists worry about it.

8 Likes

I’d suggest first checking whether asset versioning works correctly and whether the correct cache headers are being sent by the origin server: that’ll help regardless of what cache implementation you end up using, and would go a long way to solving your point number 2.

E.g. with latest Chromium doing a Lighthouse report on ocaml.org shows that quite a lot of images are not served with a long cache lifetime (although other assets like .css and .js are properly versioned based on their digest…).
Here is an example of a URL that is not cached properly: https://ocaml.org/img/home/emac-preview.png, maybe something went wrong with the routing here, there is some code there to route assets and media based on digest: ocaml.org/router.ml at 7480aca339a0340a78005248d5798d0d4706b064 · ocaml/ocaml.org · GitHub
And some code here to respond with some very basic caching headers: ocaml.org/static.ml at 7480aca339a0340a78005248d5798d0d4706b064 · ocaml/ocaml.org · GitHub
Once the code is fixed to serve headers for all static/immutable content correctly, that Dream middleware could potentially be generalized and released as its own package.

Also I notice 4 different, small .css files reported in the critical chain by Lighthouse. If they’re all static perhaps they could be merged into another static css at “build time” and serve just the one?

Reference (including some flowcharts and links to 2 more articles which contain more in-depth examples): Prevent unnecessary network requests with the HTTP Cache

2 Likes

@yawaramin makes some excellent points that should definitely be explored. If all of that fizzles out for whatever reason I’m going to assume that the web page in question is being fronted by something like haproxy or nginx both of which have built-in caching that is very configurable and performant.

2 Likes

@yawaramin, you’re completely right that sticking to something existing and battle-tested is always a safe and performant solution. For the use cases I listed, it’s clearly reasonable to use any of the solutions you listed.

@edwin Indeed, you’re completely right: the first line of defense here is proper asset versioning so that assets are served with a long cache lifetime, as we added in serve css/js assets under digest urls, cache-control by sabine · Pull Request #929 · ocaml/ocaml.org · GitHub. We didn’t go through the other assets yet, that’s an open issue which is very suitable for an OCaml newcomer to contribute to: Use `Ocamlorg_static.Asset.url` for all assets · Issue #1049 · ocaml/ocaml.org · GitHub. For the assets associated with the markdown content, it’s also a manageable issue that we would be very happy to guide and teach a contributor to work on. :slight_smile:

(When I get annoyed enough at the issue, I will fix it myself, but chances are that there will be things that are much more urgent/impactful in the bigger scheme of things, so they get worked on.)

And some code here to respond with some very basic caching headers: […]
Once the code is fixed to serve headers for all static/immutable content correctly, that Dream middleware could potentially be generalized and released as its own package.

Indeed, if there is no dream middleware that already allows you to specify cache headers using a typed interface, this sounds like a good idea! (For example, when I fiddle with nginx, I tend to end up wasting so much time debugging typos or misplaced commas, it’s not funny - maybe other people do well at that, but I really like the guard-rails that types provide).

Django, Rails and its relatives do come as a batteries-included solution that enables people who don’t yet know about all these details to build websites by providing them with facilities that impose reasonable defaults and options. Their value lies in limiting the amount of available choices to the currently reasonable ones. That comes with some maintenance work, as what is reasonable sometimes changes over time as technology evolves. IMO Dream is doing a great job at this, and I am thinking… like Django and Rails, which have a flourishing ecosystem in their own, it would be cool to have that for Dream as well.

But I’m also coming from the perspective of time being quite limited, so whatever we do it should be sensible in regards of not reinventing the wheel from ground up but instead incrementally adding to the ecosystem in places where people would actually make use of it. :slight_smile:

For example, when there are some production-ready bindings/clients to general-purpose key-value stores like memcached, Redis, my impression is that it would be reasonable to write a library that provides a generic cache-interface with pluggable backends (if a reasonable such library doesn’t exist already! If it does, we obviously use it). Checking on that, I see that a client for Redis exist: redis 0.7.1 (latest) · OCaml Package, searching for memcached, I see an abandoned-looking project built on the core library.

This cache-interface library, in turn, could be integrated into a Dream middleware to cache HTTP responses. But the same cache-abstraction could as well be used to cache other data than just HTTP responses, e.g. in a session cache.

@rbjorklin Indeed, we are in a deployment situation where we, in principle, could add nginx or haproxy to our Docker container. However, I think… this might be a situation where it’s worth to at least check if the OCaml community’s ecosystem has us covered already, and, if not, consider adding to the ecosystem. IMO, adding bindings and clients to commonly-used technology would be, in general, useful.

Considering that tools like haproxy and nginx also have a learning curve attached, I’m not instantly taking the “easy for me” route of adding nginx. Instead, I’m genuinely curious if you think, in this case, adding to the ecosystem would be both useful and feasible.

1 Like

If we take a look at Django’s caching feature: Django’s cache framework | Django documentation | Django

This is quite a lot of work to implement. Sure, maybe some of it can be omitted for an initial implementation, but I feel like a lot of it is essential complexity, e.g. caches need to respect the Vary header e.g. if we are serving localized translated versions of the pages on the same URL depending on the user’s Accept-Language header.

All I can say is that if someone has implemented and is maintaining the equivalent of this in a modern OCaml stack that is compatible with Lwt and Dream, I will be very surprised :slight_smile:

2 Likes