Job queues and schedulers in Ocsigen or Dream

Ecosystem
1

Hello,

I am curious how users of Ocsigen and Dream run background or recurring jobs. I think most app would need them to send emails or generate reports. Previous discussions mention using Lwt.Async [0][1]. For email specifically, an Ocsigen [2] and a Dream [3] use it as well. But these approaches seem pretty low level.

I was looking for something like Laravel Queues [2], which is easy to use, bundled with the framework, and can persist tasks in the application database. Other examples are Oban, Celery, Sidekiq…

I know Dream is young. I am more surprised about Ocsigen, being so mature and batteries included, not bundling a scheduler or mentioning a blessed one in the docs (though I might have missed it).

So I am really curious, how do you do jobs in production? Do you hand write them with Lwt? Do you have a bespoke Job abstraction on Lwt that is or isn’t available as library? Do you interface with cron, or any of the scheduler mentioned above?

@Vincent_Balat, I would appreciate greatly if you can share how you do it with Ocsigen please.

Thanks

[0] Recommended Practice for handling background/asynchronous tasks with Dream
[1] How to schedule a task in OCaml? - Stack Overflow
[2] ocsigen-start/src/os_email.eliom at 10f381294c9f4acf66aa1a89eaf63bc502ba56f5 · ocsigen/ocsigen-start · GitHub
[3] How to send email from Dream
[2] Queues - Laravel 11.x - The PHP Framework For Web Artisans

4 Likes
2

Hello,
Indeed, that’s not something provided by Ocsigen and I think the reason is that the basic version that we are using is extremely simple to implement (and it will probably depend on your database and the kind on actions you want).
Basically, we just record all the actions in a database, and when the server start, a loop that will take the first one and call Lwt.sleep, execute the action, then take the next one, etc.
Be Sport has a mechanism to do that, and also to run periodic actions. We would be happy to upstream it or even to implement a more complex mechanism as in Laravel if someone is interested.
Vincent