Hi all!
The Ocsigen team is organising a public meeting in which we’ll be discussing the migration from Lwt to effect-based concurrency, updates about work in progress (wasm_of_ocaml, Ocsigen-i18n, …).
We welcome user suggestions & questions, please join us Monday the 14th of April at 1pm (France/GMT+2) at the following link: https://meet.google.com/zdm-krfj-rcw
The Ocsigen public dev meeting will start in 1pm (France/GMT+2) today here: https://meet.google.com/zdm-krfj-rcw Feel free to join!
Thank you for the attendance! This was a very dense meeting 
The minutes of the meeting are available here
Extend Dune for multi-tier apps
What are the expectations for this one?
Ahrefs is using melange, so it’s a bit different, but there are a lot of similar needs. I’d be curious to know if there’s any overlap.
It’s mainly about supporting the multi-tier compiling scheme. We need to compile everything twice with different PPX extension, one for the server side and one for the client side. The current solution relies on some scripts doing things like generating a Dune file at compile time, but it is not at all satisfactory and difficult to maintain (and even to use by application developers).
I don’t know exactly how to formulate my question, and I don’t have much first-hand experience with lwt, but I’ll attempt to ask anyways : I vaguely recall reading somewhere that a point of difference between lwt and async was that in lwt, you had as part of the monad a way to represent errors. I read it as essentially saying that 'a Lwt.t is essentially closer to a 'a Or_error.t Deferred.t rather than a 'a Deferred.t. Is that right?
If so (and perhaps more generally), I was wondering what were your thoughts on this when moving from lwt to direct-style. If not resorting to using exceptions in the resulting code, you would still syntactically be really close to a monadic-concurrent style, due the handling errors still be monadic. I’m generally curious about this topic, as the direct-style appeals to me also from a code-simplifying standpoint (let us be free from monads!) but I don’t think we’re quite ready for untyped/untracked exceptions yet.
Just curious to hear your thought, from the perspective of a big refactor in a big project. Thanks!
Hey vincent,
We use copy_files in melange-land and we aren’t satisfied with the current approach. Virtual libs somehow fixes but introduce a bunch of issues as well (opaque interfaces mostly).
Did you ever designed an ideal module system for cross-targets like this scenario? I believe MirageOS have similar problems and I wonder if we could analyze what would need to change/improve in both dune and OCaml to accomplish a better situation
(Disclaimer: I’m not doing the work of migrating, so someone else might have more insight into something more specific.)
Lwt has a built-in way to represent errors, but only errors in the form of exceptions. So when you do raise Exit somewhere inside your Lwt code, the promise that the code is making progess on ends up rejected.
A rejection propagates through the mondaic binds much like an exception does through the call stack: it just traverses everything until it reaches an exception handler. The Lwt functions to install exception handlers in the monad are based on the OCaml constructs to install exception handlers in the stack. E.g.,
Lwt.catch (fun () ->
… code here …)
(function
| Invalid_argument _ -> … exception recovery here …
| exc -> Lwt.reraise exc (* can't do partial functions *)
equivalent to
try
… code here …
with
| Invalid_argument _ -> … exception recovery code here …
(It’s not that simple because there’s also Lwt.on_failure. But the bulk of exception management should be similar enough to direct-style programming that it seems mostly automatisable.)
Seeing your code illustration was helpful to me - thank you!
I’ve been reflecting on this topic while experimenting with refactoring to direct-style programs using Deferred.Or_error. For example, I’ve been working with code like this:
let open Deferred.Or_error.Let_syntax in
let%bind x = f1 () in
let%bind y = f2 () in
g x y
and wondered about missing opportunities if stopping at:
let open Or_error.Let_syntax in
let%bind x = f1 () in
let%bind y = f2 () in
g x y
thus trying to explore further more invasive refactorings based on an initial transformation such as:
Or_error.try_with (fun () ->
let x = f1 () |> ok_exn in
let y = f2 () |> ok_exn in
g x y |> ok_exn)
(and then trying to propagate the change of paradigm going inside f1, f2, etc.).
This line of questioning will primarily concern patterns where results are used inside promises, which may not represent the bulk of use cases depending on the application (are result promises common among lwt users?).
Personally I don’t know where to stand. Maybe I am wrong to try and make the switch to direct-style a time to reconsider the question of Results vs Exceptions. I will be curious to learn about “reports from the field” as people working on big refactors form opinions and experience on this. Thank you!
There’s the dedicated Lwt_result module which is part of the normal distribution of Lwt. The biggest open-source codebase I most recently contributed to used the Lwt+result monad very extensively. I’d say it’s common but not universal to use lwt+result combined.
Note that translating monadic code into direct-sytle code gives you benefits in terms of syntax (no more * after your let) but for the specific case of IO/Promise/Deffered/scheduling there are additional benefits:
For these reasons, I think it’s a good step to convert Lwt+result code into result code.
What is a Multi-Tier app?
Multi-tier programming: client and server sides of the application are written using the same language, and as a single code. Annotations in the code are used indicate on which side the code is to be run. The client side parts are automatically extracted and compiled into JavaScript to be executed in the browser. This gives a lot of flexibility to the programmer, allowing for example to generate some parts of pages either on client or server code according to your needs. It also makes communication between server and client straightforward, as you can use server side variables in client code, or call a server side OCaml functions from client code.
Source: Introduction