I just want to point out that the other tests that end in -haproxy are using all cores just in a slightly different way than what @blandinw did.
I agree that these synthetic benchmarks are very helpful.
For someone that is starting out with a language it’s not always obvious what is a good sensible base implementation of a web server, particularly in OCaml where such tutorials are absent and reaching good performance requires work
The benchmarks have also re-surfaced (since it was a known bug) that httpaf which quite a few people use (I think) had a bug with plaintext. In much the same way they can uncover more bugs or simple performance fixes of db libraries or widely used web server libraries - not everyone creates and runs benchmarks.
They also attract quite a few views -> potential new people to be interested in a language. They also provide some evidence to “this language is fast” claims and allow you to compare similar implementations across languages.
they certainly aren’t perfect, but there isn’t anything better that I know of
3 Likes
So I updated my webmachine contribution to the FrameworkBenchmarks repo to adapt @blandinw’s multiprocess solution and his use of Unix.alarm and by doing so got a 10x improvement. See before and after. Unfortunately I’m still seeing 1/10th of the performance of his httpaf implementation. Is httpaf just that much faster than cohttp?
1 Like
it might be, here’s a run that contains multiple processes for opium and @ulrikstrid 's morph:
(as of right now webmachine 's tests haven’t been reached yet, but should be available soon)
JSON’s performance is a lot more acceptable and fits my expectations - it’s still slower than ASP.NET Core, but it is 2x faster than JavaScript
The db queries performance is also faster than similar ASP.NET Core implementations (OCaml caqti and rapper is similar (still less ergonomic and more manual, but it works) to C#/F# with dapper: aspcore-mw-dap-pg)) and 2x faster than ASP.NET Core with Entity Framework (it is a full ORM and it is expected to be slower: aspcore-mvc-ef-pg)
The performance of HTML generation though is much slower than ASP.NET Core, so it’s likely that there’s a lot that can be optimized in Tyxml
Plaintext is fixed in morph’s httpaf used version
and latency is all around better than the other languages included in the comparison.
TBH I’m quite surprised how little tweaks have unlocked so much performance in OCaml.
ASP.NET Core is one of the more widely used (and fastest) frameworks, that has many people working full time and is highly optimized.
The OCaml web frameworks don’t have such a strong commercial presence and haven’t had nearly the same amount of work and optimizations, and yet - they are now performing pretty well
3 Likes
@mudrz in practice asp.net has lots of latency spikes and to get good performance requires the programmer has to use a lot of low level things. Also .net 5 gc still can have very long stw pauses compared to jvm gc’s.
@rbjorklin the webmachine results are also now included in the last run, and the results of the httpaf frameworks seems to be much better
also as you mentioned - fedora performs generally slightly better than alpine:
now I wonder if ubuntu would make any difference
haven’t encountered these issues yet, but I’ve been running asp.net core on small projects; jvm is still the default at scale at work