Are the domains that OCaml excel at, and the ones that are a poor-fit documented anywhere?
Also, are there any efforts to expand those domains, similar to the way Rust is doing with their Domain Working Groups?
Iām not sure what domains OCaml is a poor fit for. The Mirage people have even managed to build an operating system largely in OCaml. That said, Iād say that one obvious ill fit would be anything where garbage collection pauses are a really big problem ā say in tight real time systems. Mostly, though, I think OCaml is surprisingly good at a very wide variety of application domains. The compiled code is quite fast, the libraries are good, the community is fabulous, and mostly I complain about bad documentation and the fact that certain features Iād really love are a long way off.
3 Likes
Thereās this (sadly way outdated) State of the OCaml Ecosystem (2015) which listed application domains, their related technologies, and assign a rating to each. Perhaps it would be great to keep it up to date.
2 Likes
In general, OCaml will be a poor fit where multicore performance is needed, at least until the multicore branch goes mainstream. This isnāt terrible though since python, ruby, and javascript are all single core, and OCaml handily beats all of them performance-wise. There is some basic support for multi-process parallelism but itās awkward.
Additionally, OCamlās default representation of floating point values is not efficient. Heavy floating-point calculations will not be great when done in OCaml code (though Flambda is changing that gradually). However, Owl has been very successful in integrating numpy-like methods (ie. calling out to highly efficient C code when needed) for vectorized operations. So while I would not have recommended OCaml for simulations and such, Iām not sure thatās true anymore with Owl.
So in general, multicore-dependent programs are not an ideal fit (yet): operating systems (non-unikernel), game engines and such.
2 Likes
I am particularly interested in web servers, cloud-native apps (both API backends and workers), and CLIs that interact with external APIs. I encountered challenges developing anything useful in those domains due to lack of libraries, or immature/unmaintained ones.
As far as I can tell, OCaml does not support HTTP2, gRPC, Google Cloud Platform, NATSā¦
Some core infrastructure projects such as elasticsearch and redis have outdated or unmaintained libraries. And other projects, like graphql, have incomplete implementations with no production users. Even something as simple as the latest YAML standard (1.2) is not supported in OCaml. Overall, the ecosystem is so minuscule which makes introducing OCaml a dangerous business proposition.
I would love to use OCaml more. Weāre already planning to introduce it in the frontend with ReasonML, but I donāt see us using it in the backend yet. Thatās why it would be great if the OCaml community steals ideas from Rust on building and expanding the ecosystem because OCaml desperately needs more people involved.
4 Likes
As the author of the ocaml-yaml library, I can assure you that there is nothing simple about Yaml 
Out of curiosity, why do you need Yaml 1.2? Most real world use of Yaml Iāve seen is the JSON-compatible core, which is more than satisfied by the existing Yaml 1.1 implementation in OCaml.
We are liberally heading in this direction with the ongoing OCaml Platform development (tools such as Dune and opam2 are steadily maturing). Weād also welcome contributions of library implementations of (e.g.) http2 into frameworks such as cohttp.
2 Likes
When I was first learning OCaml, I decided to learn by building a CLI tool for our microservices. The CLI failed to parse a YAML file outputted by ksonnet. I honestly did not investigate at the time if it was really a YAML 1.2 issue, but I did verify that itās valid YAML, so I assumed itās a 1.2 compatibility issue.
Itās not a tooling problem. Itās mostly about stewardship of the OCaml language. Iāve joined the Rust and OCaml communities at the same time, and Iāve been blown away how different the communities are in terms of moving their respective language forward, and growing their ecosystems. I really suggest leaders in the OCaml community perusing the internal Rust discourse and subscribing to Rust weekly. Rust has a vibrant growing community. They are able to do that because some of their core team members have extensive experience building communities. Without a coordinated effort, OCaml is going to remain a fringe language used mostly by Jane Street, academics, and a few smaller companies.
Shouldnāt the OCaml community focus on http/af instead? cohttp has documented scalability problems.
Iāve used our OCaml implementation with quite a few K8s/Docker/Buildkite Yaml combinations, so this just sounds like a straightforward bug somewhere. Iād be surprised if itās Yaml 1.2 specific.
Out of curiosity, where do you think this particular Discourse setup came from? One thing to bear in mind is that Rust is a very new language, and so doesnāt have quite the same multi-decade level of backwards compatibility and compatibility needs as OCaml. So while we move at a slower pace, we are indeed moving in the right direction. What we lack are volunteers to step up and assist with elements of the community stewardship ā for example, an effort to collect an OCaml Monthly on this forum would be a huge boost to us.
Scalability bottlenecks are easily fixed with a some minds with spare time on the problem. Both libraries also have different focusses (http/af is Unix only, Cohttp compiles to JavaScript/unikernels/etc). What both libraries lack are motivated maintainers who will advance them. New contributors are extremely welcome on this front!
8 Likes
Iām not exactly sure what you mean by that. But before you spread misconceptions, letās be a bit more precise.
OCamlās float type are IEEE-754 64 bit double precision binary floating point number. There is no support for 32 bit single precision numbers.
While individual float values will in general be boxed when you pass them across functions they are not locally when you perform computations (and this long before flambda existed). floats are also special cased and stored unboxed in the float array type (though that may change in the future as it seems the feature is controversial among compiler hackers).
I believe this old document about numerical computations in OCaml is still conservatively accurate (things may have however have improved e.g. when using flambda).
3 Likes
I appreciate the details and the link, but I think my general summary was correct. OCaml would not be my choice for programs with heavy float usage. You canāt even write efficient recursive functions (one of the main paradigms of functional programming) with floats ā even with flambda. If I wanted to use for/while loops everywhere, I would have stuck with another language.
Clearly, floating point usage was not the main consideration when designing OCaml ā otherwise, we could have used something like nan tagging (which of course has its own trade-offs).
I am going to both agree with you and disagree. I fully agree that community building and a focus on strategic projects will help the community a lot. Iām going to disagree that this is something that is likely to happen because people who are very interested in engineering on the language decide to spend most of their time learning how to build a community. This is doubly true because such people usually prefer working on technical problems. Mostly a community gets built because of a virtuous circle, where strategic improvements drive new adopters which drive new strategic improvements.
1 Like
A side note: (hand) performance optimization of FP heavy code often requires selecting which floating point format to use for which case. Precision vs. cycles tradeoffs are quite real on some architectures.
Rust is a new system programming language (usually unpopular) thatās attracting developers from various communities. It has libraries for everything that I need. Those libraries usually have great documentation and many contributors. OCaml needs a coordinated and deliberate effort to match that. A lot of developers are probably dropping out because they discover that many packages are missing or have serious quality issues. The current status does not encourage contribution. The excitement about the OCaml language is dampened by the lack of packages, fragmentation, and slow pace of progress.
1 Like
The first thing I want to note is that things have really improved in the OCaml space. By a lot. People new to the language arenāt really aware of this, but itās true. Most of this I attribute to OCaml labs and the great people that have come out of there, a few companiesā investment in the community, and additionally now weāre experiencing a bonus Renaissance due to ReasonML.
At the same time, I think the main thing holding OCaml back (aside from lack of people) is a certain culture thatās pervasive, and is very common among older programmers: āWho needs this new thing? My thing works just fine for me and thereās no need to change it.ā This is followed up by āYou want to do this? Go implement it on your own and prove to me itās worthwhile before I even consider it.ā
The first problem is that this mentality ultimately saps the energy of people who want to improve things until they give up and move on, and all youāre left with is a self-selection for the same old attitude and culture. The other problem is that without change - without advancing with the time - you lose out on the most important thing, which is new users in the ecosystem. Users are everything to a language.
It took a long while until the community was convinced to try Discourse. Even now, there are old-time OCamlers who donāt show up here ā who are still clinging to the old OCaml mailing list. Was the old list āgood enoughā? Well, it was functional, but there are many advantages to Discourse, the chief one of which being that you now can attract new users who see a live, rather than a dying, community.
I would argue that the same thing needs to happen to the IRC channel. Is IRC decent? Sure. But youāre isolating yourself from the world, and you need the world a lot more than it needs you. OCaml canāt afford to look like the old man shouting at young people to get off his lawn. Itās not a good look, and it doesnāt attract anybody. How many people have not been exposed to OCaml because they didnāt stumble on a live discussion with knowledgeable users? Iād wager itās a lot.
As an example of the opposite culture, if you step into the Rust chat, you canāt help but be attracted to it. Theyāre discussing RFCs for language features in the forum, for goodnessā sake! Conversations that happen in hallways or on private email messages in the OCaml world are taking place out in the open, and interested people can get involved in whatever interests them. You canāt beat that. That infuses the community with energy.
And lest you think this is too hard to replicate, just look at ReasonML. Sure, theyāre just designing a front-end, but the energy is there because the process is open, and they have an attitude of change and growth thatās infectious.
I also want to mention that I think backwards compatibility can be overrated. I appreciate solid backwards-compatibility as much as the next person, but backwards-compatibility is most important when you have a large audience. When you have a small audience, your existing projects break down into a mix of a majority of dead and unmaintained projects left behind by people who mostly abandoned the language, and modern projects that are really interested in the changes you want to make, and would update their code to support it because they care about the future of the language. The point is, if making changes will grow your ecosystem significantly, potentially all the backwards-compatible projects that now exist will be a drop in the ocean of projects that will eventually exist.
6 Likes
Well, I agree that itās nice when people are enthusiastic, and when they can contribute design decisions and RFCs and whatnot. Rust seems amazing indeed, but as pointed out earlier itās a brand new language that has the huge advantage of hindsight.
āWho needs this new thing? My thing works just fine for me and thereās no need to change it.ā
What are you talking about? Have you seen the statistics about packages migrating to dune, for example? People are ready to move to new alternatives when they clearly seem better. If you have concrete examples of that behavior Iād like to hear them.
I would argue that the same thing needs to happen to the IRC channel.
What, should I leave IRC (and the friends I have there) just because there are newer, shinier things? What alternative would you even suggest (please donāt say āslackā)? Discourse is cool and seems to be welcoming to newcomers, that doesnāt mean we need to kill everything else.
I also want to mention that I think backwards compatibility can be overrated.
Will you please rewrite my projects on every compatibility breakage, then? The community is not very big but thereās a lot of existing OCaml code; in particular, but not only, in academia, where manpower is scarce. If you break compatibility you destroy all this work. And by the way, note that rust also makes big promises on retrocompatibility.
2 Likes
Yes, but Dune had to be built out of whole cloth and given to the community. Were it not for Jane Streetās graciousness, we would still be in the build system mess, with each person clinging to their own system. A build system had to go viral to save us from ourselves, and the energy for that viral transition came from the younger people in the community.
Iām surprised that you would say this, given your difficulty in getting changes into the stdlib or Batteries to the point that you had to do exactly what I mentioned ā you went and built a thing entirely from scratch (Containers) to prove it could be done differently. This isnāt a scalable way to do this ā not everyone can build an stdlib from scratch.
We should be open to trying out a new tool. If it doesnāt work out, ok thatās fine. But the overwhelming response is āget off my lawnā (though as I mentioned, itās changing recently). Your friends should all migrate with you, because there is a whole world of people waiting you meet you guys.
This is more of a side point. Like I said, I appreciate backwards compatibility. But fear of breaking things can be paralyzing, and all Iām saying is the community almost entirely errs on the side of being conservative in this matter. We can often come up with tools to translate old code to new (as we do now with ppx, or Reason does with refmt), and yet backwards compatibility always dominates. Not saying it shouldnāt win out most of the time, just that we could temper it with a forward-facing understanding of the potential for exponential growth.
As a part of the Reason community I have to say that an open process is by no means an integral part of the Reason or Reason React project (except for few instances when the public opinion was consulted).
Reason has grown because of its Discord channel and Reasonās creators who burned half of their productive time to respond to simple and repetitive questions. In another words itās a maniacal focus on new people and people who can convert from javascript (look syntax).
Currently the Reason community is a self propelling machine powered by mostly novice people who mutually help each other. Thereās definitely something to the Reason community but itās not openness. Iād say itās more patience of the people who started it and their incredible responsiveness who set the tone.
That said, in general hard tasks (like Bucklescript) lack contributors (a similar situation).
I believe this is right. I am not sure if people who maintain this forum have such experience. Matter of fact one of the hardest things is separating such mostly unproductive and noisy discussions from ones that can have substance. Iād say that what should be a common practice is forcefully moving discussions which get too off topic into new threads. This sub-thread for instance could be called āI want more and better librariesā. āWhat you tolerate is what you become.ā
5 Likes
While I think new and shiny methods of discussion (such as the discourse forum) are great, I have some reserves about realtime chat options. More particularly, IRC has the great advantage that there are many interfaces/frontend you can use, such as weechat, irssi, etcā¦ many of them open source, it doesnāt require subscription, and itās well adapted for multiple channels and server, i.e. you can very easily access multiple communities with the same tool.
Now if every community turns to some proprietary chat service, you need to get used to a different interface for each community, you may be required to create accounts on proprietary services, you may have to install the corresponding apps on you smartphone (with all the permission and privacy problems it can entail), etcā¦ Personally, I find this to be more of a problem than the technical aspect of figuring out how to connect to IRC.
Although I generally agree that it is good to try new things, in the case of the IRC channel it has proven useful to me a couple of times, and Iām suffering from severe chat client overload. I have far too many and donāt want more new ones. 
(This is more a factor of the fact that the instant messaging world hasnāt adopted a single reasonable standard than anything to do with OCaml. I shudder to think what it would be like if email was like this and we didnāt have SMTP and IMAP.)
I do use OCaml for scientific computations (mostly solving ODE/PDE and nonlinear optimization) and this has never been an issue. In my case (research), being able to quickly change de code to try new ideas is very important and OCaml shines at that. The small set of available scientific libraries is more a problem than raw speed. That being said, more float efficiency is good (and I hope flambda keeps improving that) but that should not deter you from writing scientific code in OCaml.
8 Likes