Expanding ocaml appeal to future open source application developers

[dimitris]

@mro

That’s more a problem of the day jobs as it is of OCaml. Mainstream is IMO another word for mediocracy. OCaml should (edit:) not strive for that.

@mro Would you like to expand a bit?

[mro]

@dimitris

expand a bit?

sure, which aspect? Here or elsewhere?

[dimitris]

@mro

sure, which aspect? Here or elsewhere?

I’m curious about the concerns you have. What kind of negative effect do you expect if ocaml becomes more widely used?

[Chet_Murthy]

I spent time in the belly of the Java beast 1996-2008ish at IBM, and I think I can speak to some of this.

Java’s evolution was hijacked by corporations, and by the architects and technologists at those corps. It suffered as a result (I have lots of receipts). If that happens to OCaml, yeah, it’d be pitiable. But I take the counter-example of Perl (or, heck, Python): widespread adoption doesn’t need to inexorably imply control by some corporate cabal.

Some receipts:

1. the way that the JDK contains so much stuff that really ought to be separate libraries
2. the continued unavoidable JIT
3. the lack of any sort of compile-time modularity (as compared with the CLR)
4. The high initial memory and CPU footprint of the JVM

Nobody can imagine writing tar(1) as Java and having it be credible. But one can imagine writing it in OCaml. I wrote zip/unzip in OCaml and it was quite credibly fast and light, where a Java implementation of Jar is a … pig.

I could go on and on … but really, these are all ramifications of the decisions made by the Java consortium: by corps.

[cjr]

@dimitris

I’m curious about the concerns you have. What kind of negative effect do you expect if ocaml becomes more widely used?

A simple one is the signal-to-noise ratio online. The more popular a language is, the more difficult it is to find good answers to your questions and separate the wheat from the chaff.

[sangwoo-joh]

A simple one is the signal-to-noise ratio online. The more popular a language is, the more difficult it is to find good answers to your questions and separate the wheat from the chaff.

I partially agree with you, but if a language is less popular, you’re the first person who faces your problem, and you should answer your question by yourself…

[bluddy]

@antron

I have to disagree with this. I’ll take Dream as an example, due to personal experience, but this undoubtedly applies to other projects as well.

The initial extensive documentation was done when Dream had only one contributor. Of course, since then, the repo got a lot more very nice examples and other documentation from more contributors.

I think that you’re the exceptional developer in this case. Most of the time from what I’ve seen, a project gains traction and users, and then it seems worthwhile for someone (sometimes the original author) to write heavy documentation. Of course, if the author is willing to put in the effort, there’s a much better chance that the users will arrive. It’s all ultimately feedback loops, as a good-looking, well maintained project will attract more users all other things being equal. It’s also possible that standards are changing for the better, and that given CI and github-pages availability, it’s become easier to get projects into a position where they’re more attractive to users.

I don’t think I’d be generalizing too much if I said that most developers dislike doing documentation and prefer to just focus on development. You often need a sufficient developer base often just to get those one or two people who are willing to put effort into the documentation (and who are sufficiently good at it).

[bluddy]

@cjr

A simple one is the signal-to-noise ratio online. The more popular a language is, the more difficult it is to find good answers to your questions and separate the wheat from the chaff.

If there’s a lot of signal-to-noise and there are many users, you’ll also get enough smart users who wish to cut through the noise and their answers will be upvoted. It’s always better to have a popular language: a bigger ecosystem means more libraries are readily available; more experts are available; the language has been used to do more things; it’s easier to get jobs; it’s easier to find solutions online; etc. A small language with a small ecosystem is always in danger of dying out once enough people abandon it.

Most importantly, language ecosystems are driven by a feedback process. That library someone made today that you don’t happen to care about will bring more people into the ecosystem, and it may have components that could be used to create something you do care about tomorrow. Every little improvement helps. It’s like gaining interest on a savings account.

[Chet_Murthy]

@bluddy

If there’s a lot of signal-to-noise and there are many users, you’ll also get enough smart users who wish to cut through the noise and their answers will be upvoted.

To add to this: Perl never suffered from this S/N ratio problem, even though it was immensely popular in the early days of the Internet. Neither did Python. There were always more-trusted sources to rely upon. For example, people are always going to pay more attention when @dbuenzli speaks, b/c his code is advance publicity for his words.

That’s right and correct.

[mro]

@bluddy

I think that you’re (@antron) the exceptional developer in this case.

Indeed and that’s what I consider very much the most appealing – having exceptional community members! OCaml has a lot of them. That’s much more than a streamlined onboarding experience or test and code formatting baked-in (as awesome as Go or Elm may have it). Or even a modern standard library with Quic etc.

And the exceptional community (you, we, me, everybody!) has not happened by chance, but by decades of being serious about elegance and reliability.

So I find the initial question about “expanding” misleading as it has a side-taste of some lacking – which doesn’t resonate with me.

[cjr]

@Chet_Murthy

Neither did Python.

It’s interesting that you would say this. I was actually specifically thinking of Python as an example here, which I worked with for probably somewhere around 15 years. As Python became increasingly popular, I found it increasingly difficult to find answers to questions because web searches would turn up so much that was only tangentially related, if at all. Same for quality material (both on- and off-line) because so much was being published and most was mediocre, even when written by prominent members of the community.

It’s curious that we would have such different views here. Mine is from the perspective of someone who programs only occasionally. I definitely found that over the years, it became harder and harder to find quality material/references/resources for Python. Not that it didn’t exist but that there was so much more muck to trudge through first.

I feel the same way about R today. It’s popularity means that whenever I forget how to do something or need to find out if some procedure exists and where, there’s just a ton of half-baked, incomplete solutions that I need to filter out first.

I’m not saying that it wouldn’t be beneficial to grow the OCaml community. But I don’t believe that to be unambiguously positive. And I don’t believe that communities are necessarily self-correcting and meritocratic. Not that we need to worry about OCaml becoming as popular as Python. (But what do I know? I was also surprised when interest in R exploded. And when Ubuntu became so popular, so quickly. And countless other things.)

[yawaramin]

@cjr

Not that we need to worry about OCaml becoming as popular as Python.

That’s the thing exactly. Some in this thread are so worried that the OCaml community will degrade into something they won’t like if it becomes more popular. IMHO, let’s worry about more realistic things than overcorrecting for OCaml’s impending (!) popularity and the incoming hordes of unwashed developers.

There’s this popular quote that does the rounds on LinkedIn:

‘What if we train our people, and they leave?’
‘What if we don’t train them, and they stay?’

What if the OCaml community becomes filled with people who like the fact that it’s niche and inaccessible to outsiders?

[Chet_Murthy]

Yawar, this is exactly right. 100%, exactly right.

And I say this, as someone who honestly doesn’t give two hoots personally about whether The Great Unwashed use OCaml. I don’t care about IDEs, merlin, LSP, Javascript, all that stuff. I use Emacs, xterms, and make. So none of it will affect me. And I am utterly unafraid of the hordes somehow crashing in and producing noise and turmoil: there will always be a core of stuff, a core of developers, that I pay attention to. For instance, I doubt that I will ever use any of the Javascript-related functionality in the OCaml ecosystem. Ever.

But two things:

1. I want OCaml to be successful, and I want the developers of OCaml to have a vibrant audience for their work. I may never use “Winders” (no “may” about it!) but I want for OCaml to be successful on Windows.

2. We shouldn’t be afraid of The Unwashed, the kids, crowding into OCaml. It’s a great thing, and can only enhance OCaml’s value and ecosystem. From those kids, will come people who want to contribute to OCaml, who want to write new tools, packages, and documentation.

Especially when it comes to documentation, I think that the arrival of hordes of newbies could help a lot. I think that we old skool folks grew up in a world of “RTFS(ource)C(ode), Luke”, and we have a constitutional allergy to writing good documentation and tutorials. Fine. The newbies won’t be like that, b/c blogging, writing tutorials, and documentation, are all part of The Way Things Work Today.

It can only help.

[Chet_Murthy]

“And I don’t believe that communities are necessarily self-correcting and meritocratic.”

It would be interesting to see examples of this in communities that aren’t led by companies. I know that Perl, for instance, has displayed precisely this sort of self-correcting behaviour. [Others may disagree with me, but] I thought the way Perl6 was evolving was “wow, that’s gonna be the kitchen sink”. And then it detoured thru Haskell, and I was even more appalled. But Perl5 kept right on truckin’, and I think this is an example of self-correction. I doubt that many people think of Perl6, unless they’re weirdo PL-turned-industrial types like me.

[Chet_Murthy]

Yawar, something else that I thought was worth saying:

I started on BSD UNIX in 1983, and with caml in 1991. From my grey-haired point-of-view, aside from the people who got PhDs in this stuff in the late 80s/early 90s, everybody is a “newbie”. Kids that I had as interns 20 years ago, are now trusted and reliable experts in areas I know nothing about, and some of them in areas where I thought I was an expert.

The lesson I draw from 25 years of watching newbies enter and create great things, is that we shouldn’t dissuade them: that encouraging newbies who will enter and eventually (soon) become old greybeards, is important and valuable for every community.

[mro]

@yawaramin

like the fact that it’s niche and inaccessible to outsiders?

niche in the sense of not being mainstream: yes, I see benefits here and dangers in the mainstream as I pointed out and others elaborated.

Being inaccessible: who ever said such? Please don’t twist the words in my mouth. I’m a outsider myself and can’t see what you mean with inaccessible.

OCaml is just rarely known, it’s not that people say – uh it lacks X or Y. They say O-what? I for example got notice of OCaml via an article of @hannes about the mirage project in the periodical of an association I’m member of. Or talks at the Chaos Communication Congresses or Jane street.

So I second the wish for visibility, talks and articles with quality well above the mediocre. Or just silly ads like the beforementioned stickers.

Don’t just focus on infrastructure, it’s already sane, that’s not what OCaml is about IMO. But rather spread the word, spread your enthusiasm to those, who never have heard of OCaml before. Talk more about the ends to achieve (e.g. small TCB, safe implementtions), not just the means like tooling.

And yes, I like things having character at the price of not being attractive for everybody. I’m a huge fan of variety. No need being the best in all details if thriving.

[Danielo_Rodriguez]

I think that having a big community usually leads to more people getting into the community because the community traction. Is kind of a self feeding loop.
More people using the language means more docs and more libraries.

To give you an example of the importance of libraries, about 6 months ago my girlfriend needed a tool to process a bunch of files quickly for her work. I needed to process and filter a bunch of KMZ files and spit out them again filtered. I would have love to do it in ocaml, but I did it in node. Why? Because all I had to do was to pick a couple of libraries, glue them and give my girlfriend something she could use in a matter of days, not weeks.
KMZ is not a popular file format, and the fact that node had libraries for it was amazing and save me a lot of time.
I don’t think I could have done it using ocaml. Sure, I can read the KMZ spec and write a parser, but then the taks would not be help my girlfriend anymore but build a KMZ parser and encoder

[mro]

@Danielo_Rodriguez

quickly

I want to build services and software running unmodified for decades. OCaml seems promising here where other options become sparse.

Choose the right tool for the job and not every tool has to be alike. They may differ. Even better if they do.

[dbuenzli]

Since this is a loose thread I’ll just tell a little semi-OT anecdote.

@Danielo_Rodriguez

To give you an example of the importance of libraries,

I think we all get the importance of libraries :–)

But what people never seem to mention is the quality of the libraries in these “batteries included” languages. Each time I reach for them I end up being disappointed by the very low power they provide – this is anecdotic of course, I’m sure there’s good software written there as well, I likely just got unlucky.

So here’s the story. Last year I needed to devise under time pressure a pipeline to produce static IIIF images (a format to serve high resolution image on the web).

So I reach for an off-the-shelf solution, lose a couple of hours (3 according to my timesheet) searching the web for something decent until I find some python code based on the PIL python library that does that. I run the software and it takes something like 9 (!) hours to produce the image with tiles of a very dubious photographic quality. The knobs to improve or tweak the results are far from being obvious and anyways the time of production was a no go. Especially since we had to experiment with the results.

So in the end I dive into the spec and produce after seven hours and a half a 150 loc OCaml program that shells out ( I know I know) to imagemagick and produces the desired file hierarchy in about 1min with all the control over quality we need (compression, color profiles etc.).

Sometimes the batteries are closer than we think :–)

[Danielo_Rodriguez]

So because you have those goals does it mean that I can not use for my purpose? As far as I know OCaml is a general purpose language, not a DSL for whatever niche usage.
The only reason I was not able to use ocaml was the lack of libraries, not because it was not possible or it would not allow me to do equally quick.
I guess you love reinventing the wheel on every project of yours and have one or two at most on a lifetime.