You might find this discussion about documented examples interesting. As mentioned in the thread, there is a long open PR for doing the kind of thing that mdx offers in .mlis with odoc, but the hacked together solution I describe in the there works pretty well for my purposes in the meantime. Since the way I’m doing it goes the reverse direction (from .ml to markup) all of the usual tooling you have available works as is.
Update!: I broke down and started a new project to split the foundations of Orsetto from its data interchange languages, i.e. CBOR and JSON. For this effort, I’m rewriting everything from the cold blocks using OCaml 5, Dune and Alcotest. My opinion of the latter two is better informed now, if not improved all that much.
Dune is still a beast, and I do not love it, but at least it seems capable of replacing OMake as my build tool, and I’ll trade its annoying opinions about how to structure a package in exchange for a decent library of plugins and tooling support. (I think wanting to use coverage and profiling tools without needing to hack with my bare hands on a bunch of squirrelly OMake library code to get there, that’s what finally pushed me to capitulate.
Alcotest is nice. About the best that I expect can be done given the limitations of the OCaml programming language. I do not miss OUnit not even a little bit, and I am grateful that Dune made it so easy for me to adopt it. I hope my experience with instrumentation will be similarly easy. Maybe I’ll even adopt ocamlformat at some point just so I can trade my 80-column margins for 132-column ones.
Look at me, living in the 21st century now.
When I last looked at MDX, it did not strike me as the sort of tool you use to achieve fullly covered and profiled testing. It’s for making sure the example code in your comments and ancillary documentation is correct, right? Short shameful confession time: not having a decent system for automatically checking the correctness of my example code has been the principal reason I’ve been hesitant to write any example code in my documents.
@mro Good point. People might remember that several years ago we had a large spurt of growth due to Reason. Eventually, we realized it wasn’t sustainable because the ecosystem wasn’t mature enough to welcome all the newcomers. Growth is good, but we shouldn’t bite more than we can chew.
@talex5 why do you prefer mdx over ppx_expect? I’d rather write my tests in OCaml than markdown given all the problems you just mentioned. I see mdx more of a literate documentation tool.
2 Likes
Comparing to Reason feels a little apples to oranges. Reason explicitly tried to create its own community. It didn’t try to grow the OCaml community. There was a small influx of people who discovered OCaml as a result but I think it’s fair to say that they have mostly been trying to contribute to and grow the OCaml community.
3 Likes
I remember when BuckleScript/Reason got popular. That was around the time I started getting into OCaml (HN has a highly voted submission about BuckleScript from 2016, which is earlier than I remember and before I started writing OCaml, if I recall correctly.) I was not a NodeJS person and my motivation for learning OCaml was to get experience with typed functional programming, not integrate with NodeJS.
At one point, I was trying to choose between BuckleScript and js_of_ocaml; the former seemed classier (Backed by Bloomberg! Readable JS!) but seemed tailored more to the NodeJS ecosystem than the existing OCaml one. I ultimately chose js_of_ocaml, which seemed to be preferred by the established OCaml ecosystem.
I remember disliking the Reason syntax. One aspect that I really disagreed to was the function application syntax, which uses a comma-separated list of arguments despite the fact that functions were still curried. If I could put into words why I found this ugly, I’d say it’s because the concrete syntax and abstract syntax aren’t isomorphic (is this right?). For example, f x y can map to either f(x, y) or f(x)(y).
The BuckleScript versus Reason distinction was confusing branding and the splinter into ReScriot was even more confusing. If I understand this correctly: BuckleScript was a compiler from OCaml to JS and Reason is an alternate syntax for OCaml that can map to OCaml source code. However, BuckleScript became ReScript, which has a new syntax which is incompatible with OCaml’s?
I feel that the BuckleScript/ReScript project largely kept to the NodeJS crowd and stayed independent of the OCaml ecosystem around what is published on OPAM. I’m worried if the BuckleScript branding confused people who could have gotten into OCaml, but I don’t think it did damage: At worst it was neutral, attracting people who mainly used NodeJS and were interested in NodeJS interoperability, then forked OCaml, taking the new NodeJS people with it.
2 Likes
I think this is the wrong analysis of what happened. Reason was killed (effectively) by its key component (Bucklescript) deciding to go their own way. While Reason was growing, we kept getting an influx of people into OCaml which was and is a huge benefit, and had it kept going, we would have seen this benefit continue. Bigger IS better in communities, with very few exceptions. More people means more contributors, more blog posts, more innovation, more competition, more applications using the language – all good things.
10 Likes
There’s really no such thing as ‘sustenance’ in languages. The reason is that the language space is massively competitive. If we cannot give people reasons to use OCaml, they will switch to other, more competitive languages. The reason the OCaml community is small is to a very large degree because we’re lagging behind other communities in key features (debugging, ecosystem, language features etc). This is a chicken and egg problem, because had we been bigger, we could have provided these features more easily, which would bring have brought in more people, etc.
6 Likes
I was talking about the community, not the language. Growth isn’t a value per se. But this would become OT.
Initially the thread was about onboarding. Which is necessary but not sufficient. In the end it’s about what priorities a community has and naturally different communities may have different ones. Maybe there is a place for those disgusted by ever breaking things. I for one value reliable results and accept initial friction.
However, what helped me navigate the confusion was
- recipes and reports what worked (the IMO not bad docs, real-word ocaml, Cornell course, blog articles, ideally found in this forum),
- early and expressive compiler messages (hm…),
- a friendly, responsive and competent crowd like the one here,
- a bit of stubbornness,
- the confidence the grass isn’t greener elsewhere.
And the end result has some properties, I don’t want to go without.
2 Likes
Indeed the balance is to grow while maintaining the things that keep the language unique and useful. I think OCaml has long known how to do the latter. The former requires listening to users and best practices and adapting.
At the end of the day, OCaml isn’t offering any particularly unique feature. What it offers is a ‘sweet spot’ - a particular mix of features that balance purity with practicality. In order to maintain this notion of a sweet spot, advances need to be made constantly to compete with other languages with bigger communities. Growth of the community is generally a sign of success - it means we’ve provided that sweet spot to more and more people, so that they find value in the language.
7 Likes
Sounds like Raku to me… which is a fine language and more people should know about it.
@talex5 why do you prefer mdx over ppx_expect?
I used ppx_expect in OCluster, and that worked too. Some advantages of MDX:
-
Projects always need examples, so you always need a dependency on MDX. Since it works for tests too, it’s good to avoid the extra dependency on ppx_expect.
-
As I recall, ppx_expect wants tests to be mingled with the source code. This requires the source to be pre-processed to get rid of the tests during a regular build, so you end up making ppx_expect a dependency for everyone, not just developers wanting to run tests. Or, you do what OCluster does, and make a fake extra library with nothing but tests. But that causes trouble too (e.g. Make obuilder only require ppx_expect for the tests by kit-ty-kate · Pull Request #139 · ocurrent/ocluster · GitHub).
-
Having the tests inside the library encourages writing tests that depend on internal APIs, which I usually prefer to avoid. With MDX, I can often just add a
val dump : t Fmt.tand use that, which is also useful for users of the library trying to debug things. -
MDX syntax is a bit nicer IMO.
If you’re running something within Lwt or Eio, ppx_expect does let you put the output exactly where it’s expected though, whereas with MDX you put it after the main-loop finishes. That could probably be fixed in MDX with a concurrency-aware version.
The lack of merlin, etc isn’t a big problem for me because test cases tend to be short, and for larger cases you can get MDX to import code from a separate .ml file.
1 Like
I just tried to get up and running with OCaml last night. For me, the difficulty was the tooling and the documentation around the tooling.
General Language documentation
- I could find virtually no information on how OCaml works with its REPL in a development workflow, beyond that “there is a REPL but we don’t use that one, we use this other one”. Trying to figure out where on the spectrum between a python REPL or a common lisp fully integrated development experience was actually was a challenge - and I still don’t know the answer. How are OCaml developers working with the REPL? Am I writing my programs from “within” my program like I would if I were writing Common Lisp or Clojure? This sort of thing is hard to find any information about.
With OPAM:
- The notion of a “switch” was an odd one. Having a global “switch” is very strange to me. Managing these is very obtuse. You need to reinstall all your development tools for a different switch version. I am used to a more project-local lock in of library versions, and a more global installation of development tooling (things like the language server protocol, utop, etc). For example, npm in the javascript universe allows you to install things globally, but also only project-locally. The tool knows which one to use based on what directory you’re in. That’s much easier and I think more sense making. Reinstalling all the dev tools if you jump switches seems kind of silly. I am sure that there are good technical reasons for this, but it is weird coming from the outside.
- Having to explicitly run
opam initseems not entirely necessary. However that leads to my next weird thing: - Having to run
eval $(opam env --shell=fish)or whatever your shell is in order to get everything in your active “switch” available on your path. Someone else somewhere in this thread rather proudly pointed out thatopam initwill modify your shell profile for you! I manage these things in version control why would I ever want some random thing changing them? That would be ridiculous. Now, this environment set up being per-user and per-shell is fine, I am used to that with node andnvm. However, this requirement to eval the result of some magic spell is very easy to miss, and I only realized it when the OCaml adjacent emacs modes were unable to stand up.
My “solution” to the above was to just install direnv and set up an .envrc file in my project’s top level to run the right eval command. Emacs has a direnv package that just makes this work. I did have to burn a not insignificant amount of time figuring this out, but it works and I don’t have to pollute my shell configuration with random stuff that IMO doesn’t really belong there.
With the editor integrations (emacs specifically)
0. I am aware that opam has something called “user setup”. Again, my emacs config is something I have under source control - why would I ever want something modifying it directly without knowing what it was going to do? That is just a total non-starter. I set it up myself. Also I don’t believe the user setup lets you use the OCaml LSP implementation for emacs but instead forces you into using something called “merlin”. OCaml is the only language ecosystem I know of that does something like this “user setup”. I was not inclined to go that route because it was unfamiliar to me and because I don’t want anything touching my configurations but me. Speaking of “merlin” though:
- The differentiation between merlin and the language server protocol implementation is very confusing. The one depends heavily on the other apparently and the difference between them isn’t really spelled out anywhere. Which one should I be using? Who knows, good luck.
- If you were like me and chose the language server protocol implementation, then you need to be aware that the LSP is only able to work if you run a build so it can I guess find build artifacts which are required for most of the OCaml LSP features. After initializing a dune project, I need to repeatedly run
dune buildwhenever I do anything apparently. Even when creating a blank file. As far as I can tell this requirement for the OCaml LSP is documented absolutely nowhere.
Actually the only place I found reference to it was the documentation for the OCaml Platform for VSCODE, under the Setting up the extension for your project section. Like, come on guys - for real?? I did find an issue in the LSP repo outlining some documentation improvements specifically in this area so I am glad that is at least being tracked by them. I appreciate that good documentation is very difficult to write. This was very confusing, as no other LSP I’ve worked with thus far needed me to run some process in my terminal. Maybe the other LSPs I work with are running something in the background. I write some clojure and I use their language server protocol implementation and no such thing is explicitly required as a user step for enabling any LSP features such as “go to definition” etc.
I feel that the LSP should be smart enough to realize that there is a dune-project file in the top level of the project and just do the right thing for me, or I should be able to communicate with Dune from within emacs in a much smoother way.
- Dune. Dune uses S-expressions. That’s fine. I come from lisp so that is not very bothersome .It is unfortunate that there are two very different sets of syntax to now worry about, but I think that is mostly manageable. It was bothersome to figure out what packages I needed in order to work with dune files. From what I recall, I found virtually no reference of the
dunepackage or its setup on the Dune build tool’s site or its documentation. It would be good to provide these things. Apparently there is ANOTHER package for formatting the dune files too but I haven’t even gotten to that yet. Perhaps also suggesting that users install and configure other things like paredit to make working with s-expressions a bit easier would be a good addition to those docs as well.
Getting from zero to the point where I could write my first function definition, and send it to the REPL was an absolute labyrinth. It was more than a little frustrating having to comb through issue trackers in various repositories, googling, taking to IRC (by the way for anyone wondering IRC is still full of rude people who will go out of their way to make you feel stupid for not knowing how the LSP works), looking for solutions to problems that should not be problems.
At this stage in my computer career, I precious limited free time. OCaml has a lot to offer but the developer experience and tool UX is not great. It is a very confusing landscape that took me a long time to navigate over a couple days. I appreciate your patience reading this word-dump. I hope it was not taken in offense, but rather a still-fresh accounting of difficulties. Thanks.
17 Likes
BTW some more discussion on reddit: https://www.reddit.com/r/ocaml/comments/zqfoq5/what_are_the_biggest_reasons_newcomers_give_up_on/
@jhw there’s probably no need to remind you, but dune doesn’t force you to delete your existing build system. It should be easy enough to maintain dune files on the side for some tooling/profiling needs you have and build the rest with omake. Could you describe some of these opinions of dune that annoyed you? I’m curious about what can be improved.
@bluddy no, size isn’t everything. Unless you think “any publicity is good publicity” and scores of jaded people with negative first impressions of the language is a good thing. The simple fact is that OCaml as a language isn’t (yet) as accessible as other languages as Python. Pretending like it is and putting potential users through pain is going to back fire on us in the long run. We should set the expectation clearly that getting up and running requires getting your hands dirty even if that isn’t going to turn away some users at first.
@talex5 #2 is an issue that is worked around by putting your ppx_expect tests in separate libraries. That doesn’t let you test internal api’s, but mdx has the same problem. I suppose another advantage of mdx is that isn’t tied to ppxlib. FWIW, it shouldn’t be too hard to add editor support to mdx files. Would be an interesting first project for lsp/merlin if anyone is interested.
4 Likes
Thanks for all the feedback, some of it is useful and I appreciate you writing it.
Really? I always found the opposite to be the case. Given your post, I have a very good idea that the interaction with “rude people” was in fact with me. I don’t think I was rude at all, but I’ll share the entire exchange with everyone else, so that they can see for themselves if IRC is as bad as you’re saying.
[19:11:39] mister_m I started a dune "lib" project, and I have emacs telling me "no config found for file" and suggests running dune build. What does this mean?
[19:12:05] rgrinberg means that you need to run $ dune build in your project
[19:12:09] mister_m why
[19:13:02] mister_m Immediately when I create a new .ml file I see this error through the lsp and flymake
[19:13:59] rgrinberg ocamllsp can only provide you with editor functionality if it knows where the build artifacts are
[19:15:54] mister_m so dune build needs to be run whenever I make a new file in a project??
[19:16:02] rgrinberg yes
[19:16:41] mister_m that's ridiculous
[19:16:52] mister_m but thank you for explaining why I am seeing that
[19:17:16] rgrinberg you're welcome
2 Likes
Hi,
Thank you for all the feedback !
One point which could help with this :
or I should be able to communicate with Dune from within emacs in a much smoother way.
You can use the watch mode dune build -w and you won’t have to rerun dune build manually (or dune build -w @check if you need it to be more snappy and it’s just for the editor).
1 Like
As a Rust expert and an OCaml newbie, I absolutely agree with this sentiment. The rest of the points you made are also very compelling, and even as an experienced developer, I’ve faced/am facing almost all of those issues as well. It’s just that my experience with a variety of language ecosystems makes it easier to handle, but I can imagine what a nighrmare (off-putting to boot) it must be for absolute beginners.
3 Likes
That’s called obsolescence, not thriving and sustenance.
This would also help with the proper lack of Windows support. More users → more Windows users → more people to help across all platforms → more users for OCaml. A veritable positive feedback loop.
(Forgive the cross-reply - the forum doesn’t allow more than 3 responses in the same thread for new users, apparently)
In general, my own experience on the IRC channel on Libera has been one of two:
- No responses whatsover, or
- Useless responses.
to the point that I’ve given up. The Discord channel (unofficial?) has been far more helpful and responsive.
Then again, I’m a beginner in OCaml. Maybe it’s more useful for veterans.
2 Likes
I dunno comrade, when a programming language community has been around as long as this one, failing to die outright from old age is thriving. This one is old enough that thinking about posterity is probably more important to us than climbing the ladder.
1 Like