How to make installing packages a better experience?

I wanted to experiment this morning with Cap’n Proto to see if it provides a viable alternative to gRPC, but I can’t even get the package to install. It conflicts with the latest JS base and core packages which I’m using, and it also uses an old version sexplib that’s not compatible with OCaml 4.06+.

This is way too common and does not instill confidence in the quality of packages. Is that indicative that the package is not used? Or, do OCaml developers upgrade very slowly?

3 Likes

It does look like someone updated upstream 4 days ago, so I suspect if you pin to the dev repo, you’ll be able to install the code.

I’ll say that my experience using opam has been pretty good. It’s very rare that I do an upgrade and end up with my packages broken, and when I find that something is out of date, a polite poke (or even better, a PR with the fixes) will often get things resolved and an update dispatched to opam.

Which is not to say that things can’t be better. I think we need to move towards having a common platform that is reliably updated with each new compiler release. But I don’t think a lagging package is necessarily a sign of a dead project. Open source developers are busy, and updating a given package may simply not be critical to them to get to for a while.

y

3 Likes

I usually open an issue whenever I encounter a broken package. I also understand that open source developers are busy. My problem this time was resolved by pinning the package to master.

I’m concerned that other developers learning OCaml might abandon it prematurely because of those problems. I don’t recall the exact number of packages that I had difficulty installing in the last few months; it’s more than a handful.

Unfortunately, most mainstream developers judge a package by its github activity, documentation, and install experience. Having a good install experience and docs for important packages will help in OCaml’s adoption.

You should also ping the maintainer that he needs to release a new version of his project.
Having the master compiling is not enough: there should be a tarball known to opam
that also compiles.
Usually, if the maintainer releases a new tag, making a PR in opam-repository
to update the package takes just one minute.

this is probably naive – could packages not be declared compatible only up to the compiler version that was used when they were published on opam? then then maintainer would have to explicitly assure that they work with a new compiler, instead of the users finding out by trial and error. is that too much of a burden (genuinely asking)?

Yes, one can do that by setting compiler constrains in OPAM. The issue here is that it usually is not that great of an idea because most packages continue to be compatible with newer compiler versions just fine, so bumping the constraints would require either busywork from the package author or the OPAM maintainers or the inability to install is due to a dependency which will be solved once the dependency itself is updated, again creating busywork for the author or OPAM maintainers to maintain the constraint at the widest range.

This could be partly automated, but it is a significant effort to constantly rebuild packages to keep these constraints up to date.

I think what we want is to encourage package maintainers who don’t have the time to do the job to move their package to ocaml-community. This way, they’d still be in charge of major changes, but simple maintenance tasks can be performed by the community.

1 Like

I suspect the issue is one of change velocity: In a recent article @lpw25 noted that strengthening the module system in 4.08 will break some usages. While the proposal make a huge amount of sense the real question is: Should a point release introduce such a significant change?

What’s more, we don’t really have the tools to handle these changes: How do I now code my package to be compatible with 4.0x < 4.08 and 4.08? cppo? Just as importantly, can we identify the “blast radius” of a compatibility change?

For me these questions represent that fact that the language has now reach a level of maturity and usage that changes to the compiler and widely used APIs have a significant impact on the community and package maintainers

Many OCaml “point releases” seem to have introduced quite significant changes in the past. Most recently, 4.06 turned on safe strings by default. 4.07 transformed the situation for the standard library.

The ppx rewrites broke multiple times as well due to internal changes, until ocaml-migrate-parsetree appeared. I think the common way is to drop support of old releases, have some code generated or use cppo.

I think the select mechanism in dune could be used in a similar way if the differences are substantial.