Serious OPAM package quality issues

My colleague has been trying to install OPAM and the packages he needs for a closed source project he’s joined. He’s not an OCaml guy, and he’s hitting pain points that I’ve only felt a little bit of, which are serious issues in our community.

The basic issue is that people just aren’t maintaining their packages. They don’t maintain their package constraints, they don’t update their code to work with recent OCaml versions, and many of them don’t even have github repos, so you can’t easily do some of the work for them. These are the pain points that used to weigh down the haskell world, turning eager people (like myself) away from their ecosystem.

The organization that my colleague is contributing to uses 4.02.0 by default in their VMs. My colleague first decided not to veer from this choice before he knew better, so he installed OPAM and then 4.02.0. Merlin, however, doesn’t support this version. So my friend said ‘to heck with it’, and installed 4.04.3. Well, there was another library that hadn’t updated their OPAM file to support anything past 4.04.1 (currently escapes me which one – I posted a github issue on their project page). As he was about to finish his build, he found out that mlgmp doesn’t support anything past 4.03. Not only that, mlgmp doesn’t even have a github site. So now he’s going to try 4.02.3 and hope for the best. At every recompilation, OPAM goes and rebuilds things over and over again, making things even slower (I made another post about this).

Needless to say, this is an atrocious user experience. I would not blame anyone for leaving OCaml and never looking back after this kind of experience.

We need to figure out how to improve this situation, but I’m not sure what to do. There needs to be some curation work here, I think. A subset of packages could attain ‘curated’ status, which would place expectations on their maintainers when they break. Some packages are already in this subset implicitly –
merlin, containers etc. At the very least, there should be a minimal expectation for every package to be on github in order to be included in OPAM. It would also be nice if there was a mechanism to pass down ownership of a package in OPAM if a package author doesn’t update his package or respond to queries.

2 Likes

Revdeps only started recently to be checked on new opam submissions so this should improve in the future.

Let’s stick to the facts. Roughly only 14% of the packages do not have a dev-repo field listed. For those who have the vast majority (99%) is available on hosted VCS:

> opam show -f package $(opam list -s --all) | wc -l 
    1592
> opam show -f dev-repo $(opam list -s --all) | grep "github\|bitbucket\|erratique\|gitlab" | wc -l 
    1342
> opam show -f dev-repo $(opam list -s --all) | grep -v "github\|bitbucket\|erratique\|gitlab" | wc -l 
      22
  1. Report concrete issues on the ocaml repository and/or if you are proficient enough to issue PR, help to adjust the constraints.
  2. Help projects that need porting to newer version of OCaml.
2 Likes

Regarding this please see fix mlgmp failing to build · Issue #9634 · ocaml/opam-repository · GitHub

There was an interesting project before, which produced a giant matrix of which packages are building on which compiler versions (Linux-only, though, I think). It may have been opam-builder, but I’m not sure, because the “latest” status page linked from the README appears to be missing. Perhaps somebody else can link to where this is now, or what other project may be doing this.

Once we find and/or restore that, it might be good to update this wiki about how to help OPAM have accurate compiler version constraints. We may also want to move the information elsewhere. I don’t think opam-bulk-logs itself is being updated. Last activity looks like it was three years ago. cc @avsm

1 Like

g[quote=“dbuenzli, post:2, topic:484, full:true”][quote=“bluddy, post:1, topic:484”]
We need to figure out how to improve this situation, but I’m not sure what to do.
[/quote]

  1. Report concrete issues on the ocaml repository and/or if you are proficient enough to issue PR, help to adjust the constraints.
  2. Help projects that need porting to newer version of OCaml.
    [/quote]

I don’t think it’s what you intended @dbuenzli, but I feel that this particular part of your response comes across as unwelcoming, or as not recognizing the possible significance of the broader issue that @bluddy was trying to raise–helpfully, I believe.

I agree that 1 and 2 are good responses for many people.

Today (départ du Tour de France) on my machine, with opam v2.0.0~beta3
I’v got : 1689 - 1404 - 285 (?)

You see, the situation is already improving …

I don’t find what is being raised here with the inflammatory discussion title and the lack of evidence about this being a general problem particularly helpful.

It may aswell just be that the newcomer mentioned in the message fell on a particular unfortunate combination — something we should try to minimize but that will always exist in any developer oriented package system that is being managed as a distributed system. I’m using the opam-repository with a lot of packages to test odig and I don’t think these problems are as acute as the initial message wants to make you believe.

In any case it’s not to say that everything is perfect but thanks to @altgr’s tireless work on opam and the CI work of many others, the tooling to curate the opam-repository is improving and the results of this may not yet have fully trickled in the repository.

However I’m afraid that the way to solve existing opam quality issues is not to raise unfocused discussion on this forum but to issue feedback on the appropriate issue trackers and help developing the tooling to improve the quality.

Concerning a tighter curation this has been discussed more than once. There’s a general lack of ressources to do so and the desire to be inclusive as to what gets in the repo (a good thing in my opinion). As a programmer using packages you should do like you do with any of the tool you use, evaluate it’s fitness, the technologies it uses, the trustability of its provider and the long term impact of using it in your project. Remember, it’s a bazaar.

Finally note that anybody willing to invest the time in managing a top-notch curated opam-repository with strict inclusion criterions is welcome to start one of its own: all the tooling to do so is freely available under open source licenses.

4 Likes

@dbuenzli touched on that a little bit, but I want to add that, since a few month, we have a new CI system for the opam-repository that runs on each PR and is really awesome. It checks for revdeps and a very wide array of systems.

You can see an overview here: https://ci.ocaml.io/pr

So yeah, the opam team is aware of the QA issue and has been working on it for quite some time.

1 Like

That makes sense to me.

I guess I’m not bothered by a newbie giving a post a relatively extreme title based on what sounds like a bad experience. It may be an isolated experience, but a newbie might not know that. It’s easy to get freaked out when you’re feeling your way in the dark with a new system. I am mostly past that point with OCaml, but I was there recently, and am new enough that it’s likely I will experience more confusion in the future. :slight_smile:

The detail in your last post provides a lot of context that could be helpful to put things in perspective for anyone new who’s struggling with some opam issue.

For what it’s worth, I have had few difficulties using opam. The fact that the system is able to coordinate dependencies of so many packages so smoothly–compiling to native code for each person’s machine, which is something that e.g. JVM languages don’t have to worry about–has been very impressive to me. Pretty amazing that it works so smoothly.

Hey everyone. Thanks for the responses. FYI, my own experience with OPAM has been good, but I also haven’t tried too many of the more exotic, less used packages myself.

I also don’t consider having an ‘alarmist’ title here inflammatory. This is the OCaml community - if we don’t discuss this here, where do we do so? The particular experience my friend had with OPAM was indeed extremely problematic. I can completely accept the answer that this was a freak occurrence - something most people don’t experience - but given the random sampling of packages that his process involved, I’d rather be safe than sorry.

@Drup, I’m really glad to see opam-repo-ci. It looks like kind of thing that’s needed, but it’d be nicer to have this URL be more accessible, and also to have the presentation on the site itself be more intuitive - for example, what are the checks and red boxes specifically referring to?

I’m also really happy to hear that QA is a priority. The haskell ecosystem has been suffering from this kind of thing for a while (not sure about the current situation with Stack), and I’d love for us to learn from their mistakes rather than recreate them.

There’s no reason to be alarmist if you don’t have solid evidence beyond anecdote. We unfortunately live in a world where perceptions are more important than facts.

No one is denying this. There are a lot of things that can be done to improve the situation which I already mentioned in my first and subsequent message; this discussion is not one of them in my opinion.

4.02.0 is a known-to-be-broken version (it doesn’t generate wrong code but is much slower to compile than all others due a quadratic-complexity bug), and we have widely discouraged people to use it – see the explicit disclaimer in ocaml.org Releases page for example.

@bluddy, the actionable advice in your log is that mlgmp should be updated to support newer OCaml releases. It looks like xnox (your friend?) and dbuenzli are taking care of it in fix mlgmp failing to build · Issue #9634 · ocaml/opam-repository · GitHub.

Yes, it is opam-builder, which I have used in the past to improve package availability for upcoming OCaml releases. The tool is currently down because its author, Fabrice Le Fessant, is working on a new version that should be easier to maintain and more flexible (the previous version depends on a fork of opam 1.2, and some of the capabilities are now available directly in opam 2.0). I’ll send him an email to ask about the current status.

1 Like

Daniel is, as always, harsher than I am comfortable with in this discussion, but I think his point has a lot of merit, and it echoes a sentiment that I have (and I’m sure others as well): we need more people taking the initiative of caring for the things that need care.

The OCaml ecosystem relies on volunteer maintainance. If you see something that is not as it should be (a package fails to build, a dependency is missing, it does not support the OCaml version you care for, it raises warnings on new OCaml releases, etc.), please consider investing a bit of time to get the issue fixed.

No one, of course, has the time to fix everything, and sometimes you must leave issues for others to resolve. But with all the efforts that have been made to have open, decentralized, contributions-welcoming basic blocks of the ecosystem, we can easily accommodate more people helping improve things.

It is natural to think of extra bureaucracy layers when we notice a problem; @bluddy above proposes a curated subset of package (curated by whom?), and a “mechanism” to handle package ownership. These all require a lot of work to design and set up, and it is not at all clear (for now) that they are necessary¹. On the other hand, lending a hand to solve the concrete problems that @bluddy’s friend encountered is actionable, easy, and doable right now – indeed this is what xnox and Daniel did in the issue pointed above.

¹: Of course it is difficult to judge when a new bureaucracy layer in fact becomes necessary. In my personal experience, when time is right a consensus often forms among the people involved in doing the corresponding work that indeed a new layer would help. Then there is also @avsm’s general approach to replace processes and mechanisms (with a rigid structure agreed upon beforehand) by tools and automation (which informs and enable people to do the work), and so far has been fairly successful for the OCaml community.

2 Likes

I’d like to add that helping triaging and fixing the opam-repository issues is a great way of making use of your procrastinating time.

3 Likes

Hi,

So here a newbie perspective.I personally can see @bluddy point. And very much so.
I came to OCaml about 3 month ago. Mainly because I’ve been looking for a functional
programming language that compiles native. And I’ve been really intrigued by Unikernals/
MirageOS.

And I really like the language, but I’m struggling with the tooling and “opam” especially.
I’ve been “fighting opam” and my development environment more then I’m actually spending
time programming.

For Example. At the moment I’m having problems installing

  • irmin-mirage (dependency issues with dns and uri)
  • mirage-solo5 (due to an compiler warning)
  • mirage-xen (due to version conflicts)

    And there is the issue with arch linux, acpcud and opam. Because it looks
    like opam dose not like acpcud if its dependency clingo is compiled with gcc
    insteed of clang. Not to mention the frequently downgrades off package
    while updating.

So delete .opam and start fresh and you and up with a whole other set of
Problems. Same goes for switching the compiler version. Oh and switching
between Linux, macos and FreeBSD is whole new ball game. These are only
some of my personal experiences. But it might simple been the fact that
I am a terrible developer?!

And now after reading this discussion I’m even more frustrate. The answer is
to tell the people to help projects and report bugs. Which is understandable.
But on the other hand I m new to the game. I’m not even sure what the
Problem is or It might been me? I know a classical chicken-and-egg-problem.

I’ve looked in other languages like Go and Rust, which have their own sets
of problems) but when it comes to tooling and packaging the situation is
much better and much easier to handle. Maybe it worth a look?

I really like OCaml but right now I’m deeply frustrated!

[…]

Yes, do report these installation issues somewhere (e.g. on the mirage issue tracker, the mirageOS mailing list or on this forum with a mirageos tag) with details otherwise 1) the situation will not improve 2) we will never be able to help you 3) It might frustrate more people in the future hitting the same issues.

The people on the venues I mention will not eat you they tend to be nice and sympathetic to newcomers. You don’t need to know what the exact problem is to report it, just tell us where you are stuck and provide the output from the tools and opam config report.

3 Likes

I recommend getting aspcud working if you haven’t already, as it can drastically improve dependency resolving in my experience.

Her we go … down the rabbit whole. No I actuallly haven’t fix aspcud yet. The workarounds
for the problem I found didn’t work (anymore?) and I’m not going to compile and then maintain
the the dependency chain be hand. I 46 years old, and I’ve be down this rabbit whole on too
many times. And yes the internal solver is not really doing a good job, which let’s me end
up with a utterly broken system.

This brings us to a another question: Is it reallly a good idea to depend on third-party-tool
at such a critical point in the tool chain? To me it sound a litlte strange the such a major
tool like opam depends on a tool the depends on python libraries with a bunch of c-bindings.

And after reading through the bug reports I can see this happend before. So, is this really
the way to go? Yes you shouldn’t invent the wheel over and over again, and usually it is a
good idea to utilize known libraries, but here I’m not sure, just a feeling.

It’s actually a bunch of C++, but I agree with your assessment that it is a dependency it would be nice to integrate directly at some stage. Unfortunately, it’s difficult to do quickly in the short term, although we’re exploring some options in Mirage specifically for not depending on opam quite so much for day-to-day development (and reserving it just for publishing and fetching archives).

In the short term though:

  • we spent quite a bit of effort packaging up aspcud on every OS distro that didn’t have it. I unfortunately don’t use Arch, so it’s not on our testing radar either. It’s probably worth adding to our Docker CI rotation for testing alongside other packages. Is anyone familiar enough with it to contribute support to opam-depext
  • if you’re having constraint issues with mirageos libraries, please do feel free to post them here under a separate topic tagged mirageos and someone will have a look. There’s a very fast pace of development at the moment with 80+ libraries in play in opam, and we try very hard to break things, but stuff does happen… user feedback is extremely useful.
  • we’re experimenting with a day-to-day build workflow around jbuilder that is significantly simpler and faster, but still preserves the ease-of-fetching that opam gives us. I’ll post more on that on this forum when we have a working prototype of the Mirage workflow.

The short answer:
Yes you are absolutly right I sound write more detailed bug reports. And yes I know
this strange people from venues won’t bit me. But in my opinion it’s a bit more
complicated than that.

On the other hand I can imagine that it must be very much frustrating for people like
you who are active in the community to reiterate theses call for help and participation.

I think you are missing my point. From my point of as a newcomer it really
looks like there is a bigger problem hiding in the bushes. Which brings me to my
long answer or the lack of.

I’m trying for ours now to formulate a more comprehensive answer, but right now
I’m completely overwhelmed with all the Problem I have in mind, but I will follow up
on it. If no one object a newbie is ranting around, that is.