Q: Is it possible that someone who owns the https://github.com/ocaml namespace could add a project to track OCaml 5 ecosystem regressions (ie. performance and Win32) in one place? Maybe https://github.com/ocaml/ocaml5-ecosystem?
Context: The recently released OCaml 5.3.0 compiler works very well out of the box for MSVC (thanks!!), but it would be nice if critical packages also work out of the box. Many critical packages like eio don’t work, and people are going to waste time and consequently get frustrated if there isn’t a single place to look for problems.
Suggestion: I’m thinking a single place to report Windows and performance issues, and maybe once a quarter to just dump the list of issues in this discussion forum. That would make the state of the OCaml 5 ecosystem visible to those still evaluating what can be ported. (I won’t be using OCaml 5 for anything essential, but even I would find a central issues list useful.)
Why now: Since OCaml 5.3.0 is at feature-parity as of yesterday, there should be no technical reason why critical packages should not work. Also, it is a lot to ask people who are not yet on OCaml 5 to follow up with issues+testing for every package that breaks or slows down … from personal experience, that doesn’t scale.
Prior Art: I propose an ecosystem-wide GitHub issues list like https://github.com/microsoft/vcpkg/issues (but only for critical ecosystem packages) where we’d be able to submit individual issues when we get problems like the following with multiple critical packages:
$ opam install odoc-driver # cff99fb826 2025-01-09
↗ ocaml-compiler 5.3.0~rc1 to 5.3.0 (pinned) [no longer available]
<><> Error report <><><><><><><><><><><><><><><><><><><><><><><><><><><><><> 🐫
┌─ The following actions failed
# `eio` really really needs to work.
│ λ build eio 1.2
# `jsoo` really really needs to work.
│ λ build js_of_ocaml-compiler 5.9.1
# JaneStreet (JS) can't and shouldn't support
# Win32 so odoc-driver and all other ocaml/*
# packages shouldn't depend on JS.
│ λ build ocaml_intrinsics_kernel v0.17.1
# Trivial problem with .opam version constraints
│ λ build ocamlgraph 1.8.8+dune
# Trivial problem
│ λ build odoc 2.4.4
Performance problems just belong to properly tagged ocaml/ocaml issues.
Installability is reported to you right away into opam when you try to install what you need. You can just then head to the issue tracker of the package with start $(opam opam show -f bug-reports $PKG) to see progress or file an issue.
Now for a more global view of 2. perhaps an instance of opam’s ci check dedicated to Windows build would be a better fit.
You can prove your point if you can show me a view of OCaml 5 related issues today (either Windows or performance).
The minimum requirement is that people have to know about it. So if it were ci check then the results need to be reported on this discussion forum periodically.
Not sure if its worth spamming the forum periodically, except perhaps to say anedoctically “hey 80% now works YAY! here’s the remaining ones”.
Really if we want make progress on this, issues belong to individual repos that break and help of motivated and skilled windows inviduals to provide us with good tools[1] and answers to make opam install as reliable as it is on unix machines.
Just an easy workflow to give me a prompt, from a unix terminal, on cmd.exe or powershell with opam installed would be golden. ↩︎
The commentary of the message’s PR link says the opposite: there hasn’t been adequate reporting of the OCaml 5 perf regressions. The lack of reporting may get fixed when the issue label gets linked in ocaml/ocaml/README.md, but today the only thing I care about as a real end-user: which of my applications can I safely upgrade to OCaml 5? And basic sanity says to check for regressions (whether installability, perf or functionality) before doing an upgrade. We can check one or two queues (so OCaml 5 perf labels in ocaml/ocaml is quite fine as one of the two queues) but checking hundreds of package queues is irrational if we are just evaluating feasibility.
I don’t think my once-a-quarter (ie. every 3 months) suggestion falls under any definition of spam. But sure, an 80% message coming from opam ci … as long as it doesn’t require end-users to do dumpster diving to find it … is fine.
I’m super skeptical that approach will work without some highly visible, global measure of “progress”. However, a very small number of issue queues and/or opam ci failure reports can work (at least for Windows regressions) because both can measure progress, and neither places an undue burden on specialized, may-not-exist/may-not-be-available individuals.
I think you are making this a bit more complicated than it sounds. Most packages do not need to do anything to work on OCaml 5 and if there’s a big performance regression it is likely more of a problem for upstream than for individual packages.
Now for the other problem which is to make opam install work for windows users:
I already pointed you to opam ci. You get a nice matrix of green and red and global oversight of what works and what doesn’t. That would perfectly fit. That’s the way OCaml releases ensure the eco-system moves with it on new releases.
To be blunt, you do not understand the problem at all. It is not about “you” and “I”. 10 words: Many people don’t know today what works and what doesn’t. If you cannot understand the adoption and awareness problem from 10 words, stop adding more words to the discussion.
For OCaml 5, 3 words: everything works fine.
For Windows: just try opam install and report what doesn’t where it belongs.
If you can’t understand these simple statements and mecanisms, stop adding more words to the discussion and make bureaucratic proposals that waste our time.
Your help in maintaining an issue (or a label) on there for something of interest to you (particularly Windows!) would be very useful indeed. At the moment, I do find it quite difficult to distinguish between Windows not working due to some transitive dependency, or if it’s a real failure for that package.
just try opam install and report what doesn’t where it belongs.
Once issues are filed where they belong, it would be great to have a GitHub tracking project that ties them together so that people can get a bird’s eye view instead of having to jump around from project to project.
This is great, but it looks like a pre-release tracker, not a post-release one.
The rightful response to this would be: ‘you’re asking people to do more bureaucratic work. Why don’t you do it?’
That’s a fair point. I think people would be willing to step up and do it if there was a publicized central place for it and if they were extended some trust and given access to be collaborators. It doesn’t have to be in the ocaml org; it could also be in the ocaml-community org if that’s more appropriate.
Let’s not mix everything here; there’s already two things that OP is mixing that should be have been kept separate.
That’s not the kind of OCaml 5 problem we were talking about. We are talking about risks of switching to OCaml 5 if you have been on the 4 line.
Now at that point (5.3) my take on this is that the worst that can happen to you is a performance regression[1]. If performance is not paramount to your business there is no risk to switch to >= 5.2 at that point[2] (the performance regression I witnessed on microbenchmarks were largely insignificant w.r.t. say network latency).
Hence my question on the 5.3 announce whether we could leave 4 behind. Because as far as my concerned 5.2 was totally fine (I’m actually super glad debian testing has 5.2, thanks to the unsung debian OCaml packagers).
So do we need a new website to announce that ? Likely not.
You get my point.
Good. But then what’s wrong with @avsm’s advice ? No need to find new place to checkout. If stuff breaks in the OCaml opam-repository then issues should definitvely go there so that proper constraints are applied. Now having an issue there with references to issues on “stuff that should be installable on Windows but is not” seems entirely sufficent.
Also as far as I’m concerned except for tsdl all my packages work on Windows. Of course they do not. But before I get issues in the proper trackers nothing will ever happen.
My anedoctical evidence, because I happened to measure stuff while I was repeatedly switching between 4.14 and 5.2 is that some programs are faster and some programs are slower. ↩︎
Being a terrible infrastructure engineer I switched my publicly deployed webapp to 5.2 without realizing and… nothing changed :–) ↩︎
Let’s look back at the context that was originally given:
The recently released OCaml 5.3.0 compiler works very well out of the box for MSVC (thanks!!), but it would be nice if critical packages also work out of the box. Many critical packages like eio don’t work, and people are going to waste time and consequently get frustrated if there isn’t a single place to look for problems.
The ppxlib issue I posted definitely falls under this category. It seems to me this is exactly the kind of OCaml 5 problem we are talking about.
what’s wrong with @avsm’s advice ? No need to find new place to checkout. If stuff breaks in the OCaml opam-repository then issues should definitvely go there
Yes, that can work too. File an opam-repository issue about breakage + link to issue filed against the package’s own repo + apply an OCaml version milestone could work. Then we can point people at the OCaml version milestones to get a bird’s eye view.
I’m not sure this helping this already hopeless thread but:
That’s not a 5 problem that’s a 5.3 problem, it’s just a new OCaml release problem. It all depends on what you want to quote from the original message, I will quote:
The problem is that the original message and request is utterly confused.
But then since apparently I do not understand the problem at all , I don’t think this thread is worthy of my attention anymore.
This is perfect. I’m fine with maintaining a new set of adoption* labels (if someone can give me access to manage labels on opam-repository) until the summer. That would mean:
If any OCaml user (including newcomers) tries out the latest release of OCaml 5.x using the latest release of opam, and a critical package has a problem, please ask them to file an issue at ISSUE TEMPLATE LINK TO FILL IN.
I’ll close it if I don’t consider the packages to be critical (examples from this thread: ppxlib, eio, odoc are critical; ocaml_intrinsics_kernel and tsdl are not). Otherwise, I’ll relabel it (ex. adoption-msvc, adoption-setup-ocaml) and tell the submitter where to make a package issue and what to say. At minimum that will include expectations that the package maintainer may take a couple months and may mark the package as unavailable if they can’t test+support the platform.
I would ask also that whenever there is a new release of either OCaml or opam, the link to the adoption labels are included.
Such a thing exists https://windows.check.ci.dev in a preliminary format. It’s setup to run every 14 days on a limited number of OCaml versions for Windows Server 2022.
This is great to answer the question “can I run my application on Windows with the mingw toolchain?” Once an application has one dependency that is yellow or red, it won’t run on Windows. The yellow (1,216) and red (354) in 3,629 packages (that is, 43.3% packages don’t build) implies that applications that have several tens of transitive dependencies have a high likelihood of not building on Windows with mingw. Did you reach the same conclusion?
The core adoption question (“should we switch today from 4.14 to 5.x”, at least for Windows users) isn’t answered with that data, but can be answered with some tweaks …
OCaml 5 specific tweaks:
Can we add 4.14 to those builds? The comparison to 4.14 is the most important question for Windows adoption (including whether we should be recommending OCaml 5.x for new Windows users today).
Can we add the MSVC toolchain and the cygwin toolchain? I’m expecting to see better coverage for cygwin, and less for MSVC. It may be that OCaml 5 with cygwin may be a better short-term recommendation.
Where is eio? It would be great if all OCaml 5 showcase packages were present in later runs.
The intention of that site is to provide the tools for checking packages work on Windows, and allow the community to fix the issues by submitting PRs. That’s a much more scalable solution for everyone involved. Currently it is building OCaml 4.14.2 mingw with 5.* coming at some point.
I did come to the same conclusion, but we need to start somewhere. If people are interested go through the failures with the most revdeps and fix those first.
Can we add the MSVC toolchain and the cygwin toolchain? I’m expecting to see better coverage for cygwin, and less for MSVC. It may be that OCaml 5 with cygwin may be a better short-term recommendation.
Technically it should be possible to have MSVC as it’s using QEMU containerisation for the builds (obuilder#195). Like with most of the CI work, the limits are on people’s time and the funding. @shonfeder would have a better idea on the prioritisation of that and what the funding looks like.
Where is eio?
This is building 4.14, once 5.* is added it will show up.
Agreed! It is a bit depressing but hopefully opam 2.2 starts a virtuous cycle where ease-of-use → more devs → more fixes → more devs. We also need to lower people’s expectations for Windows (at least for building real applications with mingw).
Yes, the packages with the most revdeps (ie. critical packages) should be the priority!
All the steps are in the log output but it isn’t that clear what is required.
The minimum steps starting from an opam 2.2 install would be:
# Create an ocaml switch for the compiler version
$ opam switch create --repositories=default 'default' 'ocaml-base-compiler.4.14.2'
# Update opam index
$ opam update --depexts
# Then install either a package or a specific version
$ opam install -y "0install.2.18"
# If this fails investigate and report it including the build log.
I’m more positive on Windows support, opam 2.2 is massively better than what came before, and while it is slow, even a Unix person like myself can get things building on Windows.
, I don’t think this thread is worthy of my attention anymore.:format(webp)/cdn.vox-cdn.com/uploads/chorus_asset/file/6438793/this-is-fine.jpg){kind=link}