I just came across an excellent idea and thought I’d share: allow self-published packages on opam but only under a namespace (i.e. a ‘scope’ like @username/packagename).
This seems like the best of both worlds:
Thoughts?
One slight modification: it might be useful if a version of a self-published package could point to a regular package (at the identical version). So that if I’m releasing a package as self-published, and I want to switch to properly-approved, then I can continue to self-publish what amount to pointers to the properly-approved package. For a while, so that users don’t get forced to update their code/configs instanter.
To turn this around a little, I’d be interested in knowing what problem you’re trying to solve by implementing this approach. The current opam-repository has several positive characteristics by being a curated database. I’m not convinced that opam namespaces are a solution that would preserve this property: bear in mind that opam was created to bring the OCaml community together, and that involved social as well as technical change over the course of several years while projects adapted to the new workflow.
I have my own ideas, but before that I’d like to hear about the problems people currently face with publishing their opam packages, and also what they like about it currently.
Note that you can already write:
opam install https://github.com/USER/PACKAGE.git#TAG
to more or less the same effect. A limitation is that this is only in the CLI, e.g. you can’t use this in depends:, but since opam 2.0 now allows a pin-depends: field that you can use to specify such dependencies.
These are ignored once published on the opam repository, so it scales better to publishing there as well.
Of course, you still can’t have chains of dependent unpublished packages this way (pin-depends are flat), but there are two important reasons for this:
One thing that is often positively (among many criticisms :–) mentioned by eco-system newcomers on the interwebs is the quality and stability of the opam repository. I don’t think we should jeopardize this in the default repo.
There are already quite a few ways of publicly providing uncurated packages @AltGr mentioned one but you can also create your own repo with alternate integration policies.
If publishing on the default repo is perceived as hard or painful, I think we should rather identify the pain points (e.g. submission of incompatible releases is one) and streamline this experience.
I’m very grateful to all the opam-repository maintainers and opam devs that I have a tool that doesn’t explode in my hands every two days and I would prefer to stay it that way.
I agree with you and @dbunzli that I wouldn’t jeopardize the stability of the main repo.
What we could probably do, is emphasize somewhere that, indeed, pin-depends and local installs of specific packages are a thing. People that want/need to stay on the bleeding-edge/unreleased side of the story could then see immediately, on the official documentation, what they need to do.
Another thing that seems relatively common, at least for big-ish projects, is to have an extra custom opam-repository to overlay to the default one. That could be thought as a sort of namespace as well. If a user wants, it takes one line to add an extra repo on top (or in place) of the default one in opam
Hi, the issues that I have in mind are:
protobuf, pb, ocaml-protoc when reimplementing Protobuf functionalityAs the opam ecosystem continues to grow, I think the above points will be helpful in scaling up.
@AltGr as you noted, pinning has several limitations to avoid fragile dependency chains. What I am suggesting is that opam namespaces can avoid the need for pinning. People can publish their namespaced packages with proper versions and hopefully use e.g. SemVer to preserve version invariants.
I should emphasize that I’m not saying that globally namespaced packages should go away; people who want to publish globally namespaced packages can go through the PR process we have today. These can be seen as similar to vanity URLs. Namespaced packages can be seen as a convenience and scalability aid.
The short answer: a bit of out-of-the-box ad-hoc discoverability for users who won’t need to reach out far nor have good Google-Fu.
Anil, you are very right, that the stability of the opam package-collection is a signal strength. The careful and comprehensive testing that occurs before a new package-version is released is a real joy to watch and benefit from. And you’re right, that there are ways to get the utility of opam for unvetted code, by pinning package-names to git URLs for example. But to your question, even though I didn’t propose this, I immediately thought of one good argument for it: ad-hoc discoverability.
It’s true, that people can just put up a public github repo and make an announcement (or not). With Google search, people can find that repo and try it out. But this is … well, this isn’t as “discoverable” as being able to publish a little blurb to a standard place, where (for instance) opam search can find the package and let users know about it.
I think it’s important to carefully weigh “easy discoverability of unvetted packages” against “stability and dependability of the repository seen as a whole”. As someone who was an early Debian user, I used to laugh at my friends would run on unstable or even experimental when they made their machines unbootable. I typically ran stable, and eventually start using testing on one machine that I was willing to reinstall if necessary. That the opam repository is “stable”, is seen as stable by newbies is a really valuable quality.
Anyway, that’s my 2c.
ETA: I’m reminded of Ubuntu PPAs. They’re not as stable, and there’s a place to look 'em up (Launchpad) so you don’t have to search all over for them.