Hi there,
Although the state of build and release tools has contiuously improved over the years (and I deeply thank all people involved in this transition), in particular regarding the simplification of steps to perform, I still find myself having trouble releasing software in Opam. So I’d like to ask here what is the current suggested process for “basic” projects relying on Dune (like my own pet project). I was even thinking a post that would permanently be updated would be cool. I know this post by @mmottl but now we have dune-release (BTW can it work across a proxy?) so I’d love to see an update, gather hints…
2 Likes
Incidentally, I finished rewriting my packages using dune, dune-release, and OPAM 2.0 just yesterday. dune-release behaves pretty much exactly like topkg on the command line. You can thus follow the usual process as outlined in my mentioned post and replace every occurence of topkg with dune-release. I did not need the CONDUIT_TLS=native environment setting anymore.
The only problem right now is that documentation generation does not work due to an apparent bug in dune-release, which will hopefully be fixed soon. This means that for the while being you can’t just execute dune-release bistro. You should instead follow the individual steps as outlined in the previous post and skip the dune-release publish doc step.
Another small difference is that dune-release is more picky about your OPAM files and may fail during lint steps. The best way to fix this is by addressing any problems raised by dune-release lint. It seems like a small bug, but this linting process does not currently account for *.descr files. Though this may likely be fixed soon, just remove the file and put its contents into the synopsis and description fields in the OPAM file instead. I actually prefer having one less file in the source tree anyway. Note that the current OPAM repository does not have any descr files anymore. I’m not sure whether they are discouraged now, but OPAM, too, obviously prefers the package documentation inside the specification file.
OPAM 2.0 also introduced some specification changes, e.g. when it comes to external dependencies. You may need to update your package specifications if this affects you. Other than that the package release process should work fine.
I’d suggest you just grab one of my recently updated and merged packages to have a template to follow. E.g. Lacaml is one of the more complex ones that have been through the process. If you need an example for how to specify external library dependencies with OPAM 2.0, you may also take a look at the still unmerged Postgresql.
The above small bugs and kinks will probably be solved soon. Specifying, building, and releasing new OCaml software is certainly substantially easier, safer, and more efficient now than ever before. I doubt you can squeeze out much more goodness at this point for the vast majority of use cases. The combination of dune, dune-release, and OPAM is at least on par if not superior to anything other languages can offer.
1 Like
Thanks for the heads up. I indeed met a bug with documentation.
How do you proceed, if the PR is refused by an automatic CI test, to update this very PR without creating a new tag?
I’ve just been contacted by the Thomas Gazagnaire (the dune-release maintainer) to discuss the recent problems. Strangely, when trying to replicate the problems to provide more details, it turned out that both the *.descr linting issue as well as the documentation publication issue have disappeared. Since even older dune-release versions don’t exhibit the issues anymore, I suspect that either used libraries or tools may have been responsible.
Anyway, the gist is that you can apparently follow the simpler release process outlined in Experience report switching to jbuilder and topkg, replacing topkg with dune-release. Just keep the above hints in mind in case the mentioned problems still bubble up with your installation.
@grayswandyr Submitting a pull request will always create a new branch in your local OPAM repository. Just check out this branch if you need to manually make changes. I don’t actually know what happens if you use dune-release again to submit updates to the same package version. I’ll probably try that some time.
Indeed, it’s not clear whether dune-release can simply update a PR branch. Anyway I guess this would be restricted to updating the OPAM file.
Because, in your process, what do you do if the OPAM checks, after submission, tell you that the version of your repo you (git) tagged is buggy? You are forced to create a new tag, right? Hence to start over your process.
If you re-submit an already existing PR, it will just update the PR (basically it will force-push the new change to your branch). The branch name is of the form release-<name>-<version> so as long as you don’t change the version it will keep pushing on the same branch and update the existing PR.
Also, the expected workflow is supposed to be:
-
add a tag in your Git repo. This should be done via
dune-release tag [tag-name]and/or manually. If you don’t supply a tag name, it will look into your change file to guess something. -
run
dune-release. This is very similar totopkg bistrobut it should handle properly repositories with multiple packages.
There are still a few fragile things in the release process, and I would be very happy to review patches and contributions 
For instance, we start to have a few unit/regressions tests, but we definitely need more.
Thanks for your answers.
@samoht is dune-release able to pas through proxies? At a moment, it was apparently considered that the package name field should not appear in opam files prior to submission: what is the policy for dune-release?
I think it is generally unusual to have a package name inside the opam file. The usual way is to name it packagename.opam in your repository, which also then allows subpackages (e.g. if you want to support Lwt or Async but don’t want to force people to install both).
It’s currently shelling out to curl and git directly, so if if your proxy is configured properly to work with these, it should work. (note: using curl directly is actually very fragile and might change in the future).
dune-releasse removes the version field but doesn’t do anything with name. But as @Leonidas said, it is usually not a good practice to repeat that name again as it already appears in the filename.