OCaml Platform Newsletter: March-May 2024

Community
1

Welcome to the eleventh edition of the OCaml Platform newsletter!

In this March-May 2024 edition, we are excited to bring you the latest on the OCaml Platform, continuing our tradition of highlighting recent developments as seen in previous editions. To understand the direction we’re headed, especially regarding development workflows and user experience improvements, check out our roadmap.

Highlights:

  • Explorations on Dune package management have reached a Minimal-Viable-Product (MVP) stage: a version of Dune that can build non-trivial projects like OCaml.org and Bonsai. With a working MVP, the team is shifting their focus to putting Dune package management in the hands of the community. To that end, we have started the Dune Developer Preview Program, where we will test Dune package management with users and refine the user experience in preparation for a final release.
  • The opam team released a second beta of opam 2.2, and with it, opened the final PR to add support for Windows OCaml to the opam-repository. Once the PR is merged, opam 2.2 will be usable with the upstream opam-repository on Windows, paving the way for a third beta very soon, and a Release Candidate next.
  • The odoc team has finalized the initial design for Odoc 3.0 and opened several RFCs to gather community input. We’ve implemented a new Odoc driver that follows the Odoc 3.0 design and have already started prototyping key parts of the design.
  • Merlin’s project-wide references query is getting very close to release. The necessary compiler PR has been merged and included in OCaml 5.2, and the Dune rules PR has been merged and included in Dune 3.16. The next steps are to merge the PR in Merlin itself and the small patch in OCaml LSP.
  • The set of standard derivers shipped with ppx_deriving.std (i.e. [@@deriving show, make, ord, eq, ...]) as well as ppx_deriving_yojson are now directly written against Ppxlib’s API. That impacts developers in two ways. First, it allows you to enjoy reliable editor features in projects with those derivers (Ppxlib preserves Merlin’s location invariants). Second, you can avoid a hard dependency on those derivers by using Ppxlib’s deriving_inline feature on them. Thanks a lot to @sim642 for all your work and very kind patience, @NathanReb for reviewing and release managing, and everyone else involved!

Releases:

[Dune] Exploring Package Management in Dune (W4)

Contributed by: @rgrinberg (Tarides), @Leonidas-from-XIV (Tarides), @gridbugs (Tarides), @Alizter

Why: Unify OCaml tooling under a single command line for all development workflows. This addresses one of the most important pain points reported by the community.

What: Prototyping the integration of package management into Dune using opam as a library. We’re introducing a dune pkg lock command to generate a lock file and enhancing dune build to handle dependencies in the lock file. More details in the Dune RFC.

Summary:

Over the past three months, significant progress has been made in adding Dune’s support for package management. We are thrilled to report that our prototypes have reached a Minimal Viable Product (MVP) stage: an experimental version of Dune package management that can be used to build non-trivial projects, including OCaml.org and Bonsai, which we are using in our tests.

There is still a long way to go, but with this milestone reached, we are now shifting our focus from prototyping to putting the feature in the hands of the community. We are moving to testing the new Dune feature with users, and in particular, now that we have a good understanding of the technical blockers and their workarounds, we will be focusing on validating and refining the developer experience (DX) of Dune package management in preparation for a first release.

To that end, the Dune team has started a Dune Developer Preview Program. We’re currently testing the Developer Preview of package management with selected beta testers, and once the biggest issues have been addressed, we’ll be opening it to the broader community.

Activities:

[opam] Native Support for Windows in opam 2.2 (W5)

Contributed by: @rjbou (OCamlPro), @kit-ty-kate (Ahrefs), @dra27 (Tarides), @AltGr (OCamlPro)

Why: Enhance OCaml’s viability on Windows by integrating native opam and opam-repository support, fostering a larger community, and more Windows-friendly packages.

What: Releasing opam 2.2 with native Windows support, making the official opam-repository usable on Windows platforms.

Summary:

The opam team is getting closer to a final release of opam 2.2 with support for Windows. In the past months, we have released a second beta of opam 2.2, addressing a number of issues reported by users on previous releases, including Windows issues.

Excitingly, we also opened the final PR adding support for Windows OCaml to opam-repository. With the PR merged, the opam team is expecting to be able to move to a Release Candidate in June.

Stay tuned for more exciting news and releases in the coming weeks and months!

Activities:

[odoc] Odoc 3.0: Unify OCaml.org and Local Package Documentation (W25)

Contributed by: @jonludlam (Tarides), @julow (Tarides), @panglesd (Tarides), Luke Maurer (Jane Street)

Why: Improving local documentation generation workflow will help package authors write better documentation for their packages, and consolidating the different odoc documentation generators will help make continuous improvements to odoc available to a larger audience.

What: We will create conventions that drivers must follow to ensure that their output will be functional. Once established, we will update the Dune rules to follow these rules, access new odoc features (e.g., source rendering), and provide similar functionalities to docs.ocaml.org (a navigational sidebar, for instance). This will effectively make Dune usable to generate OCaml.org package documentation.

Summary:

The Odoc team has made significant progress on the upcoming Odoc 3.0. We held productive in-person meetings in Paris to discuss crucial design aspects such as the CLI, source code rendering, and references. These discussions led to the publications of RFCs for the various components of the design specification.

We also started implementing a new Odoc driver that adheres to the new design for testing purposes, and began prototyping several of the new features.

While discussions on the RFCs and specific features are still ongoing, we are very excited to have a solid set of design specifications under community review and to have begun implementing key parts of the new design.

Activities:

[Merlin] Support for Project-Wide References in Merlin (W19)

Contributed by: @vds (Tarides), @Ekdohibs (OCamlPro), @Octachron (INRIA), @gasche (INRIA), @emillon (Tarides), @rgrinberg (Jane Street), @Julow (Tarides)

Why: Enhance code navigation and refactoring for developers by providing project-wide reference editor features, aligning OCaml with the editor experience found in other languages.

What: Introducing ocamlmerlin server occurrences and LSP textDocument/references support, extending compiler’s Shapes for global occurrences and integrating these features in Dune, Merlin, and OCaml LSP.

Summary:

The past few months have seen fantastic progress on releasing Merlin’s project-wide reference query: The compiler PR got merged and included in the now released OCaml 5.2; The Dune rules PR got merged, and with it significant performance improvements have been made on the indexing tool. The final PR in Merlin is open and under review. That PR as well as the small LSP patch to support the feature are about to be merged.

The PR on Merlin also adds support for the feature in the Merlin server plug-in for Emacs. Support for the Merlin server plug-in for Vim has been added separately. All editor plug-ins based on LSP will support the new feature automatically.

Activities:

  • We followed up on our compiler PR to improve performance for shape aliases weak reduction. It got merged, and made it into OCaml 5.2.0. – ocaml/ocaml#13001
  • We improved the Dune rules that drive the indexer: Simplified the rules, added benchmarks, discussed and improved performance. The PR got merged, and made it into Dune 3.16. - ocaml/dune#10422
  • We polished the indexer ocaml-index: Profiled it and improved its speed by a factor ~2, and improved its CLI.
  • We added a :MerlinOccurrencesProjectWide command to the Vim plug-in based on the Merlin server - ocaml/merlin#1767
8 Likes
2

I haven’t found anything about this. Could you expand on this topic?

3

You can find some docs here and there!

1 Like
4

As deriving_inline is an important feature, let me explain it a bit more detail. Deriving code via [@@deriving_inline ...] rather than [@@deriving ...] has two direct implications:

  • The derived code is added to your source code.
  • You usually use the following workflow: You run the deriver during dune’s lint phase. So the deriver or Ppxlib are not needed to compile the project. So there’s no need for a hard dependency on them.

The technical “secret” to how that’s achieved is as follows: When using deriving_inline, the Ppxlib driver pretty-prints the derived code rather than embedding it into the AST. So the Ppxlib driver can’t be included in the compilation itself. Instead, it’s used as a separate tool.

That allows to have the following common workflow when using deriving_inline: You invoke the Ppxlib driver as a “linter” (i.e., you have a (lint (pps <your_deriver>)) field in your dune file). Then, during development, you can run dune lint to derive the code, and dune promote to promote the derived code into your source code.

Notice that this workflow is only possible for derivers ( [@@deriving ...]), not for extension nodes (e.g. [%foo ...]).

That’s just a quick summary, though, and if you’re interested in more info, I also recommend checking out the links @panglesd has given above.

Happy to explain more, if anyone has more questions!

1 Like