Welcome to the eighth edition of the OCaml Platform newsletter!
In this November and December edition, we are excited to bring you the latest on the OCaml Platform, continuing our tradition of highlighting recent developments as seen in previous editions. To understand the direction we’re headed, especially regarding development workflows and user experience improvements, check out our roadmap.
odocteam is starting work on improving
odocperformances. After shipping important features, including a link to source code, syntax for tables, and recent support for search, they are turning their focus on consolidating the full documentation generation stack (including Dune rules and integration with OCaml.org package docs) and improving performances.
- opam 2.2~alpha3 is out! It is the last alpha release, and the opam team plans to start the beta release next cycle.
- Anticipating on the January update, the compiler PR necessary for Merlin project-wide references has been merged! This will be part of OCaml 5.2, meaning that starting with the next compiler version, OCaml developers will be able to query for project-wide references (and limited support for project-wide rename!) in their projects. Read more here to know how you can test the feature.
[Dune] Exploring Package Management in Dune (W4)
Why: Unify OCaml tooling under a single command line for all development workflows. This addresses one of the most important pain points reported by the community.
What: Prototyping the integration of package management into Dune using opam as a library. We’re introducing a
dune pkg lock command to generate a lock file and enhancing
dune build to handle dependencies in the lock file. More details in the Dune RFC.
- Introduced a locking mechanism to prevent conflicts in multiple repositories and fixed a bug in Dune’s locking code. – ocaml/dune#9140
- Enabled project locking without a network connection using locally cached
opam-repository. – ocaml/dune#9202
- Enhanced handling of
opam-repositorieswith non-standard contents (non-file objects). – ocaml/dune#9352
- Added a feature where users can set arbitrary variables which can be referred to by opam packages while solving a project’s dependencies. This gives users more control over decisions made by the solver and which dependencies their project ends up having in its
lockdir. – ocaml/dune#9325
lockdircontains all dependencies of local packages for consistency. This prevents Dune from presenting inconsistent information to users who have changed their project’s dependencies after creating a
lockdir. Instead, they’ll be prompted to recompute their
lockdir. – ocaml/dune#9156
- Ability to detect which dependencies in the
lockdirare only needed when building tests. This will be necessary to allow users to skip downloading and building packages that are only needed for testing unless they are running tests. – ocaml/dune#9095
- Improved support for Windows, focusing on better Curl integration and opam library adjustments for Windows architectures. – ocaml/dune#9252, ocaml/dune#9048
- Refined context/lock file handling in Dune package management. – ocaml/dune#9343
- Refined context/lock file handling. Before each context had an associated lock file so lock files were selected via context arguments. Now the context middlemen have been removed and the users selects lock files by specifying the lockfile itself. This applies to most pkg related commands. – ocaml/dune#9343
- Added support for the
conflictsfield in opam files – ocaml/dune#9340
- Looked into solutions to add support for deptops – ocaml/dune#9430
- Writing the PID of process that created the lock file ocaml/dune#9295 to be able to easily determine which process is holding the revision store lock
- Avoid Git translating its CLI ocaml/dune#9390. Since we use the Git binary under the hood, users with different locales might get translated Git output. Since our Git output is not shown to the user, we disable translation.
opam-repository-urloption ocaml/dune#9373 Removes all CLI options that deal with repositories. All
opam-repositoriesare now controlled in the
- Support for specifying particular branches/commits for repos ocaml/dune#9241 adds support for specifying branches and commits as
opam-repositorysources, so the user can fix one particular state of
- Enable checking out tags ocaml/dune#9471 adds the same support as above but for tags. It does so by saving the tags in per-remote namespaces, thus it works very much like branches.
- Implement downloading sources via Git ocaml/dune#9506 enables cloning source directories via the rev store, thus caching most commits when working with multiple projects from the same repo, etc.
- Read the main branch correctly even if tags exist ocaml/dune#9549 fixes an issue where the code that determines the tracking branch got confused in the presence of namespaced tags.
- Dune will compute checksums for lock files of packages which don’t already have checksums in their opam metadata. – ocaml/dune#9384
- Package metadata for a Dune project can be read from
.opamfiles rather than
- Support for conflict classes in opam files for the solver – ocaml/dune#9442
- The ability to add additional constraints to feed the solver – ocaml/dune#9337
[opam] Native Support for Windows in opam 2.2 (W5)
Why: Enhance OCaml’s viability on Windows by integrating native opam and
opam-repository support, fostering a larger community and more Windows-friendly packages.
What: Releasing opam 2.2 with native Windows support, making the official
opam-repository usable on Windows platforms.
- We’ve released opam 2.2.0~alpha3, a culmination of the last 4 months of work. This release, amongst other fixes and improvements, adds in the opam file a way to specify path environment variable rewriting rules for
envbuild:field on Windows. – #5636
- Generate a static binary for producing a Windows binary – #5680
- Fix carriage return on
opam envon cygwin – #5715
- Handle which Git to use at opam init on Windows – #5718
- We’ve also fixed a number of issues on Windows:
odoc] Add Search Capabilities to
Why: Improve usability and navigability in OCaml packages documentation, both locally and on OCaml.org, by offering advanced search options like type-based queries.
What: Implementing a search engine interface in
odoc, complete with a UI and a search index. Additionally, we’re developing a default client-side search engine based on Sherlodoc.
- After merging the PR that added support for search to
odocin October, we continued work on building an
odoc-compatible search engine based on Sherlodoc. This will give a search engine with type-based search for every package that uses
odoc. The plan is to make Sherlodoc the search engine in Dune’s documentation generation. You can try an early demo on Varray’s doc – art-w/sherlodoc#4
- Buiding on
odoc’s support for search, we merged a PR that adds occurrences information to the search index. This will allows
odocsearch engines to improve the order of search results by using the number of occurences. – ocaml/odoc#976
odoc] Syntax for Images and Assets in
Why: Empower package authors to create rich, engaging documentation by enabling the integration of multimedia elements directly into OCaml package documentation.
What: We’re introducing new syntax and support for embedding media (images, audio, videos) and handling assets within the
- Added assets in the environment to treat them similarly as other resolvable elements. This addressed the remaining feedback from reviews, and if no other blocker is found, the PR should be ready to merge. – ocaml/odoc#1002
odoc Performance (W25)
Why: Address performance issues in
odoc, particularly for large-scale documentation, to enhance efficiency and user experience, and unlock local documentation generation in large code bases.
odoc to identify the main performance bottlenecks, and optimising
odoc with the findings.
- Experimented with different data structures and algorithms for more efficient documentation generation on large files. –ocaml/odoc#1033, ocaml/odoc#1036, ocaml/odoc#1049
- Implemented item lookup improvements in signatures for faster processing. – ocaml/odoc#1049
- Developed a fix for memory issues caused by
module type ofexpressions, with promising results from testing at Jane Street. – ocaml/odoc#1042
[Dune] Generate Dependencies Documentation with Dune (W25)
Contributed by: @jonludlam (Tarides)
Why: Enhance the usability of locally-generated documentation by providing direct access to dependencies’ documentation.
What: Implementing new Dune rules for
odoc to enable efficient documentation generation and access to documentation for all opam packages in your switch.
- The new Dune rules have been merged and are available in the newest release of Dune 3.12.1. Try running
dune build @doc-newto generate your documentation and tell us what you think! – ocaml/dune#8803
- Shortly after the release, we noticed an issues with dependency handling. We’re working on a fix that should be released – ocaml/dune#9461
[Merlin] Support for Project-Wide References in Merlin (W19)
Why: Enhance code navigation and refactoring for developers by providing project-wide reference editor features, aligning OCaml with the editor experience found in other languages.
merlin single occurrences and LSP
textDocument/references support, extending compiler’s Shapes for global occurrences, and integrating these features in Dune, Merlin, and OCaml LSP.
- The first iteration on project-wide occurrences is closing-in, and we made a custom
opam-repositoryto test the feature while the changes make their way into the upstream compiler. This gave us the opportunity to test the feature on more real projects, and after another round of bug-fixing and UI improvement we opened it to the community for wider testing.
- Anticipating the January update, the compiler PR has been merged! The next steps are to revisit the patches on the other projects and to open PRs upstream. Dune is the next in line. – ocaml/ocaml#12508
[Merlin] Improving Merlin’s Performance (W19)
Why: Some Merlin queries have been shown to scale poorly in large codebases, making the editor experience subpar, and users report that they sometimes must wait a few seconds to get the answer. This is obviously a major issue that hurts developer experience, so we’re working on improving Merlin performance when it falls short.
What: Developing benchmarking tools and optimising Merlin’s performance through targeted improvements based on profiling and analysis of benchmark results.
- We’ve made the file cache lifetime configurable. Varying the lifetime will allow experimenting with Merlin’s time/space trade-off. Before, it was constantly set to 5 min. -#1698
- We’ve enriched the telemetry that comes embedded in the
- We’ve finished up Merlin’s new Fuzzy CI, a by-product of the performance work.