The end of Camlp4

I have wondered since almost the start of my involvement with OCaml why such a thing doesn’t already exist. They’re things one very clearly wants.

1 Like

It may be enough to just have a copy of ppx_deriving/lib/whatever be inside the compiler repo, with a few select ppxs, and test cases to make sure these are always up to date with the latest compiler changes.

There’s also another perspective on this, which is that the compiler versions are released too soon. I’ve mentioned this idea before, but we could have more stages to the compiler release, where the core team is no longer involved, but essential parts of the ecosystem must be updated before anyone else can even touch the new version.

Some are written.

The question is how much does it evolve over time. You can also elect to have that as a source level generation step once to make the first big shot and then let it evolve without depending on the whole ppx infrastructure.

1 Like

Same for me, perhaps it’s because it’s a problem that has no really satisfying solution. The impression I get is that in any language you can have either static type checking or you can have universal printing/equality/(de)serialization, but you can hardly have both without paying a price in terms of performance or complexity or power of the language.

2 posts were split to a new topic: Should opam use a lockfile by default?

Rust derive macros are just macros, no?

It would be much better to have a macro system, something like Rust have, maybe based on metaocaml, but with abilities to parse strings and add attributes.

Derivers are useful, but there since the language lacks dependent types, there are other useful applications of metaprogramming: format strings (for SQL, for example), parser generators etc.

2 Likes

It may be time again to remember people that the topic and focus of this discussion should be Camlp4 :slight_smile:

2 Likes

Isn’t PPX, de facto, a macro system?

I think he’s referring to an in-language macro system, as opposed to an out-of-language system like PPX that runs completely apart from the compiler. Though it seems like Rust has both this aforementioned system (which is very sound, but is limited to localized modifications) and a full PPX-like system that edits the AST.

I’d like to give a slightly different perspective on the ppx situation because I think that most the opinions here skew one way and paint an overly grim picture.

I’ve personally used ppx in several projects and will continue doing so without much hesitation. For many use cases, the amount of boilerplate that ppx eliminates is much more important than being able to instantly use the latest version of OCaml compiler. I don’t mind waiting a year (or even two) for the ppx that I care about to upgrade. In practice, since omp and ppxlib were released. it rarely took this long for the ecosystem to be updated. In fact, my upgrading bottleneck is rarely the ppx but all the other infrastructure that relies on compiler api’s: utop, merlin, ocamlformat and other breaking changes in the compiler.

So what I would like to see is a stabler api in the compiler. Not only this would help the ppx situation, but it would help to maintain all other software that’s critical to my workflow.

The real problem with ppx is just how damn ugly it is :slight_smile:

9 Likes

That seems like it would indeed make everyone’s life easier.

I think that everyone can just provide their suggestions about ppx here: The future of ppx, which is a proper place for discussing the future of this subsystem.

3 Likes

I also cleaned up the list in The end of Camlp4, removed obsolete projects and these that were fixed since already. Many of them also dropped the camlp4 dependency, but not made release yet. Added the issues links to those that are at GitHub, for tracking the progress.

3 Likes

As long as you have a dune-project file, that should be the case. The first line of the dune-project file tells Dune which version of Dune the project was written against so that future Dune can continue to understand it. Well, that’s not a 100% guarantee that everything will work exactly the same forever but at least Dune will still be able to make sense of what’s written in your dune files.

1 Like

Regarding ppx, just to state the obvious it is clearly easier to accept the tradeoff of writing ppx rewriters if you are a big company like Jane Street. For individual ppx authors, it requires much more efforts. The good news is that we are actively working on improving this and provide a stable API for ppx authors. To give an update on this work: we have a working solution on paper and are currently moving on with the implementation. Once we have more, we’ll proof test our solution by backporting it to a few past compiler versions. I’ll try to write up a quick description of our solution so that everybody can follow. We are also going to present it at the upcoming OCaml day at ICFP.

16 Likes

A lot of libraries did an update, and I almost halved the list.

1 Like

Great work. I would reckon that a good way to filter the list further is to only consider libraries downloaded in the last month. In my opinion, libraries with no active users are usually not worth our maintenance effort and should be left alone until users/maintainers step forward.

I think it worth mention Camlp4 was moved under its own organization now: https://github.com/camlp4/camlp4

1 Like

Perhaps worth mentioning briefly that for anybody who -wants- to continue using camlp4, I’m (a) maintaining camlp5 and bringing it up-to-date with everything in ocaml 4.10.0 that I can think of, and (b) I’d be happy to help them port their dependency over to camlp5.

This is not to be construed as an argument for using camlp4/5.

3 Likes

camlp4 4.10 is now available in opam. Quoting the readme :

Later releases will try to keep camlp4 buildable, by supporting new OCaml AST but not new syntax constructions, which means camlp4 will be able to parse only OCaml language up to 4.08. Rationale: existing code using camlp4 will still be buildable, but no new code should be written with camlp4.

3 Likes