Hi @nojb,
what is the story with dune? Right now, dune “just works” with ppx rewriters based on OMP, what will happen after the announced evolutions?
OMP won’t provide a driver anymore. To write a ppx rewriter that is compatible with Dune, you will have to use ppxlib. You could also write your own driver, thought I wouldn’t recommend doing that; we are trying to reduce the number of APIs rather that increase it 
Will the planned updates to ppxlib (getting rid of base, etc.) be done in time for the release of OMP 2.0.0?
Yes. @ceastlund started working on dropping the Base dependency and since it will be the same group of people doing the ppxlib and omp changes, we will make sure that there is a new version of ppxlib available on time. We plan to do things in this order:
- prepare OMP2 and an updated ppxlib
- add upper bounds to all OMP reverse dependencies in the opam repository
- release OMP2
- release an updated ppxlib shortly after OMP2 is merged in the opam repository
Would it be possible to make a minimal version of ppxlib/OMP for those that are willing to write a ppx rewriter against compiler-libs directly and do not need/want the rest of the library, but still want their rewriters to be compatible with dune?
Technically yes. That said, I wouldn’t recommend going down this route. Once most of the dependencies of ppxlib have been dropped and OMP has been simplified, the resulting OMP+ppxlib combo will be pretty lean. If there are aspects of ppxlib that would prevent people from switching to it, we are happy to consider those.
Once we have finished our work on simplifying the ppx ecosystem, I also expect that the bare ppx API in the compiler will disappear, so I would definitely not recommend to use it.
Could you say a few words on how the evolution of ppxlib is related to ppx 1?
Sure. The ppx project is part of this work. Our plan was to move to a world like this:
- the compiler offers a minimal stable Astlib library, using a dynamic AST
- the ppx project is written only against Astlib and not the rest of compiler-libs
- ppx has a stable API
- ppx rewriters are written against the ppx API
In this world, existing ppx rewriters are compatible with trunk at all times. Moreover, when a new compiler is released:
- existing released ppx rewriters are immediately compatible with the new compiler
- one can use existing ppx rewriters and the new features of the language
in other words, you can in theory release a ppx rewriters once and never touch it again. The trade-off is that it uses a technology that is quite complex and users don’t have direct access to the parsetree types.
However, we don’t expect everyone to switch to this API at once. So we will need a transition period where the existing APIs are implemented as shims on top of the new ppx API. The more ppx APIs we have to consider and the more difficult this work will be for us. So one can see this work of simplifying OMP and ppxlib as preparatory work.
Now, all of the above was our original plan. We are now considering an alternative plan that would lead to a technically simpler system that is easier to reason about. The new plan would lead to the following world:
- the compiler offers a minimal stable Astlib library, using the OMP technology
- ppxlib is written only against Astlib and not the rest of compiler-libs
- ppx rewriters are written against ppxlib
In this world, existing ppx rewriters are compatible with trunk at all times. Moreover, when a new compiler is released:
- existing released ppx rewriters are immediately compatible with the new compiler
- one cannot use existing ppx rewriters and the new features of the language until:
- ppxlib has been updated to use the new parsetree types
- the subset of ppx rewriters that are broken by this change have been updated as well
This new plan requires updating potentially many packages after each new compiler release. We know that from a technical point of view this is relatively easy. In fact, we do that all the time at Jane Street: when we bump the version of the AST, one single person sit downs and updates all the ppx rewriters at once, which doesn’t take very long. In the open-source world, this was historically impossible, though now that most OCaml projects are using dune this is becoming more realistic; we can create one big dune workspace with all the ppx rewriters and update them all at once. That said, there is a social component to this approach since the various packages are owned by various groups of people and even if one person does all the work, we still need various groups of people to take action to merge PRs and make new releases. So we plan to try this idea with one or two compiler release. In fact, @NathanReb is experimenting with this idea as we speak for the 4.10 upgrade of ppxlib. If this all works out well, we’ll continue with this new plan. Otherwise, we’ll go back to the original plan.