One hurdle to getting started with developing a ppx rewriter for use in another project seems to be that there’s little guidance in terms of how to turn it into an installable package (I’ve seen materials refer to dune, essentially saying that you must know dune before writing a ppx rewriter).
My impression is that a ppx-rewriter “skeleton”/template/“hello world” repository would be very useful to quickly get started tinkering with ppx rewriters. The repository would contain an installable ppxlib-based ppx rewriter that does something trivial (like emitting “hello world” for a given tag).
What do you think?
ETA: A few minutes after I posted, I find manual (ppxlib.manual) which seems to cover exactly what I was looking for. Regardless, coming from the perspective of someone who just quickly wants to write a PPX rewriter for practical use, I still feel it would be helpful to have a repo to clone and hack away.
The boilerplate required to start working on Ppx-writers really is a major pain point for me. Very often I think about writing a small macro for my codebase, but the activation energy required to get the ppx up and running ends up putting me off the idea.
It seems the ppxlib documentation has been improving significantly more recently, so some of that cost is being whittled away, but it would still be nice to have some more tooling support reducing the cost of writing ppxs.
Hi @sabine, if I understand you correctly, the examples folder on the ppxlib repo is what you’re looking for. It contains two different very very simple PPXs: one deriver and one extension point rewriter. So to see how the dune file for a very simple deriver looks like, you can go to the deriver’s dune file; and similarly for the extension point rewriter.
Is that what you were looking for? Or is your point that that folder should be separated into a separate repo?
In any case, even if that’s what you were looking for, your comment is very valuable since, in that case it shows us, that our examples/ folder isn’t easy to be found.
Btw, talking about troubles getting a PPX up and running: I’m interested in knowing what people mostly struggle with. @Gopiandcode, you’ve mentioned that hussle above. You’ve mentioned boilerplate. Were you referring to the PPX’s dune file? Or to registering the PPX (so e.g. in the case of a deriver calling Deriving.Generator.V2.make_noarg and then Deriving.add)? Or were you referring to getting your head around and manipulating the AST? Or to making sure that your PPX is trustable and doesn’t tinker with the rest of the program (keywords error handling and hygiene).
Also, as both of you, @sabine and @Gopiandcode, have already indirectly mentioned: @panglesd has recently done and is currently doing an amazing job at improving the ppxlib documentation on different levels! For example, see the manual @sabine has linked to above. I recommend having a look at it, it’s really good work (and he’s even still improving things and making them more complete)!
The dune file configuration for ppxs is actually pretty straighforward - I just need to add an extra option to my stanzas and then I’m writing ppxs - this works really well.
Manipulating syntax does require a little concentration to make sure you’re doing the right thing, but that’s more intrinsic to the domain of metaprogramming. My main pain points with ppxlib relate more to the bureaucracy.
Yes, this is exactly the problem - I’m not sure if this is a very constructive critisism, but in my experience, I’ve found that the API for ppxlib isn’t very “discoverable”:
one way this occurs is in the arcane sequence of incantations required to setup a ppx (some sequence of rules, rewrites, etc.)[1]
another way this arises is in the large number of modules possible to be opened at any time - it’s easy to fall into a rabbit hole of exploring the APIs modules that are completely irrelevant to most cases of writing a ppx.
For the second case, I understand that these modules are really necessary for supporting all the usecases of ppx, but sometimes as a beginner, I don’t want to have to see all of them.
I think one thing that might help is defining a module that re-exports a subset of the core modules of ppxlib that a beginner should use, and then tell beginners to primarily only use modules exported in this simplified module. (i.e only use modules under Ppxlib.Beginner.*). This would work hand in hand with merlin and make it much easier to discover the api, as autocompleting Ppxlib.Beginner.* would quickly give me a list of modules that I should be using).
The improvements to the ppxlib documentation are much appreciated! The documentation is so much easier to understand now since when I first looked at it a few years back. Major kudos to the people who have worked on it!
[1] As a side note - one thing that I consistently leaves me scratching my head, even though I’ve found the answer myself several times before, is the ~name parameter - when setting up a ppx, there are several points at which you need to supply a name. Some of these parameters are actually relevant to the usage of the ppx (such as defining the [%<name ...] forms that will run the ppx), but some others are just for debugging. Every time I write a ppx, I spend a bit of time trying to recall which ones are which.
Super stoked with all the great answers and I think they show how much has been happening behind the scenes and in places that my naive web and forum search didn’t properly reach! Thank you @davesnx for sharing your ppxlib example repo, this definitely gives me an idea how things should look like in order to use the PPX in a different codebase!
I have to admit I am an odd case: I’m a fairly new OCaml user. I have written Template Haskell before, but have never written an OCaml package. So I know exactly what I want to do and I know how metaprogramming via manipulating ASTs in general works. I have only been working on existing OCaml projects and never needed to set up my own. (Tbh, I am more interested in writing code that solves problems for people rather than learning about the ins and outs of build systems. )
With so much happening over the last years, it felt unsafe to rely on tutorials that are more than a year old (as best practices might have changed) or where it is not immediately obvious how old the content is. So in a sense this is a luxury problem that will disappear as content from people sharing their experiences takes the prime spots on search engines, and as we get ocaml.org in a shape that it helps people quickly become productive and do cool things with OCaml.
Regardless, coming from the perspective of someone who just quickly wants to write a PPX rewriter for practical use, I still feel it would be helpful to have a repo to clone and hack away. 
For example, see the manual 
)