Compiler Plugins?

So I just happened to run across this by chance:

And this does not bode well for me…

I have an OCaml plugin that uses the typedtree hook (all starting with let () = Typemod.ImplementationHooks.add_hook "elixir_of_ocaml" typemod_hook_cb) as PPX’s happen before type checking and I need information after type checking (optimally I would want it post type checking and uncurrying passes, but I’ll take what I can get, and the Simplif.Hooks.add_hook hook is way too low level and lost too much information for what allows me to write to the Beam).

So far I’d been making this on 4.7.1 then 4.8.0 beta’s and just updated to 4.8.0 rc1, but now come to find out all my work is going to be dead come 4.9.0 if I’m reading this right.

Please someone tell me that I’m reading this wrong? :frowning:

And if I’m not, how can I do this same thing without using a compiler plugin or forking the compiler (I exceptionally do not want to fork the compiler, it should work with a standard install in every way, user PPX’s, user preprocessors, Dune, etc… etc… etc…).

2 Likes

Add a comment on the PR that removed compiler plugins to register your concerns.

I too was surprised about this extension point being removed. Interestingly GHC acquired support for compiler plugins not very long ago AFAIK and here we dropped support…

2 Likes

Your best bet currently is to create a custom compiler driver that accesses the compiler-libs package. It will be a new executable, but you can theoretically do everything you’ve done so far AFAIK.

My main concern is I want it to be drop-in to the normal ocaml compiler, with no other surprises around it, same arguments, same support of tools, etc… The end goal of this is being able to output both Elixir code as well as generate a standalone BEAM node as a native compile, the plugin made that easy as I just needed to read the state of the typed AST and write it out to an appropriate file. I’d essentially be just making ocamlopt again in full with the plugin ‘built in’, which sounds a lot like forking it to me, especially as I don’t want it to require needing to recompile the toolchain or anything of the sort, it should work with the system-installed ocaml. A custom compiler driver from my understanding would need to be a separate standalone program to run.

Note that the tool support for plugins was extremely limited and that they were not really drop-in in any way.
Building an alternative driver is a bit clunky right now, but it would be straightforward to make a library that build such driver with additional hooks to emulate the plugin architecture.

The typed trees are dumped into *.cmt and *.cmti files with the -bin-annot option. That is what merlin uses, for example, and dune will automatically build and install these files. Depending on what your project does, it might be enough for your usage.

@OvermindDL1 do you have a pointer to a description of what you are doing with compiler plugins, and maybe the code? It’s hard to evalute your needs, discuss workarounds, or maybe design better approaches in the future, without a clear description.

I know you were hoping to use compiler plugins to output Elixer, but we have been extending jsoo to support other language backends (starting with Hack/php), under a project called “rehp” https://github.com/jordwalke/rehp

I think this is actually the best path to achieving total compatibility with the ecosystem because it means you can use any existing package even if they use ppx, and then start with the bytecode output for your application/dependencies, and then turn that into whatever backend language you intend on supporting. Would you like to collaborate on rehp?

3 Likes

You could use some ideas from the amazing c2rust project as well, which transpiles C to Rust, but also performs some refactoring of the resulting Rust code, as well as providing way to script the refactoring process with Lua.