I have an OCaml plugin that uses the typedtree hook (all starting with let () = Typemod.ImplementationHooks.add_hook "elixir_of_ocaml" typemod_hook_cb) as PPX’s happen before type checking and I need information after type checking (optimally I would want it post type checking and uncurrying passes, but I’ll take what I can get, and the Simplif.Hooks.add_hook hook is way too low level and lost too much information for what allows me to write to the Beam).
So far I’d been making this on 4.7.1 then 4.8.0 beta’s and just updated to 4.8.0 rc1, but now come to find out all my work is going to be dead come 4.9.0 if I’m reading this right.
Please someone tell me that I’m reading this wrong?
And if I’m not, how can I do this same thing without using a compiler plugin or forking the compiler (I exceptionally do not want to fork the compiler, it should work with a standard install in every way, user PPX’s, user preprocessors, Dune, etc… etc… etc…).
Add a comment on the PR that removed compiler plugins to register your concerns.
I too was surprised about this extension point being removed. Interestingly GHC acquired support for compiler plugins not very long ago AFAIK and here we dropped support…
Your best bet currently is to create a custom compiler driver that accesses the compiler-libs package. It will be a new executable, but you can theoretically do everything you’ve done so far AFAIK.
My main concern is I want it to be drop-in to the normal ocaml compiler, with no other surprises around it, same arguments, same support of tools, etc… The end goal of this is being able to output both Elixir code as well as generate a standalone BEAM node as a native compile, the plugin made that easy as I just needed to read the state of the typed AST and write it out to an appropriate file. I’d essentially be just making ocamlopt again in full with the plugin ‘built in’, which sounds a lot like forking it to me, especially as I don’t want it to require needing to recompile the toolchain or anything of the sort, it should work with the system-installed ocaml. A custom compiler driver from my understanding would need to be a separate standalone program to run.
Note that the tool support for plugins was extremely limited and that they were not really drop-in in any way.
Building an alternative driver is a bit clunky right now, but it would be straightforward to make a library that build such driver with additional hooks to emulate the plugin architecture.
The typed trees are dumped into *.cmt and *.cmti files with the -bin-annot option. That is what merlin uses, for example, and dune will automatically build and install these files. Depending on what your project does, it might be enough for your usage.
@OvermindDL1 do you have a pointer to a description of what you are doing with compiler plugins, and maybe the code? It’s hard to evalute your needs, discuss workarounds, or maybe design better approaches in the future, without a clear description.
I know you were hoping to use compiler plugins to output Elixer, but we have been extending jsoo to support other language backends (starting with Hack/php), under a project called “rehp” https://github.com/jordwalke/rehp
I think this is actually the best path to achieving total compatibility with the ecosystem because it means you can use any existing package even if they use ppx, and then start with the bytecode output for your application/dependencies, and then turn that into whatever backend language you intend on supporting. Would you like to collaborate on rehp?
You could use some ideas from the amazing c2rust project as well, which transpiles C to Rust, but also performs some refactoring of the resulting Rust code, as well as providing way to script the refactoring process with Lua.
I apologize for the delay in responding, my free time is minimal so I tended to only work on this project rarely.
No public code, it’s in my big playground of ‘test projects’ on bitbucket before they graduate to full projects (I.E. right now it is one hugely-massive single elixir_of_ocaml.ml file).
I’ve seen that but I was under the assumption that jsoo worked on the level closer to bytecode. I’m needing things like annotations to survive (which I think they would), things like `Blah to be able to get the name of it instead of just the integer hash of it as I map polymorphic variant names to atoms on the beam, in addition to variant head names into tuples or maps on the beam (depending on if it is a tuple head or a record head, plus annotation’s can refine that into other forms as well for greater compat with other BEAM code), plus I want the type information to survive so I can properly create spec definitions for the functions on the beam so I can have dialyzer know about the code so it can type check elixir_of_ocaml code with the rest of the beam ecosystem to confirm that non elixir_of_ocaml code can call elixir_of_ocaml code safely’ish.
I would actually love this method though, but I definitely need things like variant heads in their string form as I need to map them to BEAM atoms considering their ubiquity on both sides.
I think you should really give compiler-libs a try. The main downside is that the code using it is not ocaml-version-portable, but then plugins are not either. We are regularly cleaning up APIs in the frontend, which should make the life of the compiler-driver writer more pleasant over time.
That’s what I’ve been leaning toward so far actually, and some people in the Elixir community told me they are fine with two step compilation for both to-elixir and to-native compilations. I’m not really that big on being version portable either since as new versions of OCaml come out I want to compile any new or changed constructs better when at all possible (otherwise trying to stay up to date).
That’s awesome to hear.
Are there any docs on getting started with writing a compiler driver or should I just read the existing ocamlc source or so like how I had to do to figure out how compiler plugins work. I do actually quite prefer reading source to figure things out but some ‘getting started’ docs are immensely useful for saving the little free time I have.
Yes, I’m afraid there is not much in terms of documentation around. There is an example project compiler-libs-hack by Jun Furuse, but the code is a bit out of date (there are simpler ways to do things). Drup has written an eliom frontend that way, but I wasn’t able to find the source. I would recommend looking at driver/compile.ml, which is rather clean, but to use it from an entry point you have to figure out which bits of the global environment to initialize and how, and for this you have to read the rather gnarly driver/main.ml file.
