If I have an interpreter written in OCaml and I want it to run code faster without the burden of having to write a compiler what are my options?
I could use MetaOCaml and do staged metaprogramming. Should be easy and achieve a very respectable speedup but MetaOCaml is obscure so I assume dev tools would no longer work?
If I run my interpreter as bytecode can I generate bytecode on-the-fly and have it executed? If so, how? Can malfunction do this sort of thing?
Are there any libraries that might help like expression trees to LLVM compilers?
This series of posts might help you: How I wrote my own “proper” programming language. This uses a combination of OCaml/C++/LLVM. So not exactly what you are looking for, but definitely a valuable source of inspiration.
As an interpreter, you rely fully on OCaml’s GC and runtime. You’re going to have a very hard time compiling to LLVM without a GC, runtime etc. Your best bet is Malfunction IMO.
Yes, using Dynlink. You compile your input to a .cmo file, then dynlink it from your interpreter.
It seems like Malfunction doesn’t provide any way to output .cmo files, but you can use it with the native version of your interpreter to produce .cmxs files, that you can then dynlink.
For example, ocamlfind (with this package, which is installed automatically) and dune (see, e.g., this example) both support MetaOCaml.
I don’t know how well Merlin works with MetaOCaml, but it might work reasonably well if you use attributes rather than quotation syntax (i.e. (2 + 3) [@metaocaml.bracket] rather than .<2 + 3>.).
Libraries mostly work unmodified (since MetaOCaml is binary-compatible with OCaml), but a few libraries conflict with MetaOCaml’s additional syntax (e.g. by using >. for float comparison) and need minor modifications.