[ANN] Combining LLVM MC, lld, precompiled C and bytecode

Ecosystem
,
1

This will be much easier to just show what can be done.

# EITHER Windows PowerShell
(New-Object Net.WebClient).DownloadFile("https://aka.ms/vs/17/release/vc_redist.x64.exe", "vc_redist.exe")
./vc_redist.exe /install /passive /norestart
(New-Object Net.WebClient).DownloadFile("https://diskuv.com/a/dk-distribution/2.3.202505202143/dist/dk-windows_x86_64.exe", "dk.exe")

# -OR- macOS/Silicon (or darwin_x86_64)
curl -o dk https://diskuv.com/a/dk-distribution/2.3.202505202143/dk-darwin_arm64
chmod +x dk

# -OR- glibc Linux (or linux_x86)
curl -o dk https://diskuv.com/a/dk-distribution/2.3.202505202143/dist/dk-linux_x86_64
chmod +x dk

Then:

# Copy and paste this line
./dk -g dune -U "Tr1Stdlib_V414Io.StdIo.print_endline {|Make it so.|}" Run

# And copy from `./dk` to the end of the code block
./dk -g dune -S "
    (* Namespace imports.
       Syntax? These go inside [struct ... end] *)
    module Http = DkNet_Std.Http
    module Uri = Tr1Uri_Std.Uri
    let print_endline = Tr1Stdlib_V414Io.StdIo.print_endline
 " -U "
    (* A [unit] expression.
       Syntax? These go inside [let () = ...] *)
    print_endline @@
    Lwt_main.run @@
    Http.fetch_url ~max_sz:4096 @@
    Uri.of_string {|https://jigsaw.w3.org/HTTP/h-content-md5.html|}
 " Exe

Several platforms will be cross-compiled (darwin_* available only on macOS):

$ file target/ZzZz_Zz.Adhoc-* | cut -c1-69 | awk '{print $0 "..."}'
target/ZzZz_Zz.Adhoc-android_arm32v7a:   ELF 32-bit LSB pie executabl...
target/ZzZz_Zz.Adhoc-android_arm64v8a:   ELF 64-bit LSB pie executabl...
target/ZzZz_Zz.Adhoc-android_x86_64:     ELF 64-bit LSB pie executabl...
target/ZzZz_Zz.Adhoc-darwin_arm64:       Mach-O 64-bit executable arm...
target/ZzZz_Zz.Adhoc-darwin_x86_64:      Mach-O 64-bit executable x86...
target/ZzZz_Zz.Adhoc-linux_x86:          ELF 32-bit LSB pie executabl...
target/ZzZz_Zz.Adhoc-linux_x86_64:       ELF 64-bit LSB pie executabl...
target/ZzZz_Zz.Adhoc-windows_x86_64.exe: PE32+ executable (console) x...
target/ZzZz_Zz.Adhoc-windows_x86_64.pdb: MSVC program database ver 7....
target/ZzZz_Zz.Adhoc-windows_x86.exe:    PE32 executable (console) In...
target/ZzZz_Zz.Adhoc-windows_x86.pdb:    MSVC program database ver 7....

The executable for your platform won’t need curl, libev and other non-system libraries:

$ otool -L target/ZzZz_Zz.Adhoc-darwin_arm64
target/ZzZz_Zz.Adhoc-darwin_arm64:
        /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation (compatibility version 150.0.0, current version 2420.0.0)
        /System/Library/Frameworks/Security.framework/Versions/A/Security (compatibility version 1.0.0, current version 61123.100.169)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)
        /usr/lib/libiconv.2.dylib (compatibility version 7.0.0, current version 7.0.0)

$ target/ZzZz_Zz.Adhoc-darwin_arm64
<HTML>
...
</BODY></HTML>

I’m collecting feedback on using dk to develop programs that run as well on foreign platforms as they do on dev machines. So, I’m especially interested in constructive criticism vis-à-vis Go, Zig, etc. but I’ll take positive feedback as well (which I may shamelessly post).

Please read the release notes before using:

  1. The Windows executables generated by Exe have debug symbols which require Visual Studio when the executables are run. There is a ./dk ... -O ReleaseSmall option which will be for distributing executables without the Visual Studio debug C runtime, but that option has a bug which I’ll fix soon (not today).
  2. The disk space for the mostly immutable builds is not garbage collected yet. Clean up manually at ~/.cache/dkcoder and ~/.local/share/dkcoder (or your XDG paths) on Unix, or %LOCALAPPDATA%\Microsoft\Windows\INetCache\DkCoder and %LOCALAPPDATA%\Programs\DkCoder on Windows.
  3. Set the environment variable DK_FACADE=UPGRADE_NEVER if you do not want automatic upgrades. Upgrades are done using MlFront_Signify which is waiting in the opam repository queue.

I’ll get documentation organized depending on the interest level and what questions are asked. For now, use the ./dk --help option, post questions here and/or chat with @jonahb on Discord. Bugs go in https://github.com/diskuv/dkcoder/issues.

Engineering details: Underneath the build architecture uses lld (the clang/LLVM linker) to link native executables from precompiled C libraries, the OCaml runtime, OS sysroots and LLVM MC generated object files that are derived from OCaml bytecode. And there is no JIT or AOT compilation although the architecture supports both.

Enjoy!

3 Likes