Looking for "lovely, idiomatic" examples of Ocaml used for shell-scripting in the manner of Perl/Python (but esp. Perl)

OK, so an example (the one that motivated this post):

  1. ocamlfind constructs a series of command-fragments of the form
    ["-ppx"; "/{path-to}/ppx_import/./ppx.exe -ppx"] (that’s a list of strings; second string has a space in it) from the packages you specify on the command-line.
  2. these arguments are passed to ocamlc to invoke PPX rewriters.
  3. I’d like to take those strings, build UNIX commands out of them (which will need managing temp-files), and run those commands so I can see the result of PPX rewriting.
  4. This is a simple task, and in Perl it would be a few lines of code. It’s not as trivial in Ocaml, for many complicated reasons that boil down to “gee, string-handling is a PITA”.
  5. once I have my commands (re-)constructed, I’d like to execute them, and again, executing them and dealing with return-codes is a little bit of a PITA.

Of course, all of this is doable: I wrote code to do it, after all. But I’m acutely aware of how much easier that code would be to write in Perl, and it’s clear to me that that code would be no more reliable, for being easier-to-write.

Maybe nobody has looked at making this better. That’s OK, it’ll just go on the ever-expanding list of stuff to occupy my quarantine time, so it’s all good. But I figured I should ask around first.

bos seems like it can do a lot of what you’re looking for. It’s at least worth taking a look, though it may not be at Perl levels of concise for this kind of task.

2 Likes

Yep, I’m a big fan of bos (and everything @dbuenzli writes). But there’s a difference between writing “lovely Ocaml code” and “idiomatic script code”. And for sure, there are a ton of use-cases, where I reach for Perl instead of Ocaml, because I want an idiomatic perl/bash-script, and not an Ocaml program.

Heh, thing is, sometimes I start off with that perl-script, and then, 500 lines later, I wish I’d started in Ocaml. This is only one of the many reasons why it would be nice if I could start in Ocaml. Unfortunately, the reality is, for most such scripts, they’ll never each 500 lines, so starting them in Ocaml on the off-chance that I’ll want to grow 'em … isn’t cost-effective.

Certain languages, like perl and ruby, are optimized for string manipulation, regexp, and spawning processes, saving specific symbols for those tasks and making those tasks first class citizens of the language (usually via hacks in the language such as automatically memoizing regexp compilation). Bash/zsh are even more optimized for process spawning and piping (everything is a string by default), at the cost of their ability to handle almost all other kinds of computation. Python is one step removed from ruby/perl in its treatment of these tasks – they’re all a little more cumbersome.

I’ve long thought that it would be nice to have libraries in OCaml written specifically for shell scripting. Specifically, you’d want to not have nearly as much error checking as you’d have for the application way of doing things, and use exceptions where possible (crazy, right?). Automatic memoization of regexp would be awesome too (via globals and lazy values perhaps).

I tried to summarize my take on the subject into this gist: https://gist.github.com/mjambon/bb07b24f89fa60c973735307ce9c6cb9

I’m not aware of the existence of such tool, but this is how I might design it. This should be reminiscent of camlp4’s quotation and anti-quotation system, which allows alternating between two syntaxes within a source file.

2 Likes

I don’t think it’s lovely and I have no idea if it is idiomatic, but I made a few scripts of my own in OCaml using the same library that other mentioned: bos

  • typepass uses xdotool to type passwords from the password password manager
  • conn wraps wpa_supplicant, dhcpcd, ip, and other network management CLI
  • laptop-status fetches status information for laptops (e.g., battery level) and prints it in a nicely formatted form
  • bakelite increases or decreases screen brightness
1 Like
1 Like

I have no particular opinion about the rest, but at least on the regex side, this might be of interest: https://github.com/paurkedal/ppx_regexp

If that’s still not good enough, I would be very interested by suggestions on how to make it more convenient. :slight_smile:

1 Like

No solutions here, but I just want to put in a plug for the REXX “PARSE” feature. I hardly have used REXX and am no great fan in general, but this particular feature … for example,

Parse Var z 'key=' v ','

I guess it depends on syntactical craziness that makes it irrelevant to OCaml or any other serious language, and after all it’s pretty close to the common scanf type of parsing, but I just thought I’d mention it in case it inspires someone to greatness.

I think shexp might deserve another look. It’s not an interpreter for a sexp-based shell language, as its name might unfortunately deceivingly suggest. It’s really a DSL for constructing shell pipelines using a 'a Process.t monad. The s-expression part is advertising that you can debug and trace the actions performed using s-expressions.

I’ve found Base plus Re to be sufficient for most of my string-manipulation needs. It’s never going to be as concise as Perl’s built-in “magic” support for regexps, but you gain explicitness and clarity, which is part of the benefit of OCaml anyway.

oh, hm, I’ll have to look into this! Thank you!

Oh, also very interesting! Thank you!

This isn’t a direct answer to your question, but since people are talking about libraries, I recommend the excellent Shell library. There’s an async wrapper for it as well.

It’s a lot more comfortable to run a bunch of shell commands with this than calling the lower level standard process creation stuff, IMO.

1 Like

I’m not sure about idiomatic, but I do have a utop config that I use to do some one-off scripting in OCaml that uses shexp

#use "topfind"
#warnings "+a"
#thread
#require "ppx_jane,core"
#require "shexp.process"
#require "lambdasoup"
module List' = List
open Shexp_process
open Shexp_process.Infix
open Core

module Html = struct
    include Soup

    let of_string = parse
end

let read_lines cmd =
    eval (call cmd |- read_all)
;;

let wget url =
    read_lines ["wget"; "-O"; "-"; url]
;;

let chrome_curl url =
    read_lines ["curl"; "-k"; "-sA"; "Chrome"; "-L"; url; "-o"; "-"]
;;

let split_lines = String.split ~on:'\n'
let filter_lines substring = List.filter ~f:String.(is_substring ~substring)
let to_html = Html.of_string
let find_html pat html = Html.(html $$ pat)

let (%) = Fn.compose

Then a simple script called shexp in my path:

utop -init ~/bin/ocaml-shexp-config

I add little helper functions as I come upon them. I find it’s much easier to transition to a file, or full program when I need it. Example program:

utop # read_lines ["sensors"] |> split_lines |> filter_lines "Core 0";;
- : string list =
["Core 0:        +63.0°C  (high = +84.0°C, crit = +100.0°C)"]
3 Likes

Not exactly OCaml, but can be made with the OCaml syntax as well - see BATSH.

Really? hadn’t noticed. Ha ha.

I could never really get urge for Perl, but I use its ancestor awk a lot, and I’m trying out some awk-like simple string functions, like

let strlen = String.length
let sub s i n = let b = strlen s
     in if i < b
         then let n = min n (b - i)
         in String.sub s i n
    else ""
(* substring to end of line *)
let substr a i = if i < strlen a
     then String.sub a i ((strlen a) - i)
     else ""
let matchre t s = try
     Str.search_forward t s 0
     with | Not_found -> -1

etc.

So “open Awk” gets me a handful of more basic variations on common string functions, with less elaborate parameters, no normal exceptions, etc. Including a line by line file processing function. I have just newly started on this and haven’t used it extensively, but it seems fairly promising. No wacky syntax or hyper intelligent string processing, no packages, just a few dozen lines of cheater functions.

“Awk” is a misnomer, in that there’s little correspondence between this and awk, it was just what inspired me to try it.

2 Likes

I just found this - https://github.com/ShamoX/cash. @Chet_Murthy This may be the closest to ocaml shell scripting experience re perl.

To the extent this is about shells, maybe some would be interested to look into “es”, which I think may be the closest anyone has come to a Functional Programming shell. It’s been decades since anyone worked on it, I suppose, but still works. Lexical scope, partial application, etc., with a usable syntax for interactive use and normal shell scripts (i.e., not like Lisp.)

I really like this design. Been turning a similiar idea over in my head for a while, and this is a very nice realization of the concept. If you ever want to hack on something like this I’d be happy to contribute!

1 Like

I’m unlikely to work on this anytime soon due to other priorities but I’d use the tool if it’s solid! I must say the bar is high. Perhaps making such a tool for a more mainstream language than OCaml would be more rewarding.