[ANN] Feather 0.3.0

Community
1

I’m happy to announce Feather 0.3.0! Feather is a minimal library for bash-like scripting and process execution. (github/tutorial, documentation) This release adds two major features:

1. A new interface for collecting the exit status, stdout, and stderr of a Feather command.

For example, you can easily print a process’s stderr if it exits non-zero:

open Feather;;
let stderr, status =
  process "ls" [ "/tmp/does-not-exist" ] |> collect stderr_and_status
in
if status <> 0 then failwith ("ls failed with stderr:\n" ^ stderr)

where the types are

val process : string -> string list -> cmd

type 'a what_to_collect
val stderr_and_status : (string * int) what_to_collect

val collect :
  ?cwd:string ->
  ?env:(string * string) ->
  'a what_to_collect ->
  cmd ->
  'a

as you can imagine, we expose several of these what_to_collect's. Here’s the full set:

val stdout : string what_to_collect
val stderr : string what_to_collect
val status : int what_to_collect

val stdout_and_stderr : (string * string) what_to_collect
val stdout_and_status : (string * int) what_to_collect
val stderr_and_status : (string * int) what_to_collect

type everything = { stdout : string; stderr : string; status : int }
val everything : everything what_to_collect

We considered different design approaches here. I think what we landed on keeps the call site readable and the types of the interface simple.

It should be noted: the simplest way to run a command without collecting anything is to use Feather.run.

2. The ability to wait on background processes and collect their output.

Starting with Feather 0.1.0, you were able to start processes in the background, but the only way to wait for them to complete was to use Feather’s async wrapper. For those wanting an async-less, direct-style interface, we now expose new methods to do this properly:

type 'a background_process

val run_in_background :
  ?⁠cwd:string ->
  ?⁠env:(string * string) Base.list ->
  cmd ->
  unit background_process

val collect_in_background :
  ?cwd:string ->
  ?env:(string * string) list ->
  'a what_to_collect ->
  cmd ->
  'a background_process

val wait : 'a background_process -> 'a
val wait_all : unit -> unit

where an example use might be

let server_process =
   process "my-server.exe" [] |> collect_in_background stdout_and_status
in
... do other things ...
match Feather.wait server_process with
| (stdout, 0) -> ...
| (_, 1) -> ...

Thanks again to @Firobe and @tmarti2 for their contributions to this release! I think we’ve made a lot of progress here and I’m excited to see where things go :slight_smile:

19 Likes
2

Why not allow users to do this:

Feather.process "cat text.txt"
3

We could probably expose Feather.bash "cat text.txt"

let bash program = process "bash" [ "-c"; program ]

along with something like Core.Sys.quote so it could be used safely.

So far we’ve avoided parsing / quoting in Feather and I think that makes things simpler both for the user and for the implementation. (Though maybe a quoting function would be good for people who do use bash already?)

I think a good pattern to get help with ergonomics is to write helper functions for the processes that get used a lot. Even if it’s just let ssh args = process "ssh" args. Then the call site is just

ssh [ "test@host"; "--"; "etc" ]

which looks cleaner. But it hasn’t been hard to come up with more semantic OCaml interfaces and then you get merlin support!

We already expose some helpers; I’d be interested in maintaining more / better ones as time goes on.

1 Like
4

Maybe it is not safe, but I often construct command lines using sprinf:

let cmd = Printf.sprintf "full_command_line_format_with_percents" arg1 arg2 arg3 in
Sys.command cmd
1 Like
5

I was about to suggest / ask after an lwt variant, but I see that’s already noted @ Feather_lwt · Issue #22 · charlesetc/feather · GitHub :slight_smile:

1 Like
6

I think it would be extremely useful for some applications but:

  • Isn’t it out of scope for the Feather project? (although this may get you some users just for this function :sweat_smile:)
  • Would it be completely reliable? e.g. if someone writes unintentionally sprintf "echo '%s'" (quote msg), we’re back to being vulnerable to code injections.
1 Like
7

Thanks for making Feather! I’ve been having a blast playing around with it recently, and it’s since then become my go-to source for doing anything more complex on the shell. (Initially, there was some friction in terms of setting up an environment in which I could write standalone scripts without dune etc., but I eventually figured out a combination of a local opam switch + evaluation bash script that works).

Here’s a few functions that I’ve written recently for automating some grading tasks that I thought feather allowed me to express quite elegantly:

let collect_submissions n =
  let base_path = rank_files_path n in
  base_path
  |> ls
  |> collect stdout
  |> lines
  |> List.sort String.compare
  |> List.map (fun path -> base_path ^ path)

let no_pages pdf =
  process "pdftk" [pdf; "dump_data"]
  |. grep "NumberOfPages"
  |> collect stdout
  |> String.split_on_char ':'
  |> function
  | ["NumberOfPages"; no_pages] ->
    Int.of_string_exn (String.trim no_pages)
  | ls -> failwith @@ "unexpected output from pdftk:" ^ (String.concat " " ls)
4 Likes
8

Ah I’m glad to hear you’re enjoying it! :tada: