Points-free style considered harmful?

I just spent an hour debugging an issue that arose from using refs with point-free style. I am quite ashamed that this happened to me after years of “experience” with OCaml, so I’m posting here mainly to have you kind folks tell me, “it’s okay, it happens to the best of us” (even if it doesn’t :slight_smile:).

Simplified version of what happened:

let x = ref 5 
    
let addx_p y = y + !x
             
let addx = (+) !x
             
let () =
  Printf.printf "%d\n%!" (addx_p 3);
  Printf.printf "%d\n%!" (addx 3);
  x := 6;
  Printf.printf "%d\n%!" (addx_p 3);
  Printf.printf "%d\n%!" (addx 3)

prints:

8
8
9
8

I was using the equivalent of addx in my code and scratching my head about why my state wasn’t updated.

Since points-free is not just a “style” in OCaml, it has an observably different meaning in the presence of side-effects - should it be actively discouraged?

3 Likes

I would think that it’s the other way around: use of mutable datastructures makes pointsfree style problematic. So if you’re going to use pointsfree style, don’t use mutables (or only with great care).

More theoretically, the conversion from lambda-calculus to combinators is only valid for functional programs, not for imperative ones, right? So the argument for pointsfree doesn’t work unless you’re already functional-only.

8 Likes

Since points-free is not just a “style” in OCaml, it has an observably different meaning in the presence of side-effects - should it be actively discouraged?

In my opinion: yes :slight_smile: .

It can change semantics, and OCaml isn’t particularly good at optimizing tangles of functional operators; not to the same level as Haskell.

2 Likes

Interesting question. I find it does more good than harm in OCaml and I regularly exploit exactly the kind of thing you’re doing. So I say embrace it!

Couple of thoughts:

I would not call the definition of addx an instance of point-free style; it’s just vanilla partial application, creating a closure with a subset of a function’s parameters. That is often a useful tactic, and its efficiency and ease in OCaml (and similar FP langs) is one of its landmark strengths IMO. To my mind, point-free style is where you create new definitions by directly composing chains of existing functions, e.g. let positives = filter (< 0) %> sort %> uniq (using the %> operator in Containers.Fun for composition there).

I’m not clear on this at all; what semantics are changed via partial application or function composition? Maybe you want to say that the (+) !x verse might be easy to slip up on (maybe ! is easy to miss / become visually blind to?), but there’s nothing semantically unusual going on. To @Chet_Murthy’s point, it’s the mutability that is the source of any surprising outcomes here; if the expression were (+) (some_pure_fun 1 2 3), I don’t think anyone would wonder if the partial application was problematic in and of itself.

Likewise, there’s nothing special to optimize here; it’s a basic partial application (i.e. just a closure), which is a fixture in so many OCaml programs (maybe most commonly when using various “pipelining” operators like |>, |., etc.

1 Like

I’m not clear on this at all; what semantics are changed via partial application or function composition? Maybe you want to say that the (+) !x verse might be easy to slip up on (maybe ! is easy to miss / become visually blind to?), but there’s nothing semantically unusual going on. To @Chet_Murthy’s point, it’s the mutability that is the source of any surprising outcomes here; if the expression were (+) (some_pure_fun 1 2 3), I don’t think anyone would wonder if the partial application was problematic in and of itself.

Sure, I started from the assumption that OCaml code is not necessarily
pure. The semantics that is affected by partial applications appear both
on the type level (value restriction), where you might expect more
polymorphism than you actually get; but also, with side effects, it’s
very easy to make a mistake about when evaluation takes place.

A classic mistake I’ve made in the past:

# List.map (Format.sprintf "coucou %d") [1;2;3];;
- : string list =
["coucou 1"; "coucou 1coucou 2"; "coucou 1coucou 2coucou 3"]

(because Format.sprintf — which imho should never be used, use
Format.asprintf instead — does not handle its internal buffer intuitively.)

In general, partial application and imperative features are powerful
together but also error prone.

Likewise, there’s nothing special to optimize here; it’s a basic partial application (i.e. just a closure), which is a fixture in so many OCaml programs (maybe most commonly when using various “pipelining” operators like |>, |., etc.

Well, one could say that an expression like succ %> (( * ) x) (where %>
is the trivial composition operator, and double x = x * 2) should be
optimized into (fun y -> (x+1)*y). That gives you a single closure
instead of two. I’d expect GHC to do this kind of simplifications.

Also, OCaml treats |> and @@ specially (they’re primitives known to the
compiler, which knows to turns them into regular applications).

3 Likes

You’re right, of course, I should stop using words I don’t understand.

You’re right again, it’s the mutability that’s the real problem, as @Chet_Murthy said. Here’s something closer to what actually happened:

module Store = struct
  type t = (string,string) Hashtbl.t

  let db : t ref = ref (Hashtbl.create 100)

  let add = Hashtbl.add !db

  let iter f = Hashtbl.iter f !db

  let mem = Hashtbl.mem !db

  let read f = db := In_channel.with_open_bin f Marshal.from_channel |> Hashtbl.rebuild

  let write f = Out_channel.with_open_bin f (fun oc -> Marshal.to_channel oc !db [])
end

Here, after you Store.read from a file, Store.iter will work with the new state, but Store.mem and Store.add won’t. Imagine my surprise that I can print all the keys/values, but when I search for one of them, it can’t find any of them. I legit thought for some time that Marshal that was broken. If I’d written mem and add to take arguments, it’d work though:

module Store = struct
...
  let add k v = Hashtbl.add !db k v
...
  let mem k = Hashtbl.mem !db k
...
end

The difference between let mem = Hashtbl.mem !db and let mem k = Hashtbl.mem !db k seems so small when you’re typing, after all, the resulting types are the same, the sub-expressions are all the same, what could go wrong? But they’re oh so different. And it seems so tempting to save time by typing two characters fewer, so you can spend hours on the important stuff - debugging.

@c-cube 's sprintf example is nice too - I can totally see myself doing that.

Yeah, code like this gives me a great deal of anxiety :sweat_smile: Hashtbl is itself mutable, so putting it into a ref makes for a very error-prone situation.

What is the rationale for the ref in the first place?

No good rationale tbh, I could have added a create function, and passed it around as an arg, but as always, typing fewer characters is my usual motivation for doing dumb stuff :laughing:

FWIW, with what you’ve posted so far, you don’t need a create function either. Just let db = Hashtbl.create 100, remove all the bangs since there’s no ref anymore, and change read so it reads from the channel, resets db, and adds in all of the read hashtable’s entries (using to_seq and add_seq).

1 Like

Do you mean something like this?

let read f =
  Hashtbl.reset db;
  In_channel.with_open_bin f Marshal.from_channel
  |> Hashtbl.to_seq 
  |> Hashtbl.add_seq db

Doesn’t that use double the space, and and make me pay twice for building a Hashtbl.t, once in Marshal.from_channel and then in add_seq?

Or did you mean something more clever?

I would only issue the reset after the table has been successfully unmarshalled, but yes, that’s roughly what I meant. You’re right that it’s not clever, and that it carries inefficiencies; it’s the tradeoff of not using a ref, and storing the hashtbl in its entirety. You could avoid those inefficiencies by reading and writing streams of hashtbl entries, but unless the table is holding a very, very large number of entries, the advantage to optimizing any of this is probably inconsequential.

1 Like