$ cat hello.mli
type error = [ `Msg of string ]
val to_string : int -> (string, [> `Msg of string ]) result
val f : int -> (string -> ('a, [> error ]) result) -> ('a -> 'b) -> ('b, [> error]) result

$ cat hello.ml
type error = [`Msg of string]
let to_string i =
if i < 10 then Result.ok (string_of_int i) else Result.error (`Msg "invalid number")
let f : int -> (string -> ('a, [> error]) result) -> ('a -> 'b) -> ('b, [> error]) result =
fun i f1 f2 -> Result.bind (to_string i) (fun s -> f1 s) |> function
| Ok a -> Ok (f2 a)
| Error e -> Error e

However, I keep getting the following compilation error,

Values do not match:
val f : int -> (string -> ('a, [> error ] as 'c) result) -> ('a -> 'b) -> ('b, 'c) result
is not included in
val f : int -> (string -> ('a, [> error ]) result) -> ('a -> 'b) -> ('b, 'c) result

It seems the ocaml compiler generalizes (?) [> error] to 'c which makes it not compile to the given hello.mli file. Is there a way to fix this without changing the signature?

Also does 'c still conform to [> error] after it being generalized to 'e?

The inferred type of f indicates that if the f1 argument returns more than the tags in error, then so will f itself. The 'c type variable is expressing that relationship. The type without it is not correct. You either need to constrain f1 by removing < or constrain f with 'c.

@jjbâ€™s reply is excellent, but letâ€™s give a couple extra details. Your example boils down to something like:

apply : (unit -> [> error]) -> [> error])

When you write [> error], you have to understand this as a universal quantification: an instance of [> error] is any type which contains at least error (for example [ error ] or [ error | `Foo | `Bar ]. As with polymorphic type variables in signature, the caller gets to choose the instance they want at function application time. The two separate occurrences of [> error] correspond to two distinct/independent quantifications.

So the caller could instantiate this to, for example:

and clearly this is incorrect (with the â€śobviousâ€ť implementation fun f -> f ()): if the function parameter returns `Foo, then the return type must allow `Foo. In other words, the two occurrences of [> error] must be instantiated with the same type. This corresponds to the following, less general (and more correct) signature:

val apply : (unit -> ([> error ] as 'a)) -> ([> error ] as 'a)