Test Mocks for Eio.Flow and StdEnv to write to a file

Hello,

When I was exploring mock tests I tried to use the mock Flow by trying to mock StdEnv and using Eio_Mock.Flow

I mistakenly believed that I could switch between Flow and Eio_Mock.Flowin the actual code so that I can either write to a file or the buffer. The buffer contents can be inspected for testing.

But I didn’t achieve that. StdEnv isn’t mocked fully even in Eio_Mock. So I now try to write two differnet types of tests without involving the SUT.

And I came up with this type of pattern encapsulating the contents to be writtern and the Flow.

Is this the best approach possible ?


let create_keyvalue  (type t) (type f) k v flow  =
  let module Key_value = struct
    type key_value = t
    type flow = f
    let key =  k
    let value = v
    let flow_type = flow
  end in
  (module Key_value : KEYVALUE with type key_value = t and type flow = f)

(* A test to use the mock and one more to write to a file by switching to a real Flow *)
let%expect_test "Test Set and Get keys"=
 Eio_main.run @@ fun env ->
let module MockWalWriter =
  (val create_keyvalue (Bytes.make 1 (Char.chr 1)) (Bytes.make 1 (Char.chr 2))  (Eio_mock.Flow.make "mock-stdout")
    : KEYVALUE with type key_value = Bytes.t and type flow = Eio_mock.Flow.t) in
  Eio.Flow.copy_string (Bytes.to_string MockWalWriter.key)  (Eio.Stdenv.stdout env)

Thanks.

Hi @Mohan_Radhakrishnan,

I think we need a few more bits of information here about what you are trying to achieve and what is not working for you. I also think a little more specificity in your title would make this much more searchable later (e.g. “How to use Eio’s mock flows to write data”).

I’m not sure what “like this” is referring to here?

The idea with mock flows is that anywhere that accepts a flow will also accept a mock flow that you can instrument in some way to test. Your actual code remains agnostic to whatever flow it accepts. Consider the following self-contained example:

type t = { name : string; age : int }

let to_json t = Fmt.str {|{ "name": "%s", "age": %i }|} t.name t.age

(* Read CSV from stdin *)
let parse_from_stdin stdin =
  let buf = Eio.Buf_read.of_flow ~max_size:max_int stdin in
  let rec loop acc =
    try
      let line = Eio.Buf_read.line buf in
      match String.split_on_char ',' line with
      | [ name; age ] -> loop ({ name; age = int_of_string age } :: acc)
      | _ -> invalid_arg line
    with End_of_file -> List.rev acc
  in loop []

(* Copy to stdout *)
let dump_to_stdout stdout ts =
  Eio.Buf_write.with_flow stdout @@ fun bw ->
  Eio.Buf_write.string bw "[";
  let rec loop = function
    | [] -> ()
    | [ t ] -> Eio.Buf_write.string bw (to_json t)
    | t :: rest ->
    Eio.Buf_write.string bw (to_json t);
    Eio.Buf_write.string bw ",\n";
    loop rest
  in
  loop ts;
  Eio.Buf_write.string bw "]"

It is a rather simple parser of a CSV-like list of people’s names and ages. We might read those from command line and print them as JSON to stdout (I’ve used buffered readers and writers here only to show this works when you are doing more with your flows).

let main stdin stdout =
  parse_from_stdin stdin |> dump_to_stdout stdout 

let () =
  Eio_main.run @@ fun env ->
  main env#stdin env#stdout

If we wanted to test our parse_from_stdin we could instead instrument a mock flow to do so.

let test1 () =
  let mock = Eio_mock.Flow.make "mock-stdin" in
  Eio_mock.Flow.on_read mock [ 
    `Return "alice,42";
    `Raise End_of_file
  ];
  let people = parse_from_stdin mock in
  assert (people = [{ name = "alice"; age = 42 }])

We no longer need all of the OS resources that something like Eio_main provides us. We could just run this as let () = test1 (), but to future-proof the code (in case Fibers are involved) we can wrap it in the provided mock backend.

let () =
  Eio_mock.Backend.run @@ fun () ->
  test1 ()

To achieve what you want for a “mock” writer, we can use Eio.Flow.buffer_sink.

let test2 () =
  let people = [ { name = "alice"; age = 42 } ] in
  let buffer = Buffer.create 16 in
  dump_to_stdout (Eio.Flow.buffer_sink buffer) people;
  assert (Buffer.contents buffer = {|[{ "name": "alice", "age": 42 }]|})

Hopefully that helps a little? Please do add more information if this does not answer your question.

2 Likes

I have added some details to the question.

Since I use Eio.Path.( / ) and Eio.Stdenv.fs env I couldn’t understand how a Mock could replace the real Flow.

So I decided not to touch the SUT and inject the Mock Flow. Won’t there be a condition to check if I need a buffer or the real File System ?

Thanks.

Thanks for the extra details @Mohan_Radhakrishnan, that is super helpful. I also put two and two together and realised this is (in part) a continuation of: How do I pass Eio 'env'?.

In light of that I think the question becomes a lot clearer. And yes, if your point of abstraction is an _ Eio.Path.t then at the moment eio.mock does not offer ways to mock the required interfaces (directories and files). This actually gets very complicated very quickly as you end up needing to mock Eio.Fs.Pi.DIR and Eio.File.Pi.RO/RW. The logic between all of your mocks will need to take into account file creation, handlers for calls to stat etc. If you are curious to head down this route, I started mocking Fs.Dir and File.Ro, and I think opening an issue on Eio would be the correct next step. Here is what that might then look like:

module type WalWriter = sig
  type t

  val v : _ Eio.Path.t -> t
  val write : t -> Cstruct.t -> unit
  val read : t -> string
end

let make_stat ~size kind : Eio.File.Stat.t = {
    dev = 0L;
    ino = 0L;
    kind;
    perm = 0;
    nlink = 0L;
    uid = 0L;
    gid = 0L;
    rdev = 0L;
    size = Optint.Int63.of_int size;
    atime = 0.;
    mtime = 0.;
    ctime = 0.;
}

let test1 wal =
  assert (String.length (WalWriter.read wal) = 4)

let test () = 
  let mock_reader = Mock.File.make_ro "wal-reader" in
  Mock.File.on_read mock_reader [ `Return "1234" ];
  let t = Mock.Fs.make "mock-wal" in
  let path : _ Eio.Path.t = ((t :> Eio.Fs.dir_ty r), "/mock.log") in
  Mock.Fs.on_open_in t [ `Return mock_reader ];
  (* For your stat to check if the file exists, not strictly necessary *)
  Mock.Fs.on_stat t [ `Return (make_stat ~size:4 `Regular_file) ];
  (* For the file loading stat *)
  Mock.File.on_stat mock_reader [ `Return (make_stat ~size:4 `Regular_file) ];
  let wal = WalWriter.v path in
  test1 wal 
 
let () =
   Eio_mock.Backend.run @@ fun () ->
   test () 

Pretty gnarly… I think you would be better off using the filesystem directly in your tests instead of trying to mock it. Note as well that I changed your module signature a little to be more conventional. I hope this helps.

1 Like