# Binary encoding using Stdint

**URL:** <https://discuss.ocaml.org/t/binary-encoding-using-stdint/1349>\
**Category:** Learning\
**Created:** [December 27, 2017, 10:16am UTC](https://discuss.ocaml.org/t/binary-encoding-using-stdint/1349 "2017-12-27T10:16:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![pukkamustard](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/pukkamustard/32/2330_2.png) [@pukkamustard](https://discuss.ocaml.org/u/pukkamustard)\
**Post date:** [December 27, 2017, 10:16am UTC](https://discuss.ocaml.org/t/binary-encoding-using-stdint/1349/1 "2017-12-27T10:16:05Z")

</div>

Hello!

I am playing around with some binary protocols and would like to have a nice interface for decoding various sized integers using Angstrom ([https://github.com/inhabitedtype/angstrom](https://github.com/inhabitedtype/angstrom)) and encoding them in a nice way as well. For this I am using the stdint library [https://github.com/andretnth/ocaml-stdint](https://github.com/andretnth/ocaml-stdint)) and extending it with a Functor. This is working very nicely for decoding. For encoding I am observing unexpected behavior and need some help figuring out what is going on.

Following is the extension to stdint with runnable example (requires angstrom, stdint and hex):

```auto
let make_decoder bits of_bytes = 
  let open Angstrom in
  let byte_length = bits / 8 in
  Angstrom.count byte_length any_char >>= fun char_list ->
  of_bytes
    (Bytes.init byte_length (List.nth char_list))
    0
  |> return

let make_encode bits to_bytes =
  let byte_length = bits / 8 in
  let bytes = Bytes.create byte_length in
  fun v -> (
      to_bytes v bytes 0;
      bytes
    )

module Extend(A: Stdint.Int) = struct
  include A

  let decoder_little_endian = 
    make_decoder A.bits A.of_bytes_little_endian

  let decoder_big_endian = 
    make_decoder A.bits A.of_bytes_big_endian

  let encode_little_endian =
    make_encode A.bits A.to_bytes_little_endian

  let encode_big_endian =
    make_encode A.bits A.to_bytes_big_endian
end

module Int8 = Extend(Stdint.Int8)
type int8 = Int8.t

module Uint8 = Extend(Stdint.Uint8)
type uint8 = Uint8.t

module Int16 = Extend(Stdint.Int16)
type int16 = Int16.t

module Uint16 = Extend(Stdint.Uint16)
type uint16 = Uint16.t

module Uint32 = Extend(Stdint.Uint32)
type uint32 = Uint32.t

let () =
  let n = 42 in
  let hexdump = fun s -> s |> Hex.of_string |> Hex.hexdump in
  n |> Int16.of_int |> Int16.encode_little_endian |> hexdump;
  n + 1 |> Int16.of_int |> Int16.encode_little_endian |> hexdump;
  [n; n + 1] |> List.map (fun x -> x |> Int16.of_int |> Int16.encode_little_endian) |> String.concat "" |> hexdump;

```

The program outputs:

```auto
00000000: 2a00 *
00000000: 2b00 +
00000000: 2b00 2b00 ++

```

Here `[n; n+1]` encodes (incorrectly to) `2b00 2b00` even tough `n` encodes to `2a00` and `n+1` to `2b00`.

I would expect:

```auto
00000000: 2a00 *
00000000: 2b00 +
00000000: 2a00 2b00 *+

```

So encoding `n` and `n+1` separately is ok, but doing it together in `List.map` does weird stuff. This is very side-effecty and I can’t figure out how or why.

---

<div class="post-metadata">

**Author:** ![andrenth](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/andrenth/32/546_2.png) [@andrenth](https://discuss.ocaml.org/u/andrenth)\
**Post date:** [December 27, 2017, 11:29am UTC](https://discuss.ocaml.org/t/binary-encoding-using-stdint/1349/2 "2017-12-27T11:29:38Z")

</div>

In `make_encode` you’re using a single buffer, so when you make multiple calls and then print the result, you only get the last encoded value.

This change fixes it:

```diff
 let make_encode bits to_bytes =
   let byte_length = bits / 8 in
- let bytes = Bytes.create byte_length in
   fun v -> (
+ let bytes = Bytes.create byte_length in
       to_bytes v bytes 0;
       bytes
     )

```

---

<div class="post-metadata">

**Author:** ![pukkamustard](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/pukkamustard/32/2330_2.png) [@pukkamustard](https://discuss.ocaml.org/u/pukkamustard)\
**Post date:** [December 27, 2017, 5:46pm UTC](https://discuss.ocaml.org/t/binary-encoding-using-stdint/1349/3 "2017-12-27T17:46:53Z")

</div>

Beautiful. Thank you very much!
