Canonical JSON with Yojson for signature verification

Learning
1

I’m signing JSON in OCaml (yojson) and verifying it in Rust (serde_json::to_vec()). Even using Yojson.Safe.to_string, the outputs differ: key ordering, spacing, etc. Result: signature verification fails.

I need a way to emit deterministic/canonical JSON from Yojson.Safe.t. Anyone got a minimal serializer or working approach?

Thanks!

2

Actually, I am thinking of just shelling out from Rust.

3

Except malicious users might just replace the binary and then I am screwed.

4

Or I can embed the OCaml binary in the Rust binary, maybe?

5

AFAIK people don’t sign the JSON object itself but the values inside it. See eg RFC 7515 - JSON Web Signature (JWS)

1 Like
6

Fair enough. It’s just that I am not serializing and signing primitives/tokens, I need to sign an entire structure.

7

If you are in Rust (and a few other languages) you may want to use Amazon Ion Hash. Ion is a superset of JSON and has a C binding (none for OCaml) and Ion Hash is a tiny addition to that. The hashing was used in the now-defunct Quantum Ledger DB, among other things.

Or if adding encryption to the JSON signing helps avoids the spawning processes threat in your threat model, perhaps look at GitHub - getsops/sops: Simple and flexible tool for managing secrets with age keys.

1 Like
8

Another idea: don’t sign the direct JSON output, but convert the JSON output into a deterministic form that can be signed reliably. Eg, if you have:

{"foo": "bar", "baz": 1}

Convert it into:

["foo", "bar", "baz", 1]

Sign this version and verify it on the Rust side by doing the same JSON transformation there.

1 Like
9

The Yojson.Safe.t type is fairly small and easy to work with. It wouldn’t be hard to write a custom function which outputs what you want. From a quick test a LLM seems to be able to one shot it.

  • always output on a the same line
  • do not put any spaces in between elements
  • sort the object keys
  • disallow types such as float which might have tiny differences depending on the environment?

But I don’t know if it will be as easy to write the exact same function in rust to be sure that the outputs are matching, I’ve no experience with serde_json.

1 Like
10

Sharing link to PRs currently on hold, for reference.

11

I do not understand that. Why do you need to do that? I think I would solve this as @yawaramin suggests: take the values, sign them, transmit the signature. Then on the Rust side, parse the JSON and verify.

That said, Yojson should in general output object keys in the same order they are specified in the list that is passed to Assoc.

If indeed you need to make sure that the structure is unchanged, JSON is the wrong format because there is no canonical representation of data in JSON. If you want that, other formats are better options like maybe ProtoBuf.

12

Makes sense, not to mention floating point numbers, etc.

On the other hand, I could make the OCaml logic available as a service through a Unix domain socket…

I’ve been thinking of wrapping it in a Debian package, whereby I’d create a new group and user as well as promoting it to keeper of the keys.

Thoughts on this?

13

Signature verification logic in Unix socket works like a charm.

14

I’m not an expert in json at all, but I’ve come across RFC 8785: JSON Canonicalization Scheme (JCS) (plus all the JWK/JWS work RFC 7515: JSON Web Signature (JWS) which seem to have some canonicalization embedded as well).

I don’t know whether there is an implementation of canonicalization of JSON in OCaml, for JWS there’s GitHub - ulrikstrid/ocaml-jose which may be a good starting point?

2 Likes
15

Hmm. Yeah, you’re right, I wasn’t aware of it. However it is outside the JSON spec (RFC 8259) itself itself and only describes a way to do that by encoding it in a special way (a bit like @yawaramin was suggesting, but valid JSON and backed by an RFC so I’d say it’s a nicer way).

If someone wants to contribute a printer that outputs JSON in a JCS-compliant manner to Yojson I’d most definitely be interested in including such a feature! It shouldn’t be too hard, the main part is sorting and deduplicating keys in objects and printing in the exact way RFC 8785 specifies.

3 Likes
16

That would definitely be a useful addition, though I’ll admit this is well above my OCaml pay grade.

In my case, I sidestepped the whole issue by wrapping the logic behind a Unix domain socket and signing (and verifying) the IR output from there.

Still, if someone does contribute a JCS-compliant printer to Yojson, I’ll totally include it!!!