Text-mode config-file format (like JSON but better) but supporting raw strings?

Ecosystem
1

[I’ll leave this up, but after some thinking I realized I could just -very rapidly- write an alternative parser for s-expressions (that supported raw-strings) and solve my problem. It worked, and I’m moving along with writing test-cases (which was the original problem).]

Does anybody know of a file-format that meets this criterion ( (1) raw-strings) ? I know that there’s (of course) YAML, but what I didn’t mention in the title is that it also needs to support (2) recursive data-structures.

(a) JSON supports (2), but not (1)

(b) YAML supports (1) but not (not really) (2). And it’s ugggggly besides.

(c) s-expressions are like JSON.

and I can’t think of anything else.

Unstated is that it’d be nice if there were a PPX deriver to de/serialize the format from/to OCaml types, but really, that’s the easy part. Anybody got any pointers?

Why do I want it?

Imagine you want to put the type harness_t on a text-file, so that you can edit it. The field groupfile is a pair of a filename and its contents. You want those contents to be easily-editable, and you don’t want to be futzing around with escaping all the time. Meanwhile, the field attributes is an “environment” of a sort, mapping variable-names to values. So we need both raw-strings (for groupfile) and hierarchical data (for attributes).

module Value = struct
type t =
  STRING of string
| BOOL of bool
| DICT of (string * t) list
| LIST of t list
| NULL
[@@deriving show,yojson]
end

module Environ = struct
type frame_t = (string * Value.t) list
[@@deriving show,yojson]
type t = frame_t list
[@@deriving show,yojson]
end

type harness_t =
  {
    name : string
  ; template_s : string
  ; attributes : Environ.frame_t
  ; groupfile : (string * string) option
  }
[@@deriving show,yojson]

P.S. what’s it for? I’m implementing an OCaml version of StringTemplate4 (a Java thingie), and I want to be able to define a test-case in this format, and -emit- the Java version of the test-case (as well as drive the OCaml version), then compare the results of the two. And of course, I want the source to be in OCaml, with Java code only transiently existing during test-runs.

P.P.S. obvs. I can implement such a format. Indeed, I have two available as part of pa_ppx: JSON and sexp, both with locations, and both with PPX derivers to/from OCaml data-types. The parsers today don’t support OCaml raw-strings, but that’s a minor change. So failing anything else, I’ll do that. But I figured I’d ask, b/c maybe I was unaware of some file-format that meets the spec.

2

I haven’t had my first coffee of the day yet, but I can’t see why YAML can’t handle the attributes. A list of dicts of dicts isn’t anything unusual. Of course the indentation can get quite deep with a complex enough structure, but any human-readable format struggles in some way as you get more complex.

But - do you really need a config format at all? If this is just for writing test-cases you could just write the data-structures as ocaml. I should think anyone wanting to tweak/add test-cases would be able to cope even without knowing any ocaml (but they’ve presumably managed to install it and compile something).

3

These are good questions.

(1) “do you need recursive data-structures”?

I think I do, and don’t want to limit myself there.

(2) “YAML is ugly, but really, anything would be”

I’ve started down the road with JSON, and oof, it’s ugly to encode constructor data-types in it. Ugggggly. So now I’m upgrading my SEXP parser so I can use that instead. I do know that s-expressions are pretty nice for recursive data – after all, it’s where we all started back when we were in college, eh?

(3) “do you really need a text test-file format? Can’t you just write OCaml test-cases?”

I think that textual test-cases are valuable, and I’d point at a different use of them in the ANTLR corpus (which is what I’m porting to OCaml).

Here’s a somewhat involved test: tests/fixtures/custom-descriptors/antlr4/STG1.txt

That’s got five different test-cases.

Here’s a directory (of directories) of similar test-cases: antlr-ocaml/tests/fixtures/descriptors at master · chetmurthy/antlr-ocaml · GitHub

I admit that before I started working on this project, I also was wont to just write OCaml test-cases. But it’s pretty nice to have a text-file format and just write 'em out.

4

I thought I should add that I’m not a big fan of YAML, and that’s also what drives me away from it. Specifically, I found (past tense) YAML’s equivalent of ‘raw strings’ to be … unpleasant. So much so that I designed a modified YAML format I called “YAY” (Yay Ain’t YAML) to fix the problems I saw, wrote an OCaml implementation, etc. Of course, the idea of getting the world to care about something like that is … ludicrous, so I didn’t try to do that. Just wanted prove to myself that something YAML-like could be made less … awful.

5

Have you considered KDL?

There are OCaml binding for it - ocaml-kdl. While I’ve used them only once, I was quite happy with the experience.

6

Did you take a look on [ANN] bcfg, a Boringly Simple ConFiGuration format?

7

I’d bite the bullet and use lua bindings. Lua is half config half programming language already, it’s popular, it has good dictionary and raw string support, and it’ll give you sharing and recursion in a robust way.

8

grin too late, SEXP+raw-strings FTW! Also though, in addition to a config-file format, I’d need the type-deriver for de/serialization. Since I already had it for sexp, I could just add the raw-strings (a few minutes’ coding) and boom, all done, back to writing tests.

 ("TestCoreBasics-testMap"
  ((classname hello)
   (template_s {|<test(name)>|})
   (attributes ((name (MV ((STRING Ter) (STRING Tom) (STRING Sumana))))))
   (groupfile (("a.stg" 
                {| 
   inc(x) ::= "[<x>]"
   test(name) ::= "hi <name:inc()>!"
 |})))
   (expected {bar|hi [Ter][Tom][Sumana]!|bar}))
  )
 ("TestCoreBasics-testAttrIsList"
  ((classname hello)
   (template_s {|hi <name>!|})
   (attributes ((name (MV ((LIST ((STRING Ter) (STRING Tom))) (STRING Sumana))))))
   (groupfile ())
   (expected {bar|hi TerTomSumana!|bar})))

The thing I like about having a test-runner -tool- is that for the most part, I’m not writing OCaml code. I’m focusing on the tests (written in Java) and extracting the relevant strings and copying them into this test-format definition. So there’s a clean separation between “how to run a test” and “how to specify what needs testing”.