Benchmark based regression testing?

Is there a nice library where I can time a piece of code and then create a regression test with it that alerts if that piece of code gets slower (or faster)?

Would be an amazing thing to hook into a CI test.

I don’t know about this kind of library but it would be great to have one.
Do you have any examples of this kind of libraries in other languages/infrastructures?

core_bench might have what you’re looking for.

Hmm, actually, some cool guy told me once that the reason Chrome is so fast is because they have regression tests like this that prevent you from checking code in that makes stuff slower.

Never found confirmation though but it certainly sounds plausible. Uh, now that I’m saying this out loud I realize it wouldn’t actually be hard to look up since Chrome is an open source project?

EDIT: might have been talking about this thing they call Perf Trybots which benchmarks both the base of your patch and tip against a collection of devices and tells you what the performance impact of the patch is. Not quite what I was talking about but perhaps a more sensible approach.

You may also be interested by lucene’s approach. I like the fact that label the graph to explain some of the changes.

1 Like