Coincidentally after my Go Fuzz post yesterday, today I see:
Why Did the OpenSSL Punycode Vulnerability Happen
Buffer overruns aren't a problem with Go (unless you're using unsafe) but the lesson still applies. I should spend more time writing fuzz tests. Especially when the Go tools make it easy.
Go 1.18 included a new fuzz testing facility. (overshadowed by generics.) I haven't used it a lot but it has been helpful in a few cases. For example, I used it to test gSuneido's regular expression code and found a number of obscure bugs.
A few days ago, one of our customers got a runtime bounds error from parsing a date. This code has been running in production for years without seeing this error so presumably it was something obscure. The date parsing code is a little complicated because dates can be written many different ways. The error log didn't have the bad input, but it did have a call stack, so it probably wouldn't have been too hard to find the issue manually. But I was curious whether fuzzing would find it. And if there was one bug, maybe there were more.
| func FuzzParseDate(f *testing.F) { | |
| f.Fuzz(func(t *testing.T, s string) { | |
| ParseDate(s, "yMd") | |
| }) | |
| } |
To keep it simple, I didn't check for correct results, so the test was just looking for panics.
One of the things that should be simple, but I found confusing, was how to actually run fuzz tests. I'm not sure if it's the simplest, but this seems to work:
go test -fuzz=FuzzParseDate -run=FuzzParseDate
Within a few seconds the fuzz test found an input that would cause a panic.
"0A0A0A0A0A0A0A0A0A00000000"
That's probably not what the customer entered, but it was the same bug, and easily fixed once I could recreate it.
I let it run until it was no longer finding "interesting" inputs which took a few minutes. It didn't find any more bugs.
Ideally the test would be checking for correct output, not just lack of panics. But that requires a second implementation, which I don't have. Sometimes you can round-trip e.g. encode/decode but that wouldn't work in this case.