THRIFT-6211: Add native Go fuzz targets that run their seed corpus in CI

Client: go

The Go tree had only go-fuzz style targets, behind the gofuzz build tag and
consumed by the OSS-Fuzz build. Nothing here fuzzed them: "make check" ran
TestFuzz, which calls FuzzTutorial once with three bytes. There was also no
corpus in the tree, so an input that OSS-Fuzz found and we fixed had nowhere
to land as a regression test.

Add native testing.F targets alongside them. Run without -fuzz they replay
their seed corpus as ordinary test cases, so the existing make check gets
regression coverage on both Go versions in the matrix at no extra runtime,
and a committed testdata/fuzz/<Target>/ input keeps a fixed bug fixed.

lib/go/thrift/fuzz_test.go needs no generated code: it drives the read paths
through ReadMessageBegin and Skip, which is what generated struct-read code
amounts to. Twelve targets, covering ground no existing target reaches --
THeaderProtocol (client type detection, transform list, info headers),
TFramedTransport, TSimpleJSONProtocol, a server side dispatch hop through
TMultiplexedProcessor, and ParseTuuid.

lib/go/test/fuzz/fuzz_native_test.go adds native equivalents of the existing
struct read and round trip targets, which do need gen-go. The file carries no
build tag, so one "go test -tags gofuzz" runs both sets and the make target
needs no change.

The go-fuzz targets are untouched; OSS-Fuzz keeps consuming them.

Verified: 18 targets, 60s each, ~19M executions, no failures. Seed corpus
replay is 147 cases in lib/go/thrift and 60 in lib/go/test/fuzz.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Jens GeyerandClaude Opus 5 authored and Jens Geyer committed 2026-09-09 22:41:08 +02:00
1 parent 5c5e93e1eb
commit 2f75006ca8
5 files changed
+487 -11

No files matched your search

+9 -1
View File
@@ -48,6 +48,13 @@ To ensure fuzzing can find issues as soon as possible, we will enable fuzzing su
Currently the only convenient, formally supported build with fuzzing support enabled is via the oss-fuzz workflow. For languages where local fuzzing is practical, documentation is provided alongside the fuzzers. For example, C++ builds libFuzzer binaries directly, while Ruby exposes `make` targets that wrap Ruzzy.
Go additionally has native `testing.F` targets, in `lib/go/thrift/fuzz_test.go` and
`lib/go/test/fuzz/fuzz_native_test.go`. Those need no external tooling, and run without `-fuzz`
they replay their seed corpus as ordinary test cases -- so `make check` gets regression coverage
from them for free, and a fixed input stays fixed once its file under `testdata/fuzz/<Target>/`
is committed. To fuzz one for real, `go test -run '^$' -fuzz <Target> -fuzztime 2m`. See
`lib/go/test/fuzz/README.md`.
## OSS-Fuzz Integration
Our fuzzers run continuously on OSS-Fuzz. To view build status:
@@ -68,7 +75,8 @@ we would still appreciate contributions that:
1. Add new fuzzers for unsupported languages
2. Improve existing fuzzers
3. Add test cases to corpus
3. Add test cases to corpus -- where a language keeps its corpus in the tree, as Go does under
`testdata/fuzz/<Target>/`, an input that once failed belongs there
If you do add or change a fuzzer, please remember to make corresponding changes to the oss-fuzz build script in case they are needed.
+1
View File
@@ -34,4 +34,5 @@ distdir:
EXTRA_DIST = \
fuzz.go \
fuzz_test.go \
fuzz_native_test.go \
go.mod
+51 -10
View File
@@ -1,15 +1,56 @@
# Go fuzzing README
To build the fuzz targets, simply run `make check` in this directory
There are two kinds of fuzz target in the Go tree, and they are built and run differently.
To reproduce a bug, update the code in the fuzz_test.go file, pass the right input buffer in there and update the code to call the relevant function.
## Native targets (`go test -fuzz`)
We currently have the following fuzz targets:
These are ordinary `testing.F` targets. They need no external tooling, and they have a property
worth knowing about: **run without `-fuzz` they replay their seed corpus as normal test cases**,
so they act as regression tests in CI at no cost. `make check` runs them.
* FuzzTutorial -- a fuzzer which spins up an mini server and fuzzes it with random data, following the tutorial example
* FuzzParseCompact -- fuzzes the deserialization of the Compact protocol
* FuzzParseBinary -- fuzzes the deserialization of the Binary protocol
* FuzzParseJson -- fuzzes the deserialization of the JSON protocol
* FuzzRoundtripCompact -- fuzzes the roundtrip of the Compact protocol
* FuzzRoundtripBinary -- fuzzes the roundtrip of the Binary protocol
* FuzzRoundtripJson -- fuzzes the roundtrip of the JSON protocol
They live in two places:
* `lib/go/thrift/fuzz_test.go` — targets that need no generated code, driving the protocol and
transport read paths directly. These run with the library's own tests (`make -C lib/go check`).
* `FuzzReadBinary`, `FuzzReadBinaryNonStrict`, `FuzzReadBinaryStruct`
* `FuzzReadCompact`, `FuzzReadCompactStruct`
* `FuzzReadJSON`, `FuzzReadJSONStruct`, `FuzzReadSimpleJSON`
* `FuzzReadHeader` — client type detection, transform list, info headers
* `FuzzReadFramed` — the frame length taken off the wire
* `FuzzServerDispatch` — a server side hop through `TMultiplexedProcessor`
* `FuzzParseTuuid`
* `fuzz_native_test.go` in this directory — reading and round-tripping a generated struct.
* `FuzzStructReadBinary`, `FuzzStructReadCompact`, `FuzzStructReadJSON`
* `FuzzStructRoundtripBinary`, `FuzzStructRoundtripCompact`, `FuzzStructRoundtripJSON`
To fuzz one for real:
```bash
# in lib/go
go test ./thrift -run '^$' -fuzz FuzzReadHeader -fuzztime 2m
# in lib/go/test/fuzz (run "make check" once first, so gen-go exists)
go test -run '^$' -fuzz FuzzStructRoundtripCompact -fuzztime 2m
```
When a run fails, Go writes the input to `testdata/fuzz/<Target>/` next to the target. **Commit
that file with the fix** — from then on every `go test` replays it.
## go-fuzz targets (OSS-Fuzz)
`fuzz.go` holds the older [go-fuzz](https://github.com/dvyukov/go-fuzz) style targets, behind the
`gofuzz` build tag. These are what the OSS-Fuzz build consumes; `make check` only compiles and
smoke-tests them here.
* `FuzzTutorial` — spins up a mini server and feeds it random data, following the tutorial example
* `FuzzParseBinary`, `FuzzParseCompact`, `FuzzParseJson` — deserialization per protocol
* `FuzzRoundtripBinary`, `FuzzRoundtripCompact`, `FuzzRoundtripJson` — round trip per protocol
To reproduce a bug in one of these, update `fuzz_test.go` to pass the input buffer to the relevant
function.
## Building
Run `make check` in this directory. It generates the code the targets need, then runs both sets.
See `FUZZING.md` at the top of the repository for the wider fuzzing setup.
+169
View File
@@ -0,0 +1,169 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
// Native Go fuzz targets for reading and round-tripping a generated struct.
//
// These carry no build tag, so unlike the go-fuzz targets in fuzz.go they run
// under a plain "go test": each one replays its seed corpus plus anything under
// testdata/fuzz/<Target>/ as an ordinary test case. "make check" runs them
// alongside the go-fuzz build check. To fuzz for real:
//
// go test -run '^$' -fuzz FuzzStructReadBinary -fuzztime 2m
//
// A failing input is written to testdata/fuzz/<Target>/ and should be committed
// with the fix.
//
// Targets that need no generated code live in lib/go/thrift/fuzz_test.go and
// run with the library's own tests.
package fuzz
import (
"context"
"testing"
"github.com/apache/thrift/lib/go/test/fuzz/gen-go/fuzztest"
"github.com/apache/thrift/lib/go/thrift"
)
// nativeFuzzConf keeps a made-up size on the wire from costing memory rather
// than fuzzer time.
func nativeFuzzConf() *thrift.TConfiguration {
return &thrift.TConfiguration{
MaxMessageSize: 1 << 18,
MaxFrameSize: 1 << 18,
}
}
func nativeFuzzSeeds(f *testing.F) {
f.Add([]byte{})
// An empty struct, in binary and in compact.
f.Add([]byte{0x00})
// One i32 field, then stop.
f.Add([]byte{0x08, 0x00, 0x01, 0x00, 0x00, 0x00, 0x2a, 0x00})
// One string field, then stop.
f.Add([]byte{0x0b, 0x00, 0x02, 0x00, 0x00, 0x00, 0x03, 'a', 'b', 'c', 0x00})
// A string field claiming a negative length.
f.Add([]byte{0x0b, 0x00, 0x02, 0xff, 0xff, 0xff, 0xff, 0x00})
// A list claiming a huge element count.
f.Add([]byte{0x0f, 0x00, 0x03, 0x08, 0x7f, 0xff, 0xff, 0xff, 0x00})
// A map with a large count and mismatched types.
f.Add([]byte{0x0d, 0x00, 0x04, 0x0b, 0x0c, 0x7f, 0xff, 0xff, 0xff, 0x00})
// Compact: an i32 field carrying a varint, then stop.
f.Add([]byte{0x15, 0x54, 0x00})
// JSON.
f.Add([]byte(`{"1":{"i32":42}}`))
f.Add([]byte(`{"2":{"str":"abc"}}`))
}
// readStruct reads a generated struct out of data, and reports whether that
// succeeded.
func readStruct(t *testing.T, data []byte, factory thrift.TProtocolFactory) (*fuzztest.FuzzTest, bool) {
t.Helper()
trans := thrift.NewTMemoryBufferLen(len(data))
defer func() {
trans.Close()
trans.Buffer.Reset()
}()
trans.Write(data)
value := fuzztest.NewFuzzTest()
if err := value.Read(context.Background(), factory.GetProtocol(trans)); err != nil {
return nil, false
}
return value, true
}
// roundtrip writes value back out, reads it again, and requires the two to be
// equal.
func roundtrip(t *testing.T, value *fuzztest.FuzzTest, factory thrift.TProtocolFactory) {
t.Helper()
ctx := context.Background()
out := thrift.NewTMemoryBuffer()
if err := value.Write(ctx, factory.GetProtocol(out)); err != nil {
// Not every struct that reads is one this protocol can write back;
// that is the writer's business, not a round-trip failure.
return
}
serialized := out.Bytes()
in := thrift.NewTMemoryBufferLen(len(serialized))
in.Write(serialized)
reread := fuzztest.NewFuzzTest()
if err := reread.Read(ctx, factory.GetProtocol(in)); err != nil {
t.Fatalf("re-reading what we just wrote failed: %v", err)
}
if !value.Equals(reread) {
t.Fatalf("round trip changed the value:\n before: %v\n after: %v", value, reread)
}
}
func FuzzStructReadBinary(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTBinaryProtocolFactoryConf(nativeFuzzConf())
f.Fuzz(func(t *testing.T, data []byte) {
readStruct(t, data, factory)
})
}
func FuzzStructReadCompact(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTCompactProtocolFactoryConf(nativeFuzzConf())
f.Fuzz(func(t *testing.T, data []byte) {
readStruct(t, data, factory)
})
}
func FuzzStructReadJSON(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTJSONProtocolFactory()
f.Fuzz(func(t *testing.T, data []byte) {
readStruct(t, data, factory)
})
}
func FuzzStructRoundtripBinary(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTBinaryProtocolFactoryConf(nativeFuzzConf())
f.Fuzz(func(t *testing.T, data []byte) {
if value, ok := readStruct(t, data, factory); ok {
roundtrip(t, value, factory)
}
})
}
func FuzzStructRoundtripCompact(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTCompactProtocolFactoryConf(nativeFuzzConf())
f.Fuzz(func(t *testing.T, data []byte) {
if value, ok := readStruct(t, data, factory); ok {
roundtrip(t, value, factory)
}
})
}
func FuzzStructRoundtripJSON(f *testing.F) {
nativeFuzzSeeds(f)
factory := thrift.NewTJSONProtocolFactory()
f.Fuzz(func(t *testing.T, data []byte) {
if value, ok := readStruct(t, data, factory); ok {
roundtrip(t, value, factory)
}
})
}
+257
View File
@@ -0,0 +1,257 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
// Native Go fuzz targets for the protocol and transport read paths.
//
// These need no generated code: a reader that calls ReadMessageBegin and then
// Skip walks the same library code that generated struct-read code does, since
// Skip dispatches on the wire-supplied type and reads containers at
// wire-supplied sizes. The targets in lib/go/test/fuzz complement these by
// reading an actual generated struct.
//
// Run as ordinary tests -- go test ./thrift, which is what "make check" does --
// they replay the seed corpus below plus anything under
// testdata/fuzz/<Target>/, so a fixed input stays fixed. To fuzz for real:
//
// go test ./thrift -run '^$' -fuzz FuzzReadBinary -fuzztime 2m
//
// A failing input is written to testdata/fuzz/<Target>/ and should be committed
// with the fix.
package thrift
import (
"context"
"testing"
)
// fuzzConf is what a server gets by default, with the ceilings pulled down so
// that a made-up size costs the fuzzer time rather than memory.
func fuzzConf() *TConfiguration {
return &TConfiguration{
MaxMessageSize: 1 << 18,
MaxFrameSize: 1 << 18,
}
}
func fuzzBuffer(data []byte) *TMemoryBuffer {
trans := NewTMemoryBufferLen(len(data))
trans.Write(data)
return trans
}
// fuzzReadMessage reads an enveloped message and walks its payload.
func fuzzReadMessage(p TProtocol) {
ctx := context.Background()
if _, _, _, err := p.ReadMessageBegin(ctx); err != nil {
return
}
if err := SkipDefaultDepth(ctx, p, STRUCT); err != nil {
return
}
p.ReadMessageEnd(ctx)
}
// fuzzReadStruct reads a bare struct, the way TDeserializer does.
func fuzzReadStruct(p TProtocol) {
SkipDefaultDepth(context.Background(), p, STRUCT)
}
// fuzzSeeds covers the shapes worth starting from: nothing, a well formed call
// in each dialect, and the sizes and types that have to be rejected rather than
// believed.
func fuzzSeeds(f *testing.F) {
f.Add([]byte{})
// Strict binary "ping" call with an empty argument struct.
f.Add([]byte{0x80, 0x01, 0x00, 0x01, 0x00, 0x00, 0x00, 0x04, 'p', 'i', 'n', 'g', 0x00, 0x00, 0x00, 0x00, 0x00})
// Non-strict binary: bare name length, no version word.
f.Add([]byte{0x00, 0x00, 0x00, 0x04, 'p', 'i', 'n', 'g', 0x01, 0x00, 0x00, 0x00, 0x00, 0x00})
// Compact "ping" call.
f.Add([]byte{0x82, 0x21, 0x01, 0x04, 'p', 'i', 'n', 'g', 0x00})
// JSON and simple-JSON envelopes.
f.Add([]byte(`[1,"ping",1,0,{}]`))
f.Add([]byte(`[1,"ping",1,0,{"1":{"str":"x"}}]`))
// A header frame: frame size, header magic, flags, sequence id, header
// length, then no transforms and no info headers.
f.Add([]byte{0x00, 0x00, 0x00, 0x0e, 0x0f, 0xff, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00})
// A string field claiming a negative length.
f.Add([]byte{0x0b, 0x00, 0x01, 0xff, 0xff, 0xff, 0xff, 0x00})
// A string field claiming more bytes than the message can hold.
f.Add([]byte{0x0b, 0x00, 0x01, 0x7f, 0xff, 0xff, 0xff, 0x00})
// A list header claiming a huge element count.
f.Add([]byte{0x0f, 0x00, 0x01, 0x08, 0x7f, 0xff, 0xff, 0xff, 0x00})
// A map header with mismatched key/value types and a large count.
f.Add([]byte{0x0d, 0x00, 0x01, 0x0b, 0x0c, 0x7f, 0xff, 0xff, 0xff, 0x00})
// Nested structs, to walk into the depth limit.
f.Add([]byte{0x0c, 0x00, 0x01, 0x0c, 0x00, 0x01, 0x0c, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00})
// A frame header on its own, with nothing behind it.
f.Add([]byte{0x7f, 0xff, 0xff, 0xff})
}
func FuzzReadBinary(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadMessage(NewTBinaryProtocolConf(fuzzBuffer(data), fuzzConf()))
})
}
func FuzzReadBinaryNonStrict(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
conf := fuzzConf()
strict := false
conf.TBinaryStrictRead = &strict
fuzzReadMessage(NewTBinaryProtocolConf(fuzzBuffer(data), conf))
})
}
func FuzzReadBinaryStruct(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadStruct(NewTBinaryProtocolConf(fuzzBuffer(data), fuzzConf()))
})
}
func FuzzReadCompact(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadMessage(NewTCompactProtocolConf(fuzzBuffer(data), fuzzConf()))
})
}
func FuzzReadCompactStruct(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadStruct(NewTCompactProtocolConf(fuzzBuffer(data), fuzzConf()))
})
}
func FuzzReadJSON(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadMessage(NewTJSONProtocol(fuzzBuffer(data)))
})
}
func FuzzReadJSONStruct(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadStruct(NewTJSONProtocol(fuzzBuffer(data)))
})
}
func FuzzReadSimpleJSON(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadMessage(NewTSimpleJSONProtocol(fuzzBuffer(data)))
})
}
// FuzzReadHeader covers THeaderProtocol, which decides the client type from the
// first bytes and, for a header frame, parses the transform list, the info
// headers and the padding before any payload is read.
func FuzzReadHeader(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
fuzzReadMessage(NewTHeaderProtocolConf(fuzzBuffer(data), fuzzConf()))
})
}
// FuzzReadFramed covers the frame length taken off the wire by
// TFramedTransport, underneath a protocol that then reads the frame.
func FuzzReadFramed(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
trans := NewTFramedTransportConf(fuzzBuffer(data), fuzzConf())
fuzzReadMessage(NewTBinaryProtocolConf(trans, fuzzConf()))
})
}
// FuzzServerDispatch covers the server side hop: client type detection, then a
// processor that dispatches on the peer-supplied method name, and the reply
// written back for a name it does not know.
func FuzzServerDispatch(f *testing.F) {
fuzzSeeds(f)
f.Fuzz(func(t *testing.T, data []byte) {
in := NewTHeaderProtocolConf(fuzzBuffer(data), fuzzConf())
out := NewTHeaderProtocolConf(NewTMemoryBuffer(), fuzzConf())
processor := NewTMultiplexedProcessor()
processor.RegisterProcessor("svc", &fuzzProcessor{})
processor.RegisterDefault(&fuzzProcessor{})
// A single buffer can hold more than one message; stop at the first
// one the processor rejects.
for range 4 {
if ok, err := processor.Process(context.Background(), in, out); err != nil || !ok {
return
}
}
})
}
// fuzzProcessor stands in for generated dispatch code: read the arguments,
// write an empty reply.
type fuzzProcessor struct {
funcs map[string]TProcessorFunction
}
func (p *fuzzProcessor) ProcessorMap() map[string]TProcessorFunction {
return p.funcs
}
func (p *fuzzProcessor) AddToProcessorMap(name string, fn TProcessorFunction) {
if p.funcs == nil {
p.funcs = make(map[string]TProcessorFunction)
}
p.funcs[name] = fn
}
func (p *fuzzProcessor) Process(ctx context.Context, in, out TProtocol) (bool, TException) {
name, _, seqID, err := in.ReadMessageBegin(ctx)
if err != nil {
return false, NewTProtocolException(err)
}
if err := SkipDefaultDepth(ctx, in, STRUCT); err != nil {
in.ReadMessageEnd(ctx)
return false, NewTProtocolException(err)
}
if err := in.ReadMessageEnd(ctx); err != nil {
return false, NewTProtocolException(err)
}
if err := out.WriteMessageBegin(ctx, name, REPLY, seqID); err != nil {
return false, NewTProtocolException(err)
}
out.WriteStructBegin(ctx, "result")
out.WriteFieldStop(ctx)
out.WriteStructEnd(ctx)
out.WriteMessageEnd(ctx)
out.Flush(ctx)
return true, nil
}
// FuzzParseTuuid covers the uuid text parser, which TSimpleJSONProtocol feeds
// straight from the wire.
func FuzzParseTuuid(f *testing.F) {
f.Add("6ba7b810-9dad-11d1-80b4-00c04fd430c8")
f.Add("")
f.Add("------------------------------------")
f.Add("6ba7b810-9dad-11d1-80b4-00c04fd430")
f.Fuzz(func(t *testing.T, s string) {
ParseTuuid(s)
})
}