This is a comparison of different binary and text encodings.
We compare the codecs provided by github.com/ugorji/go/codec package, against other libraries:
github.com/ugorji/go/codec provides:
- msgpack: [http://github.com/msgpack/msgpack]
- binc: [http://github.com/ugorji/binc]
- cbor: [http://cbor.io] [http://tools.ietf.org/html/rfc7049]
- simple:
- json: [http://json.org] [http://tools.ietf.org/html/rfc7159]
Other codecs compared include:
- github.com/vmihailenco/msgpack/v4
- github.com/globalsign/mgo/bson
- go.mongodb.org/mongo-driver/bson
- github.com/davecgh/go-xdr/xdr2
- github.com/Sereal/Sereal/Go/sereal
- bitbucket.org/bodhisnarkva/cbor/go
- github.com/tinylib/msgp
- github.com/pquerna/ffjson/ffjson
- github.com/json-iterator/go
- github.com/mailru/easyjson
- github.com/fxamacker/cbor/v2
The data being serialized is a TestStruc randomly generated values.
See values_test.go for the
definition of the TestStruc.
See bench.sh for how to download the external libraries which we benchmark against, generate the files for the types when needed, and run the suite of tests.
The 3 suite of benchmarks are
- CodecSuite
- XSuite
- CodecXSuite
# Note that `bench.sh` may be in the codec sub-directory, and should be run from there.
# download the code and all its dependencies
./bench.sh -d
# code-generate files needed for benchmarks against ffjson, easyjson, msgp, etc
./bench.sh -c
# run the full suite of tests (not including external formats)
./bench.sh -s
# Below, see how to just run some specific suite of tests, knowing the right tags and flags ...
# See bench.sh for different iterations
# Run suite of tests in default mode (selectively using unsafe in specific areas)
go test -tags "alltests x" -bench "CodecXSuite" -benchmem
# Run suite of tests in safe mode (no usage of unsafe)
go test -tags "alltests x codec.safe" -bench "CodecXSuite" -benchmem
# Run suite of tests in codecgen mode, including all tests which are generated (msgp, ffjson, etc)
go test -tags "alltests x generated" -bench "CodecXGenSuite" -benchmem
The following issues are seen currently (11/20/2014):
- code.google.com/p/cbor/go fails on encoding and decoding the test struct
- github.com/davecgh/go-xdr/xdr2 fails on encoding and decoding the test struct
- github.com/Sereal/Sereal/Go/sereal fails on decoding the serialized test struct
Please see the benchmarking blog post for detailed representative results.
A snapshot of some results on my Core-i5 2018 Dell Inspiron laptop is below.
Note: errors are truncated, and lines re-arranged, for readability.
What you should notice:
- Results get better with codecgen, resulting in about 50% performance improvement. Users should carefully weigh the performance improvements against the usability and binary-size increases, as performance is already extremely good without the codecgen path.
See bench.out.txt for representative result from running bench.sh as below, as of 2020-11-11.
./bench.sh -z > bench.out.txtsnippet of benchmark output, running without codecgen (2021-02-04)
note that the first 5 are from codes (and the following are from other libraries (as named).
BenchmarkCodecXSuite/options-false.../
Benchmark__Msgpack____Encode-8 17764 67504 ns/op 24 B/op 1 allocs/op
Benchmark__Binc_______Encode-8 16442 72011 ns/op 24 B/op 1 allocs/op
Benchmark__Simple_____Encode-8 17060 69965 ns/op 24 B/op 1 allocs/op
Benchmark__Cbor_______Encode-8 16450 70783 ns/op 24 B/op 1 allocs/op
Benchmark__Json_______Encode-8 7303 145232 ns/op 24 B/op 1 allocs/op
Benchmark__Std_Json___Encode-8 5308 229397 ns/op 79216 B/op 556 allocs/op
Benchmark__Gob________Encode-8 7278 165572 ns/op 174169 B/op 683 allocs/op
Benchmark__JsonIter___Encode-8 7950 154060 ns/op 54437 B/op 80 allocs/op
Benchmark__Bson_______Encode-8 3780 305520 ns/op 242263 B/op 1181 allocs/op
Benchmark__Mgobson____Encode-8 3040 397415 ns/op 300345 B/op 1877 allocs/op
Benchmark__VMsgpack___Encode-8 5572 218910 ns/op 164607 B/op 360 allocs/op
Benchmark__Fxcbor_____Encode-8 10000 114321 ns/op 51947 B/op 400 allocs/op
Benchmark__Sereal_____Encode-8 3063 350328 ns/op 277718 B/op 3388 allocs/op
Benchmark__Msgpack____Decode-8 8372 144322 ns/op 37412 B/op 315 allocs/op
Benchmark__Binc_______Decode-8 8487 141145 ns/op 37413 B/op 315 allocs/op
Benchmark__Simple_____Decode-8 8542 139446 ns/op 37413 B/op 315 allocs/op
Benchmark__Cbor_______Decode-8 7989 154240 ns/op 37414 B/op 315 allocs/op
Benchmark__Json_______Decode-8 2880 416769 ns/op 71880 B/op 592 allocs/op
Benchmark__Std_Json___Decode-8 1207 988487 ns/op 137146 B/op 3113 allocs/op
Benchmark__Gob________Decode-8 4083 300238 ns/op 159394 B/op 2329 allocs/op
Benchmark__JsonIter___Decode-8 2607 476016 ns/op 132690 B/op 2654 allocs/op
Benchmark__Bson_______Decode-8 2418 508113 ns/op 190987 B/op 4544 allocs/op
Benchmark__Mgobson____Decode-8 2217 543875 ns/op 168741 B/op 6776 allocs/op
Benchmark__VMsgpack___Decode-8 3133 391164 ns/op 100126 B/op 2030 allocs/op
Benchmark__Fxcbor_____Decode-8 4645 251379 ns/op 72515 B/op 1395 allocs/op
- snippet of bench.out.txt, running with codecgen (2021-02-04) *
BenchmarkCodecXGenSuite/options-false.../
Benchmark__Msgpack____Encode-8 26810 45254 ns/op 24 B/op 1 allocs/op
Benchmark__Binc_______Encode-8 25396 48931 ns/op 24 B/op 1 allocs/op
Benchmark__Simple_____Encode-8 28516 46036 ns/op 24 B/op 1 allocs/op
Benchmark__Cbor_______Encode-8 26923 44930 ns/op 24 B/op 1 allocs/op
Benchmark__Json_______Encode-8 9980 124222 ns/op 24 B/op 1 allocs/op
Benchmark__Msgp_______Encode-8 36180 33802 ns/op 0 B/op 0 allocs/op
Benchmark__Easyjson___Encode-8 10000 114262 ns/op 50690 B/op 12 allocs/op
Benchmark__Ffjson_____Encode-8 4051 299364 ns/op 128399 B/op 1121 allocs/op
Benchmark__Msgpack____Decode-8 13057 94631 ns/op 35909 B/op 308 allocs/op
Benchmark__Binc_______Decode-8 12678 91177 ns/op 35909 B/op 308 allocs/op
Benchmark__Simple_____Decode-8 13144 90958 ns/op 35910 B/op 308 allocs/op
Benchmark__Cbor_______Decode-8 10000 102062 ns/op 35911 B/op 308 allocs/op
Benchmark__Json_______Decode-8 3019 365442 ns/op 70469 B/op 589 allocs/op
Benchmark__Msgp_______Decode-8 14877 79750 ns/op 68788 B/op 969 allocs/op
Benchmark__Easyjson___Decode-8 3150 389608 ns/op 70531 B/op 475 allocs/op
Benchmark__Ffjson_____Decode-8 2758 435859 ns/op 95178 B/op 1290 allocs/op