By Ugorji Nwoke   Thu, 18 Dec 2014 08:00:00 -0700   /blog   technology go-codec

Code Generation using go-codec - for 2-20X performance improvement

View articles in the go-codec series, source at http://github.com/ugorji/go

go-codec supports compile-time generation of encoders and decoders for named types, which does not incur the overhead of reflection in the typical case, giving 2X-20X performance improvement over the idiomatic runtime introspection mode.

Idiomatic encoding and decoding types within go typically relies on the reflection capabilities of the go runtime. This affords flexible performance without the need for a pre-compilation step; the go types contain all the information needed and the runtime exposes the full types via reflection. However, introspecting the runtime to get this information has a noticeable overhead, which can be eliminated by a pre-compilation/code-generation step.

To eliminate that overhead, a pre-compilation step must be done to create the code which would have been inferred at runtime. This is why Protocol Buffers, Avro, etc have better performance than runtime-based systems. go-codec now provides the same capabilities, with the accompanying 2X-20X performance improvement depending on the size and structure of the named type.

Show me some performance numbers. Whet my appetite …

Let us start with some benchmark numbers to whet your appetite.

Encoding - Runtime

Benchmark__Msgpack____Encode	   10000	     66858 ns/op	    8816 B/op	      58 allocs/op
Benchmark__Binc_NoSym_Encode	   10000	     66061 ns/op	    8848 B/op	      58 allocs/op
Benchmark__Cbor_______Encode	   10000	     66168 ns/op	    8816 B/op	      58 allocs/op
Benchmark__Json_______Encode	   10000	     86525 ns/op	    8896 B/op	      58 allocs/op

Benchmark__Std_Json___Encode	   10000	     91267 ns/op	   14136 B/op	     123 allocs/op
Benchmark__Gob________Encode	    5000	    139903 ns/op	   10162 B/op	     222 allocs/op
Benchmark__Bson_______Encode	    5000	    164735 ns/op	   31120 B/op	     728 allocs/op

Encoding - CodeGen

Benchmark__Msgpack____Encode	   50000	     18374 ns/op	     224 B/op	       3 allocs/op
Benchmark__Binc_NoSym_Encode	   50000	     18342 ns/op	     256 B/op	       3 allocs/op
Benchmark__Cbor_______Encode	   50000	     16696 ns/op	     224 B/op	       3 allocs/op
Benchmark__Json_______Encode	   20000	     33873 ns/op	     304 B/op	       3 allocs/op

Decoding - Runtime

Benchmark__Msgpack____Decode	   10000	     87545 ns/op	    9616 B/op	     253 allocs/op
Benchmark__Binc_NoSym_Decode	   10000	     92794 ns/op	    9648 B/op	     253 allocs/op
Benchmark__Cbor_______Decode	   10000	     88502 ns/op	    9616 B/op	     253 allocs/op
Benchmark__Json_______Decode	    5000	    152253 ns/op	   11648 B/op	     330 allocs/op

Benchmark__Std_Json___Decode	    2000	    288132 ns/op	   13784 B/op	     493 allocs/op
Benchmark__Gob________Decode	    2000	    420523 ns/op	   70171 B/op	    1745 allocs/op
Benchmark__Bson_______Decode	    5000	    177946 ns/op	   15928 B/op	    1046 allocs/op

Decoding - CodeGen

Benchmark__Msgpack____Decode	   20000	     42610 ns/op	    6808 B/op	     134 allocs/op
Benchmark__Binc_NoSym_Decode	   20000	     45270 ns/op	    6840 B/op	     134 allocs/op
Benchmark__Cbor_______Decode	   20000	     44329 ns/op	    6792 B/op	     134 allocs/op
Benchmark__Json_______Decode	    5000	    103326 ns/op	    8680 B/op	     210 allocs/op

The table below compares encode using runtime support only against a baseline of code generation.

Time Memory Allocations
Msgpack 3.6 X 39 X 19 X
Binc 3.6 X 38 X 19 X
Cbor 4.0 X 39 X 19 X
Json 2.6 X 29 X 19 X

The table below compares decode using runtime support only against a baseline of code generation.

Time Memory Allocations
Msgpack 2.0 X 1.4 X 1.9 X
Binc 2.1 X 1.4 X 1.9 X
Cbor 2.0 X 1.4 X 1.9 X
Json 1.5 X 1.3 X 1.6 X

There is very clear benefit to code generation. Code generation gives you better performance in clock time, cpu time and memory usage/allocations. The benefits are more pronounced during encoding than during decoding.

Wow. Is this performance gap because of reflection??? Is it that slow???

I call sheninegens! reflection in go is not slow. In fact, interfaces/type-switch/etc use the same runtime introspection mechanism under the hood that reflection does.

Let me explain. In go, reflection is a thin layer of runtime introspection support. There is a small computational cost to compute or expose requested information about the types already known to the runtime, or to create new values and return a wrapper (reflect.Value) around them.

reflection is an intrinsic part of the go runtime, and used in fundamental packages like fmt.

In the current implementation, the main cost of reflection is in allocation. My guess is that faster allocation expected in the go 1.5 timeframe should make this better.

OK. How does codecgen work?

codecgen works off a single interface.

type Selfer interface {
	CodecEncodeSelf(*Encoder)
	CodecDecodeSelf(*Decoder)
}

When encoding or decoding a type, if it implements the codec.Selfer interface above, then it will handle its own encoding and decoding. The Encoder/Decoder checks this before extension support or if the type also implements encoding.(Text|Binary)(M|Unm)arshaler interfaces.

NOTE: the Canonical option is ignored (not supported AT THIS TIME). If you need Canonical support (e.g. for cbor), then do not use codecgen.

codecgen uses this knowledge to generate type-safe code which does exactly what the regular runtime introspection code does at run-time. It is an amazing feat.

With codecgen, the full feature-set of codec is still supported, including:

  1. Indefinite-length formats
  2. Anonymous (embedded fields)
  3. All field types supported (including interfaces)
  4. Standard field renaming via tags
  5. Omit Empty fields, configured in Handle or in struct tags
  6. Struct To Array, configured in Handle or in struct tags

Exciting!!! Tell me more about how it works.

codecgen builds fully atop the go-codec package. We needed it to work exactly as the runtime introspection works, so we can leverage all the IP built into the package already.

go-codec at runtime will parse each type needed and create an in-memory structure specifying all important information about the type. codecgen uses all that information and replicates the runtime logic exactly.

codecgen runs in multiple phases:

  1. Parse the input go files using go/ast package
  2. For each named type in there tha matches the regex passed,
    1. Add the type to the list of named types that need a codec.Selfer implementation
  3. Create a transient file
    1. This file calls codec.Gen(...) function, passing in all the types gathered
  4. Run the transient file, using go run -tags=XYZ transient-file.go
    1. This will create the generated file with codec.Selfer implementations
  5. Delete the transient file

The transient file looks like this (*error handling removed for conciseness):

	fout, err := os.Create("values_codecgen_generated_test.go")
	var out bytes.Buffer
	var typs []reflect.Type 
	var t0 codec.AnonInTestStruc
	typs = append(typs, reflect.TypeOf(t0))
	var t1 codec.AnonInTestStrucIntf
	typs = append(typs, reflect.TypeOf(t1))
    // <snip>
	codec.Gen(&out, "codecgen", "codec", false, typs...)
	bout, err := format.Source(out.Bytes())
	fout.Write(bout)

The generated file looks like this (details elided):

func (x *MyType) CodecEncodeSelf(e *Encoder) {
}
func (x *MyType) CodecDecodeSelf(e *Decoder) {
}

Awesome!!!! How do I use it?

Using codecgen is very straightforward.

Download and install the tool

go get -u github.com/ugorji/go/codec/codecgen

Run the tool on your files

The command line format is:

codecgen [options] (-o outfile) (infile ...)

% codecgen -?
Usage of codecgen:
  -c string
    	codec path (default "ugorji.net/codec")
  -d int
    	random identifier for use in generated code
  -o string
    	out file
  -r string
    	regex for type name to match (default ".*")
  -rt string
    	tags for go run
  -st string
    	struct tag keys to introspect (default "codec,json")
  -t string
    	build tag to put in file
  -u	Use unsafe, e.g. to avoid unnecessary allocation on []byte->string
  -x	keep temp file

% codecgen -o values_codecgen.go values.go values2.go moretypedefs.go

That is it

Please explain the options better

Option Description
-o codecgen will generate a single output file.
-u When decoding a struct, we read the encoded field name, and use a type switch to match it to the right field. Copying the string is a performance hit. We can use unsafe to wrap the []byte as a string and consequently bypass the []byte->string allocation.
-c If you have used vendored the codec package into a different place, use this option to specify a different package path for the codec package. Most users do not need this.
-t Users may want to only use the code generated file when specific build tags are specified. You can pass some tags and the generated file will have them.
-st Users can customize the struct tags keys to introspect
-rt codecgen runs by creating a temporary file, and then using go run to execute it. If the file that you are generating values against needs a build tag, specify it to the codecgen tool.
-x This is a debugging switch to not delete the transient file which must be passed to go run.
-d Specify the random integer used during codecgen. This helps reduce churn in generated output, etc.

Can I use this with ‘go generate’?

Yes.

codecgen can be used easily with go generate.

The easiest way is to create a file, add the generate tag to it, and call codecgen in it. A sample file looks like this:

//+build generate

package mypackage
//go:generate codecgen -o values.generated.go file1.go file2.go file3.go

Run go generate in the directory containing the file.

How do I know if I have to re-generate the code?

go-codec updates an internal version each time an incompatible change occurs to the library.

Within an init function, we check that the generated code matches the current supporting library. If the check fails, we panic in the init so that the application never starts until the user updates.

The error message looks like: codecgen version mismatch: current: 1, need 2. Re-generate file: /home/ugorji/depot/repo/src/ugorji.net/codec/values_codecgen_generated_test.go

If you get a similar panic message, please use an old library or regenerate your file.

What about other code-generation libraries

There are a few other code-generation libraries created for specific formats. They had issues which I will list below:

msgp https://github.com/philhofer/msgp/

  • This was the best performing of the bunch
  • It supports messagepack only
  • It lacks features e.g. struct renaming via tags, anonymous field support
  • It has limited support for interfaces.
    It only expects the following as interfaces:
    number, bool, string, []byte, map[string]string, map[string]interface{}
  • It expects every field in the struct which is a named type to also have code generated for it. codec falls back to runtime introspection in this case.
  • It requires that users annotate which structs are extensions in the source code.
    This means that types which are not under the developer control (e.g. third party types) cannot be made extensions.

The others were non-starters, as they failed to generate implementations for TestStruc.

megajson https://github.com/benbjohnson/megajson

  • Supports json only.
  • Failed to generate for TestStruc successfully:
    Error: Field contains no name: &{<nil> [] AnonInTestStruc <nil> <nil>}:

ffjson https://github.com/pquerna/ffjson

  • It is an encoder only, and does not handle decoding
  • It also lacks features e.g. struct renaming via tags, anonymous field support, interfaces
  • It also failed to generate for TestStruc successfully:
    panic: runtime error: index out of range

bsongen http://godoc.org/github.com/youtube/vitess/go/cmd/bsongen

  • It supports bson only
  • It only supports very simple types. It failed to generate for TestStruc successfully:
    &{Struct:696 Fields:0xc208063380 Incomplete:false} is not a simple type
Tags: technology go-codec


Subscribe: Technology
© Ugorji Nwoke