Announcing go-msgpack

Announcing go-msgpack, a rich msgpack codec for Go. Supports encoding/decoding to msgpack binary format, and use for net/rpc communication.

https://github.com/ugorji/go-msgpack
http://gopkgdoc.appspot.com/pkg/github.com/ugorji/go-msgpack

It provides features similar to encoding packages in the standard library (ie json, xml, gob, etc).

Supports:

1. Standard Marshal/Unmarshal interface.
2. Standard field renaming via tags
3. Encoding from any value
   (struct, slice, map, primitives, pointers, interface{}, etc)
4. Decoding into a pointer to any non-nil value
   (struct, slice, map, int, float32, bool, string, etc)
5. Decoding into a nil interface{} (big)
6. Handles time.Time transparently.
7. Provides a Server and Client Codec so msgpack can be used as communication protocol for net/rpc.

Usage:

// v can be interface{}, int32, bool, map[string]bool, etc
dec = msgpack.NewDecoder(r, nil)
err = dec.Decode(&v)

enc = msgpack.NewEncoder(w, nil)
err = enc.Encode(v)

//methods below are convenience methods over functions above.
data, err = msgpack.Marshal(v, nil)
err = msgpack.Unmarshal(data, &v, nil)
//RPC Communication
conn, err = net.Dial("tcp", "localhost:5555")
rpcCodec := msgpack.NewRPCCodec(conn)
client := rpc.NewClientWithCodec(rpcCodec)
...

Why?

I was initially looking at different binary serialization formats for an application I’m developing. I looked at Thrift, Protocol Buffers, BSON, Msgpack, etc.

I finally decided on msgpack:

1. compact
2. supports basic types (just like JSON. Numbers, Bool, Null, Bytes/String, Map, List) from which other data structures else can be assembled
3. raw bytes support (which can represent binary data or strings)
4. no schema needed (just like JSON)
5. cross-language support
6. has pretty good mindshare

Unfortunately, the Go library on the msgpack.org site is old, does not build, and is not “smart” like the encoding/json package. Specifically, it doesn’t allow me decode into a typed object (e.g. struct, bool, etc).

I wrote go-msgpack to give the same conveniences we’ve gotten used to (spoiled by using the encoding/json package), while being really performant.

Summary of my testing:

1. Decodes significantly faster (120% faster) and encodes only slightly slower (20% slower) than encoding/json
2. Uses less disk space (40% less)
3. May not require compression (compression only gave a 10% reduction in size).
   Since compression/decompression time may be significant, this may be an important win.

Hope folks use it and enjoy using it. I know I will. Please feel free to send me feedback.