Binc is a lightweight, compact, limitless, schema-free, precise, binary, high-performance, feature-rich, language-independent, multi-domain, extensible, data interchange format for structured data.
See the format documented at http://www.ugorji.net/project/binc
See Announcement of enhancements. Highlights:
Let’s talk about each of these descriptions one by one:
In tests, Binc encoding has been shown to take up less than 60% the size of JSON, BSON and other lightweight encodings.
Care was taken to support compact encodings for common values. For example, signed integers from -1 to 16, booleans, and other special values are encoded with only one byte. For small containers, the size(length) is encoded into a single byte.
Binc allows for extremely high precision integers (up to 2^15 bits of precision) both signed and unsigned, and the full spectrum of IEEE 754 floating point types (including decimals, extended precision binary floats, etc). Maps and arrays can have lengths that fit into a unsigned 64 bit integer value.
Just like JSON, a schema is not required. This is conceptually an advantage of others like protocol buffers, Thrift, etc which require a schema and a compilation step before use.
Binc aims to remove all ambiguity in the format. There are distinct signed and unsigned integers, distinct precisions, distinct unicode strings (utf8 vs utf16LE, utf16BE, utf32LE, utf32BE), distinct bytearray (binary) type, etc.
Binc is a binary encoding format. This affords significant benefits in space (encoding size) and time (encoding and decoding real and cpu time).
By “packing” the bits intelligently while still allowing easy traversal, encoding and decoding performance is achieved. We have tests that show encoding and decoding the same structure taking about less than 40% the time that JSON takes.
Binc is not limited by any specific language. Instead, implementations are free to expose the extent of their support.
By supporting arbitrary precision integers, it is a good fit for scientific data interchange. By supporting precise decimal types, it is a good fit for financial data interchange. Different domains would require different levels of support.
Binc natively supports user-defined extensions. This allows users to transfer custom types and expose how they will be encoded and decoded.
The decision to create Binc was not done lightly. A lot of analysis for features and performance was done. Other schema-free binary codecs were evaluated before Binc was created. These include:
In particular, my application use-case required extreme compactness and high encoding/decoding performance without compression. I also required precise support for timestamps, user-defined extensions, and distinct binary and string types. None of thees encodings supported these features natively.
The closest I got was msgpack which I had standardized on and engaged the community and author to include timestamp and distinct binary and string types. However, after a few months working on it, progress just halted and could not be jumpstarted (see https://github.com/msgpack/msgpack/issues/128).
However, I believe Binc has significant features beyond those provided by msgpack, and stands tall on its own.
We implemented a Binc encoder/decoder using the same high-performance codec library used to build the de-facto and best performing msgpack encoder/decoder for the Go Language, and ran extensive benchmarks agains other encoders. The results are reproduced below, and show the 40% savings in data size and 60% savings in time for encoding and decoding vs others.
.............................................. Benchmark: Struct recursive Depth: 2 ApproxDeepSize Of benchmark Struct: 136311 bytes Benchmark One-Pass Run (with Unscientific Encode/Decode times): msgpack: len: 72599 bytes, encode: 518.595µs, decode: 485.455µs binc: len: 65777 bytes, encode: 270.782µs, decode: 457.233µs simple: len: 74465 bytes, encode: 264.539µs, decode: 407.068µs cbor: len: 72131 bytes, encode: 242.007µs, decode: 479.807µs json: len: 92013 bytes, encode: 635.15µs, decode: 839.028µs std-json: len: 92429 bytes, encode: 684.876µs, decode: 2.338646ms gob: len: 64701 bytes, encode: 524.783µs, decode: 428.706µs v-msgpack: len: 72573 bytes, encode: 1.184547ms, decode: 974.376µs bson: len: 100582 bytes, encode: 884.651µs, decode: 1.19786ms .............................................. Benchmark__Msgpack____Encode-8 10000 183961 ns/op 10224 B/op 75 allocs/op Benchmark__Binc_______Encode-8 10000 206362 ns/op 12551 B/op 80 allocs/op Benchmark__Simple_____Encode-8 10000 193966 ns/op 10224 B/op 75 allocs/op Benchmark__Cbor_______Encode-8 10000 192666 ns/op 10224 B/op 75 allocs/op Benchmark__Json_______Encode-8 3000 475767 ns/op 10352 B/op 75 allocs/op Benchmark__Std_Json___Encode-8 3000 525223 ns/op 256049 B/op 835 allocs/op Benchmark__Gob________Encode-8 5000 270550 ns/op 333548 B/op 959 allocs/op Benchmark__Bson_______Encode-8 2000 747360 ns/op 715539 B/op 5629 allocs/op Benchmark__VMsgpack___Encode-8 2000 637388 ns/op 320385 B/op 542 allocs/op Benchmark__Msgpack____Decode-8 5000 370340 ns/op 120352 B/op 1210 allocs/op Benchmark__Binc_______Decode-8 3000 443650 ns/op 126144 B/op 1263 allocs/op Benchmark__Simple_____Decode-8 3000 381155 ns/op 120352 B/op 1210 allocs/op Benchmark__Cbor_______Decode-8 5000 370754 ns/op 120352 B/op 1210 allocs/op Benchmark__Json_______Decode-8 2000 719658 ns/op 159289 B/op 1478 allocs/op Benchmark__Std_Json___Decode-8 1000 2204258 ns/op 276336 B/op 6959 allocs/op Benchmark__Gob________Decode-8 5000 383884 ns/op 256684 B/op 3261 allocs/op Benchmark__Bson_______Decode-8 2000 1146851 ns/op 373121 B/op 15703 allocs/op
We hope you find good use for the Binc format.
If you are looking for a high-performance library for it, please check out the Go Library codec at https://github.com/ugorji/go/tree/master/codec#readme . You can find API docs for it at http://godoc.org/github.com/ugorji/go/codec .