Data Serialization in Go
Minimum price
Suggested price

Data Serialization in Go

The Complete Guide to JSON and Other Serialization Formats in Go

About the Book

Strict types, a lack of object-oriented features, and no generics are just some of the limitations that apply to data serialization in Go. Proper and efficient data serialization can be daunting even for the most experienced Go developer. For the newcomer, it's often overwhelming and infuriating.

This book grew out of several years of experience building and maintaining Kivik (, the popular CouchDB driver/SDK for Go. In working with CouchDB, which relies heavily on sophisticated manipulation of JSON documents, the author developed several novel techniques and approaches to solve common problems encountered in data serialization.

This work-in-progress book consolidates all of the author's knowledge, research, and tips and tricks on the subject, in a single place.

About the Author

Jonathan Hall
Jonathan Hall

Hello, world! I’m Jonathan Hall, a software developer and DevOps advocate originally from the midwest United States, living in the Netherlands since 2015.

I remember the day I wrote my first line of code. I was eight years old, and the night before I had been falling asleep in the bedroom adjacent the living room where my father had his Commodore 64. I kept hearing the computer make strange beeping noises, so I got out of bed and went into the living room to inquire.

“I’m typing in some programs from the computer manual. Go to bed, and tomorrow you can take a look,” was his answer.

So the next day, I began reading through the Commodore 64 User’s Guide, which was more than a User’s Guide, it also contained an introduction to Commodore 64 BASIC.

I had quickly finished all the programming examples, and was hungry for more. Thus began my regular visits to the public library, to borrow every book on BASIC.

Jump ahead a few years, and now I'm now an avid Gopher (Go developer), and DevOps advocate.

Table of Contents

    • Preface
    • Contents
    • Introduction (WIP)
      • What is Data Serialization
        • Subtleties
        • Serialization is key
        Uses of Data Serialization
      • History (TODO)
      Data Formats (WIP)
        • When the decision has been made for you
        • Strengths
        • Weaknesses
        • Strengths
        • Weaknesses
        • Strengths
        • Weaknesses
        INI (TODO)
      • BSON (TODO)
      • MessagePack (TODO)
      • Smile (TODO)
      • Ion (TODO)
      • CBOR (TODO)
      • XML (TODO)
      • Protobuf (TODO)
      • CSV (TODO)
      • Custom \& proprietary formats (TODO)
    The Basics
    • Getting Started with JSON (TODO)
      • The encoding/json Package (TODO)
      • Marshaling JSON (TODO)
      • Unmarshaling JSON (TODO)
      • Creating a Custom Marshaler (TODO)
      • Creating a Custom Unmarshaler (TODO)
      • Relationship to the encoding Package (TODO)
      Streaming JSON (TODO)
      • json.Encoder Options (TODO)
      • json.Decoder Options (TODO)
      • Streaming Limitations (TODO)
      Maintaining Clean Code (TODO)
      • Best Practices for Readable Serialization Code (TODO)
      • Using a Linter (TODO)
    Data Serialization Toolbox
    • Data Serialization Toolbox (WIP)
      • Pointer vs Value Receivers (TODO)
      • The Self-Referencing (Un)Marshaler
      • Configurable Marshalers (TODO)
      • Validation of input and output (TODO)
      • Embedding fields (TODO)
      • Internal types (TODO)
      • Combining JSON Objects
    • Working with Simple Types (TODO)
      • Standard Types (TODO)
      • Numbers (TODO)
      Working with Complex Types
      • Wrapping Existing Types
        • Regular Expressions
        • time.Time
        JSON Array as Go Struct
      • Extending an Embedded Marshaler
      • Combining Multiple Structs Into a Single JSON Object
      • A Single JSON Object to Multiple Go Structs
      Working with Unknown Input Types (TODO)
    • Streaming JSON Output (WIP)
      • Producing an io.Reader from an Object
      • Providing a net/http.Request.GetBody Function
      • Optimizing for Memory Usage (TODO)
      Streaming JSON Input (TODO)
      • json.NewEncoder For Shorter Code (TODO)
      • Optimizing for Memory Usage (TODO)
      • JSON Tokenizer (TODO)
    Working with YAML, BSON, MessagePack, and Others
    • Working with YAML (TODO)
      • What is YAML (TODO)
      • When to use YAML (TODO)
      • Converting JSON to YAML (TODO)
      • Converting YAML to JSON (TODO)
      Working with BSON (TODO)
      • What is BSON (TODO)
      • When to use BSON (TODO)
      • Converting JSON to BSON (TODO)
      • Converting BSON to JSON (TODO)
      Working with MessagePack (TODO)
      • What is MessagePack (TODO)
      • When to use MessagePack (TODO)
      • Converting JSON to MessagePack (TODO)
      • Converting MessagePack to JSON (TODO)
      Working with CBOR (TODO)
      • What is CBOR (TODO)
      • When to use CBOR (TODO)
      • Converting JSON to CBOR (TODO)
      • Converting CBOR to JSON (TODO)
      Working with TOML (TODO)
      • What is TOML (TODO)
      • When to use TOML (TODO)
      • Converting JSON to TOML (TODO)
      • Converting TOML to JSON (TODO)
      Working with XML (TODO)
      • What is XML (TODO)
      • When to use XML (TODO)
      Working with ProtoBuf (TODO)
      • What is ProtoBuf (TODO)
      • When to use ProtoBuf (TODO)
      • Converting to ProtoBuf (TODO)
      • Converting from ProtoBuf (TODO)
      Other Serialization Formats (TODO)
      • INI (TODO)
      • Smile (TODO)
      • Comma-Separated Values (CSV) (TODO)
      • Custom \& Proprietary Formats (TODO)
      • Related Formats and Concepts (TODO)
        • Binary-to-Text: URL Encoding, Base64, Etc (TODO)
        • VarInt (TODO)
        • MIME (TODO)
        • Data Compression (TODO)
    Third Party-Libraries
    • Third-Party Libraries (TODO)
      • (TODO)
      • (TODO)
      • ffjson (TODO)
      • Others (TODO)
    • Alphabetical Index

The Leanpub 60-day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

See full terms

80% Royalties. Earn $16 on a $20 book.

We pay 80% royalties. That's not a typo: you earn $16 on a $20 sale. If we sell 5000 non-refunded copies of your book or course for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earnedover $12 millionwriting, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub