Introducing Go
In this section we’re going to explore Go through a series of increasingly complex programs which we can interact with either through the command line or across a network.
Getting Started
If you’re of an impatient nature (my teenage self certainly was) temper that enthusiasm for a second and read Google’s install docs to get your Go install up-and-running. You might also like to install LiteIDE which is an excellent cross-platform development environment for Go which also happens to be written in Go.
This book is laid out as a series of parts focused on different topics I’m interested in and the order you read it in is up to you with one clear exception: Hello World is designed as a quick tutorial introduction to Go so if you don’t know the language that’s the place to start. Even if you do know Go you’ll probably find other stuff in Hello World which will be useful as it covers network communications and cryptography.
Now Go have fun messing with my code.
So what is Go?
Well obviously it’s a programming language, which means an artificial language intended for humans to read and write but which can be usefully translated into the 1s and 0s understood by digital computers.
But is it a good programming language?
Opinion on the internet has been bitterly divided on this question (which is hardly unusual) since Go was released in November 2009 and the language has many detractors who either see it as deficient in some crucial feature without which efficient programs can’t possibly be productively developed, or else as a wholehearted nostalgia trip back into the 1980s by a team who’ve failed to move with the times.
So let’s look at the reality and see if these criticisms are justified.
Open source
Go’s developed as an open source project so anyone with an interest can read through its source code, make modifications for their own purposes, or get involved in its future development. However it’s also a carefully curated project and unlike many popular languages there’s a tightly defined language specification which is intended to be read and understood by anyone working with Go.
These are pragmatic choices. Not only do many eyes make light work of code quality, a clear and readable language specification makes it very clear what Go is and isn’t trying to achieve.
Efficient compilation
Work started on the language at Google in 2007 as a way to address three key problems in systems programming.
First there’s efficient compilation, the process of taking the source code for a program and turning it into an executable form with a minimum of fuss. As increasingly complex programs are developed in traditional systems languages like C and C++ they become dependent on equally complex build tools and compilation times can stretch into minutes or hours.
This isn’t a particularly new phenomenon. The systems I was working on in the 90s often had compile times in the order of 10 to 30 minutes so builds were a great excuse to make a cup of tea whilst waiting to see if things compiled successfully. Still there’s only so many cups of tea you can make in a day and maintain any semblance of productivity.
Sadly one of the main reasons for the popularity of interpreted languages like Ruby and Python (both of which have other much more compelling reasons to explore) is the interactive nature of developing with them. This basically allows devs to embrace the same smugness the Lisp crowd exude without the intellectual snobbery or taste the pioneering spirit of Forth without the serious WTF? factor when rereading old code.
In interpreted languages small chunks of code can be developed independently of each other and tested immediately. This fast feedback cycle makes it much easier to explore a problem domain and figure out how to work with it effectively (or if we’re lucky elegantly). However interpreted languages have a reputation for being memory hungry and slow.
Efficient execution
This leads us to the second problem Go addresses: efficient execution. Systems programming is all about programs which other programs are going to rely on for key services so it’s highly desirable that languages addressing these problems produce programs which run in a timely fashion and with predictable memory overhead.
Compilation provides a huge boost to runtime performance compared to interpretation and the standard Go compiler features a range of optimisations which make it roughly competitive with C++ or Java. And because Go was developed with multi-core processors in mind it uses their resources effectively.
One area which raised concerns when Go was first released is its use of garbage collection rather than traditional programmer-controlled memory allocation. Garbage collection used to be a significant cost with 10ms set aside from every 50ms of program execution but in recent years there have been huge improvements by switching to a concurrent design and tailoring this to other features of Go’s design.
Ease of programming
The majority of systems languages were designed at a time when memory was limited and processors could generally only execute one instruction at a time (and that glacially slowly by modern standards). As a result these languages provide programmers with very primitive abstractions which no longer even vaguely resemble what’s really going on in most general purpose computers, and to make matters worse they often entail a lot of careful book-keeping just to track memory allocations and prevent leaks which will otherwise cause systems to become unstable.
Go was developed after these changes took place by a team with a long history of working on systems software and embraces programmer-friendly conveniences such as garbage collection and CSP concurrency which ease much of this burden. Its design also includes other features which aid ease of programming such as first-class functions, slices, hash maps and structural typing which can greatly simplify code analysis and reuse.
We’re going to have a lot of fun with all these features throughout this book.
And where did it come from?
Originally Go was developed in-house at Google, a company which thanks to its position in web search and online advertising is heavily tech driven. Google’s systems operate at ROFLscale - a scale which most of us can only dream of messing with - and its codebases measure multiple BLOC (that’s billion lines of code). The aspirations at the heart of Go’s design are essentially attempts to reign in the complexity of numerous codebases scattered throughout Google’s data-centres and written in C++, Python, and Java.
The initial development team comprised three industry veterans with a record of both innovation and successful commercial implementation: Ken Thompson, Rob Pike, and Robert Griesemer.
Ken Thompson is a legendary hacker, the original creator of UNIX whose B programming language was a strong influence on the design of hacker favourite C. Rob Pike meanwhile was deeply involved with the development of the UNIX, Plan 9 (along with Ken) and Inferno operating systems as well as the Dis virtual machine and a number of programming languages focused on concurrency. Robert Greisemer cut his teeth working on Java’s HotSpot VM and Google’s V8 JavaScript runtime, so between the three of them there’s a wealth of experience in systems software design and performance engineering.
At first glance Go is similar to another of Rob’s languages Limbo which he created for the Inferno operating system and it features the CSP approach to concurrency he and Ken had been playing with since their UNIX days. CSP (or Communicating Sequential Processes) was formally defined by British computer scientist Tony Hoare in the late 70s and was key to the technical success of the multi-processor InMos Transputer machines of the 1980s (though it’s programming language Occam was pretty nasty to work with).
However Go is a much smaller language than Limbo thanks to a key decision made at the start of the project: no suggested feature would be included in Go unless all three of the core team agreed it should be included. This makes for a highly conservative language design focused on must have features to achieve its stated design goals rather than nice to have features which might be at odds with them. This is quite a departure from the usual feature creep and special pleading seen in other languages.
Into the wild
When Go was first announced to the world on November 10th 2009 it was already a robust language but with a few rough edges like the use of make files for project builds, a weekly source release cycle which often saw package library APIs changing, and some fairly fiddly configuration via environment variables.
However key features like the language specification, structural typing, garbage collection, and goroutines were already fixed and a community of passionate users started to grow up around the language. Some of these came from the bifurcating Python scene in search of stability and static typing, others like me had need for a systems language but wanted to keep many of the free-wheeling conveniences we enjoyed in higher-level languages. Then there were the C/C++ and Java devs initially attracted by the reputations of Go’s designers who then stuck around for the fun they could have.
Looking back I think all three groups were pulling not only in different directions to each other, but also in different directions to the core team and it’s a testament to the flexibility of Go that a dedicated Rubyist like me can be as comfortable working with it as the most static-typing obsessed escapee from Python or generics-missing Java head.
The early buzz around Go wasn’t nearly as loud as TIOBE ratings suggested and the language would probably have remained a curiosity even with Google’s backing if it hadn’t been for the widespread adoption of Docker by the DevOps community.
Docker popularised containers, a standardised way of installing and running one or more lightweight virtual machines on a server. These virtual machines aren’t to be confused with the kind used by languages like Java or Erlang (and which form the meat of the software machines section of this book) but are better understood as an outgrowth of UNIX chroot jails based on the resource isolation features present in the Linux kernel. This allows one server to run multiple independent environments each with well-defined privileges and managed resources without requiring the overhead associated with more traditional virtualisation hypervisors like Xen or VMWare.
Google themselves are no strangers to containerisation and in 2015 released Kubernetes, an orchestration system for deploying and managing containers across large-scale clusters inspired by a previous in-house tool named Borg. Kubernetes is written in Go and in combination with Docker has helped simplify the once black art of microservice design. Not only are these powerful tools, their use of Go makes them great advertisements for the language.
Where next?
At the time of writing Go has been in the public eye for nine years. The release of version 1 in March 2012 came with the promise that there’d be no breaking language changes until a version 2 release and since version 1.1 development has followed a tight release schedule focused primarily on improving runtime performance and the standard library. Good software engineering practices aren’t particularly glamorous but the end results are impressive as the garbage collector and goroutine scheduler demonstrate.
It’s still far from clear when version 2 of Go will materialise or the extent of any changes to the language which will accompany it. One much-anticipated addition is some kind of support for generic types, the absence of which has been controversial since Go was first released. Generics are one of a number of different ways to enable compile-time metaprogramming and are popular in a number of languages such as C++ and Java, however they generally make the compilation process more complex and trade compile-time cost for runtime efficiency.
Go already allows metaprogramming at runtime via its reflection and unsafe APIs with minimal impact on compilation speeds but there is a runtime performance hit for any significant use of reflection which starts to make higher-level interpreted languages look competitive. And thanks to structural typing via interfaces it’s possible to write reasonably generic code based upon capability, though yet again there are runtime costs associated with this approach.
Personally I’d like Go version 2 to introduce runtime optimisations for both reflection and structural typing - maybe with some form of JIT compilation. This might well be possible for type switches by using call-site caching and making changes to the reflection API inspired by virtual machine designs.
There are also a number of active projects focused on richer compile-time metaprogramming using everything from code templating (the C++ approach) to Lisp-inspired macros. These all involve code generation and subsequent compilation and it would be interesting to see direct compile-time metaprogramming integrated into the language specification.
Then there’s enumeration via range statements which could be opened up to a wider selection of user-defined types. Conceptually this could be simple enough: add an interface to the standard library which yields successive elements and allow the compiler to spot this is implemented for a given type. But it’s hard to see how this approach wouldn’t get pretty ugly pretty fast.
A possible first step might be to add nodes and lists to the core types defined in the language specification as a analogous abstraction arrays & slices so that at least singly-linked and doubly-linked lists could be used in range statements.
Another area where Go could be improved is for mobile development. There’s quite a bit of work that’s gone on in this area over the past few years but Go still isn’t a first-class language on Android or iOS so it would be good to see something official in this direction.
Things are much further along with desktop applications such as LiteIDE which uses Qt5 bindings for its cross-platform GUI whilst TinyGo is bringing a decent subset of the language to minimalist embedded devices like the BBC micro:bit. Likewise GopherJS has pioneered Go in the web browser whilst Go version 1.11 now includes experimental support for WebAssembly. There are even bindings for the Vulkan graphics API.
So hopefully the version 2 release will make Go a credible language to use all the way from deep embedded systems right the way through to web browser front-ends. Who knows, perhaps we’ll even see some graphically intensive games being developed in Go.