Hacking TensorFlow Internals


This book is no longer available for sale.

Hacking TensorFlow Internals

An Insider’s Commentary on A Learning System

About the Book

This book is an attempt to decipher the internals of TensorFlow in gory details, including (but not limited to) what, how and why from a hacker’s perspective, explaining in detail the nucleus of one of the most interesting learning systems to appear in recent years. This analyses the kernel and reveals the system’s innards including architecture, programming model, tensors, graphs (computational and calculational), gradients, optimizers, clusters and other data structures with algorithms in play. This provides illustrated commentary on code snippets with annotations. The commentary also remarks on how the code might be improved.

These topics will help the programmer to learn, appreciate, modify and extend the TensorFlow Core, which in turn will help improve the TensorFlow system design and performance optimization. C++ Code enthusiasts (both inside and outside the Google Inc.) will be better equipped to hack it to their tastes and needs.

After reading this book, the reader will be on par with the core team of TensorFlow who conceptualized and crafted a new programming model to address problems in machine learning, deep learning, computer vision along with related sub-disciples and will be able to extend it further by sharing the vision.

  • First book of its kind on TensorFlow Internals
  • It will attract C++ programmers too. Typically this field is donned by Python and R programmers.
  • The only exposition of the workings of a 'real' learning system.
  • The only TensorFlow kernel documentation available outside Google. (I doubt if one such exists inside Google !)

Typical books on TensorFlow focus on its usage, whereas this book will allow classroom use of the source code.

This book is primarily for the C++ programmer who is keen to unravel the mystical nuances buried deep inside the code of TensorFlow Core. Familiarity with programming in C++ and python with some background in linear algebra, calculus, statistics and machine learning is assumed. Other data science practitioners and instructors may also get benefited by embracing the only commentary available on TensorFlow internals.

In my opinion, it is highly beneficial for practitioners of data science to have the opportunity to study a working learning system in all its aspects.

Moreover it is undoubtedly good for students majoring in Data Science, to be confronted at least once in their careers, with the task of reading and understanding a learning program of major dimensions.

About the Author

Chandra Shekhar Kumar
Chandra Shekhar Kumar

Chandra Shekhar Kumar is Staff Software Architect @ GE Healthcare (Ultrasound Digital Solutions). In an innovator role, he is actively involved in digital Innovation in Ultrasound ecosystem (premise, edge and cloud) including (but not limited to) C++17/20/23, Boost C++ Libraries, WineLib, Qt, wxWidgets, WebAssembly, NATS.io, Hashicorp Nomad and Rust. Motto is to build once and run everywhere using the same code base (using and extending WebAssembly Infrastructure)

He is Co-Founder of Ancient Science Publishers (estd.2014), a venture to publish monographs on mathematics, physics and computer science and render services related to hiring technical talents, training for competitive programming, algorithms, programming interviews, IITJEE and Olympiads. Inspiration for this undertaking came from the writings of Leonhard Euler (mathematics), Richard Phillips Feynman (physics) and Edsger Wybe Dijkstra (computer science).


He is Founder of Ancient Kriya Yoga Mission (estd.2013), a venture to disseminate simple techniques of ancient science of living and publish kriya yoga scriptures and commentaries.

He holds a degree of Integrated M.Sc.(5 yrs) in Physics from IIT Kanpur.

He has worked with software companies like Trilogy, Oracle and few start-ups.

He has been programming in C++ for the last 22 years. He loves to hack gcc, gdb, valgrind, clang, boost, TeX, LaTeX and pours inside the works of Dijkstra and Knuth.

Table of Contents

Chapter 1 : Genesis

  • DistBelief

Chapter 2 : Introduction

Chapter 3 : Programming Model

  • Kernels
  • Operations
  • Sessions
  • Variables
  • Gradients
  • Execution

Chapter 4 : System Architecture

  • Single Machine
  • Distributed

Chapter 5 : Source Code Structure

Chapter 6 : Data Structures (300+)

  • Tensors
  • Graph
  • Computational
  • Calculational
  • Queue
  • Container

Chapter 7 : Algorithms (400+)

Chapter 8 : Source Code Analysis

Chapter 9 : Programming Idioms

Chapter 10 : Design Patterns

Chapter 11 : Optimization

Chapter 12 : Visualization

Chapter 13 : APIs

Chapter 14 : Alternatives

Chapter 15 : Extensions

Chapter 16 : Traps and Pitfalls

Appendix A : Linear Algebra

Appendix B : Matrix Calculus

Appendix C : Probability

Appendix D : Statistics

Appendix E : SWIG

The Leanpub 60 Day 100% Happiness Guarantee

Within 60 days of purchase you can get a 100% refund on any Leanpub purchase, in two clicks.

Now, this is technically risky for us, since you'll have the book or course files either way. But we're so confident in our products and services, and in our authors and readers, that we're happy to offer a full money back guarantee for everything we sell.

You can only find out how good something is by trying it, and because of our 100% money back guarantee there's literally no risk to do so!

So, there's no reason not to click the Add to Cart button, is there?

See full terms...

80% Royalties. Earn $16 on a $20 book.

We pay 80% royalties. That's not a typo: you earn $16 on a $20 sale. If we sell 5000 non-refunded copies of your book or course for $20, you'll earn $80,000.

(Yes, some authors have already earned much more than that on Leanpub.)

In fact, authors have earnedover $12 millionwriting, publishing and selling on Leanpub.

Learn more about writing on Leanpub

Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

Write and Publish on Leanpub

You can use Leanpub to easily write, publish and sell in-progress and completed ebooks and online courses!

Leanpub is a powerful platform for serious authors, combining a simple, elegant writing and publishing workflow with a store focused on selling in-progress ebooks.

Leanpub is a magical typewriter for authors: just write in plain text, and to publish your ebook, just click a button. (Or, if you are producing your ebook your own way, you can even upload your own PDF and/or EPUB files and then publish with one click!) It really is that easy.

Learn more about writing on Leanpub