Five: Clortex - A Complementary Project
Introduction
Clortex is a new design for HTM, designed to provide a basis for communication of and research into the theory, as well as exploration and development of new applications.
Design Goals
Directly Analogous to HTM/CLA Theory
In order to be a platform for demonstration, exploration and experimentation of Jeff Hawkins’ theories, the system must at all levels of relevant detail match the theory directly (ie 1:1). Any optimisations introduced may only occur following an effectively mathematical proof that this correspondence is maintained under the change.
There are several benefits to this requirement. Firstly, during development, this requirement provides a rigid and testable constraint on the options for implementation. With a good model of the theory in mind, we may proceed with confidence to use transparently analogous data structures and algorithms, leaving the question of computational performance for a later day.
Secondly, this requirement will ensure that the system at its heart remains a working implementation of the theory as it develops. In addition, because of this property, it will be directly usable by Jeff and any co-workers (including us) in extending and experimenting with new ideas in the theoretical space. This will enhance support for the new project, and encourage the HTM community to consider the new project as a parallel or alternative way to realise their own goals.
Thirdly, the software will provide a runnable explanation of the theory, with real working code (see next requirement) replacing the pseudocode and providing live imagery instead of diagrams (see later requirement).
Lastly, we feel that the theory deserves software of similar quality, and that the lack of this has slowed the realisation of the goals of all concerned. The development of a true analogue in software will pave the way for a rapid expansion in interest in the entire project. In particular, this will benefit anyone seeking to exploit the commercial advantages which the CLA offers.
Transparently Understandable Implementation in Source Code
All source code must at all times be readable by a non-developer. This can only be achieved if a person familiar with the theory and the models (but not a trained programmer) can read any part of the source code and understand precisely what it is doing and how it is implementing the algorithms.
This requirement is again deliberately very stringent, and requires the utmost discipline on the part of the developers of the software. Again, there are several benefits to this requirement.
Firstly, the extreme constraint forces the programmer to work in the model of the domain rather than in the model of the software. This constraint, by being adhered to over the lifecycle of the project, will ensure that the only complexity introduced in the software comes solely from the domain. Any other complexity introduced by the design or programming is known as incidental complexity and is the cause of most problems in software.
Secondly, this constraint provides a mechanism for verifying the first requirement. Any expert in the theory must be able to inspect the code for an aspect of the system and verify that it is transparently analogous to the theory.
Thirdly, anyone wishing to extend or enhance the software will be presented with no introduced obstacles, leaving only their level of understanding of the workings of the theory.
Finally, any bugs in the software should be reduced to breaches of this requirement, or alternatively, bugs in the theory.
Directly Observable Data
All relevant data structures representing the computational model must be directly observable and measurable at all times. A user must be able to inspect all this data and if required, present it in visual form.
This requirement ensures that the user of the platform can see what they’re doing at all times. The software is essentially performing a simulation of a simplified version of the neocortex as specified in the CLA, and the user must be able to directly observe how this simulation is progressing and how her choices in configuring the system might affect the computation.
The benefits of this requirement should be reasonably obvious. Two in particular: first, during development, a direct visual confirmation of the results of changes is a powerful tool; and secondly, this answers much of the representation problem, as it allows an observer to directly see how the models in the theory work, rather than relying on analogy.
Sufficiently Performant
The system must have performance sufficient to provide for rapid development of configurations suitable to a user task. In addition, the performance on large or complex data sets must be sufficient to establish that the system is succeeding in its task in principle, and that simply by scaling or optimising it can perform at production levels.
What this says is that the system must be a working prototype for how a more finely tuned or higher-performance equivalent will perform. Compute power and memory are cheap, and software can always be made faster relatively easily. The question a user has when using the system is primarily whether or not (and how well) the system can solve her problem, not whether it takes a few seconds or a few hours.
This constraint requires that the software infrastructure be designed so as to allow for significant raw performance improvements, both by algorithm upgrades and also by using concurrency and distribution when the user has the resources to scale the system.
Useful Metrics
The system must include functionality which allows the user to assess the effectiveness of configuration choices on the system at all relevant levels.
At present, NuPIC has some metrics but they are either difficult to understand and interpret, inappropriate, or both. The above requirement must be answered using metrics which have yet to be devised, so we have no further detail at this stage.
Appropriate Platform
The development language(s) and runtime platform must ensure ease of deployment, robust execution, easy maintenance and operation, reliability, extensibility, use in new contexts, portability, interoperability, and scaleable performance.
Quite a list, but each failure in the list reduces the potential mindshare of the software and raises fears for new adopters. Success in every item, along with the other requirements, ensures maximal usefulness and easy uptake by the rising ramp of the adoption curve.
Design Philosophy
It’s odd, but Clortex’ journey began when I followed a link to a talk Jeff gave at GOTO Aarhus 2013, and decided to watch one, then two, and finally all three talks given by Russ Miles at the same event. If you’re only able to watch one, the one to watch is Architectural Simplicity through Events. In that talk, Russ outlines his axioms for building adaptable software:
1. Your Software’s First Role is to be Useful
Clearly, NuPIC is already useful, but there is a huge opportunity for Clortex to be useful in several new ways:
a) As a Teaching Tool to help understand the CLA and its power. HTM and CLA are difficult to understand at a deep level, and they’re very different from traditional Neural Networks in every way. A new design is needed to transparently communicate an intuitive view of CLA to layman, machine learning expert, and neuroscientist alike. The resulting understanding should be as clear to an intelligent and interested viewer as it is to Jeff himself.
b) As a Research and Development platform for Machine Intelligence. Jeff has recently added – literally – a whole set of layers to his theory, involving a new kind of temporal pooling, sensorimotor modelling, multilayer regions, behaviour, subcortical connections, and hierarchy. This is all being done with thought experiments, whiteboards, pen and paper, and slides. We’ll see this in software sometime, no doubt, but that process has only begun. A new system which allows many of these ideas to be directly expressed in software and tested in real time will accelerate the development of the theory and allow many more people to work on it.
c) As a Production Platform for new Use Cases. NuPIC is somewhat optimised for a certain class of use cases – producing predictions and detecting anomalies in streaming machine-generated numerical data. It’s also been able to demonstrate capabilities in other areas, but there is a huge opportunity for a new design to allow entirely new types of information to be handled by HTM and CLA techniques. These include vision, natural language, robotics, and many other areas to which traditional AI and ML techniques have been applied with mixed results. A new design, which emphasises adaptability, flexibility, scaleability and composability, will allow CLA to be deployed at whatever scale (in terms of hierarchy, region size, input space etc as well as machine resources) is appropriate to the task.
2. The best software is that which is not needed at all
Well, we have our brains, and the whole point of this is to build software which uses the principles of the brain. On the other hand, we can minimise over-production by only building the components we need, once we understand how they work and how they contribute to the overall design. Clortex embraces this using a design centred around immutable data structures, surrounded by a growing set of transforming functions which work on that data.
3. Human Comprehension is King
This axiom is really important for every software project, but so much more so when the thing you’re modelling is so difficult to understand for many. The key to applying this axiom is to recognise that the machine is only the second most important audience for your code – the most important being other humans who will interact with your code as developers, researchers, and users. Clortex has as its #1 requirement the need to directly map the domain – Jeff’s theory of the neocortex – and to maintain that mapping at all costs. This alone would justify building Clortex for me.
4. Machine Sympathy is Queen
This would seem to contradict Axiom 3, but the use of the word “Queen” is key. Any usable system must also address the machine environment in which it must run, and machine sympathy is how you do that. Clortex’ design is all about turning constraints into synergies, using the expressive power and hygiene of Clojure and its immutable data structures, the unique characteristics of the Datomic database system, and the scaleability and portability characteristics of the Java Virtual Machine. Clortex will run on Raspberry Pi, a version of which will run in browsers and phones, yet it will scale layers and hierarchies across huge clusters to deliver real power and test the limits of HTM and CLA in production use.
5. Software is a Process of R&D
This is obviously the case when you’re building software based on an evolving theory of how the brain performs analogous functions. Russ’ key point here is that our work always involves unknowns, and our software and processes must be designed in such a way as not to slow us down in our R&D work. Clortex is designed as a set of loosely coupled, interchangeable components around a group of core data structures, and communicating using simple, immutable data.
6. Software Development is an Extremely Challenging Intellectual Pursuit
Again, this is so true in this case, but the huge payoff you can derive if you can come up with a design which matches the potential of the CLA is hard to beat. I hope that Clortex can meet this most extreme of challenges.
I met Russ in Spring 2014 when he visited Dublin to give a talk, and we had a great discussion (as ever) in the pub afterwards, about Simplicity, Antifragile Software and, of course, how HTM, and understanding the brain, is going to change the world. Russ is putting together his ideas and methods in his own Leanpub book, Antifragile Software, which I’d strongly encourage you to read.
Platform - Clojure, Functional Programming on the Java Virtual Machine
NuPIC is a hybrid, written in Python and partly re-implemented (nupic.core) in C++ for performance and cross-platform applicability. Python is a great language, but it’s at heart an Object-oriented (OO) language, and NuPIC has a complex OO-style structure as a result. As I was investigating possible platforms for a new HTM system, it became clearer and clearer to me that the world of software is undergoing a revolution, shedding decades of distraction and complexity in favour of a return to its roots: Functional Programming.
Clojure is a relatively young language (first released in 2008), but in fact it’s an update of the second-oldest popular language ever - LISP - which dates back to 1958.
It turns out that Rich Hickey, Clojure’s author, is also a neuroscience nut.
Architecture - Russ Miles’ Life Preserver
Russ’ idea for Antifragile Software is called the Life Preserver (aka the Lifebelt). The design was originally conceived to provide a simple answer to the question “where do I put this?” when architecting a software system. Russ’ answer is that each module, object, or subsystem in a design belongs either in the centre, or core, or else in the ring around the core, in an integration domain.
After many weeks of thought (simplicity is not easy!), the architecture of Clortex practically designed itself. The core of Clortex is simply a big data structure, which contains all the layers (called patches in Clortex) of neurons, each layer organised into columns as in the theory. Each neuron has a proximal dendrite and a number of distal dendrites, and each dendrite contains a set of simple synapses which connect with other neurons.
This data structure is stored in a new kind of database - Datomic. I’ll go into more detail about Datomic later, but for now here are some of the key attributes which Datomic brings to Clortex: