###### Linear Algebra: Theory, Intuition, Code

### Linear Algebra: Theory, Intuition, Code

###### Wholesome mathy goodness for everyone.

# About the Book

Linear algebra is perhaps the most important branch of mathematics for computational sciences, including machine learning, AI, data science, statistics, simulations, computer graphics, multivariate analyses, matrix decompositions, signal processing, and so on.

The way linear algebra is presented in traditional textbooks is different from how professionals use linear algebra in computers to solve real-world applications in machine learning, data science, statistics, and signal processing. For example, the "determinant" of a matrix is important for linear algebra theory, but should you actually use the determinant in practical applications? The answer may surprise you!

If you are interested in learning the mathematical concepts linear algebra and matrix analysis, but also want to apply those concepts to data analyses on computers (e.g., statistics or signal processing), then this book is for you. You'll see all the math concepts implemented in MATLAB and in Python.

Unique aspects of this book:

- Clear and comprehensible explanations of concepts and theories in linear algebra.

- Several distinct explanations of the same ideas, which is a proven technique for learning.

- Visualization using graphs, which strengthens the geometric intuition of linear algebra.

- Implementations in MATLAB and Python. Com'on, in the real world, you never solve math problems by hand! You need to know how to implement math in software!

- Beginner to intermediate topics, including vectors, matrix multiplications, least-squares projections, eigendecomposition, and singular-value decomposition.

- Strong focus on modern applications-oriented aspects of linear algebra and matrix analysis.

- Intuitive visual explanations of diagonalization, eigenvalues and eigenvectors, and singular value decomposition.

- Codes (MATLAB and Python) are provided to help you understand and apply linear algebra concepts on computers.

- A combination of hand-solved exercises and more advanced code challenges. Math is not a spectator sport!

#### Table of Contents

- 0.1 Front matter
- 0.2 Dedication
- 0.3 Forward
- 1 Introduction
- 1.1 What is linear algebra and why learn it?
- 1.2 About this book
- 1.3 Prerequisites
- 1.4 Exercises and code challenges
- 1.5 Online and other resources

- 2 Vectors
- 2.1 Scalars
- 2.2 Vectors: geometry and algebra
- 2.3 Transpose operation
- 2.4 Vector addition and subtraction
- 2.5 Vector-scalar multiplication
- 2.6 Exercises
- 2.7 Answers
- 2.8 Code challenges
- 2.9 Code solutions

- 3 Vector multiplication
- 3.1 Vector dot product: Algebra
- 3.2 Dot product properties
- 3.3 Vector dot product: Geometry
- 3.4 Algebra and geometry
- 3.5 Linear weighted combination
- 3.6 The outer product
- 3.7 Hadamard multiplication
- 3.8 Cross product
- 3.9 Unit vectors
- 3.10 Exercises
- 3.11 Answers
- 3.12 Code challenges
- 3.13 Code solutions

- 4 Vector spaces
- 4.1 Dimensions and fields
- 4.2 Vector spaces
- 4.3 Subspaces and ambient spaces
- 4.4 Subsets
- 4.5 Span
- 4.6 Linear independence
- 4.7 Basis
- 4.8 Exercises
- 4.9 Answers

- 5 Matrices
- 5.1 Interpretations and uses of matrices
- 5.2 Matrix terms and notation
- 5.3 Matrix dimensionalities
- 5.4 The transpose operation
- 5.5 Matrix zoology
- 5.6 Matrix addition and subtraction
- 5.7 Scalar-matrix mult.
- 5.8 "Shifting" a matrix
- 5.9 Diagonal and trace
- 5.10 Exercises
- 5.11 Answers
- 5.12 Code challenges
- 5.13 Code solutions

- 6 Matrix multiplication
- 6.1 "Standard" multiplication
- 6.2 Multiplication and eqns.
- 6.3 Multiplication with diagonals
- 6.4 LIVE EVIL
- 6.5 Matrix-vector multiplication
- 6.6 Creating symmetric matrices
- 6.7 Multiply symmetric matrices
- 6.8 Hadamard multiplication
- 6.9 Frobenius dot product
- 6.10 Matrix norms
- 6.11 What about matrix division?
- 6.12 Exercises
- 6.13 Answers
- 6.14 Code challenges
- 6.15 Code solutions

- 7 Rank
- 7.1 Six things about matrix rank
- 7.2 Interpretations of matrix rank
- 7.3 Computing matrix rank
- 7.4 Rank and scalar multiplication
- 7.5 Rank of added matrices
- 7.6 Rank of multiplied matrices
- 7.7 Rank of A, A', A'A, and AA'
- 7.8 Rank of random matrices
- 7.9 Boosting rank by "shifting"
- 7.10 Rank difficulties
- 7.11 Rank and span
- 7.12 Exercises
- 7.13 Answers
- 7.14 Code challenges
- 7.15 Code solutions

- 8 Matrix spaces
- 8.1 Column space of a matrix
- 8.2 Column space: A AA'
- 8.3 Determining whether v is in C(A)
- 8.4 Row space of a matrix
- 8.5 Row spaces of A'A and A
- 8.6 Null space of a matrix
- 8.7 Geometry of the null space
- 8.8 Orthogonal subspaces
- 8.9 Matrix space orthogonalities
- 8.10 Dimensionalities of matrix spaces
- 8.11 More on Ax=b and Ay=0
- 8.12 Exercises
- 8.13 Answers
- 8.14 Code challenges
- 8.15 Code solutions

- 9 Complex numbers
- 9.1 Complex numbers
- 9.2 What are complex numbers?
- 9.3 The complex conjugate
- 9.4 Complex arithmetic
- 9.5 Complex dot product
- 9.6 Special complex matrices
- 9.7 Exercises
- 9.8 Answers
- 9.9 Code challenges
- 9.10 Code solutions

- 10 Systems of equations
- 10.1 Algebra and geometry of eqns.
- 10.2 From systems to matrices
- 10.3 Row reduction
- 10.4 Gaussian elimination
- 10.5 Row-reduced echelon form
- 10.6 Gauss-Jordan elimination
- 10.7 Possibilities for solutions
- 10.8 Matrix spaces, row reduction
- 10.9 Exercises
- 10.10 Answers
- 10.11 Coding challenges
- 10.12 Code solutions

- 11 Determinant
- 11.1 Features of determinants
- 11.2 Determinant of a 2x2 matrix
- 11.3 The characteristic polynomial
- 11.4 3x3 matrix determinant
- 11.5 The full procedure
- 11.6 Delta of triangles
- 11.7 Determinant and row reduction
- 11.8 Delta and scalar multiplication
- 11.9 Theory vs practice
- 11.10 Exercises
- 11.11 Answers
- 11.12 Code challenges
- 11.13 Code solutions

- 12 Matrix inverse
- 12.1 Concepts and applications
- 12.2 Inverse of a diagonal matrix
- 12.3 Inverse of a 2x2 matrix
- 12.4 The MCA algorithm
- 12.5 Inverse via row reduction
- 12.6 Left inverse
- 12.7 Right inverse
- 12.8 The pseudoinverse, part 1
- 12.9 Exercises
- 12.10 Answers
- 12.11 Code challenges
- 12.12 Code solutions

- 13 Projections
- 13.1 Projections in R2
- 13.2 Projections in RN
- 13.3 Orth and par vect comps
- 13.4 Orthogonal matrices
- 13.5 Orthogonalization via GS
- 13.6 QR decomposition
- 13.7 Inverse via QR
- 13.8 Exercises
- 13.9 Answers
- 13.10 Code challenges
- 13.11 Code solutions

- 14 Least-squares
- 14.1 Introduction
- 14.2 5 steps of model-fitting
- 14.3 Terminology
- 14.4 Least-squares via left inverse
- 14.5 Least-squares via projection
- 14.6 Least-squares via row-reduction
- 14.7 Predictions and residuals
- 14.8 Least-squares example
- 14.9 Code challenges
- 14.10 Code solutions

- 15 Eigendecomposition
- 15.1 Eigenwhatnow?
- 15.2 Finding eigenvalues
- 15.3 Finding eigenvectors
- 15.4 Diagonalization
- 15.5 Conditions for diagonalization
- 15.6 Distinct, repeated eigenvalues
- 15.7 Complex solutions
- 15.8 Symmetric matrices
- 15.9 Eigenvalues singular matrices
- 15.10 Eigenlayers of a matrix
- 15.11 Matrix powers and inverse
- 15.12 Generalized eigendecomposition
- 15.13 Exercises
- 15.14 Answers
- 15.15 Code challenges
- 15.16 Code solutions

- 16 The SVD
- 16.1 Singular value decomposition
- 16.2 Computing the SVD
- 16.3 Singular values and eigenvalues
- 16.4 SVD of a symmetric matrix
- 16.5 SVD and the four subspaces
- 16.6 SVD and matrix rank
- 16.7 SVD spectral theory
- 16.8 Low-rank approximations
- 16.9 Normalizing singular values
- 16.10 Condition number of a matrix
- 16.11 SVD and the matrix inverse
- 16.12 MP Pseudoinverse, part 2
- 16.13 Code challenges
- 16.14 Code solutions

- 17 Quadratic form
- 17.1 Algebraic perspective
- 17.2 Geometric perspective
- 17.3 The normalized quadratic form
- 17.4 Evecs and the qf surface
- 17.5 Matrix definiteness
- 17.6 The definiteness of A'A
- 17.7 Eigenvalues and definiteness
- 17.8 Code challenges
- 17.9 Code solutions

- 18 Covariance matrices
- 18.1 Correlation
- 18.2 Variance and standard deviation
- 18.3 Covariance
- 18.4 Correlation coefficient
- 18.5 Covariance matrices
- 18.6 Correlation to covariance
- 18.7 Code challenges
- 18.8 Code solutions

- 19 PCA
- 19.1 PCA: interps and apps
- 19.2 How to perform a PCA
- 19.3 The algebra of PCA
- 19.4 Regularization
- 19.5 Is PCA always the best?
- 19.6 Code challenges
- 19.7 Code solutions

- 20 The end.
- 20.1 The end... of the beginning!
- 20.2 Thanks!

### The Leanpub 45-day 100% Happiness Guarantee

Within **45 days of purchase** you can get a **100% refund** on any Leanpub purchase, in **two clicks**.

See full terms

### Do Well. Do Good.

#### Authors have earned$10,803,456writing, publishing and selling on Leanpub, earning **80% royalties** while saving up to **25 million pounds of CO2** and up to **46,000 trees**.

**Learn more about writing on Leanpub**

### Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers), EPUB (for phones and tablets) and MOBI (for Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

### Top Books

### Aprendiendo Git

Miguel Angel Durán GarcíaGit no es

*complicado*...**¡Si lo entiendes!**😜¿Sientes que sabes usarlo porque

**has memorizado todos los comandos que necesitas**? ¡Pero no entiendes qué hace cada cosa y*por qué*! Así es normal que, cuando exista un problema, te cueste resolverlo.¡Con este libro vas a entender de una vez por todas todo lo que es Git y cómo sacarle provecho!

### C++20

Rainer GrimmC++20 is the next big C++ standard after C++11. As C++11 did it, C++20 changes the way we program modern C++. This change is, in particular, due to the big four of C++20: ranges, coroutines, concepts, and modules.

### Functional Programming Made Easier

Charles ScalfaniA Functional Programming book from beginner to advanced without skipping a single step along the way.

In my 40 years of programming, I've felt that programming books always let me down, especially Functional Programming books. So, I wrote the book I wish I had 5 years ago.

Functional Programming will never be easy, but it can be easier.

### Introducing EventStorming

Alberto BrandoliniThe deepest tutorial and explanation about EventStorming, straight from the inventor.

### Windows 10 System Programming, Part 2

Pavel Yosifovich### Ansible for DevOps

Jeff GeerlingAnsible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.

### Thinking with Types

Sandy MaguireThis book aims to be

*the*comprehensive manual for type-level programming. It's about getting you from here to there---from a competent Haskell programmer to one who convinces the compiler to do their work for them.### node-opcua by example

Etienne RossignonGet the best out of node-opcua through a set of documented examples by the author himself that will allow you to create stunning OPCUA Servers or Clients.

### OpenIntro Statistics

David Diez, Christopher Barr, Mine Cetinkaya-Rundel, and OpenIntro### C++20 - The Complete Guide

Nicolai M. Josuttis

### Top Bundles

- #1
### Software Architecture for Developers: Volumes 1 & 2 - Technical leadership and communication

2 Books

"Software Architecture for Developers" is a practical and pragmatic guide to modern, lightweight software architecture, specifically aimed at developers. You'll learn:The essence of software architecture.Why the software architecture role should include coding, coaching and collaboration.The things that you really need to think about before... - #2
### CCIE Service Provider Ultimate Study Bundle

2 Books

Piotr Jablonski, Lukasz Bromirski, and Nick Russo have joined forces to deliver the only CCIE Service Provider training resource you'll ever need. This bundle contains a detailed and challenging collection of workbook labs, plus an extensively detailed technical reference guide. All of us have earned the CCIE Service Provider certification... - #3
### Modern C++ Collection

3 Books

Get All about Modern C++C++ Standard Library, including C++20Concurrency with Modern C++, including C++20C++20Each book has about 200 complete code examples. Updates are included. When I update one of the books, you immediately get the updated bundle. You can expect significant updates to each new C++ standard (C++23, C++26, .. ) and also... - #4
### The Python Craftsman

3 Books

The Python Craftsman series comprises The Python Apprentice, The Python Journeyman, and The Python Master. The first book is primarily suitable for for programmers with some experience of programming in another language. If you don't have any experience with programming this book may be a bit daunting. You'll be learning not just a programming... - #7
### 10 Books Bundle - A New Career in Tech 🚀

10 Books

Do you want to create yourself a new and profitable skillset - in one year or less? Do you want to position yourself on the right side of change finally? Do you want to work from home and set your own work schedule while choosing fun and interesting coding projects? Software developers and freelance developersearn six figuresin the US, according... - #9
### Pattern-Oriented Memory Forensics and Malware Detection

2 Books

This training bundle for security engineers and researchers, malware and memory forensics analysts includes two accelerated training courses for Windows memory dump analysis using WinDbg. It is also useful for technical support and escalation engineers who analyze memory dumps from complex software environments and need to check for possible...