###### Linear Algebra: Theory, Intuition, Code

### Linear Algebra: Theory, Intuition, Code

###### Wholesome mathy goodness for everyone.

# About the Book

Linear algebra is perhaps the most important branch of mathematics for computational sciences, including machine learning, AI, data science, statistics, simulations, computer graphics, multivariate analyses, matrix decompositions, signal processing, and so on.

The way linear algebra is presented in traditional textbooks is different from how professionals use linear algebra in computers to solve real-world applications in machine learning, data science, statistics, and signal processing. For example, the "determinant" of a matrix is important for linear algebra theory, but should you actually use the determinant in practical applications? The answer may surprise you!

If you are interested in learning the mathematical concepts linear algebra and matrix analysis, but also want to apply those concepts to data analyses on computers (e.g., statistics or signal processing), then this book is for you. You'll see all the math concepts implemented in MATLAB and in Python.

Unique aspects of this book:

- Clear and comprehensible explanations of concepts and theories in linear algebra.

- Several distinct explanations of the same ideas, which is a proven technique for learning.

- Visualization using graphs, which strengthens the geometric intuition of linear algebra.

- Implementations in MATLAB and Python. Com'on, in the real world, you never solve math problems by hand! You need to know how to implement math in software!

- Beginner to intermediate topics, including vectors, matrix multiplications, least-squares projections, eigendecomposition, and singular-value decomposition.

- Strong focus on modern applications-oriented aspects of linear algebra and matrix analysis.

- Intuitive visual explanations of diagonalization, eigenvalues and eigenvectors, and singular value decomposition.

- Codes (MATLAB and Python) are provided to help you understand and apply linear algebra concepts on computers.

- A combination of hand-solved exercises and more advanced code challenges. Math is not a spectator sport!

#### Table of Contents

- 0.1 Front matter
- 0.2 Dedication
- 0.3 Forward
- 1 Introduction
- 1.1 What is linear algebra and why learn it?
- 1.2 About this book
- 1.3 Prerequisites
- 1.4 Exercises and code challenges
- 1.5 Online and other resources

- 2 Vectors
- 2.1 Scalars
- 2.2 Vectors: geometry and algebra
- 2.3 Transpose operation
- 2.4 Vector addition and subtraction
- 2.5 Vector-scalar multiplication
- 2.6 Exercises
- 2.7 Answers
- 2.8 Code challenges
- 2.9 Code solutions

- 3 Vector multiplication
- 3.1 Vector dot product: Algebra
- 3.2 Dot product properties
- 3.3 Vector dot product: Geometry
- 3.4 Algebra and geometry
- 3.5 Linear weighted combination
- 3.6 The outer product
- 3.7 Hadamard multiplication
- 3.8 Cross product
- 3.9 Unit vectors
- 3.10 Exercises
- 3.11 Answers
- 3.12 Code challenges
- 3.13 Code solutions

- 4 Vector spaces
- 4.1 Dimensions and fields
- 4.2 Vector spaces
- 4.3 Subspaces and ambient spaces
- 4.4 Subsets
- 4.5 Span
- 4.6 Linear independence
- 4.7 Basis
- 4.8 Exercises
- 4.9 Answers

- 5 Matrices
- 5.1 Interpretations and uses of matrices
- 5.2 Matrix terms and notation
- 5.3 Matrix dimensionalities
- 5.4 The transpose operation
- 5.5 Matrix zoology
- 5.6 Matrix addition and subtraction
- 5.7 Scalar-matrix mult.
- 5.8 "Shifting" a matrix
- 5.9 Diagonal and trace
- 5.10 Exercises
- 5.11 Answers
- 5.12 Code challenges
- 5.13 Code solutions

- 6 Matrix multiplication
- 6.1 "Standard" multiplication
- 6.2 Multiplication and eqns.
- 6.3 Multiplication with diagonals
- 6.4 LIVE EVIL
- 6.5 Matrix-vector multiplication
- 6.6 Creating symmetric matrices
- 6.7 Multiply symmetric matrices
- 6.8 Hadamard multiplication
- 6.9 Frobenius dot product
- 6.10 Matrix norms
- 6.11 What about matrix division?
- 6.12 Exercises
- 6.13 Answers
- 6.14 Code challenges
- 6.15 Code solutions

- 7 Rank
- 7.1 Six things about matrix rank
- 7.2 Interpretations of matrix rank
- 7.3 Computing matrix rank
- 7.4 Rank and scalar multiplication
- 7.5 Rank of added matrices
- 7.6 Rank of multiplied matrices
- 7.7 Rank of A, A', A'A, and AA'
- 7.8 Rank of random matrices
- 7.9 Boosting rank by "shifting"
- 7.10 Rank difficulties
- 7.11 Rank and span
- 7.12 Exercises
- 7.13 Answers
- 7.14 Code challenges
- 7.15 Code solutions

- 8 Matrix spaces
- 8.1 Column space of a matrix
- 8.2 Column space: A AA'
- 8.3 Determining whether v is in C(A)
- 8.4 Row space of a matrix
- 8.5 Row spaces of A'A and A
- 8.6 Null space of a matrix
- 8.7 Geometry of the null space
- 8.8 Orthogonal subspaces
- 8.9 Matrix space orthogonalities
- 8.10 Dimensionalities of matrix spaces
- 8.11 More on Ax=b and Ay=0
- 8.12 Exercises
- 8.13 Answers
- 8.14 Code challenges
- 8.15 Code solutions

- 9 Complex numbers
- 9.1 Complex numbers
- 9.2 What are complex numbers?
- 9.3 The complex conjugate
- 9.4 Complex arithmetic
- 9.5 Complex dot product
- 9.6 Special complex matrices
- 9.7 Exercises
- 9.8 Answers
- 9.9 Code challenges
- 9.10 Code solutions

- 10 Systems of equations
- 10.1 Algebra and geometry of eqns.
- 10.2 From systems to matrices
- 10.3 Row reduction
- 10.4 Gaussian elimination
- 10.5 Row-reduced echelon form
- 10.6 Gauss-Jordan elimination
- 10.7 Possibilities for solutions
- 10.8 Matrix spaces, row reduction
- 10.9 Exercises
- 10.10 Answers
- 10.11 Coding challenges
- 10.12 Code solutions

- 11 Determinant
- 11.1 Features of determinants
- 11.2 Determinant of a 2x2 matrix
- 11.3 The characteristic polynomial
- 11.4 3x3 matrix determinant
- 11.5 The full procedure
- 11.6 Delta of triangles
- 11.7 Determinant and row reduction
- 11.8 Delta and scalar multiplication
- 11.9 Theory vs practice
- 11.10 Exercises
- 11.11 Answers
- 11.12 Code challenges
- 11.13 Code solutions

- 12 Matrix inverse
- 12.1 Concepts and applications
- 12.2 Inverse of a diagonal matrix
- 12.3 Inverse of a 2x2 matrix
- 12.4 The MCA algorithm
- 12.5 Inverse via row reduction
- 12.6 Left inverse
- 12.7 Right inverse
- 12.8 The pseudoinverse, part 1
- 12.9 Exercises
- 12.10 Answers
- 12.11 Code challenges
- 12.12 Code solutions

- 13 Projections
- 13.1 Projections in R2
- 13.2 Projections in RN
- 13.3 Orth and par vect comps
- 13.4 Orthogonal matrices
- 13.5 Orthogonalization via GS
- 13.6 QR decomposition
- 13.7 Inverse via QR
- 13.8 Exercises
- 13.9 Answers
- 13.10 Code challenges
- 13.11 Code solutions

- 14 Least-squares
- 14.1 Introduction
- 14.2 5 steps of model-fitting
- 14.3 Terminology
- 14.4 Least-squares via left inverse
- 14.5 Least-squares via projection
- 14.6 Least-squares via row-reduction
- 14.7 Predictions and residuals
- 14.8 Least-squares example
- 14.9 Code challenges
- 14.10 Code solutions

- 15 Eigendecomposition
- 15.1 Eigenwhatnow?
- 15.2 Finding eigenvalues
- 15.3 Finding eigenvectors
- 15.4 Diagonalization
- 15.5 Conditions for diagonalization
- 15.6 Distinct, repeated eigenvalues
- 15.7 Complex solutions
- 15.8 Symmetric matrices
- 15.9 Eigenvalues singular matrices
- 15.10 Eigenlayers of a matrix
- 15.11 Matrix powers and inverse
- 15.12 Generalized eigendecomposition
- 15.13 Exercises
- 15.14 Answers
- 15.15 Code challenges
- 15.16 Code solutions

- 16 The SVD
- 16.1 Singular value decomposition
- 16.2 Computing the SVD
- 16.3 Singular values and eigenvalues
- 16.4 SVD of a symmetric matrix
- 16.5 SVD and the four subspaces
- 16.6 SVD and matrix rank
- 16.7 SVD spectral theory
- 16.8 Low-rank approximations
- 16.9 Normalizing singular values
- 16.10 Condition number of a matrix
- 16.11 SVD and the matrix inverse
- 16.12 MP Pseudoinverse, part 2
- 16.13 Code challenges
- 16.14 Code solutions

- 17 Quadratic form
- 17.1 Algebraic perspective
- 17.2 Geometric perspective
- 17.3 The normalized quadratic form
- 17.4 Evecs and the qf surface
- 17.5 Matrix definiteness
- 17.6 The definiteness of A'A
- 17.7 Eigenvalues and definiteness
- 17.8 Code challenges
- 17.9 Code solutions

- 18 Covariance matrices
- 18.1 Correlation
- 18.2 Variance and standard deviation
- 18.3 Covariance
- 18.4 Correlation coefficient
- 18.5 Covariance matrices
- 18.6 Correlation to covariance
- 18.7 Code challenges
- 18.8 Code solutions

- 19 PCA
- 19.1 PCA: interps and apps
- 19.2 How to perform a PCA
- 19.3 The algebra of PCA
- 19.4 Regularization
- 19.5 Is PCA always the best?
- 19.6 Code challenges
- 19.7 Code solutions

- 20 The end.
- 20.1 The end... of the beginning!
- 20.2 Thanks!

### The Leanpub 60-day 100% Happiness Guarantee

Within **60 days of purchase** you can get a **100% refund** on any Leanpub purchase, in **two clicks**.

See full terms

### Do Well. Do Good.

#### Authors have earned$11,955,464writing, publishing and selling on Leanpub, earning **80% royalties** while saving up to **25 million pounds of CO2** and up to **46,000 trees**.

**Learn more about writing on Leanpub**

### Free Updates. DRM Free.

If you buy a Leanpub book, you get free updates for as long as the author updates the book! Many authors use Leanpub to publish their books in-progress, while they are writing them. All readers get free updates, regardless of when they bought the book or how much they paid (including free).

Most Leanpub books are available in PDF (for computers) and EPUB (for phones, tablets and Kindle). The formats that a book includes are shown at the top right corner of this page.

Finally, Leanpub books don't have any DRM copy-protection nonsense, so you can easily read them on any supported device.

Learn more about Leanpub's ebook formats and where to read them

### Top Books

### OpenIntro Statistics

David Diez, Christopher Barr, Mine Cetinkaya-Rundel, and OpenIntro### Deep Learning with PyTorch Step-by-Step

Daniel Voigt GodoyIn 2019, I published a PyTorch tutorial on Towards Data Science and I was amazed by the reaction from the readers! Their feedback motivated me to write this book to help beginners start their journey into Deep Learning and PyTorch. I hope you enjoy reading this book as much as I enjoy writing it.

### Linear Algebra

Elira Curri and Tanush ShaskaAnswers to selected problems included.

### Modern IT Automation with PowerShell

The DevOps Collective, Inc. and Michael ZanattaA PowerShell Textbook written by the community for the community!

### Ansible for DevOps

Jeff GeerlingAnsible is a simple, but powerful, server and configuration management tool. Learn to use Ansible effectively, whether you manage one server—or thousands.

### R Programming for Data Science

Roger D. PengThis book brings the fundamentals of R programming to you, using the same material developed as part of the industry-leading Johns Hopkins Data Science Specialization. The skills taught in this book will lay the foundation for you to begin your journey learning data science. Printed copies of this book are available through Lulu.

### Code Your Own Synth Plug-Ins With C++ and JUCE

Matthijs HollemansLearn the fundamentals of audio programming by building a fully-featured software synthesizer plug-in, with every step explained along the way. The plug-in can be used in all the popular DAWs and is made using the industry standard tools for audio development: JUCE framework and the C++ programming language. Not too much math, lots of explanations!

### Introducing EventStorming

Alberto BrandoliniThe deepest tutorial and explanation about EventStorming, straight from the inventor.

### Software Architecture for Developers

Simon BrownA developer-friendly, practical and pragmatic guide to lightweight software architecture, technical leadership and the balance with agility.

### The Rails 7 Way

Obie Fernandez, Lucas Dohmen, and Tom Henrik AadlandThe Rails™ 7 Way is the comprehensive, authoritative reference guide for professionals delivering production-quality code using modern Ruby on Rails. It illuminates the entire Rails 7 API, its most powerful idioms, design approaches, and libraries. Building on the previous editions, this edition has been heavily refactored and updated.

### Top Bundles

- #1
### Software Architecture

2 Books

"Software Architecture for Developers" is a practical and pragmatic guide to modern, lightweight software architecture, specifically aimed at developers. You'll learn:The essence of software architecture.Why the software architecture role should include coding, coaching and collaboration.The things that you really need to think about before... - #2
### CCIE Service Provider Ultimate Study Bundle

2 Books

Piotr Jablonski, Lukasz Bromirski, and Nick Russo have joined forces to deliver the only CCIE Service Provider training resource you'll ever need. This bundle contains a detailed and challenging collection of workbook labs, plus an extensively detailed technical reference guide. All of us have earned the CCIE Service Provider certification... - #3
### Pattern-Oriented Memory Forensics and Malware Detection

2 Books

This training bundle for security engineers and researchers, malware and memory forensics analysts includes two accelerated training courses for Windows memory dump analysis using WinDbg. It is also useful for technical support and escalation engineers who analyze memory dumps from complex software environments and need to check for possible... - #4
### Static Analysis and Automated Refactoring

2 Books

As PHP developers we are living in the "Age of Static Analysis". We can use a tool like PHPStan to learn about potential bugs before we ship our code to production, and we can enforce our team's programming standards using custom PHPStan rules. Recipes for Decoupling by Matthias Noback teaches you in great detail how to do this, while also... - #5
### Practical FP in Scala + Functional event-driven architecture

2 Books

Practical FP in Scala (A hands-on approach) & Functional event-driven architecture, aka FEDA, (Powered by Scala 3), together as a bundle! The content of PFP in Scala is a requirement to understand FEDA so why not take advantage of this bundle!? - #6
### Modern C++ Collection

3 Books

Get All about Modern C++C++ Standard Library, including C++20Concurrency with Modern C++, including C++20C++20Each book has about 200 complete code examples. Updates are included. When I update one of the books, you immediately get the updated bundle. You can expect significant updates to each new C++ standard (C++23, C++26, .. ) and also... - #7
### All the Books of The Medical Futurist

6 Books

We put together the most popular books from The Medical Futurist to provide a clear picture about the major trends shaping the future of medicine and healthcare. Digital health technologies, artificial intelligence, the future of 20 medical specialties, big pharma, data privacy, digital health investments and how technology giants such as Amazon... - #9
### Statistics - All Lee Baker's Books

10 Books

This bundle includes all of the statistics books that Lee Baker has published at LeanPub 13 Statistics Books*1 PlaceNo Hassle! All in both epub and mobi formats - perfect for all devices and screen sizes. --------------------------------------------------------- *The book listed as Getting Started With Statistics is a bundle of 4 books...