Linear Algebra: A Fundamental Approach

Full Table of Contents (500-page plan)

Part I: Vectors, Spaces, and Core Geometry

1. Vectors, Subspaces, and Projections

2. Linear Equations and Matrix Row Reduction

3. Determinants, Eigenvalues, and Diagonalization

Part II: Matrix Structure and Computation

4. Singular Value Decomposition and Least Squares

5. Orthogonality, Norms, and Best Approximations

6. Symmetric and Positive (Semi)Definite Matrices

Part III: Data Science and Learning Geometry

7. Feature Maps, Distances, and Kernels

8. PCA and Dimensionality Reduction

9. Classification with Linear Models

10. Regularization and Stability

Part IV: Cybersecurity and Signal Thinking

11. Linear Algebra for Cryptography Basics

12. Attacks via Linear Dependence and Rank

13. Error-Correcting Codes with Subspaces

14. Geometry of Detection and Filtering

Part V: Quantum Computing Foundations

15. Quantum States, Operators, and Linear Algebra

16. Entanglement as Subspace Structure

17. Measurement, Projections, and Decision Rules

18. Algorithms Built from Linear-Algebraic Building Blocks

Part VI: Advanced Connections and Research Threads

19. Tensor Products and Multilinear Structure

20. Random Matrices and Concentration Intuition

21. Open Problems and Modern Research Spotlight

---

Chapter 1: Vectors, Subspaces, and Projections

A good way to understand linear algebra is to stop treating vectors as “lists of numbers” and start treating them as directions that can be combined. Once you do that, many later topics - least squares, dimensionality reduction, even quantum measurement - become different views of the same geometry. Subspaces are where “directions stay closed under mixing,” and projections are the mechanism for splitting space into “what lies in the subspace” and “what points away from it.”

In this chapter you build that geometric picture carefully, and you also keep it honest with rigorous proofs. You will see how orthogonality (perpendicularity) is not just a picture detail: it turns “closest point” into a clean statement and makes projections behave predictably.

Core objects: vectors, spans, and subspaces

A vector space (often shortened to “space”) is a set of vectors where you can add vectors and scale them without leaving the set. In plain terms: if the vectors represent directions or signals, the space contains all mixtures you can form using those directions. Formally, a set V is a vector space over a field F if it is closed under addition and scalar multiplication and it satisfies the usual axioms (associativity, commutativity, distributivity, and existence of a zero vector and additive inverses). Ask yourself a quick check: if you take two allowed directions and mix them, do you still stay inside the allowed directions?

Now focus on how spaces are built. Given vectors v1, v2, ..., vk in a vector space, the span of these vectors is the set of all linear combinations:

c1 v1 + c2 v2 + ... + ck vk

where each ci is a scalar. The span is the smallest subspace containing the vectors. That “smallest” part matters because it gives you a recipe: if you want a subspace generated by certain directions, span is the construction.

A subspace is a subset W of a vector space V that is itself a vector space under the same operations. So to prove something is a subspace, you do not need every axiom from scratch. Instead, you typically check closure properties and the presence of the zero vector (which is forced by closure). A common workflow is: verify that the zero vector is in W, that u and v in W implies u + v is in W, and that u in W and scalar c implies c u is in W.

One concrete way to anchor this is to think of points in R2 or R3. In R2, a subspace is either the zero vector alone, a line through the origin, or all of R2. In R3, a subspace is either {0}, a line through the origin, a plane through the origin, or all of R3. These are exactly the shapes you get when you take spans of one vector, two non-parallel vectors, or three vectors that generate the whole space.

[INSERT FIGURE: 2D geometric representation of a span and a subspace in R^2; show a line through the origin and the set of all scalar multiples]

Practical takeaway: whenever you see a statement like “the set of all mixtures of these vectors,” translate it into “span,” and whenever you see “subset that is closed under mixing,” translate it into “subspace.”

Orthogonality and orthogonal complements: separating “in” from “out”

To talk about projections, you need orthogonality. Two vectors u and w are orthogonal if their dot product is zero. In Rn with the standard dot product, u · w = 0 means they point in perpendicular directions. This gives you a way to define “perpendicular to a whole subspace,” not just to a single vector.

...