Matrices and Transformations
A matrix is not a grid of numbers — it is a transformation of space. This topic builds geometric intuition for what matrices do to vectors and why the columns of a matrix tell you everything about its effect. The machinery here is at the heart of every linear layer in a neural network.
- Vectors Required — Matrices act on vectors — you need to know what a vector is first
What Does a Matrix Do?
A 2×2 matrix is a recipe for transforming space. Every matrix rotates, scales, shears, or reflects — and the geometry of what it does is encoded directly in its columns. This chapter builds the intuition for matrix-vector multiplication before introducing the four fundamental transformation types and connecting them to the weight matrices at the heart of every neural network.
Matrix Multiplication as Composition
Two matrices can be combined into a single matrix that performs both transformations in sequence. This operation — matrix multiplication — is not commutative: AB and BA describe different sequences and generally produce different results. Understanding composition is the key to understanding deep neural networks, where each layer applies a matrix and the sequence of layers defines a pipeline of transformations.
Eigenvectors — Directions That Don't Rotate
Most vectors change direction when a matrix is applied to them. Eigenvectors are the exception: they stay on the same line through the origin, only changing in length. The amount of stretching is the eigenvalue. These special directions reveal the skeleton of a transformation and underpin the most important dimensionality-reduction algorithm in data science: PCA.