LINEAR ALGEBRA / 2. MATRICES & TRANSFORMATIONS

Matrices — Linear Transformations

3Blue1Brown: a matrix is a transformation of space

EXPLANATION

3Blue1Brown's core insight: don't think of a matrix as a grid of numbers. Think of it as a transformation — a function that takes vectors and moves them in space.

Every matrix encodes a linear transformation:
• Where does [1,0] (the x-hat vector) land?
• Where does [0,1] (the y-hat vector) land?
• The columns of the matrix tell you exactly this.

A 2×2 matrix [[a,b],[c,d]] means:
• x-hat [1,0] → [a,c]
• y-hat [0,1] → [b,d]
• Any vector [x,y] → x*[a,c] + y*[b,d]

Types of transformations:
• Identity matrix I → no change
• Scaling → stretch/compress along axes
• Rotation → rotate space
• Shear → slant space
• Projection → flatten onto a lower dimension

This visual understanding is critical for ML:
• Each neural network layer applies a matrix transformation
• W in y = Wx + b rotates and scales the input space
• Deep networks compose many transformations in sequence

DIAGRAM

Matrix as transformation (2D):

  Identity [[1,0],[0,1]]:    Rotation 90°: [[0,-1],[1,0]]:
  x-hat → [1,0]              x-hat → [0,1]
  y-hat → [0,1]              y-hat → [-1,0]
  Space unchanged            Space rotated 90°

  Scaling [[2,0],[0,3]]:     Shear [[1,1],[0,1]]:
  x stretched ×2             x-hat unchanged
  y stretched ×3             y-hat → [1,1]

  Reading column by column:
  A = [[a, b],    col1=[a,c]: where x-hat lands
       [c, d]]    col2=[b,d]: where y-hat lands

  Matrix × vector = applying the transformation:
  [[2,0],[0,3]] × [1,2] = [2,6]  (x scaled ×2, y scaled ×3)

CODE

PYTHON
1import numpy as np
2import matplotlib.pyplot as plt
3
4# ── Matrix as transformation ──────────────────────────────────────
5# 3B1B insight: columns tell you where basis vectors land
6I    = np.eye(2)                     # identity
7scale = np.array([[2, 0], [0, 3]])   # stretch x×2, y×3
8rot90 = np.array([[0, -1], [1, 0]])  # rotate 90°
9shear = np.array([[1, 1], [0, 1]])   # shear
10
11# Apply to a vector
12v = np.array([1, 2])
13print("Transformations of v=[1,2]:")
14print(f"  Identity:   {I @ v}")       # [1, 2]
15print(f"  Scale(2,3): {scale @ v}")   # [2, 6]
16print(f"  Rotate 90°: {rot90 @ v}")   # [-2, 1]
17print(f"  Shear:      {shear @ v}")   # [3, 2]
18
19# ── Reading columns = where basis vectors land ─────────────────────
20print("
21Column view of transformation matrix:")
22A = np.array([[3, 1], [-1, 2]])
23print(f"  A = {A}")
24print(f"  x-hat [1,0] → {A @ np.array([1,0])} (first column)")
25print(f"  y-hat [0,1] → {A @ np.array([0,1])} (second column)")
26
27# ── Matrix operations ─────────────────────────────────────────────
28A = np.array([[1, 2], [3, 4]])
29B = np.array([[5, 6], [7, 8]])
30
31print("
32Matrix operations:")
33print(f"  A + B =
34{A + B}")
35print(f"  A @ B =
36{A @ B}")     # matrix multiplication
37print(f"  Aᵀ    =
38{A.T}")       # transpose
39print(f"  2*A   =
40{2*A}")
41
42# ── Rank — number of independent dimensions ───────────────────────
43full_rank = np.array([[1, 0], [0, 1]])
44rank_1    = np.array([[1, 2], [2, 4]])   # row2 = 2*row1
45
46print(f"
47Rank of full rank matrix: {np.linalg.matrix_rank(full_rank)}")  # 2
48print(f"Rank of rank-1 matrix:    {np.linalg.matrix_rank(rank_1)}")      # 1
49# Rank-1 matrix collapses 2D space to a 1D line
50
51# ── Composing transformations ─────────────────────────────────────
52# Rotate then scale = scale @ rotate (applied right to left)
53rotate = np.array([[0, -1], [1, 0]])
54scale2 = np.array([[2, 0], [0, 2]])
55composed = scale2 @ rotate   # first rotate, then scale
56
57v_test = np.array([1, 0])
58print(f"
59Composed (rotate then scale) applied to [1,0]:")
60print(f"  Step 1 rotate: {rotate @ v_test}")
61print(f"  Step 2 scale:  {scale2 @ (rotate @ v_test)}")
62print(f"  Composed:      {composed @ v_test}")  # same result
← PREV1. Vectors & OperationsNEXT →3. Matrix Multiplication