Determinants

3Blue1Brown's insight: the determinant of a matrix is the factor by which the transformation scales areas (2D) or volumes (3D).

det(A) = 2 → transformation doubles all areas
det(A) = 0.5 → transformation halves all areas
det(A) = 0 → transformation squishes space to a lower dimension (a line, or a point)
det(A) < 0 → transformation flips orientation of space

det(A) = 0 means:
• The matrix is singular (not invertible)
• Columns are linearly dependent
• The transformation collapses space to lower dimensions
• The system Ax = b may have no solution or infinitely many

For 2×2 matrix [[a,b],[c,d]]:
det = ad - bc

Geometric meaning: the parallelogram formed by the two column vectors has area |det|.

In ML:
• det = 0 → loss of information (bad!)
• Condition number = σ_max/σ_min → relates to det via singular values
• Log-determinant appears in multivariate Gaussians: log P(x) = -0.5 log det(Σ) + ...

1import numpy as np
2
3# ── Determinant basics ────────────────────────────────────────────
4A = np.array([[3., 1.], [0., 2.]])
5B = np.array([[2., 4.], [1., 2.]])   # singular
6C = np.array([[1., 2., 3.], [4., 5., 6.], [7., 8., 9.]])  # rank-deficient
7
8print("Determinants:")
9print(f"  det(A) = {np.linalg.det(A):.4f}")   # 6 (doubles area)
10print(f"  det(B) = {np.linalg.det(B):.4f}")   # 0 (singular!)
11print(f"  det(C) = {np.linalg.det(C):.6f}")   # ~0 (rank-deficient)
12
13# 2x2 manual: ad - bc
14a, b, c, d = 3, 1, 0, 2
15manual_det = a*d - b*c
16print(f"
17Manual det([[3,1],[0,2]]) = {a}*{d} - {b}*{c} = {manual_det}")
18
19# ── Geometric interpretation ──────────────────────────────────────
20# Unit square area = 1
21# After transformation by A, area = |det(A)|
22unit_square_area = 1.0
23transformed_area = abs(np.linalg.det(A)) * unit_square_area
24print(f"
25Unit square area: {unit_square_area}")
26print(f"After transform by A (det={np.linalg.det(A):.1f}): area = {transformed_area}")
27
28# ── Det for checking invertibility ────────────────────────────────
29matrices = {
30    "identity":         np.eye(3),
31    "scaling":          np.diag([2., 3., 4.]),
32    "singular":         np.array([[1.,2.,3.],[4.,5.,6.],[7.,8.,9.]]),
33    "nearly singular":  np.array([[1.,0.,0.],[0.,1.,0.],[0.,0.,1e-10]]),
34}
35
36print("
37Invertibility check:")
38for name, M in matrices.items():
39    d = np.linalg.det(M)
40    invertible = abs(d) > 1e-10
41    print(f"  {name:20s}: det={d:12.6f} → {'invertible' if invertible else 'SINGULAR'}")
42
43# ── Determinant properties ────────────────────────────────────────
44A = np.random.randn(3, 3)
45B = np.random.randn(3, 3)
46
47# det(AB) = det(A) * det(B)
48print(f"
49det(A)*det(B) = {np.linalg.det(A)*np.linalg.det(B):.6f}")
50print(f"det(A@B)      = {np.linalg.det(A@B):.6f}")  # same
51
52# det(Aᵀ) = det(A)
53print(f"
54det(A)  = {np.linalg.det(A):.6f}")
55print(f"det(Aᵀ) = {np.linalg.det(A.T):.6f}")  # same