Vectors & Vector Operations

A vector is an ordered list of numbers. Geometrically, it's an arrow pointing from the origin to a point in n-dimensional space.

3Blue1Brown's key insight: think of vectors as transformations, not just lists of numbers. Adding two vectors = following one arrow then the other. Scaling = stretching or flipping.

Basic operations:
• Addition: [a,b] + [c,d] = [a+c, b+d]  (tip-to-tail of arrows)
• Scalar multiplication: k·[a,b] = [ka, kb]  (stretch by k)
• Dot product: a·b = Σaᵢbᵢ = ||a||·||b||·cos(θ)

Dot product is the most important operation:
• Measures alignment between vectors
• a·b > 0 → point in same general direction
• a·b = 0 → perpendicular (orthogonal)
• a·b < 0 → point in opposite directions
• Normalized: cos(θ) = (a·b) / (||a||·||b||) → cosine similarity

Norms measure vector length:
• L1: ||x||₁ = Σ|xᵢ|
• L2: ||x||₂ = √(Σxᵢ²)  ← most common, Euclidean distance
• L∞: ||x||∞ = max|xᵢ|

1import numpy as np
2
3# ── Vector basics ────────────────────────────────────────────────
4a = np.array([2.0, 1.0, 3.0])
5b = np.array([1.0, 4.0, -1.0])
6
7print("Vector operations:")
8print(f"  a + b    = {a + b}")
9print(f"  a - b    = {a - b}")
10print(f"  3 * a    = {3 * a}")
11print(f"  a · b    = {np.dot(a, b)}")           # dot product
12
13# ── Norms ─────────────────────────────────────────────────────────
14print("
15Norms of a =", a)
16print(f"  L1 norm  ||a||₁ = {np.linalg.norm(a, ord=1):.4f}")
17print(f"  L2 norm  ||a||₂ = {np.linalg.norm(a, ord=2):.4f}")
18print(f"  L∞ norm  ||a||∞ = {np.linalg.norm(a, ord=np.inf):.4f}")
19
20# ── Dot product and angle ─────────────────────────────────────────
21cos_theta = np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
22angle_deg = np.degrees(np.arccos(np.clip(cos_theta, -1, 1)))
23print(f"
24Angle between a and b: {angle_deg:.2f}°")
25print(f"cos(θ) = {cos_theta:.4f}")
26
27# ── Cosine similarity (used in RAG, NLP) ─────────────────────────
28def cosine_similarity(u, v):
29    return np.dot(u, v) / (np.linalg.norm(u) * np.linalg.norm(v))
30
31# Sentence embeddings example
32embed_cat    = np.array([0.8, 0.6, 0.1, 0.2])
33embed_kitten = np.array([0.7, 0.7, 0.1, 0.1])
34embed_car    = np.array([0.1, 0.1, 0.9, 0.8])
35
36print("
37Cosine similarities (toy embeddings):")
38print(f"  cat vs kitten : {cosine_similarity(embed_cat, embed_kitten):.4f}")  # high
39print(f"  cat vs car    : {cosine_similarity(embed_cat, embed_car):.4f}")     # low
40
41# ── Unit vectors and projection ───────────────────────────────────
42a_unit = a / np.linalg.norm(a)   # normalize to unit vector
43print(f"
44Unit vector: {a_unit}")
45print(f"  ||unit|| = {np.linalg.norm(a_unit):.6f}")   # 1.0
46
47# Projection of b onto a
48proj = (np.dot(b, a) / np.dot(a, a)) * a
49print(f"
50Projection of b onto a: {proj}")
51
52# ── Linear combination & span ────────────────────────────────────
53# Any vector in R² can be written as c₁v₁ + c₂v₂ if v₁,v₂ are linearly independent
54v1 = np.array([1, 0])
55v2 = np.array([0, 1])
56target = np.array([3, 5])
57# target = 3*v1 + 5*v2 → coefficients are [3, 5]
58print(f"
59Linear combination: {target} = 3{v1} + 5{v2}: {np.allclose(3*v1 + 5*v2, target)}")