MACHINE LEARNING / 7. CLUSTERING

Clustering

Finding structure in unlabeled data

EXPLANATION

Clustering groups similar data points together without any labels. Unsupervised learning — you don't tell the algorithm what the groups are.

K-Means:
• Pick K centroids randomly
• Assign each point to nearest centroid
• Move centroids to mean of assigned points
• Repeat until convergence
• Problem: you must choose K, sensitive to initialization, assumes spherical clusters

DBSCAN (Density-Based):
• Groups points that are close together (dense regions)
• Points in sparse regions = noise/outliers
• No need to specify K
• Handles arbitrary cluster shapes and outliers

Choosing K for K-Means:
• Elbow method: plot inertia vs K, pick the elbow
• Silhouette score: measures how similar a point is to its cluster vs others. Range (-1, 1), higher is better

Use cases: customer segmentation, document grouping, anomaly detection, data compression.

DATA FLOW

K-Means (K=3):

  Iteration 0: random centroids ✕
  ● ● ✕      ○ ○       ✕ ■ ■
  ● ●    ○ ✕ ○ ○      ■ ■ ■

  Iteration N (converged): centroids at cluster centers
  ● ● ✕      ○ ○       ■ ■ ✕
  ● ●    ○ ○ ✕ ○      ■ ■ ■

  DBSCAN:
  Core point: has min_samples within eps radius
  Border point: within eps of core but not core itself
  Noise point: isolated, not within eps of any core → outlier

CODE

PYTHON
1import numpy as np
2from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
3from sklearn.preprocessing import StandardScaler
4from sklearn.metrics import silhouette_score
5from sklearn.datasets import make_blobs, make_moons
6import matplotlib.pyplot as plt
7
8# ── K-Means ───────────────────────────────────────────────────────
9X, y_true = make_blobs(n_samples=500, centers=4, random_state=42)
10
11kmeans = KMeans(
12    n_clusters=4,
13    init="k-means++",     # smart initialization (not random)
14    n_init=10,            # run 10 times, keep best
15    max_iter=300,
16    random_state=42,
17)
18kmeans.fit(X)
19labels    = kmeans.labels_
20centroids = kmeans.cluster_centers_
21
22print(f"Inertia       : {kmeans.inertia_:.2f}")   # within-cluster sum of squares
23print(f"Silhouette    : {silhouette_score(X, labels):.4f}")
24
25# ── Elbow method to find optimal K ────────────────────────────────
26inertias    = []
27silhouettes = []
28K_range = range(2, 10)
29
30for k in K_range:
31    km = KMeans(n_clusters=k, n_init=10, random_state=42)
32    km.fit(X)
33    inertias.append(km.inertia_)
34    silhouettes.append(silhouette_score(X, km.labels_))
35
36# Best K: elbow in inertia OR peak in silhouette score
37
38# ── DBSCAN (handles non-spherical clusters) ───────────────────────
39X_moons, _ = make_moons(n_samples=400, noise=0.05)
40X_moons    = StandardScaler().fit_transform(X_moons)
41
42dbscan = DBSCAN(
43    eps=0.2,             # neighborhood radius
44    min_samples=5,       # min points to form a core point
45)
46labels_db = dbscan.fit_predict(X_moons)
47
48n_clusters = len(set(labels_db)) - (1 if -1 in labels_db else 0)
49n_noise    = (labels_db == -1).sum()
50print(f"\nDBSCAN clusters: {n_clusters}, noise points: {n_noise}")
51# -1 label = outlier/noise
← PREV6. SVMNEXT →8. PCA & Dimensionality Reduction