Chi-Squared Test

The Chi-squared (χ²) test works on categorical data — counts and frequencies.

Two main uses:

1. Goodness of Fit test:
"Does my observed data fit the expected distribution?"
H₀: data follows the expected distribution
χ² = Σ (Observed - Expected)² / Expected

2. Test of Independence:
"Are two categorical variables related?"
H₀: the two variables are independent
χ² = Σ (Oᵢⱼ - Eᵢⱼ)² / Eᵢⱼ
where Eᵢⱼ = (row total × col total) / grand total

Degrees of freedom:
• Goodness of fit: df = k - 1 (k = number of categories)
• Independence: df = (rows-1) × (cols-1)

Large χ² → observed data is far from expected → evidence against H₀.

Rule of thumb: expected frequency in each cell should be ≥ 5 for the test to be valid.

In ML: used for feature selection — is this feature statistically associated with the target? sklearn's chi2 selector uses this.

1import numpy as np
2from scipy import stats
3import pandas as pd
4
5# ── Chi-squared Goodness of Fit ───────────────────────────────────
6# Is a die fair? Roll it 120 times
7observed = np.array([18, 22, 15, 20, 17, 28])
8expected = np.array([20, 20, 20, 20, 20, 20])   # uniform
9
10chi2_stat, p_val = stats.chisquare(observed, f_exp=expected)
11print("Goodness of Fit — Is the die fair?")
12print(f"  Observed: {observed}")
13print(f"  Expected: {expected}")
14print(f"  χ² = {chi2_stat:.4f}")
15print(f"  df = {len(observed)-1}")
16print(f"  p-value = {p_val:.4f}")
17print(f"  Decision: {'Reject H₀ (biased)' if p_val < 0.05 else 'Fail to reject (appears fair)'}")
18
19# Manual calculation
20chi2_manual = np.sum((observed - expected)**2 / expected)
21print(f"
22  Manual χ² = {chi2_manual:.4f}  (matches scipy)")
23
24# ── Chi-squared Test of Independence ─────────────────────────────
25# Are gender and preference independent?
26contingency = np.array([
27    [80, 10],   # row: contains "$"  → [spam, ham]
28    [20, 90],   # row: no "$"
29])
30
31chi2, p, dof, expected_freq = stats.chi2_contingency(contingency)
32print(f"
33Test of Independence — $ sign vs spam/ham:")
34print(f"  χ² = {chi2:.4f}")
35print(f"  df = {dof}")
36print(f"  p-value = {p:.6f}")
37print(f"  Expected frequencies:
38{expected_freq}")
39print(f"  Decision: {'Reject H₀ ($ and spam ARE related)' if p < 0.05 else 'Independent'}")
40
41# ── Chi-squared for feature selection (ML use case) ───────────────
42from sklearn.feature_selection import chi2 as sklearn_chi2
43from sklearn.datasets import load_iris
44from sklearn.preprocessing import MinMaxScaler
45
46iris    = load_iris()
47X, y    = iris.data, iris.target
48X_scaled = MinMaxScaler().fit_transform(X)   # chi2 needs non-negative
49
50chi2_scores, p_values = sklearn_chi2(X_scaled, y)
51print(f"
52Feature selection via Chi-squared (Iris):")
53for feat, score, p in zip(iris.feature_names, chi2_scores, p_values):
54    print(f"  {feat:25s}: χ²={score:.2f}, p={p:.6f}")
55# Higher chi2 = more associated with target = better feature