CALCULUS / OVERVIEW

Calculus — The Full Map

The mathematics of change — essential for understanding ML optimization

EXPLANATION

Calculus is the mathematics of change and accumulation. For machine learning, it is not optional — every training algorithm is calculus in disguise.

Why calculus matters for ML:
• Gradient descent → follows the negative gradient of the loss function
• Backpropagation → chain rule applied recursively through a computation graph
• Loss surfaces → understanding minima, maxima, saddle points
• Optimization → knowing when and why algorithms converge

Two branches:
• Differential calculus → rates of change, derivatives, gradients
• Integral calculus → accumulation, areas, expectations in probability

The key insight: a derivative tells you the slope at a point — which direction makes the function increase fastest. Gradient descent goes the opposite direction to minimize loss.

This appendix focuses on differential calculus — the branch directly used in deep learning. We follow Sebastian Raschka's notation from "Introduction to Artificial Neural Networks and Deep Learning."

DIAGRAM

DIFFERENTIAL CALCULUS        INTEGRAL CALCULUS
  ─────────────────────────    ──────────────────────────
  Limits & continuity           Antiderivatives
  Derivatives                   Definite integrals
  Differentiation rules         Area under curve
  Partial derivatives           Expected value (probability)
  Gradients ← ML critical       Fundamental theorem
  Chain rule ← backprop
  Hessian ← 2nd order opt
  ─────────────────────────
  ML connection:
  f(x) = loss function
  f'(x) = gradient
  x = x - lr * f'(x) ← gradient descent step

CODE

PYTHON

1import numpy as np
2import matplotlib.pyplot as plt
3from sympy import *    # symbolic math
4
5x = symbols('x')
6
7# The core question calculus answers:
8# "How fast is this function changing at this exact point?"
9
10f = x**3 - 2*x**2 + x
11df = diff(f, x)           # derivative: 3x^2 - 4x + 1
12print(f"f(x)  = {f}")
13print(f"f'(x) = {df}")
14print(f"f'(2) = {df.subs(x, 2)}")   # slope at x=2
15
16# Gradient descent in 1D — pure calculus in action
17def loss(w):     return (w - 3)**2         # minimum at w=3
18def grad(w):     return 2 * (w - 3)        # derivative of loss
19
20w, lr = 0.0, 0.1
21for step in range(20):
22    g = grad(w)
23    w = w - lr * g
24    print(f"step {step:2d}: w={w:.4f}, loss={loss(w):.6f}")

NEXT →1. Limits & Continuity