Start here: notation and Python: Quick recap

01 How to learn here

Recognition (it looks familiar) is not recall (you can produce it). Re-reading trains recognition and creates an illusion of fluency.
Retrieval practice (answering from memory) and spacing (study spread over days) are the two study methods with the strongest evidence (Dunlosky et al., 2013).
Pretesting (guess before you learn), interleaving (mix problem types) and desirable difficulties (effortful but successful recall) also improve long-term memory.
Simplified forgetting model: $R(t) = e^{-(t - t_0)/S}$ ; a review that succeeds multiplies the stability $S$ by more when recall was harder; a review that fails starts over.
This site's scheduler (an SM-2 variant): a card rated Good every time returns after 1, 3, 8, 20, 50, 125 days; Again brings it back in 10 minutes and restarts it at 1 day with a lower ease.
The review deck lives only in your browser; save it to a file to move it or keep a copy.
An active quick look: cover the summary, recall, then compare. Daily: review first (10–15 minutes), then new material.

02 Reading math notation

A set has no order and no repeats. $x \in A$ : element; $A \subseteq B$ : subset; $\cup$ (or), $\cap$ (and), $\setminus$ (without); $\lvert A \rvert$ : size; $\{x \in \mathbb{R} : x > 0\}$ = "all real $x$ such that $x > 0$ ".
$f: X \to Y$ gives each input exactly one output. The range is the set of outputs that occur. $(f \circ g)(x) = f(g(x))$ : $g$ acts first. $f(x; \theta)$ : input $x$ , parameters $\theta$ .
$X \in \mathbb{R}^{m \times n}$ : $m$ rows (samples), $n$ columns (features). $X_{ij} = x^{(i)}_j$ is Python X[i-1, j-1]. $x^{(i)}$ is sample $i$ , not a power.
$\sum_{i=1}^{n} a_i$ is a loop that adds the terms. Rules: $\sum (c\,a_i + b_i) = c \sum a_i + \sum b_i$ ; $\sum_{i=1}^{n} c = nc$ ; $\sum_i \sum_j a_i b_j = (\sum_i a_i)(\sum_j b_j)$ , but $\sum_i a_i b_i \ne (\sum a_i)(\sum b_i)$ in general.
$\sum_{i=1}^{n} i = n(n+1)/2$ . The empty sum is 0 and the empty product is 1; $\log \prod_i a_i = \sum_i \log a_i$ for $a_i > 0$ .
$\max_x f(x)$ is a value; $\arg\max_x f(x)$ is the set of places where it is reached (it can have several elements). Training: $\theta^\star = \arg\min_\theta L(\theta)$ .
Context decides the meaning of a symbol: $\sigma$ (standard deviation or sigmoid), $\Sigma$ (sum or covariance matrix), $\lambda$ (regularization or eigenvalue), $\lvert \cdot \rvert$ (absolute value or set size).

03 Python and NumPy for math

A NumPy array has a shape (the length of each axis), a dtype and ndim. $x \in \R^n$ is shape (n,); $A \in \R^{m \times n}$ is shape (m, n); math index $x_i$ is x[i-1].
Slices a[i:j] exclude j; negative indices count from the end. An integer index removes an axis, a slice keeps it. Slices are views that share memory; masks return copies; use .copy() to be safe.
A reduction with axis=k removes axis k ((m, n) → (n,) for axis=0); keepdims=True keeps it with length 1.
Broadcasting: align shapes on the right; sizes must be equal or one of them 1 (missing counts as 1); size-1 dimensions are stretched. (n,) with (n, 1) silently gives (n, n).
* is elementwise; @ is the matrix product, (m, p) @ (p, n) → (m, n); for 1-D arrays x @ y is the dot product $\sum_i x_i y_i$ .
np.arange(a, b, step) excludes b; np.linspace(a, b, n) includes b and gives n points.
rng = np.random.default_rng(seed) makes random numbers reproducible: the same seed gives the same sequence.
Floats are not exact: compare with np.isclose/np.allclose; np.exp overflows to inf above about 709.78.

04 Lines, powers, exp and log

The line through $(x_1, y_1)$ and $(x_2, y_2)$ has slope $w = (y_2 - y_1)/(x_2 - x_1)$ and intercept $b = y_1 - w x_1$ ; in ML, $\hat y = wx + b$ has weight $w$ and bias $b$ .
$a\,f(c(x - h)) + k$ moves the graph of $f$ right by $h$ and up by $k$ , stretches it vertically by $a$ and squeezes it horizontally by $c$ .
Power rules: $a^m a^n = a^{m+n}$ , $(a^m)^n = a^{mn}$ , $a^0 = 1$ , $a^{-n} = 1/a^n$ ; for the exponential, $e^{a+b} = e^a e^b$ and $e^x > 0$ for every $x$ .
$\log_b y$ is the power $x$ with $b^x = y$ ; it is defined only for $y > 0$ . In ML, $\log$ means the natural log (base $e$ ).
Log rules: $\log(ab) = \log a + \log b$ , $\log(a^k) = k\log a$ , $\log_b y = \log y / \log b$ . But $\log(a + b) \ne \log a + \log b$ .
$\log$ is strictly increasing, so $\arg\max_\theta \log L(\theta) = \arg\max_\theta L(\theta)$ ; and $\log \prod_i p_i = \sum_i \log p_i$ avoids underflow.
Exponential data are straight on a semilog plot (slope $\log b$ ); a power law $y = ax^k$ is straight on a log-log plot with slope $k$ .
Every exponential $b^x$ with $b > 1$ eventually beats every power $x^k$ ; $2^x$ passes $x^{10}$ only at $x \approx 58.77$ .

05 Sigmoid, softmax, log-sum-exp

The sigmoid $\sigma(z) = 1/(1 + e^{-z})$ maps a real score to $(0, 1)$ , with $\sigma(0) = 1/2$ and $\sigma(-z) = 1 - \sigma(z)$ .
The inverse of the sigmoid is the logit, $\log\frac{p}{1-p}$ (the log-odds); the slope is $\sigma'(z) = \sigma(z)(1 - \sigma(z)) \le 1/4$ , and it is tiny for large $|z|$ (saturation).
$\operatorname{softmax}(z)_k = e^{z_k}/\sum_j e^{z_j}$ gives positive entries that sum to 1 and keeps the order of the logits.
Softmax is shift invariant: $\operatorname{softmax}(z + c\mathbf 1) = \operatorname{softmax}(z)$ ; with two classes, $\operatorname{softmax}(z_1, z_2)_1 = \sigma(z_1 - z_2)$ .
Temperature: $\operatorname{softmax}(z/T)$ tends to a one-hot vector at the argmax as $T \to 0$ and to the uniform vector as $T \to \infty$ .
$\operatorname{LSE}(z) = \log\sum_j e^{z_j}$ satisfies $\max_j z_j \le \operatorname{LSE}(z) \le \max_j z_j + \log n$ , and $\log\operatorname{softmax}(z)_k = z_k - \operatorname{LSE}(z)$ .
The cross-entropy loss from logits is $\operatorname{LSE}(z) - z_y$ .
Stable code subtracts $m = \max_j z_j$ : $\operatorname{LSE}(z) = m + \log\sum_j e^{z_j - m}$ ; in float64, $e^z$ overflows above about 709.78 (float32: 88.72).

06 A gentle tour of vectors

A vector $x \in \R^n$ is a list of $n$ numbers. In 2-D and 3-D it is also an arrow from the origin; in higher dimensions we compute with the list.
Addition and scalar multiplication work entry by entry. As arrows, addition is tip to tail, and $u - v$ goes from the tip of $v$ to the tip of $u$ .
The norm is $\|x\| = \sqrt{x_1^2 + \dots + x_n^2}$ (Pythagoras). $x/\|x\|$ is the unit vector in the direction of $x$ , and $\|x - y\|$ is the distance.
The dot product is $x^\top y = \sum_i x_i y_i = \|x\|\|y\|\cos\theta$ . Its sign tells an acute (+), right (0) or obtuse (−) angle; $x^\top x = \|x\|^2$ .
$\|x \pm y\|^2 = \|x\|^2 + \|y\|^2 \pm 2x^\top y$ . With the law of cosines, this proves the geometric formula of the dot product.
Cosine similarity $x^\top y / (\|x\|\|y\|) \in [-1, 1]$ compares directions and ignores length. For unit vectors, $\|x - y\|^2 = 2 - 2\cos(x, y)$ , so distance and cosine give the same ranking.
A linear model $\hat y = w^\top x + b$ is a dot product: each weight $w_j$ is the change of $\hat y$ per unit of feature $j$ .

07 Slope, derivative, gradient

The average rate of change $\frac{f(x+h) - f(x)}{h}$ is the slope of a secant line. The derivative $f'(x)$ is its limit as $h \to 0$ , the slope of the tangent line.
Local linear approximation: $f(x + h) \approx f(x) + f'(x)h$ ; for smooth $f$ the error shrinks like $h^2$ . Example: $\sqrt{4.1} \approx 2.025$ .
Rules: $(x^n)' = nx^{n-1}$ , $(e^x)' = e^x$ , $(\log x)' = 1/x$ , derivatives of sums and constant multiples, and the chain rule $(f(g(x)))' = f'(g(x))g'(x)$ , which gives $\sigma' = \sigma(1 - \sigma)$ .
At a smooth inner minimum or maximum, $f'(x) = 0$ ; the converse is false ( $x^3$ at 0).
The central difference $\frac{f(x+h) - f(x-h)}{2h}$ has error of order $h^2$ , the forward difference of order $h$ ; too small an $h$ adds round-off error, so use $h \approx 10^{-5}$ (central).
A partial derivative $\partial f/\partial x_j$ changes $x_j$ and holds the other inputs fixed. The gradient $\nabla f$ collects them into a vector in the input space.
$D_u f = \nabla f^\top u = \|\nabla f\|\cos\theta$ : the gradient is the direction of steepest ascent with slope $\|\nabla f\|$ , $-\nabla f$ is steepest descent, and $\nabla f$ is perpendicular to the contour lines.
Gradient descent $x \leftarrow x - \eta\nabla f(x)$ on $x^2$ multiplies $x$ by $1 - 2\eta$ each step: it converges for $0 < \eta < 1$ , zig-zags for $0.5 < \eta < 1$ , and diverges for $\eta > 1$ .

08 Chance and data

Probability is long-run relative frequency. The frequency of an event in $n$ independent trials has typical error $\sqrt{p(1-p)/n}$ : it shrinks like $1/\sqrt n$ .
Rules: $P(\text{not } A) = 1 - P(A)$ ; $P(A \text{ or } B) = P(A) + P(B) - P(A \text{ and } B)$ ; $P(A \text{ and } B) = P(A)P(B)$ only for independent events.
Mean $\bar x = \frac1n\sum_i x_i$ (balance point); variance $\frac1n\sum_i (x_i - \bar x)^2$ ; standard deviation = its square root, in the unit of the data. The median is robust to outliers; the mean and std are not.
Sample variance divides by $n - 1$ and is unbiased. NumPy uses $n$ (ddof=0) by default, pandas uses $n - 1$ (ddof=1).
If $y = ax + b$ , then $\bar y = a\bar x + b$ and $\sigma_y = |a|\sigma_x$ . For independent variables, variances add, so the mean of $n$ has std $\sigma/\sqrt n$ .
A density histogram has bar areas that add up to 1. For a continuous variable, probability = area under the density; a density value can be larger than 1.
Normal $\mathcal N(\mu, \sigma^2)$ : z-score $z = (x - \mu)/\sigma$ ; within $1, 2, 3$ std: $0.6827, 0.9545, 0.9973$ .
CLT: the mean of $n$ independent, identically distributed variables with finite variance is approximately $\mathcal N(\mu, \sigma^2/n)$ for large $n$ . It says nothing about the data being normal.

09 Reading and writing proofs

A theorem says: if the assumptions hold, then the conclusion holds. A proof is a chain of steps, each justified by a definition, an assumption or an earlier result.
$P \Rightarrow Q$ is false only when $P$ is true and $Q$ is false. Its contrapositive $\neg Q \Rightarrow \neg P$ is equivalent; its converse $Q \Rightarrow P$ is a different claim. "Iff" needs both directions.
$\neg(\forall x\, P(x)) \equiv \exists x\, \neg P(x)$ and $\neg(\exists x\, P(x)) \equiv \forall x\, \neg P(x)$ . The order of $\forall$ and $\exists$ matters.
Direct proof: from the assumptions to the conclusion. Contrapositive: from "not $Q$ " to "not $P$ ". Contradiction: assume the claim is false and derive something impossible.
Induction: prove the base case $P(n_0)$ and the step $P(k) \Rightarrow P(k+1)$ for every $k \ge n_0$ . Both are needed; the step alone proves nothing.
One counterexample disproves a "for all" claim. Any number of checked examples does not prove it.
Useful results proved here: $\sqrt 2$ is irrational; $1 + \dots + n = n(n+1)/2$ ; $2^n > n^2$ for $n \ge 5$ ; the mean minimizes $\sum_i (x_i - c)^2$ ; one linear threshold unit cannot compute XOR.