How to read and write a proof
Before this lesson: lesson 02 (sets, , , , ). Some examples use lesson 05 (softmax) and lesson 08 (the mean). Time: about 60 minutes, plus the notebook. Next: the numbered proofs in Linear Algebra lesson 01 and in every other track.
Why this matters #
Every later track states theorems and proves them, and ML papers use the same language. A theorem tells you exactly when a method works. For example: "if is convex and its gradient is -Lipschitz, then gradient descent with step size satisfies ". To use it, you must check the assumptions. The loss of a neural network is not convex, so this theorem does not apply to it directly; that is a research question, not a detail. The Optimization track proves this result.
Proofs also help you write correct code. A loop is correct by an induction argument (a loop invariant). Backpropagation is correct because the chain rule holds at every layer, one layer after another. And a single counterexample can stop a wrong idea early, such as "a single linear unit can learn XOR" (section 4b).
1. Predict first #
2. The picture #
2.1 An implication is a statement about sets #
"For every : if , then " is written . Think of two sets: the numbers where holds and the numbers where holds. The claim says that the -set lies inside the -set. A counterexample is a number in the -set but outside the -set.
Choose a claim, then switch the form. Blue dots are numbers where the hypothesis holds; an orange ring means the conclusion holds; a magenta cross is a counterexample. For "divisible by 4 ⇒ even" every blue dot has a ring. Switch to the converse: the numbers 2, 6, 10, … become counterexamples. Switch to the contrapositive: the counterexamples are always the same as for the original claim.
2.2 Induction is a row of dominoes #
Let be a statement for each , for example "". Picture one domino per statement. The base case pushes the first domino: we prove . The induction step puts the dominoes close enough together: we prove that if domino falls, domino falls, for every . Then all dominoes fall. Both parts are needed.
Predict, then press Push. A domino turns blue when its statement is proved. The yellow arrow is the base case; a magenta × between two dominoes is a missing step. Try no base case (nothing falls), step fails at 1 → 2 (only the first falls) and the preset for , where the base case is .
2.3 A picture proof of a sum #
Why is ? Draw the sum as a staircase of squares: column has squares. Turn a copy of the staircase upside down and put it on top. The two staircases fill a rectangle with columns and rows. So two staircases have squares, and one has .
Move the slider . The blue staircase has squares; the orange turned copy has the same number. Together they fill the rectangle. Switch to induction step: the yellow column of squares turns the staircase for into the staircase for .
"If is divisible by 4, then is even." Which statement means exactly the same thing?
The contrapositive of is : "not even ⇒ not divisible by 4". It has the same counterexamples as the original (none). The first option is the converse; it fails at . The third option fails at : not divisible by 4, but even.
A proof shows , and it shows for every . What does it prove?
The step from 1 to 2 is missing. Domino 1 falls, but it does not knock over domino 2, and nothing pushes domino 2. The steps for are "if then ", and so on, but itself is never proved.
3. Definitions and main results #
3.1 The words of mathematics #
- A definition gives a precise meaning to a word. "An integer is even if for some integer ." Proofs about even numbers start from this sentence.
- A theorem is an important true claim with a proof. A lemma is a smaller result used to prove a theorem. A corollary follows quickly from a theorem. A conjecture is a claim that nobody has proved yet.
- A proof ends with (or "QED"). In this site, long proofs are numbered step by step.
Most theorems have the form "assumptions conclusion". In "if is convex and differentiable, then every point with is a global minimum", the assumptions are "convex" and "differentiable", and the conclusion is "global minimum".
3.2 Implications, converse, contrapositive #
A statement is true or false. For statements and :
| converse | contrapositive | |||
|---|---|---|---|---|
| true | true | true | true | true |
| true | false | false | true | false |
| false | true | true | false | true |
| false | false | true | true | true |
Here means "not ". Read the table row by row.
- is false only in one case: true and false. When is false, the implication makes no promise, so it counts as true.
- The contrapositive has the same column as : they are equivalent. A proof of one is a proof of the other.
- The converse has a different column. It is a different claim.
- If and only if ("iff", ) means both and . A proof of an iff has two parts, one for each direction.
3.3 Quantifiers and how to negate them #
means "for every "; means "there exists (at least one) ". To negate a statement, swap the quantifiers and negate the inside:
So "not every number is even" means "some number is odd". This is why one counterexample disproves a "for all" claim.
The order matters. On the real numbers:
- ("for every number there is a larger one") is true: take .
- ("there is a number larger than every number") is false: no is larger than .
In the first statement, may depend on . In the second, one must work for all . Negating the first statement step by step: .
What is the negation of ""?
Negation swaps each quantifier (), keeps their order, and negates the inside ( becomes ). The result says: "some is at least as large as every ", which is false on , as it must be, because the original statement is true.
3.4 How to read a proof #
- Restate the claim in your own words. Name the assumptions and the conclusion.
- Try a small case with numbers. It shows what the claim says and makes the steps concrete.
- For each line, ask: which fact gives this? A definition, an assumption, an earlier line, or a known result. If you cannot answer, slow down; that line is where the idea is.
- Find where each assumption is used. If an assumption is never used, either the theorem is more general than stated, or you missed a step. If you drop an assumption, look for a counterexample.
3.5 Direct proof #
Start from the assumptions and go step by step to the conclusion.
Claim. The sum of two even integers is even. Proof. Let and be even. By the definition, and for some integers . Then , and is an integer. So is even.
Claim. For all real : . Proof. A square is never negative, so . Expand: . Add to both sides: . (Equality holds exactly when . Finding the right starting fact, here , is the creative part.)
Claim (softmax is shift invariant, lesson 05). For every vector and every number : , where adds to every entry. Proof. For each ,
We used , pulled the common factor out of the sum, and cancelled it (, so it is not 0).
3.6 Proof by contrapositive #
To prove , prove instead (section 3.2). Use it when "not " gives you something concrete to work with.
Claim. For every integer : if is even, then is even. Proof. We prove the contrapositive: if is odd, then is odd. Let be odd, so for an integer . Then , which is odd.
A direct proof is hard here: "" says little about itself. The contrapositive starts from .
3.7 Proof by contradiction #
To prove a statement , assume and derive something impossible: a statement and its negation at the same time. Then cannot hold, so is true.
Theorem. is irrational: there are no integers with and .
Proof.
- Assume the opposite: for some integers with .
- Cancel common factors, so that the fraction is in lowest terms: and have no common factor larger than 1. (Every fraction can be reduced like this.)
- Square: , so . Hence is even.
- By the claim of section 3.6, is even: for an integer .
- Put this into step 3: , so . Hence is even, and (section 3.6 again) is even.
- Now and are both even, so 2 is a common factor. This contradicts step 2.
- So the assumption in step 1 is false, and is irrational.
There are infinitely many prime numbers (Euclid)
- Assume the opposite: there are only finitely many primes, .
- Let . Then , so has at least one prime factor .
- is one of the , because the list contains all primes. But dividing by any leaves the remainder 1, so does not divide . Contradiction.
- So there are infinitely many primes.
Careful: the proof does not say that itself is prime. .
In the proof that is irrational, which assumption does the final contradiction break?
Steps 3–5 show that and are both even. Then 2 divides both, so the fraction was not in lowest terms, as step 2 required. Every fraction can be reduced, so the only assumption left to reject is itself.
3.8 Proof by induction #
Principle of induction. Let be a statement for each integer . If
- base case: is true, and
- induction step: for every , ,
then is true for every .
In the step, is called the induction hypothesis. Assuming it is not circular: the step does not claim that is true. It proves the implication "if , then ". Truth enters only through the base case, and then flows along the implications: , then , and so on.
Theorem. For every : .
Proof.
- Base case, : the left side is 1 and the right side is .
- Step. Let and assume (induction hypothesis).
- Then by the hypothesis.
- , which is the formula for .
Step 3 is the yellow column of the staircase figure.
The base case matters. Claim: for every .
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | |
|---|---|---|---|---|---|---|---|
| 2 | 4 | 8 | 16 | 32 | 64 | 128 | |
| 1 | 4 | 9 | 16 | 25 | 36 | 49 | |
| ? | yes | no | no | no | yes | yes | yes |
The step works for every (section 4c), but and are false, so the chain can only start at . A correct step without a true base case proves nothing.
A broken induction: "all horses have the same colour". Let be "in every group of horses, all horses have the same colour". is true. "Step": take a group of horses. Remove the first horse: the remaining have one colour, by . Remove the last horse instead: those also have one colour. The two groups overlap, so all horses have the same colour.
Where is the error? For (going from 1 to 2 horses), the two groups are {horse 2} and {horse 1}. They do not overlap, so the colours need not match. The step fails at , so nothing beyond follows. The step must work for every .
3.9 Counterexamples #
One counterexample proves that a "for all" claim is false (section 3.3). Examples:
- " is prime for every integer ." It is prime for , but .
- "." False: gives on the left and on the right. The true rule is .
- " for square matrices." False: , give and .
Many examples never prove a "for all" claim. A famous case is Pólya's conjecture from number theory: it holds for every from 2 up to 906 150 256, and fails at . Checking is the best way to find counterexamples, and a good way to test a claim before you try to prove it. It is not a proof.
Pick a claim and press Scan. Each cell is one case: aqua means the claim holds there, magenta is a counterexample. Click a cell to see the computation. For "" and "6 divides " the scan finds nothing; the widget then shows the real proof, which covers every case and not only the ones in the grid.
Which pair is a counterexample to "for all real : if , then "?
A counterexample must make the hypothesis true and the conclusion false. For : , but . For the hypothesis is false, so the pair says nothing about the claim. The correct statement is: or .
Answers to section 1. (1) No. That is the converse, and is a counterexample: it is even but not divisible by 4. (2) No. A check of finitely many cases cannot cover all integers; the next case may fail, as in Pólya's conjecture. You need a proof. (3) No. The step proves only the implication "if , then ". The base case supplies the first true statement (section 3.8).
4. Worked example #
(a) The mean minimizes the squared error (used in lesson 08 and in every regression model).
Claim. Let be real numbers with mean , and let . Then for every real , with equality only for .
Small case first. For the data : , , , . The value at the mean is the smallest of the three.
Proof.
- Write . (Add and subtract ; this is the idea of the proof.)
- Square: .
- Sum over . The middle term is , and (lesson 08, section 3.3). The last term does not depend on , so it gives .
- So .
- , with equality only when . So , with equality only for .
Check with the small case: ✓. Where was each fact used? Step 3 uses the definition of ; step 5 uses "a square is never negative" and .
(b) A single linear unit cannot compute XOR (proof by contradiction). A linear threshold unit outputs 1 if and 0 otherwise. XOR outputs 1 exactly when one input is 1 and the other is 0.
- Assume the opposite: some give XOR on all four inputs.
- Input must give 0, so .
- Inputs and must give 1, so and .
- Input must give 0, so .
- Add the two inequalities of step 3: , so .
- By step 2, . So . This contradicts step 4.
- So no such exist.
This is why networks need a hidden layer (Minsky and Papert, 1969). A two-layer network computes XOR easily.
(c) The induction step for , written in full.
Claim. For every : .
- Base case, : .
- Step. Let and assume .
- Then by the hypothesis.
- We want . Expand: , so we need , that is, , that is, .
- For , . (Step 4 even holds for every .)
- So : this is .
5. In code #
Code cannot prove a "for all" claim, but it finds counterexamples quickly and checks claims on many cases before you try to prove them. The notebook for this lesson has these cells and the exercises.
import itertools
import numpy as np
def is_prime(m):
return m >= 2 and all(m % d for d in range(2, int(m**0.5) + 1))
# a counterexample: the first n with n^2 + n + 41 not prime
n = next(n for n in itertools.count() if not is_prime(n * n + n + 41))
print(n, n * n + n + 41, 41 * 41) # 40 1681 1681
# 2^n > n^2 holds for n = 1 and n >= 5 (checked here up to 60; the proof covers all n)
print([n for n in range(1, 61) if not 2**n > n**2]) # [2, 3, 4]
# Gauss's formula, checked for n = 1..1000 (a check, not a proof)
print(all(sum(range(1, n + 1)) == n * (n + 1) // 2 for n in range(1, 1001))) # True
# truth table: P => Q is "not P or Q"; the contrapositive has the same column
imp = lambda p, q: (not p) or q
for P, Q in itertools.product([True, False], repeat=2):
print(P, Q, imp(P, Q), imp(Q, P), imp(not Q, not P))
# True True True True True
# True False False True False
# False True True False True
# False False True True True
# the mean minimizes the sum of squared distances (on a grid of candidates c)
x = np.array([1., 2., 6.])
c = np.linspace(0, 6, 601)
f = ((x[:, None] - c[None, :])**2).sum(axis=0)
print(c[f.argmin()], f.min()) # 3.0 14.0
6. Where it shows up #
- Reading theorems in the other tracks. Every lesson in Linear Algebra states and proves results with numbered steps. For example, the spectral theorem is proved by induction on the size of the matrix: a symmetric problem is reduced to a symmetric one.
- Convergence results. Optimization theorems ("gradient descent converges if …") have precise assumptions: convexity, smoothness, a step size bound. Checking them is the first thing to do when you apply such a result.
- Algorithms. A loop invariant is an induction: it holds before the first iteration (base case) and each iteration keeps it (step). Dynamic programming and the Bellman equation in reinforcement learning are argued the same way, one step at a time.
- Backpropagation. The gradient at layer is correct if the gradient at layer is correct and the chain rule holds for that layer: an induction over the layers, from the output back to the input.
- Impossibility results. The XOR proof of section 4b, the no-free-lunch theorems and lower bounds on sample complexity are proofs that something cannot be done. They tell you which model class or how much data you need.
- Counterexamples in practice. A failing unit test is a counterexample. Adversarial examples are counterexamples to "the classifier is correct near every training image".
7. Common mistakes #
- Proving the converse. You want but you show . Write the assumptions and the conclusion on top of your proof, and check that you start from the assumptions.
- "I checked many cases, so it is true." Examples only show that no counterexample was found yet.
- Forgetting the base case, or checking the wrong one. For , the step works from , but the claim is false for and .
- A step that does not work for every . The horses argument fails at . Check the step for the smallest too.
- Assuming what you want to prove. In a proof by contradiction you assume the negation of the claim, not the claim. In an induction step you may use , not .
- Negating a statement wrongly. The negation of "every has property " is "some does not have ", not "no has ".
- Hidden division by zero. A chain of equations that divides by is wrong when .
- Thinking Euclid's is prime. It only has a prime factor outside the list; .
8. Check yourself #
- Write the converse and the contrapositive of "if a function is differentiable at , then it is
continuous at ". Which of the two is true?
Answer
Converse: "if it is continuous at , then it is differentiable at ". It is false: is continuous at 0 but has a corner there. Contrapositive: "if it is not continuous at , then it is not differentiable at ". It is true, because it is equivalent to the original statement, which is a theorem of calculus. - Negate "every student passed at least one exam". Write it with quantifiers first.
Answer
: passed . The negation is : did not pass . In words: "some student failed every exam". - Prove that the product of two odd integers is odd.
Answer
Let and . Then , and is an integer. So is odd. - Prove by induction: for every .
Answer
Base case: gives . Step: assume . Add the next odd number, : the sum becomes . - Find the error: "Let . Then , so , so , so , so , so ."
Answer
The step from to divides by , which is 0 because . Division by zero is not allowed, so the chain breaks there. - Is " is prime for every integer " true?
Answer
No. For : , which is not prime. (It is prime for , which shows again that many checked cases prove nothing.)
What is the contrapositive of , and is it equivalent to it?
. Yes, it is equivalent: it is true in exactly the same cases.
What is the converse of , and is it equivalent to it?
. Not in general: "divisible by 4 ⇒ even" is true, "even ⇒ divisible by 4" is false ().
When is the implication false?
Only when is true and is false.
Negate .
: there is at least one counterexample.
What are the two parts of a proof by induction?
The base case , and the step: for every .
Why is assuming in an induction step not circular?
The step only proves the implication "if then ". Truth comes from the base case and passes along the chain.
What is the structure of a proof by contradiction?
Assume the claim is false, derive an impossible statement (something and its negation), and conclude that the claim is true.
What does one counterexample do to a "for all" claim, and what do many examples do?
One counterexample proves it false. Many examples (even millions) do not prove it true.
In the proof that is irrational, what is contradicted?
That is in lowest terms: forces both and to be even.
Compute .
, from .
9. Interview angle #
- "Prove that is irrational." — Contradiction with a fraction in lowest terms (section 3.7). Know which step uses " even ⇒ even" and how that is proved (contrapositive).
- "Prove ." — Induction, or the picture with two staircases, or Gauss's trick: write the sum forwards and backwards and add.
- "Why can a single perceptron not learn XOR?" — The four inequalities of section 4b contradict each other: XOR is not linearly separable.
- "Show that the mean minimizes the squared error, and the median the absolute error." — Section 4a for the mean. For the median: moving to the right by a small step changes by (points left of − points right of ) × step, so the minimum is where both counts are equal.
- "Your colleague tested the claim on 10 000 random inputs. Is it correct?" — Not proved. Ask for the argument, look for edge cases (0, negative numbers, ties, empty input), and try to build a counterexample.
10. Practice #
The notebook for this lesson runs in Colab or in your browser. Code checks claims; it does not prove them. The notebook says this again in each exercise. Five exercises, each with a check cell and a folded solution:
- Find the first for which is not prime.
- Check and for every .
- Build the truth tables of , its converse and its contrapositive with
itertools.product. - Find the smallest such that for every from to 200.
- Check on a grid that the mean minimizes , then read the proof in the solution.
11. Watch and read #
- Richard Hammack, Book of Proof, 3rd edition (free PDF): chapter 2 (logic), chapter 4 (direct proof), chapter 5 (contrapositive), chapter 6 (contradiction), chapter 9 (disproof and counterexamples) and chapter 10 (induction).
- Lehman, Leighton and Meyer, Mathematics for Computer Science (free PDF): chapter 1 (proofs) and chapter 5 (induction).
- Daniel Solow's and Daniel Velleman's books (How to Read and Do Proofs, How to Prove It) are good printed guides if your library has them.
Runs in your browser with nothing to install. The notebook repeats the key demos and has exercises with checks; solutions are inside, folded.
No Google account needed: Python runs inside your browser. The first start downloads about 40 MB; your changes are saved in this browser only.