A gentle tour of vectors: arrows, lists, length and the dot product

Before this lesson: lesson 02 (the notation $\R^n$, indices and $\sum$) and lesson 03 (NumPy arrays). From school you need the theorem of Pythagoras and the cosine of an angle. Time: about 55 minutes, plus the notebook.

---

Why this matters #

• Data are vectors. A data set with $n$ features stores each sample as a vector $x \in \R^n$. An $8 \times 8$ grey image is a vector in $\R^{64}$. A language model stores each word piece (token) as an embedding, a vector with hundreds or thousands of entries (768 in BERT-base).
• Models are dot products. A linear model, and every neuron of a neural network, computes $w^\top x + b$ before its activation function. In a transformer, the attention score of a query $q$ and a key $k$ is the dot product $q^\top k$, scaled by $1/\sqrt{d}$.
• Similarity is geometry. $k$-nearest neighbours uses the distance $\|x - y\|$. Retrieval for chatbots (RAG), recommendation and face recognition compare embeddings with cosine similarity.
• Size is a norm. Weight decay and ridge regression add $\lambda \|w\|^2$ to the loss, so they keep the weights short.

This lesson is a first look. The Linear Algebra track starts where it ends.

1. Predict first #

2. The picture #

Two views of one vector #

The vector $u = (3, 1)$ is a list of two numbers. Draw it as an arrow: start at the origin, go 3 to the right and 1 up. The numbers are the entries (or components) $u_1 = 3$ and $u_2 = 1$.

The two views help in different ways. The arrow gives intuition in 2-D and 3-D. The list works in any number of dimensions. A house with area 120 m², 3 rooms and age 15 years is the vector $(120, 3, 15) \in \R^3$. An $8 \times 8$ image is a list of 64 grey values, so it is a vector in $\R^{64}$. We cannot draw $\R^{64}$. But every operation in this lesson is defined by the entries, so it works for any $n$. We draw the 2-D case only as a guide.

Adding, subtracting and scaling #

• Addition is entry by entry: $(3, 1) + (1, 2) = (4, 3)$. As arrows: put the tail of $v$ at the tip of $u$ ("tip to tail"). The sum goes from the origin to the new tip. It is the diagonal of the parallelogram with sides $u$ and $v$.
• Scalar multiplication multiplies every entry by one number $c$ (a scalar): $2 \cdot (3, 1) = (6, 2)$. The arrow gets 2 times longer. A negative $c$ flips the arrow to the opposite direction.
• Subtraction is $u - v = u + (-1)v$. As arrows, $u - v$ goes from the tip of $v$ to the tip of $u$, because $v + (u - v) = u$.

Interactive figure: loading…

A sum of scaled vectors, such as $c_1 u + c_2 v$, is a linear combination of $u$ and $v$. Linear combinations are the main object of Linear Algebra lesson 1.

Because the operations work entry by entry, they work on data too. Mixing two images, $(1 - t)\,a + t\,b$, is one scaling of each image and one addition:

Interactive figure: loading…

Length and distance #

The arrow $(3, 4)$ is the long side (hypotenuse) of a right triangle with the sides 3 and 4. By Pythagoras, its length is $\sqrt{3^2 + 4^2} = \sqrt{25} = 5$. We write the length of $x$ as $\|x\|$ (the norm of $x$).

In 3-D, use Pythagoras twice. For $x = (1, 2, 2)$, the shadow of $x$ on the floor is $(1, 2, 0)$, with length $\sqrt{1^2 + 2^2} = \sqrt 5$. The vertical part has length 2, and it is at a right angle to the floor. So $\|x\| = \sqrt{(\sqrt 5)^2 + 2^2} = \sqrt{9} = 3$. The same pattern continues in any dimension: square the entries, add them, take the square root.

The distance between two points $x$ and $y$ is the length of the arrow between them, $\|x - y\|$. For $(1, 1)$ and $(4, 5)$: $(4, 5) - (1, 1) = (3, 4)$, so the distance is 5.

The dot product as a shadow #

The dot product of $u$ and $v$ multiplies the entries pair by pair and adds the products: $u^\top v = u_1 v_1 + u_2 v_2$. For $u = (3, 1)$ and $v = (1, 2)$: $3 \cdot 1 + 1 \cdot 2 = 5$.

There is a second way to get the same number. Let $\theta$ be the angle between the two arrows. Drop $v$ straight down (at a right angle) onto the line of $u$. The result is the shadow, or projection, of $v$ on $u$. Its signed length is $\|v\| \cos\theta$. Then

In words: $u^\top v$ is the length of $u$ times the signed length of the shadow of $v$.

Interactive figure: loading…

So the sign of the dot product tells you the type of angle:

$u^\top v$	angle $\theta$	the two arrows
$> 0$	$0° \le \theta < 90°$ (acute)	point roughly the same way
$= 0$	$\theta = 90°$ (right)	are orthogonal (perpendicular)
$< 0$	$90° < \theta \le 180°$ (obtuse)	point roughly opposite ways

The table assumes that $u$ and $v$ are not the zero vector. For the zero vector the angle is not defined.

3. Definitions and main results #

Vectors and their operations #

A vector $x \in \R^n$ is an ordered list of $n$ real numbers $x_1, \dots, x_n$. We write it as a column, and $x^\top$ (the transpose) is the same numbers as a row:

In running text we write $x = (x_1, \dots, x_n)$; it still means a column. The zero vector $0$ has all entries 0. For $x, y \in \R^n$ and a scalar $c \in \R$:

Both operations work entry by entry, so they follow the rules of ordinary numbers: $x + y = y + x$, $(x + y) + z = x + (y + z)$, $c(x + y) = cx + cy$, and $x + 0 = x$. Only vectors of the same size can be added.

Norm, unit vector, distance #

Definition. The norm (Euclidean length) of $x \in \R^n$ is

Other norms exist (the Linear Algebra lesson on norms compares them). In this track, $\|x\|$ always means this one, also written $\|x\|_2$. Do not confuse it with $|a|$, the absolute value of one number.

Two properties follow directly from the definition.

1. $\|x\| \ge 0$, and $\|x\| = 0$ only for $x = 0$. A sum of squares is zero only if every square is zero.
2. $\|c\,x\| = |c|\,\|x\|$. Proof: $\|cx\| = \sqrt{\sum_i c^2 x_i^2} = \sqrt{c^2}\sqrt{\sum_i x_i^2} = |c|\,\|x\|$, because $\sqrt{c^2} = |c|$ (for example $\sqrt{(-2)^2} = 2$). $\blacksquare$

A unit vector has norm 1. For $x \ne 0$, the vector $\hat x = x / \|x\|$ is a unit vector in the direction of $x$: by property 2, $\|x / \|x\|\| = \|x\| / \|x\| = 1$. For example $(3, 4)/5 = (0.6, 0.8)$. Dividing by the norm is called normalizing.

The (Euclidean) distance between $x$ and $y$ is $\|x - y\|$.

The dot product #

Definition. The dot product (or inner product) of $x, y \in \R^n$ is the number

Books also write $x \cdot y$ or $\langle x, y \rangle$. The notation $x^\top y$ is "row times column": a $1 \times n$ row times an $n \times 1$ column gives a $1 \times 1$ result, one number. In NumPy it is x @ y or np.dot(x, y).

Rules. For vectors $x, y, z$ and a scalar $c$:

• symmetric: $x^\top y = y^\top x$;
• linear in each vector: $x^\top (y + z) = x^\top y + x^\top z$ and $x^\top (c y) = c\, x^\top y$;
• length: $x^\top x = \sum_i x_i^2 = \|x\|^2$.

Each rule follows from the same rule for numbers, applied to every term of the sum.

Expanding a squared length. The rules give a formula that we use again and again:

Proof (for the minus sign; the plus sign is the same with $-y$ replaced by $y$).

1. Use the length rule: $\|x - y\|^2 = (x - y)^\top (x - y)$.
2. Use linearity to multiply out, like $(a - b)(a - b)$ for numbers: $x^\top x - x^\top y - y^\top x + y^\top y$.
3. Use symmetry, $y^\top x = x^\top y$, and the length rule: $\|x\|^2 - 2\,x^\top y + \|y\|^2$. $\blacksquare$

It is the vector version of $(a - b)^2 = a^2 - 2ab + b^2$.

The geometric formula #

Theorem. Let $x, y \in \R^n$ be nonzero, and let $\theta \in [0°, 180°]$ be the angle between them. Then

Why this is true, in one picture. The arrows $x$, $y$ and $x - y$ form a triangle. The expansion above gives the length of the third side from the dot product. The law of cosines gives the length of the same side from the angle. The two formulas must agree, and that forces $x^\top y = \|x\|\|y\|\cos\theta$.

Proof with the law of cosines

1. Draw the triangle with corners $0$, $x$ and $y$. Its sides have the lengths $a = \|x\|$ and $b = \|y\|$, and the side from the tip of $y$ to the tip of $x$ is the arrow $x - y$, with length $c = \|x - y\|$. The angle at the corner $0$ is $\theta$.
2. The law of cosines (school geometry; it is Pythagoras when $\theta = 90°$) says $c^2 = a^2 + b^2 - 2ab\cos\theta$, that is, $\|x - y\|^2 = \|x\|^2 + \|y\|^2 - 2\|x\|\|y\|\cos\theta$.
3. The expansion of the previous section says $\|x - y\|^2 = \|x\|^2 + \|y\|^2 - 2\,x^\top y$.
4. The left sides of steps 2 and 3 are equal. Subtract $\|x\|^2 + \|y\|^2$ from both right sides and divide by $-2$: $x^\top y = \|x\|\|y\|\cos\theta$. $\blacksquare$

If $x$ and $y$ lie on one line, the "triangle" is flat ($\theta = 0°$ or $180°$). The formula still holds: for $y = cx$ with $c > 0$, $x^\top y = c\|x\|^2 = \|x\|\,\|y\|\cos 0°$, and the case $c < 0$ is the same with $\cos 180° = -1$.

In $\R^n$ with $n > 3$ we cannot measure the angle with a protractor. There, we turn the theorem around and use it to define the angle:

This is a valid definition because the fraction is always between $-1$ and $1$ (the Cauchy–Schwarz inequality, in the folded block below).

Consequences.

• $x^\top y > 0$, $= 0$ or $< 0$ exactly when the angle is acute, right or obtuse (for nonzero $x$, $y$), because $\|x\|\|y\| > 0$ and $\cos\theta$ has the same sign as $x^\top y$.
• $x$ and $y$ are orthogonal (perpendicular) when $x^\top y = 0$. Then $\|x + y\|^2 = \|x\|^2 + \|y\|^2$: this is Pythagoras in $\R^n$.
• The scalar projection of $y$ on the direction of $x$ is $\|y\|\cos\theta = x^\top y / \|x\|$, and the projection vector is $\dfrac{x^\top y}{\|x\|^2}\,x$. This is the "shadow" of the figure.

Cosine similarity #

Definition. The cosine similarity of two nonzero vectors is

It is 1 when $x$ and $y$ point in the same direction, 0 when they are orthogonal, and $-1$ when they point in opposite directions. It ignores length: for $c > 0$, $\cos(cx, y) = \dfrac{c\,x^\top y}{c\|x\|\,\|y\|} = \cos(x, y)$.

This is why it is used for texts and embeddings. A long article and a short note about the same topic have word-count vectors that point the same way but have different lengths. Cosine similarity calls them similar. The distance $\|x - y\|$ calls them far apart, because it also compares the lengths.

Interactive figure: loading…

For unit vectors the two measures agree. If $\|x\| = \|y\| = 1$, the expansion gives

So a smaller distance means a larger cosine. This is why many systems normalize all embeddings to length 1 first and then use either measure.

A linear model is a dot product #

A linear model predicts a number from the feature vector $x \in \R^n$ with weights $w \in \R^n$ and a bias $b \in \R$:

Each weight $w_j$ says how much feature $j$ counts: if $x_j$ grows by 1 and the other features stay the same, $\hat y$ changes by $w_j$. For a house with $x = (1.2, 4)$ (area in 100 m², rooms) and $w = (3, 0.5)$, $b = 1$ (prices in 100 000 CHF): $\hat y = 3 \cdot 1.2 + 0.5 \cdot 4 + 1 = 6.6$.

The geometric formula gives a second reading. For a fixed length of $x$, the score $w^\top x$ is largest when $x$ points in the direction of $w$. So a neuron "looks for" inputs that point like its weight vector.

Cauchy–Schwarz, and angles in high dimension

Cauchy–Schwarz inequality. For all $x, y \in \R^n$: $|x^\top y| \le \|x\|\,\|y\|$.

Proof. If $y = 0$ both sides are 0. Otherwise, for every number $t$, a squared length is not negative: $0 \le \|x - t y\|^2 = \|x\|^2 - 2t\,x^\top y + t^2 \|y\|^2$. Choose $t = x^\top y / \|y\|^2$ (the $t$ that makes the right side smallest). This gives $0 \le \|x\|^2 - (x^\top y)^2 / \|y\|^2$, so $(x^\top y)^2 \le \|x\|^2\|y\|^2$. Take square roots. $\blacksquare$

So $x^\top y / (\|x\|\|y\|)$ always lies in $[-1, 1]$, and the definition of the angle above makes sense. It does not need a picture, so it works for every $n$.

Random vectors in high dimension are almost orthogonal. Draw two vectors with independent standard normal entries. In $\R^2$ their angle can be anything from 0° to 180°. In $\R^{1000}$ it is almost always close to 90°: in a simulation with 2000 pairs, the angles had mean 90.0° and standard deviation 1.8°. The dot product is a sum of $n$ random terms with mean 0, so it grows like $\sqrt n$, while $\|x\|\|y\|$ grows like $n$. The notebook draws the histogram. Linear Algebra lesson 18 uses this fact.

Answers to section 1. (1) $45°$: $\cos\theta = \dfrac{1 \cdot 1 + 0 \cdot 1}{1 \cdot \sqrt 2} = \dfrac{1}{\sqrt 2}$. (2) $\|2x\| = 2\|x\| = 2 \cdot 5 = 10$; directly, $\sqrt{36 + 64} = 10$. (3) Exactly 1: cosine similarity ignores length, and $2x$ points in the same direction as $x$.

4. Worked example #

Take $x = (2, 1)$ and $y = (1, 3)$.

(a) Operations.

(b) Lengths and distance.

The unit vector in the direction of $x$ is $x / \|x\| = (2, 1)/\sqrt 5 \approx (0.894, 0.447)$.

(c) Dot product and angle.

so $\theta = 45°$.

(d) Check with the expansion. $\|x - y\|^2 = \|x\|^2 + \|y\|^2 - 2\,x^\top y = 5 + 10 - 2 \cdot 5 = 5$. This matches (b): $(\sqrt 5)^2 = 5$. ✓

(e) Projection. The projection of $y$ on the direction of $x$ is

with scalar projection $x^\top y / \|x\| = 5/\sqrt 5 = \sqrt 5 \approx 2.236$. The rest, $y - (2, 1) = (-1, 2)$, is orthogonal to $x$: $(2, 1)^\top(-1, 2) = -2 + 2 = 0$. ✓

(f) Cosine versus distance. The query $q = (2, 1)$ is compared with two documents of the figure: D2 $= (8, 4)$ and D6 $= (1, 1)$.

• D2 $= (8, 4)$: $\;q^\top D2 = 2 \cdot 8 + 1 \cdot 4 = 20$ and $\|D2\| = \sqrt{80}$, so $\cos(q, D2) = \dfrac{20}{\sqrt 5 \cdot \sqrt{80}} = \dfrac{20}{\sqrt{400}} = 1$. The difference is $q - D2 = (-6, -3)$, so the distance is $\sqrt{36 + 9} = \sqrt{45} \approx 6.708$.
• D6 $= (1, 1)$: $\;q^\top D6 = 2 + 1 = 3$ and $\|D6\| = \sqrt 2$, so $\cos(q, D6) = \dfrac{3}{\sqrt 5 \cdot \sqrt 2} = \dfrac{3}{\sqrt{10}} \approx 0.949$. The difference is $q - D6 = (1, 0)$, so the distance is $1$.

Cosine similarity prefers D2 (same direction: $D2 = 4q$). Distance prefers D6 (close to the tip of $q$).

5. In code #

NumPy checks the hand computations above. The notebook for this lesson has these cells and the exercises.

import numpy as np

x = np.array([2., 1.])
y = np.array([1., 3.])

print(x + y, x - y, 2 * x)                 # [3. 4.] [ 1. -2.] [4. 2.]
print(np.linalg.norm(x), np.linalg.norm(y))  # 2.236... 3.162...
print(np.linalg.norm(x - y))               # 2.236...  (distance)
print(x / np.linalg.norm(x))               # [0.894 0.447]  (unit vector)

dot = x @ y                                # same as np.dot(x, y) and np.sum(x * y)
cos = dot / (np.linalg.norm(x) * np.linalg.norm(y))
print(dot, cos, np.degrees(np.arccos(cos)))  # 5.0 0.7071... 45.0...

# the expansion ||x - y||^2 = ||x||^2 + ||y||^2 - 2 x.y
print(np.isclose(np.sum((x - y)**2), x @ x + y @ y - 2 * (x @ y)))  # True

# projection of y on the direction of x, and the orthogonal rest
p = (x @ y) / (x @ x) * x
print(p, (y - p) @ x)                      # [2. 1.] 0.0

# cosine similarity versus distance
q, D2, D6 = np.array([2., 1.]), np.array([8., 4.]), np.array([1., 1.])
cosine = lambda a, b: a @ b / (np.linalg.norm(a) * np.linalg.norm(b))
print(cosine(q, D2), cosine(q, D6))        # 1.0 0.9486...
print(np.linalg.norm(q - D2), np.linalg.norm(q - D6))  # 6.708... 1.0

The angle may print as 45.00000000000001: floats are not exact (lesson 03). Compare floats with np.isclose.

6. Where it shows up #

• Features and embeddings. A row of a data matrix $X$ is a feature vector $x^{(i)}$. Word embeddings place words with similar meaning at similar directions, so cosine similarity finds related words. Sentence embeddings and vector databases do the same for whole texts; this is the retrieval step of RAG.
• Nearest neighbours. $k$-NN classification and clustering with $k$-means compute distances $\|x - y\|$ between feature vectors. The notebook finds the nearest neighbour of a digit image by cosine similarity.
• Neurons and attention. A neuron computes $w^\top x + b$. Scaled dot-product attention computes $q^\top k / \sqrt d$ for every query and key, then a softmax (lesson 05). A large dot product means "this key matches this query".
• Regularization. Ridge regression and weight decay add $\lambda\|w\|^2 = \lambda\, w^\top w$ to the loss.
• ML for science. A molecule or a material is often stored as a fingerprint vector, and similar compounds are found with a similarity of these vectors. A wind field is a 2-D or 3-D vector at each grid point, and the speed is its norm.

7. Common mistakes #

• Adding the entries to get a length. $\|(3, 4)\| = 5$, not $3 + 4 = 7$. Square, add, then take the square root.
• Forgetting the square root. $x^\top x = \|x\|^2$, not $\|x\|$.
• Confusing $\|x\|$ and $|x|$. $\|x\|$ is the length of a vector. $|a|$ is the absolute value of a number. They agree only for $n = 1$.
• Thinking that the dot product is a vector. $x^\top y$ is one number. The entrywise product x * y in NumPy is a vector; its sum is the dot product.
• Asking for the angle with the zero vector. $\cos\theta = x^\top y / (\|x\|\|y\|)$ divides by 0. The zero vector has no direction.
• Using cosine similarity when length matters. Two shopping baskets with 1 apple and 100 apples point in the same direction, so their cosine similarity is 1. If the amount matters, use a distance (or compare the norms too).

8. Check yourself #

1. Compute $u + v$, $u - v$ and $-2u$ for $u = (1, -2, 3)$ and $v = (4, 0, -1)$.
   Answer

   $u + v = (5, -2, 2)$, $u - v = (-3, -2, 4)$, $-2u = (-2, 4, -6)$.
2. Find the unit vector in the direction of $(1, 2, 2)$.
   Answer

   $\|(1, 2, 2)\| = \sqrt{1 + 4 + 4} = 3$, so the unit vector is $(1/3, 2/3, 2/3)$.
3. For which number $a$ are $(2, a)$ and $(3, -6)$ orthogonal?
   Answer

   $2 \cdot 3 + a \cdot (-6) = 6 - 6a = 0$, so $a = 1$. Check: $(2, 1)^\top (3, -6) = 6 - 6 = 0$.
4. $\|x\| = 3$, $\|y\| = 4$ and $x^\top y = 0$. What is $\|x + y\|$?
   Answer

   $\|x + y\|^2 = \|x\|^2 + \|y\|^2 + 2x^\top y = 9 + 16 + 0 = 25$, so $\|x + y\| = 5$ (Pythagoras).
5. Two embeddings are normalized to length 1, and their distance is $\|x - y\| = 1$. What is their cosine similarity, and what is the angle?
   Answer

   $\|x - y\|^2 = 2 - 2\cos(x, y)$ gives $1 = 2 - 2\cos$, so $\cos(x, y) = 0.5$ and $\theta = 60°$.
6. A linear model has $w = (2, -1)$ and $b = 0.5$. Compute $\hat y$ for $x = (3, 4)$. Which change of $x$ increases $\hat y$ fastest, for a step of length 1?
   Answer

   $\hat y = 2 \cdot 3 + (-1) \cdot 4 + 0.5 = 2.5$. A step $s$ changes $\hat y$ by $w^\top s = \|w\|\|s\|\cos\theta$. For $\|s\| = 1$ this is largest when $s$ points along $w$: $s = w/\|w\| = (2, -1)/\sqrt 5$, and $\hat y$ grows by $\|w\| = \sqrt 5 \approx 2.236$. Lesson 07 uses the same argument for the gradient.

Recall cardsAnswer in your head first, then reveal. Add the cards to your review deck to see them again just before you would forget.

1. What is the norm (length) of $x \in \R^n$?

   Show answer

2. Compute $\|(3, 4)\|$ and the unit vector in its direction.

   Show answer

3. Give the two formulas for the dot product $x^\top y$.

   Show answer

4. What does $x^\top y = 0$ mean for two nonzero vectors?

   Show answer

5. Expand $\|x - y\|^2$ with dot products.

   Show answer

6. What is the cosine similarity of $x$ and $y$, and what range does it have?

   Show answer

7. Why is the cosine similarity of $x$ and $3x$ equal to 1?

   Show answer

8. How are distance and cosine similarity related for unit vectors?

   Show answer

9. Compute $(1, 2)^\top (3, -1)$.

   Show answer

10. Write a linear model as a dot product. What does the weight $w_j$ mean?

    Show answer

Add these cards to my reviewOpen the review →

9. Interview angle #

• "What is the difference between cosine similarity and Euclidean distance? When do you use which?" — Cosine compares directions and ignores length; distance compares positions. Use cosine for texts and embeddings where length reflects size, not meaning. For unit vectors they give the same ranking, because $\|x - y\|^2 = 2 - 2\cos(x, y)$.
• "Why is the dot product $\|x\|\|y\|\cos\theta$?" — Expand $\|x - y\|^2 = \|x\|^2 + \|y\|^2 - 2x^\top y$ and compare with the law of cosines on the triangle $0, x, y$.
• "Why does attention divide by $\sqrt{d}$?" — For random vectors with independent entries of variance 1, $q^\top k$ is a sum of $d$ terms with variance 1 each, so its standard deviation is $\sqrt d$. Dividing keeps the scores in a range where the softmax is not saturated.
• "Two random vectors in 1000 dimensions: what angle do you expect?" — Close to $90°$; the cosine is of size $1/\sqrt n$.

10. Practice #

The notebook for this lesson runs in Colab or in your browser. Four exercises, each with a check cell and a folded solution:

1. Write norm, dot and angle_deg with a loop or np.sum, and check them against np.linalg.norm and @.
2. Check $\|u + v\|^2 = \|u\|^2 + \|v\|^2 + 2u^\top v$ on random vectors.
3. Build the cosine-similarity matrix of the rows of a matrix with broadcasting, and find the nearest neighbour of a digit image from load_digits.
4. Draw random vectors in $n = 2, 10, 1000$ dimensions and plot the histogram of their angles.

11. Watch and read #

Vectors | Chapter 1, Essence of linear algebra3Blue1Brown · Essence of linear algebra, chapter 1 · 10 min

The arrow view and the list view of a vector, addition tip to tail, and scaling.

Dot products and duality | Chapter 9, Essence of linear algebra3Blue1Brown · Essence of linear algebra, chapter 9 · 14 min

The dot product as a projection (the "shadow" of this lesson). The duality part is optional for now.

Cosine Similarity, Clearly Explained!!!StatQuest with Josh Starmer · 10 min

Cosine similarity on word counts of short texts, the same setting as the cosine figure above.

• Boyd and Vandenberghe, Introduction to Applied Linear Algebra (VMLS), free PDF: chapter 1 (vectors, addition, inner product) and chapter 3 (norm, distance, angle).
• Deisenroth, Faisal and Ong, Mathematics for Machine Learning, free PDF: §3.1–3.4 (norms, inner products, lengths and distances, angles and orthogonality).
• Next step: Linear Algebra lesson 1 (matrices as maps, span and basis).