Symmetry, group representations and E(3)-equivariant networks

Before this lesson: lesson 09 (real spherical harmonics, rotations of the coefficients), linear algebra lesson 04 (orthogonal matrices) and linear algebra lesson 19 (equivariance and the numerical test). Helpful: deep learning lesson 13 (message passing in graph neural networks). Time: about 110 minutes, plus the notebook.

---

Why this matters #

• Interatomic potentials. ML interatomic potentials such as NequIP, Allegro and MACE predict the energy of a configuration of atoms and get the forces by automatic differentiation. They are used in molecular dynamics of materials, catalysts and liquids. NequIP (Batzner et al., 2022) reports that adding vector ($l = 1$) features to a scalars-only ($l = 0$) network lowers the force error for every training-set size they tried.
• Data efficiency. The symmetry does not have to be learned, so fewer training examples are needed, and the model is exact for orientations it never saw.
• Geoscience. A three-component seismogram is a vector. The seismic moment tensor of an earthquake and the stress tensor are symmetric $3 \times 3$ matrices: an isotropic part (1 number) plus a deviatoric part (5 numbers). Wind is a vector field on the sphere.
• The same algebra as lesson 09. The reason why an SFNO filter may depend on the degree $l$ but not on the order $m$ is Schur's lemma, proved here.

1. Predict first #

2. The picture #

Invariance and equivariance #

Let $x = (x_1, \dots, x_n)$ be the positions of $n$ atoms. Move all atoms by a rotation $R$ and a shift $t$: $x_i \mapsto Rx_i + t$. The energy $E(x)$ must not change: $E$ is invariant. The force on atom $i$ must turn with the atoms: $F_i(Rx + t) = R\,F_i(x)$. The forces are equivariant. The two facts are linked: forces are $F_i = -\partial E/\partial x_i$, and the gradient of an invariant function is equivariant (Theorem 5 below).

Interactive figure: loading…

Rotations of the plane: a first look at irreps #

Start in 2D. Let $R_\alpha$ rotate the plane by the angle $\alpha$. Write a 2D vector $(a, b)$ as the complex number $z = a + ib$; then $R_\alpha$ multiplies $z$ by $e^{i\alpha}$. Call a quantity a quantity of frequency $m$ if a rotation multiplies it by $e^{im\alpha}$:

• $m = 0$: a scalar (temperature, pressure); it does not change.
• $m = 1$: a vector (a horizontal wind).
• $m = 2$: a quantity that returns to itself after a rotation by $180°$, for example the traceless part of a symmetric $2 \times 2$ strain matrix, or an axis without a direction.

Rotations never mix different frequencies. Two facts follow, and both carry over to 3D:

• Linear maps. A linear map that commutes with all rotations sends frequency $m$ only to frequency $m$, and there multiplies by one fixed number. So no nonzero equivariant linear map turns a vector into a scalar.
• Products. If $z_1$ has frequency $m_1$ and $z_2$ has frequency $m_2$, then $z_1 z_2$ has frequency $m_1 + m_2$ and $\bar z_1 z_2$ has frequency $m_2 - m_1$. For two vectors $u \leftrightarrow z_1$ and $v \leftrightarrow z_2$, $\bar z_1 z_2 = u\cdot v + i\,(u_1v_2 - u_2v_1)$ has frequency 0 (the dot product and the 2D cross product), and $z_1 z_2$ has frequency 2. So the 4 numbers of $uv^\top$ split as $1 + 1 + 2$.

Three dimensions: degrees and parity #

Rotations in 3D do not commute, but the same structure holds. The "frequencies" become the degrees $l = 0, 1, 2, \dots$ of lesson 09. A degree-$l$ quantity has $2l + 1$ components:

• $l = 0$: 1 number (energy, charge, a distance);
• $l = 1$: 3 numbers (a position difference, a force, a velocity);
• $l = 2$: 5 numbers (a symmetric traceless $3 \times 3$ matrix: deviatoric stress, a quadrupole).

A rotation mixes the $2l + 1$ components of one degree among themselves, and never mixes different degrees. Reflections add one more label, the parity: what happens under the inversion $x \mapsto -x$. A vector changes sign (odd). The cross product of two vectors does not, since $(-u) \times (-v) = u \times v$ (even). The cross product is a pseudovector.

Interactive figure: loading…

Nonlinearities need care #

A network needs nonlinearities. On scalars any function is fine, because a rotation does not change a scalar. On vectors, a function applied to each component separately depends on the choice of axes. Two safe choices for a vector $v$: a gate $\sigma(s)\,v$, where $s$ is a scalar feature, and a norm nonlinearity $\phi(\|v\|)\,v$. Both multiply $v$ by a rotation-invariant number.

Interactive figure: loading…

The design of an e3nn-type network #

Put together, the rules give the whole design of an equivariant network in the style of the e3nn library:

1. Store each atom's features as a list of irreps, for example 16 scalars, 8 vectors and 4 degree-2 features (16x0e + 8x1o + 4x2e).
2. Describe each edge (pair of neighbouring atoms) by its length (invariant) and its direction, encoded by the spherical harmonics $Y_l$ of the unit edge vector.
3. Combine edge directions with neighbour features by Clebsch–Gordan tensor products.
4. Mix channels within each degree.
5. Apply nonlinearities to scalars only, and scale the higher degrees by scalar gates.

3. Definitions and main results #

3.1 Groups, representations and equivariance #

A group $G$ is a set of transformations that contains the identity and is closed under composition and inverses. A representation of $G$ on a vector space $V$ gives each $g \in G$ an invertible matrix $\rho(g)$ with $\rho(gh) = \rho(g)\rho(h)$. A map $f$ is equivariant if

and invariant if $\rho_{\text{out}}(g) = I$ (linear algebra lesson 19).

The groups of 3D space:

• $O(3)$: all orthogonal $3 \times 3$ matrices $Q$ ($Q^\top Q = I$). $SO(3)$: those with $\det Q = +1$, the rotations.
• $E(3)$: all maps $x \mapsto Qx + t$ with $Q \in O(3)$ and $t \in \R^3$. $SE(3)$: the same with $Q \in SO(3)$.
• In 3D, $\det(-I) = -1$, so every $Q$ with $\det Q = -1$ is $-I$ times a rotation. $O(3)$ is the rotations plus the inversion.

Translations are handled by the inputs. A network that sees only relative vectors $x_j - x_i$ is invariant to $t$, because these vectors do not change. So the real work is $O(3)$.

3.2 Irreducible representations #

A subspace $U \subseteq V$ is invariant if $\rho(g)\,U \subseteq U$ for all $g$. A representation is irreducible (an irrep) if its only invariant subspaces are $\{0\}$ and $V$.

For a compact group such as $SO(3)$ there is an inner product in which every $\rho(g)$ is orthogonal (average any inner product over the group). Then the orthogonal complement of an invariant subspace is invariant too. So every finite-dimensional representation is a direct sum of irreps: in a suitable basis all $\rho(g)$ are block diagonal with irrep blocks.

The irreps of $SO(3)$ (we state this without proof): there is exactly one for each degree $l = 0, 1, 2, \dots$, of dimension $2l + 1$. We call its matrices $D^l(R)$. The irreps of $O(3)$ add a parity $p = \pm 1$: the inversion acts as $p\,I$. The e3nn library writes them 0e, 1o, 2e, … (e = even, $p = +1$; o = odd, $p = -1$).

Irrep	Components	Examples
0e	1	energy, charge, a distance $\|x_i - x_j\|$, a dot product $u\cdot v$
0o	1	pseudoscalar: the triple product $(u \times v)\cdot w$, the sign of a chirality
1o	3	vector: position difference, force, velocity, a three-component seismogram
1e	3	pseudovector: $u \times v$, angular momentum, the magnetic field
2e	5	symmetric traceless matrix: deviatoric stress or moment tensor, $uu^\top - \tfrac13\|u\|^2 I$

3.3 Real spherical harmonics and the Wigner-D matrix #

Lesson 09 defined the real orthonormal spherical harmonics $Y_l^m$, $-l \le m \le l$. Stack the $2l + 1$ harmonics of degree $l$ into a vector $Y_l(\hat x) \in \R^{2l+1}$, for a unit vector $\hat x$. Two facts:

• The components of $Y_l$ are homogeneous polynomials of degree $l$ in $(x, y, z)$ with zero Laplacian, restricted to the sphere. They are a basis of all such polynomials (a space of dimension $2l + 1$).
• For $l = 1$ they are $\sqrt{3/(4\pi)}$ times the coordinates, in the order $(y, z, x)$ for $m = -1, 0, 1$. The e3nn library reorders them to $(x, y, z)$.

Definition. The real Wigner-D matrix $D^l(R) \in \R^{(2l+1)\times(2l+1)}$ is the matrix with

For $l = 0$, $D^0(R) = 1$. For $l = 1$ in the order $(x, y, z)$, $Y_1(R\hat x) \propto R\hat x$, so $D^1(R) = R$.

Proof of Theorem 1

1. Existence and uniqueness. A component of $Y_l(R\hat x)$ is a homogeneous polynomial of degree $l$ after a linear change of variables, so it is still homogeneous of degree $l$. It still has zero Laplacian, because the Laplacian commutes with rotations. So it lies in the span of the components of $Y_l$, and its coefficients in that basis are unique. They are the rows of $D^l(R)$.
2. Homomorphism. $Y_l(R_1R_2\hat x) = D^l(R_1)\,Y_l(R_2\hat x) = D^l(R_1)\,D^l(R_2)\,Y_l(\hat x)$. By the uniqueness of step 1, $D^l(R_1R_2) = D^l(R_1)\,D^l(R_2)$.
3. Orthogonality. Orthonormality says $\int_{S^2} Y_l\,Y_l^\top\,d\Omega = I$. Substitute $\hat x \to R\hat x$; the area element does not change under rotations, so $I = \int Y_l(R\hat x)\,Y_l(R\hat x)^\top\,d\Omega = D^l(R)\big(\int Y_l Y_l^\top d\Omega\big)D^l(R)^\top = D^l(R)\,D^l(R)^\top$. $\blacksquare$

Each space of degree-$l$ harmonics is irreducible (we do not prove it), so $D^l$ is the degree-$l$ irrep. From the definition you also get a formula to compute it: $D^l(R) = \int_{S^2} Y_l(R\hat x)\,Y_l(\hat x)^\top\,d\Omega$, an integral that the Gauss grid of lesson 09 computes exactly.

Link to lesson 09. A real field $f = \sum_l a_l^\top Y_l$ rotated to $f(R^{-1}\cdot)$ has the coefficients $D^l(R^{-1})^\top a_l = D^l(R)\,a_l$. This is the rule $a_l \mapsto D^l(R)\,a_l$ used there.

Parity. $Y_l(-\hat x) = (-1)^l\,Y_l(\hat x)$, because the components are homogeneous of degree $l$. So spherical harmonics have the parities 0e, 1o, 2e, 3o, …

Characters. The character of a representation is $\chi(R) = \operatorname{tr} D(R)$. For a rotation by the angle $\theta$ about any axis,

Reason: about the $z$-axis, a rotation turns each pair $(Y_l^m, Y_l^{-m})$ by the angle $m\theta$ (lesson 09), a $2 \times 2$ block with trace $2\cos m\theta$, and leaves $Y_l^0$ alone. A rotation by $\theta$ about another axis is $SR_z(\theta)S^{-1}$, and the trace does not change under this similarity. For $l = 1$, $\chi_1 = 1 + 2\cos\theta = \operatorname{tr} R$.

Interactive figure: loading…

3.4 Schur's lemma and equivariant linear layers #

Proof of Theorem 2

1. $\ker W$ is invariant: if $Wv = 0$, then $W D_1(g)v = D_2(g)\,Wv = 0$. Since $D_1$ is irreducible, $\ker W = \{0\}$ or $V_1$.
2. $\operatorname{im} W$ is invariant: $D_2(g)\,Wv = W D_1(g)v \in \operatorname{im} W$. So $\operatorname{im} W = \{0\}$ or $V_2$.
3. If $W \ne 0$, then $\ker W \ne V_1$ and $\operatorname{im} W \ne \{0\}$. So $\ker W = \{0\}$ and $\operatorname{im} W = V_2$: $W$ is invertible. This is (a).
4. For (b), $W$ has a complex eigenvalue $\lambda$. Then $W - \lambda I$ is also equivariant and not invertible, so by (a) it is 0. $\blacksquare$

For the real irreps $D^l$ of $SO(3)$, part (b) also holds with a real $\lambda$, because they stay irreducible over $\C$. For 2D rotations it does not: the real $2 \times 2$ block of frequency $m \ge 1$ commutes with every rotation–scaling (the complex numbers of section 2).

Corollary (equivariant linear layers). Let a feature have $c_l$ channels of degree $l$: $h = (h^{(0)}, h^{(1)}, \dots)$ with $h^{(l)} \in \R^{c_l \times (2l+1)}$, where a rotation acts by $h^{(l)} \mapsto h^{(l)}D^l(R)^\top$ (each row is one channel). A linear map is $SO(3)$-equivariant exactly when

It mixes channels within each degree, uses the same weights for all $2l + 1$ components, and never maps one degree to another. A bias is allowed only on scalars. With parity ($O(3)$), channels of different parity do not mix either.

Proof. Split the map into blocks from one copy of degree $l$ to one copy of degree $l'$. Each block is equivariant, so by Schur's lemma it is 0 if $l \ne l'$ and $\lambda I$ if $l = l'$. Collect the numbers $\lambda$ into $W_l$. A constant output $b$ is equivariant only if $D^l(R)\,b = b$ for all $R$, which forces $b = 0$ for $l \ge 1$. $\blacksquare$

The same argument is behind lesson 09: a spectral filter on the sphere is equivariant only if it is $c_l I$ on each degree.

3.5 The Clebsch–Gordan tensor product #

Take $u$ of degree $l_1$ and $v$ of degree $l_2$. Their tensor product $u \otimes v$ (the outer product $uv^\top$, flattened: $(2l_1 + 1)(2l_2 + 1)$ numbers) transforms by the Kronecker product $D^{l_1}(R) \otimes D^{l_2}(R)$. This representation is reducible.

Proof by characters (take $l_1 \ge l_2$)

1. The trace of a Kronecker product is the product of the traces. So the character of $D^{l_1} \otimes D^{l_2}$ is $\chi_{l_1}(\theta)\,\chi_{l_2}(\theta) = \sum_{m_1}\sum_{m_2} e^{i(m_1 + m_2)\theta}$ (write each $\chi_l$ as $\sum_{|m| \le l} e^{im\theta}$).
2. Count how often each total $M = m_1 + m_2$ appears, with $|m_1| \le l_1$ and $|m_2| \le l_2$. For $|M| \le l_1 + l_2$ the count is $\min(l_1 + l_2 - |M|,\ 2l_2) + 1$.
3. In $\sum_{L = l_1 - l_2}^{l_1 + l_2}\chi_L(\theta) = \sum_L\sum_{|M| \le L} e^{iM\theta}$, the term $e^{iM\theta}$ appears once for each $L$ in the range with $L \ge |M|$. That is $l_1 + l_2 - \max(|M|, l_1 - l_2) + 1 = \min(l_1 + l_2 - |M|,\ 2l_2) + 1$ times. So the two characters are equal.
4. For a compact group, the character fixes a representation up to a change of basis (the number of copies of $D^L$ inside $D$ is $\int \chi_D\,\chi_L\,dR$, by the orthogonality of characters). Every rotation is a rotation by some angle, so the characters agree on all of $SO(3)$. $\blacksquare$

Clebsch–Gordan coefficients. For each allowed $L$ there is an array $C^L \in \R^{(2L+1)\times(2l_1+1)\times(2l_2+1)}$ that extracts the degree-$L$ part:

$D^L$ appears once, so Schur's lemma makes $C^L$ unique up to a factor. In practice $C^L$ can be computed as the null space of the linear equations $C\,(D^{l_1}(R) \otimes D^{l_2}(R)) = D^L(R)\,C$ for a few random $R$ (the notebook does this). For $L$ outside the range the null space is $\{0\}$: this is the selection rule. Parities multiply: $p_{\text{out}} = p_1p_2$. So 1o ⊗ 1o → 0e + 1e + 2e, and 1o ⊗ 2e → 1o + 2o + 3o.

The example to remember: two vectors. For $u, v \in \R^3$ (in the basis where $D^1 = R$),

Each piece transforms on its own: $(Ru)\cdot(Rv) = u\cdot v$; $(Ru)\times(Rv) = R\,(u \times v)$ for $\det R = 1$; and the symmetric traceless part $S$ goes to $RSR^\top$. So $9 = 1 + 3 + 5$, and the dot and cross products are the Clebsch–Gordan products $1 \otimes 1 \to 0$ and $1 \otimes 1 \to 1$.

Interactive figure: loading…

The cost of tensor products, and the eSCN trick

A full tensor product of two features that contain all degrees up to $L_{\max}$ has $O(L_{\max}^3)$ paths $(l_1, l_2) \to L$, each a dense contraction with $O(L_{\max}^3)$ terms: $O(L_{\max}^6)$ work in total (Passaro and Zitnick, 2023). This is why most e3nn-style networks use $L_{\max}$ between 1 and 3.

The eSCN method rotates each edge onto the polar axis before the product. There the spherical harmonics of the edge direction are zero except for $m = 0$ (lesson 09: the factor $\sin^{|m|}\theta$ vanishes at the pole). The product becomes sparse, the $SO(3)$ convolution becomes an $SO(2)$ one, and the cost drops to $O(L_{\max}^3)$. EquiformerV2 uses this.

3.6 An equivariant message-passing layer #

Atoms $i = 1, \dots, n$ have positions $x_i \in \R^3$ and features $h_i = (h_i^{(0)}, h_i^{(1)}, \dots)$. Each atom receives messages from its neighbours $j \in \mathcal N(i)$ within a cutoff radius $r_c$. For an edge, let $r_{ij} = x_j - x_i$, $d_{ij} = \|r_{ij}\|$ and $\hat r_{ij} = r_{ij}/d_{ij}$. One layer of a Tensor Field Network or NequIP-type model:

1. Message. $m_{ij}^{(L)} = \sum_{l_1, l_2} w^{(l_1l_2L)}(d_{ij})\;C^L\big(Y_{l_1}(\hat r_{ij}),\ h_j^{(l_2)}\big)$. The weights $w$ come from a small MLP applied to a radial basis expansion of $d_{ij}$, with a smooth cutoff.
2. Aggregate. $M_i^{(L)} = \sum_{j \in \mathcal N(i)} m_{ij}^{(L)}$.
3. Mix and gate. $h_i^{(L)} \leftarrow h_i^{(L)} + \operatorname{Gate}\big(W_L M_i^{(L)}\big)$. $W_L$ mixes channels (the Corollary). Scalars go through a nonlinearity $\sigma$. Each channel $c$ of degree $L \ge 1$ is multiplied by $\sigma(g_c)$, where $g_c$ is an extra scalar channel (the gate).
4. Readout. Energy $E = \sum_i \operatorname{MLP}(h_i^{(0)})$ (invariant); forces $F_i = -\partial E/\partial x_i$ (equivariant).

Proof.

1. $r_{ij} = x_j - x_i \mapsto Rr_{ij}$: the translation cancels. So $d_{ij}$ and every weight $w(d_{ij})$ are invariant.
2. $Y_{l_1}(R\hat r_{ij}) = D^{l_1}(R)\,Y_{l_1}(\hat r_{ij})$ (the definition of $D^l$, Theorem 1).
3. $C^L(D^{l_1}a,\ D^{l_2}b) = D^L\,C^L(a, b)$ (Theorem 3). So each term of the message moves by $D^L(R)$, times an invariant weight.
4. A sum of terms that all move by $D^L(R)$ moves by $D^L(R)$: messages and aggregates are equivariant.
5. $W_L$ acts on channels and $D^L$ on components, so they commute (Corollary). A gate multiplies by the invariant number $\sigma(g_c)$, which commutes with $D^L$. On scalars $D^0 = 1$, so any $\sigma$ works.
6. A composition of equivariant maps is equivariant. The energy is a function of scalars only, so it is invariant. $\blacksquare$

For $O(3)$ also track parity: $Y_l$ has parity $(-1)^l$, positions are 1o, and each path must produce the parity of its output channel. An $O(3)$-equivariant network whose outputs are only 0e scalars gives the same output for a molecule and its mirror image; chiral properties need pseudoscalars (0o).

Proof. Differentiate $E(Qx_1 + t, \dots, Qx_n + t) = E(x_1, \dots, x_n)$ with respect to $x_i$. The chain rule gives $Q^\top\,(\nabla_i E)(Qx + t) = (\nabla_i E)(x)$. Multiply by $Q$ (using $QQ^\top = I$): $(\nabla_i E)(Qx + t) = Q\,(\nabla_i E)(x)$. Change the sign. $\blacksquare$ Translation invariance also gives $\sum_i F_i = 0$ (differentiate $E(x + t) = E(x)$ in $t$).

Cheaper models. EGNN (Satorras et al., 2021) uses only scalars and the vectors $x_i - x_j$: messages $m_{ij} = \phi_e(h_i, h_j, \|x_i - x_j\|^2)$, a position update $x_i \leftarrow x_i + C\sum_j (x_i - x_j)\,\phi_x(m_{ij})$ ($C$ a constant), and a scalar update $h_i \leftarrow \phi_h(h_i, \sum_j m_{ij})$. It is an $l \le 1$ model without explicit Clebsch–Gordan products. The review of Duval et al. (2024) calls such models (with PaiNN and TorchMD-Net) Cartesian, the irrep models (TFN, NequIP, MACE, Equiformer) spherical, the distance-and-angle models (SchNet, DimeNet) invariant, and models that learn the symmetry by data augmentation unconstrained.

3.7 How to test equivariance #

1. Draw random inputs, random $Q \in O(3)$ (both signs of $\det Q$) and a random $t \in \R^3$.
2. Compute $e = \max\big|f(Qx + t) - \rho(Q)f(x)\big|$. On an output of type $(l, p)$, $\rho(Q) = D^l(Q)$ if $\det Q = +1$, and $\rho(Q) = p\,D^l(-Q)$ if $\det Q = -1$ (then $-Q$ is a rotation).
3. In float64 an exactly equivariant model gives $e$ near $10^{-16}$ to $10^{-12}$ (relative); in float32, near $10^{-7}$. Also check $D^l(R)\,D^l(R)^\top = I$.
4. Run the same test on a model that is not equivariant and see it fail. A test that cannot fail proves nothing.

Answers to section 1. (1) No. Equivariance needs $\tanh(Rf) = R\tanh(f)$ for all $R$. For $f = (2, 0, 0)$ and the rotation by $45°$ about $z$: $Rf = (\sqrt2, \sqrt2, 0)$, so $\tanh(Rf) \approx (0.888, 0.888, 0)$, but $R\tanh(f) = \tanh(2)\,(1, 1, 0)/\sqrt2 \approx (0.682, 0.682, 0)$. Use a gate $\sigma(s)\,f$ or a norm nonlinearity. (2) Into $1 + 3 + 5$: the dot product ($L = 0$), the cross product ($L = 1$) and the symmetric traceless part ($L = 2$). This is $1 \otimes 1 = 0 \oplus 1 \oplus 2$. (3) Zero. By Schur's lemma, a linear equivariant map from degree 1 to degree 0 is 0. To get a scalar from vectors you need a nonlinear operation, such as $\|v\|^2$ or a dot product of two vectors.

4. Worked example #

(a) Two vectors, by hand. Take $u = (1, 0, 0)$ and $v = (1, 1, 0)$. Then $uv^\top = \begin{pmatrix}1&1&0\\0&0&0\\0&0&0\end{pmatrix}$.

• $L = 0$: $u\cdot v = 1$, so the piece is $\tfrac13 I$.
• $L = 1$: $u \times v = (0, 0, 1)$, and the piece is $A = \tfrac12(uv^\top - vu^\top) = \begin{pmatrix}0&\frac12&0\\-\frac12&0&0\\0&0&0\end{pmatrix}$.
• $L = 2$: $S = \tfrac12(uv^\top + vu^\top) - \tfrac13 I = \begin{pmatrix}\frac23&\frac12&0\\\frac12&-\frac13&0\\0&0&-\frac13\end{pmatrix}$: symmetric, trace $\tfrac23 - \tfrac13 - \tfrac13 = 0$.
• The pieces are orthogonal in the Frobenius inner product, so the squared norms add up: $\|uv^\top\|_F^2 = 2 = \tfrac13 + \tfrac12 + \tfrac76$. Check: $\|\tfrac13 I\|_F^2 = 3 \cdot \tfrac19$, $\|A\|_F^2 = 2 \cdot \tfrac14$, $\|S\|_F^2 = \tfrac49 + 2 \cdot \tfrac14 + \tfrac19 + \tfrac19 = \tfrac76$.

Now turn both vectors by $90°$ about $z$, with $R = \begin{pmatrix}0&-1&0\\1&0&0\\0&0&1\end{pmatrix}$: $Ru = (0, 1, 0)$ and $Rv = (-1, 1, 0)$.

• $Ru\cdot Rv = 0 \cdot (-1) + 1 \cdot 1 + 0 = 1$: unchanged.
• $Ru \times Rv = (1 \cdot 0 - 0 \cdot 1,\ 0 \cdot (-1) - 0 \cdot 0,\ 0 \cdot 1 - 1 \cdot (-1)) = (0, 0, 1) = R\,(0, 0, 1)$: it turns with the vectors (here the axis of $R$ is $z$, so $(0, 0, 1)$ stays).
• The new symmetric traceless part is $\begin{pmatrix}-\frac13&-\frac12&0\\-\frac12&\frac23&0\\0&0&-\frac13\end{pmatrix}$, which equals $RSR^\top$.

(b) Characters. For a rotation by $\theta = 90°$: $\chi_0 = 1$, $\chi_1 = 1 + 2\cos 90° = 1$, $\chi_2 = 1 + 2\cos 90° + 2\cos 180° = -1$, $\chi_3 = 1 + 2(\cos 90° + \cos 180° + \cos 270°) = -1$.

• $1 \otimes 1$: $\chi_1^2 = 1$ and $\chi_0 + \chi_1 + \chi_2 = 1 + 1 - 1 = 1$. ✓
• $1 \otimes 2$: $\chi_1\chi_2 = -1$ and $\chi_1 + \chi_2 + \chi_3 = 1 - 1 - 1 = -1$. ✓
• At $\theta = 120°$: $\chi_1 = 0$, $\chi_2 = -1$, $\chi_3 = 1$, and again $\chi_1\chi_2 = 0 = \chi_1 + \chi_2 + \chi_3$. ✓

(c) Weights of an equivariant linear layer. Input and output 16x0e + 8x1o + 4x2e: that is $16 + 8 \cdot 3 + 4 \cdot 5 = 60$ numbers. A general linear map has $60^2 = 3600$ weights. The equivariant one has $16^2 + 8^2 + 4^2 = 336$, plus 16 biases on the scalars.

(d) A seismic moment tensor. The moment tensor $M$ of an earthquake is a symmetric $3 \times 3$ matrix (6 numbers). Under a rotation $R$ of the source, $M \mapsto RMR^\top$, and the inversion leaves it unchanged ($(-R)M(-R)^\top = RMR^\top$). So $M = \tfrac13\operatorname{tr}(M)\,I + M_{\text{dev}}$ is 0e ⊕ 2e: the isotropic part (a volume change; an explosion $M = I$ is pure $l = 0$) and the deviatoric part (5 numbers). A vertical strike-slip double couple $\begin{pmatrix}0&1&0\\1&0&0\\0&0&0\end{pmatrix}$ has trace 0, so it is pure $l = 2$. An equivariant network that predicts moment tensors outputs one 0e and one 2e channel.

5. In code #

One message step on atoms with scalar and vector features, tested under random orthogonal matrices of both determinants and random shifts. The gated layer passes; a layer that adds $\hat r \times v$ (a pseudovector, 1e) to a vector channel (1o) passes for rotations and fails for reflections; a componentwise $\tanh$ fails always.

import numpy as np
from scipy.stats import ortho_group

def layer(X, s, V, mode):
    """One message step: positions X (n,3), scalars s (n,), vectors V (n,3) -> new vectors (n,3)."""
    D = X[None, :, :] - X[:, None, :]                       # D[i, j] = x_j - x_i
    r = np.linalg.norm(D, axis=-1)
    np.fill_diagonal(r, 1.0)
    phi = np.exp(-r**2) * (1 - np.eye(len(X)))              # invariant radial weight, no self-loop
    rhat = D / r[..., None]                                 # unit edge vectors (1o)
    m0 = (phi * (s[None, :] + (rhat * V[None]).sum(-1))).sum(1)          # 0e message: s_j + r_hat . v_j
    m1 = (phi[..., None] * (s[None, :, None] * rhat + V[None])).sum(1)   # 1o message: s_j r_hat + v_j
    if mode == "cross":                                     # adds r_hat x v_j, a pseudovector (1e)
        m1 = m1 + (phi[..., None] * np.cross(rhat, V[None])).sum(1)
    if mode == "tanh":                                      # tanh on each component
        return np.tanh(m1)
    return m1 / (1 + np.exp(-m0))[:, None]                  # gate: vector times sigmoid(scalar)

rng = np.random.default_rng(0)
X, s, V = rng.standard_normal((6, 3)), rng.standard_normal(6), rng.standard_normal((6, 3))
for mode in ("gate", "cross", "tanh"):
    errs = []
    for k in range(4):
        Q = ortho_group.rvs(3, random_state=10 + k)         # random orthogonal matrix, det +1 or -1
        t = rng.standard_normal(3)
        out = layer(X, s, V, mode)
        out_moved = layer(X @ Q.T + t, s, V @ Q.T, mode)    # move the input: x -> Qx + t, v -> Qv
        errs.append(f"det {round(np.linalg.det(Q)):+d}: {np.abs(out_moved - out @ Q.T).max():.0e}")
    print(f"{mode:5s}", ", ".join(errs))
# gate  det +1: 3e-16, det +1: 3e-16, det +1: 3e-16, det -1: 2e-16    O(3)-equivariant
# cross det +1: 4e-16, det +1: 4e-16, det +1: 4e-16, det -1: 1e+00    only SO(3): parity bug
# tanh  det +1: 1e-01, det +1: 2e-01, det +1: 2e-01, det -1: 2e-01    not equivariant

The cross layer is a typical bug. A test with only $\det Q = +1$ would not find it. The notebook also computes Wigner-D matrices and Clebsch–Gordan coefficients (as a null space) and builds an EGNN layer and a force field.

6. Where it shows up #

• ML interatomic potentials. NequIP, Allegro and MACE predict an invariant energy from equivariant features and get the forces by autodiff. MACE builds many-body messages from tensor products of a per-atom "density" of neighbours.
• Catalysis and proteins. eSCN and EquiformerV2 were developed on the Open Catalyst datasets (OC20, OC22). AlphaFold 3 has no built-in equivariance in its diffusion module; it trains with random rotations and translations of the structures (the unconstrained approach).
• Geoscience. Moment tensors and stress (0e + 2e), three-component seismograms and their polarization (1o), and anisotropy tensors. On the sphere, the degree-wise filters of SFNO (lesson 09) are the Schur's-lemma answer.
• When the symmetry is broken on purpose. The atmosphere is not rotation-symmetric: Earth's rotation axis, the Coriolis parameter, orography and the land–sea mask fix directions and places. Weather models give these fields as inputs, which breaks the symmetry where the physics breaks it (lesson 16).
• Practice tools. e3nn (PyTorch): o3.Irreps, o3.spherical_harmonics, o3.FullyConnectedTensorProduct, nn.Gate, and assert_equivariant for tests.

7. Common mistakes #

• Componentwise nonlinearities or normalisation on vectors. $\tanh$, ReLU or a per-component scale break equivariance. Use gates or functions of the norm.
• Mixing degrees or parities in a linear layer. Schur's lemma forbids it. 1e and 1o look the same under rotations and differ under reflections.
• Testing too little. One random rotation, only $\det Q = +1$, or no translation. Test reflections and shifts, and make sure the test can fail.
• Basis conventions. The order of the $l = 1$ components ($(y, z, x)$ or $(x, y, z)$), the normalisation of $Y_l$, and $D^l(R)$ versus $D^l(R^{-1})$ differ between libraries. Fix one convention and test it.
• Hard cutoffs. A hard cutoff at $r_c$ makes the energy jump when an atom crosses it, so the forces are wrong there. Use a smooth envelope.
• Assuming a symmetry the data does not have. Gravity, an external field or Earth's rotation break $O(3)$. Give them as inputs: an equivariant model with the gravity vector as an input feature is still exactly equivariant.
• Direct force heads by default. A head that predicts forces directly saves a backward pass, but its forces are not the gradient of an energy, so molecular dynamics with them does not conserve energy. Choose on purpose.

8. Check yourself #

1. Which degrees appear in $2 \otimes 2$, and does the dimension count work?
   Answer

   $L = 0, 1, 2, 3, 4$, and $1 + 3 + 5 + 7 + 9 = 25 = 5 \times 5$.
2. How many weights does an $O(3)$-equivariant linear map from 8x1o to 1x0e have?
   Answer

   Zero: the only such map is 0 (Schur's lemma; the irreps differ). To get an invariant from vectors you need a nonlinear operation, for example $\|v\|^2$ or the tensor product $1 \otimes 1 \to 0$ of two vectors (a dot product).
3. Compute $\chi_2$ for a rotation by $180°$, and check $1 \otimes 1 = 0 \oplus 1 \oplus 2$ at this angle.
   Answer

   $\chi_2(\pi) = 1 + 2\cos\pi + 2\cos 2\pi = 1$ and $\chi_1(\pi) = 1 + 2\cos\pi = -1$. So $\chi_1^2 = 1 = \chi_0 + \chi_1 + \chi_2 = 1 - 1 + 1$.
4. A layer passes the equivariance test for all rotations but fails for $Q = -I$. Give a likely cause and a fix.
   Answer

   A parity error: an even quantity (for example $\hat r \times v$, `1e`) is added to an odd channel (`1o`), or a pseudoscalar is used as a scalar. Track the parity of every path ($p_1p_2$) and keep `1e` and `1o` in separate channels.
5. An $O(3)$-equivariant network predicts the moment tensor of an earthquake. Which output irreps does it need?
   Answer

   `1x0e + 1x2e` (6 numbers): the isotropic part (the trace) and the deviatoric part. A symmetric matrix has no antisymmetric (`1e`) part, and the moment tensor does not change under the inversion, so both parts are even.
6. Your float32 model gives a relative equivariance error of $3 \times 10^{-7}$. A model trained with random rotations gives $2 \times 10^{-2}$. What do you conclude?
   Answer

   The first is exactly equivariant: the error is at float32 rounding level. The second is only approximately equivariant: augmentation teaches the symmetry on the training distribution, with no guarantee elsewhere.

Recall cardsAnswer in your head first, then reveal. Add the cards to your review deck to see them again just before you would forget.

1. What does it mean that a map $f$ is equivariant to a group $G$?

   Show answer

2. What is the dimension of the irrep of $SO(3)$ of degree $l$?

   Show answer

3. What do the labels 1o and 1e mean, with an example of each?

   Show answer

4. State Schur's lemma for an equivariant linear map between two irreps.

   Show answer

5. Which output degrees $L$ does the tensor product of degrees $l_1$ and $l_2$ contain?

   Show answer

6. Into which pieces does $uv^\top$ split for two vectors $u, v$?

   Show answer

7. Why are forces computed as $-\nabla E$ from an invariant energy equivariant?

   Show answer

8. How do you test a model for E(3) equivariance numerically?

   Show answer

9. Why can you not apply ReLU to each component of a vector feature?

   Show answer

Add these cards to my reviewOpen the review →

9. Interview angle #

• "Why can you not apply a ReLU to a vector feature?" $\sigma(Rv) \ne R\sigma(v)$ in general. Use gates $\sigma(s)\,v$ with an invariant $s$, or norm nonlinearities $\phi(\|v\|)\,v$.
• "What is an irrep, and why does e3nn keep the degrees separate?" The smallest blocks that rotations cannot split further. By Schur's lemma, equivariant linear maps cannot mix them.
• "What is a Clebsch–Gordan tensor product?" The equivariant bilinear product of two irreps, projected onto each degree $L$ with $|l_1 - l_2| \le L \le l_1 + l_2$. For two vectors: the dot product, the cross product and the symmetric traceless part.
• "Built-in equivariance or data augmentation?" Built-in: exact symmetry, better data efficiency (NequIP's $l = 0$ versus $l \ge 1$ comparison), conservative and equivariant forces from an invariant energy, but tensor products cost $O(L_{\max}^6)$ ($O(L_{\max}^3)$ with eSCN). Augmentation: any architecture and good scaling with data (AlphaFold 3), but the symmetry is only approximate.
• "Would you make a global weather model $SO(3)$-equivariant?" Only partly. Earth's rotation, orography and land–sea contrasts break the symmetry. Use equivariant building blocks (SFNO filters) plus inputs that carry the broken symmetry.

10. Practice #

The notebook for this lesson runs in Colab and in the browser. It has five exercises, each with a check cell and a folded solution:

1. Write the numerical equivariance test and use it to find the parity bug and the tanh bug in three given layers.
2. Compute real Wigner-D matrices by quadrature; check orthogonality, the homomorphism property, the character formula and $D^1 = PRP^\top$ in the $(y, z, x)$ order.
3. Split $uv^\top$ into its $0 \oplus 1 \oplus 2$ parts and check how each part transforms under $O(3)$.
4. Write an EGNN layer (scalars and relative positions only) and test its E(3) equivariance.
5. Write a pair-potential energy and its analytic forces; check them against finite differences and test that the forces are equivariant and sum to zero.

11. Watch and read #

e3nn Tutorial MRS 2021 Fall Meeting -- Tutorial 1/6 -- Tess Smidte3nn · 46 min

Watch for how geometric tensors are sorted into irreps, and why the network keeps them separate.

e3nn Tutorial MRS 2021 Fall Meeting -- Tutorial 2/6 -- Mario Geigere3nn · 49 min

The e3nn library in practice: irreps, spherical harmonics, tensor products and equivariance tests.

Group Equivariant Deep Learning - Lecture 1.2: Group theory (product, inverse, representations)Erik Bekkers · 23 min

A short, careful introduction to groups and representations, the language of section 3.1.

AMMI Course "Geometric Deep Learning" - Lecture 8 (Groups & Homogeneous spaces) - Taco CohenMichael Bronstein · 66 min

The general theory behind equivariant networks: groups acting on spaces, and why convolutions are equivariant.

Reading

• Duval et al., A Hitchhiker's Guide to Geometric GNNs for 3D Atomic Systems (2024): the Cartesian, spherical, invariant and unconstrained model families, tensor products and gating. arXiv 2312.07511
• The e3nn user guide: irreducible representations, convolutions and equivariance testing. docs.e3nn.org
• Batzner et al., E(3)-equivariant graph neural networks for data-efficient and accurate interatomic potentials (NequIP, 2022). arXiv 2101.03164
• Bronstein, Bruna, Cohen and Veličković, Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges (2021). arXiv 2104.13478
• Passaro and Zitnick, Reducing SO(3) Convolutions to SO(2) for Efficient Equivariant GNNs (eSCN, 2023). arXiv 2302.03655