Matrix Multiplication and Its Properties

Matrix multiplication is not element-wise multiplication—it computes dot products between rows and columns. It models feature weighting, coordinate transformations, and multi-layer neural network propagation.

I’ll start by writing (m × n) @ (n × p) = (m × p). This dimensional rule is more reliable than memorizing formulas.

In previous chapters, we covered the fundamentals of matrix addition and scalar multiplication. Matrix multiplication is a core operation in linear algebra—and especially critical in machine learning and computer science. In this chapter, we will delve deeply into the definition, properties, and practical applications of matrix multiplication.

Definition of Matrix Multiplication

Matrix multiplication is an operation that multiplies two matrices—but not all matrices can be multiplied. Suppose we have two matrices $A$ and $B$, of dimensions $m \times n$ and $n \times p$, respectively. Their product $C = AB$ yields an $m \times p$ matrix. Formally, matrix multiplication is defined as:

When learning matrix multiplication, first verify that the number of columns in the left matrix matches the number of rows in the right matrix; then confirm the resulting dimensions and associativity. Do not confuse element-wise multiplication with matrix multiplication.

$$C_{ij} = \sum_{k=1}^{n} A_{ik} B_{kj}$$

This means the entry $C_{ij}$—located in row $i$, column $j$ of matrix $C$—is the sum of the products of corresponding elements from row $i$ of matrix $A$ and column $j$ of matrix $B$.

Example

Consider two matrices:

$$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}$$

We compute $C = AB$:

1. Compute $C_{11}$: $C_{11} = 1 \cdot 5 + 2 \cdot 7 = 5 + 14 = 19$
2. Compute $C_{12}$: $C_{12} = 1 \cdot 6 + 2 \cdot 8 = 6 + 16 = 22$
3. Compute $C_{21}$: $C_{21} = 3 \cdot 5 + 4 \cdot 7 = 15 + 28 = 43$
4. Compute $C_{22}$: $C_{22} = 3 \cdot 6 + 4 \cdot 8 = 18 + 32 = 50$

Thus, matrix $C$ is:

$$C = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}$$

Properties of Matrix Multiplication

Matrix multiplication possesses several important properties—widely used in theory and practice. Here are key ones:

When reviewing Matrix Operations: Matrix Multiplication and Properties, you need not tackle large projects at once. Start with one simple example to verify whether the core logic is clear.

If Matrix Operations: Matrix Multiplication and Properties hasn’t yet been fully internalized, revisit the four actions on this card to retrace your learning path.

After finishing Matrix Operations: Matrix Multiplication and Properties, treat the flowchart in this figure as a checklist: Is the problem well-defined? Are operations concretely implemented? Can the evaluation criteria be reused?

1. Associativity: Matrix multiplication is associative: $(AB)C = A(BC)$.
2. Distributivity: Matrix multiplication distributes over addition:
   $A(B + C) = AB + AC$ and $(A + B)C = AC + BC$.
3. Non-commutativity: In general, matrix multiplication is not commutative: $AB \neq BA$.
4. Identity Matrix: There exists an identity matrix $I_n$ such that for any $n \times n$ matrix $A$, $AI_n = I_n A = A$.

Associativity Example

Consider three matrices:

$$A = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix}, \quad B = \begin{pmatrix} 5 & 6 \\ 7 & 8 \end{pmatrix}, \quad C = \begin{pmatrix} 9 & 10 \\ 11 & 12 \end{pmatrix}$$

We compute both $(AB)C$ and $A(BC)$ to verify equality.

1. Compute $AB$:

$$AB = \begin{pmatrix} 1 \cdot 5 + 2 \cdot 7 & 1 \cdot 6 + 2 \cdot 8 \\ 3 \cdot 5 + 4 \cdot 7 & 3 \cdot 6 + 4 \cdot 8 \end{pmatrix} = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix}$$

2. Compute $(AB)C$:

$$(AB)C = \begin{pmatrix} 19 & 22 \\ 43 & 50 \end{pmatrix} \begin{pmatrix} 9 & 10 \\ 11 & 12 \end{pmatrix} = \begin{pmatrix} 19 \cdot 9 + 22 \cdot 11 & 19 \cdot 10 + 22 \cdot 12 \\ 43 \cdot 9 + 50 \cdot 11 & 43 \cdot 10 + 50 \cdot 12 \end{pmatrix}$$

Carrying out the arithmetic yields:

$$(AB)C = \begin{pmatrix} 209 & 244 \\ 553 & 640 \end{pmatrix}$$

3. Compute $BC$:

$$BC = \begin{pmatrix} 5 \cdot 9 + 6 \cdot 11 & 5 \cdot 10 + 6 \cdot 12 \\ 7 \cdot 9 + 8 \cdot 11 & 7 \cdot 10 + 8 \cdot 12 \end{pmatrix} = \begin{pmatrix} 99 & 108 \\ 119 & 140 \end{pmatrix}$$

4. Compute $A(BC)$:

$$A(BC) = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \begin{pmatrix} 99 & 108 \\ 119 & 140 \end{pmatrix} = \begin{pmatrix} 1 \cdot 99 + 2 \cdot 119 & 1 \cdot 108 + 2 \cdot 140 \\ 3 \cdot 99 + 4 \cdot 119 & 3 \cdot 108 + 4 \cdot 140 \end{pmatrix}$$

Computing gives:

$$A(BC) = \begin{pmatrix} 337 & 388 \\ 603 & 712 \end{pmatrix}$$

Wait—this contradicts associativity! Let’s double-check our arithmetic.

Actually, the earlier calculation of $(AB)C$ contains an error. Correct computation:

• $C_{11} = 19 \cdot 9 + 22 \cdot 11 = 171 + 242 = 413$
• $C_{12} = 19 \cdot 10 + 22 \cdot 12 = 190 + 264 = 454$
• $C_{21} = 43 \cdot 9 + 50 \cdot 11 = 387 + 550 = 937$
• $C_{22} = 43 \cdot 10 + 50 \cdot 12 = 430 + 600 = 1030$

So $(AB)C = \begin{pmatrix} 413 & 454 \\ 937 & 1030 \end{pmatrix}$.

Now recompute $A(BC)$ correctly:

• $BC = \begin{pmatrix} 5\cdot9+6\cdot11 & 5\cdot10+6\cdot12 \\ 7\cdot9+8\cdot11 & 7\cdot10+8\cdot12 \end{pmatrix} = \begin{pmatrix} 45+66 & 50+72 \\ 63+88 & 70+96 \end{pmatrix} = \begin{pmatrix} 111 & 122 \\ 151 & 166 \end{pmatrix}$

Then:

• $A(BC) = \begin{pmatrix} 1 & 2 \\ 3 & 4 \end{pmatrix} \begin{pmatrix} 111 & 122 \\ 151 & 166 \end{pmatrix} = \begin{pmatrix} 1\cdot111 + 2\cdot151 & 1\cdot122 + 2\cdot166 \\ 3\cdot111 + 4\cdot151 & 3\cdot122 + 4\cdot166 \end{pmatrix} = \begin{pmatrix} 111+302 & 122+332 \\ 333+604 & 366+664 \end{pmatrix} = \begin{pmatrix} 413 & 454 \\ 937 & 1030 \end{pmatrix}$

✅ Indeed, $(AB)C = A(BC) = \begin{pmatrix} 413 & 454 \\ 937 & 1030 \end{pmatrix}$. This confirms associativity.

Real-World Application Example

In machine learning, matrix multiplication underpins numerous algorithms. For instance, consider a simple linear regression model:

$$Y$$