Older
MIT_OCW_Linear_Algebra_18_06 / I_04_Matrix_multiplication_Inverses.ipynb
juanklopper on 13 May 2015 18 KB MIT_OCW_Linear_Algebra_18_06
not connected
In [1]
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
Out [1]
In [2]
from sympy import init_printing, Matrix, symbols, eye, Rational
from warnings import filterwarnings
In [3]
init_printing(use_latex = 'mathjax')
filterwarnings('ignore')

# Matrix multiplication, inverse and transpose

## Multiplying matrices

### Method 1

• Consider multiply matrices A and B to result in C
• We have already seen that the column size of the first must equal the row size of the second, nA must equal mB
$${m}{A}\times{n}{A}\cdot{m}{B}\times{n}{B} \ = {m}{A}\cdot{n}{B}$$
• C will then be of size mA × nB
• Every position cij, with i as the row position and j as the column position is calculated by taking the dot product (i.e. each element times it's corresponding element, all added), cij = (row i in A ⋅ column j of B)
• Here we calculate the row 2, column 1 position in C by the dot product of row 2 in A by column 1 in B
$${ \begin{bmatrix} \cdots & \cdots & \cdots \ 3 & 2 & -1 \ \cdots & \cdots & \cdots \ \cdots & \cdots & \cdots \end{bmatrix} }{ 4\times 3 }{ \begin{bmatrix} 1 & \vdots \ 2 & \vdots \ 1 & \vdots \end{bmatrix} }{ 3\times 2 }={ \begin{bmatrix} { c }{ 11 } & { c }{ 12 } \ \left( 3\times 1 \right) +\left( 2\times 2 \right) +\left( -1\times 1 \right) & { c }{ 22 } \ { c }{ 31 } & { c }{ 32 } \ { c }{ 41 } & { c }{ 42 } \end{bmatrix} }{ 4\times 2 }$$
• Notice how this multiplication is only possible because the row size of A equals the column size of B
$${ \begin{bmatrix} \cdots & \cdots & \cdots \ { a }{ 21 } & { a }{ 22 } & { a }{ 23 } \ \cdots & \cdots & \cdots \ \cdots & \cdots & \cdots \end{bmatrix} }{ 4\times 3 }{ \begin{bmatrix} { b }{ 11 } & \vdots \ { b }{ 21 } & \vdots \ { b }{ 31 } & \vdots \end{bmatrix} }{ 3\times 2 }={ \begin{bmatrix} { c }{ 11 } & { c }{ 12 } \ \left( { a }{ 21 }{ b }{ 11 } \right) +\left( { a }{ 22 }{ b }{ 21 } \right) +\left( { a }{ 23 }{ b }{ 31 } \right) & { c }{ 22 } \ { c }{ 31 } & { c }{ 32 } \ { c }{ 41 } & { c }{ 42 } \end{bmatrix} }{ 4\times 2 }\ =\sum { k=1 }^{ n }{ { a }{ 2k }{ b }_{ k1 } }$$

### Method 2

• In this method we note that each column in C is the result of the matrix A times the corresponding column in B
• This is akin to a matrix multiplied by a vector Ax=b
• We see B as made up of vector columns
• The columns of C are thus combinations of columns of A
• The numbers in the corresponding columns in B is this combination

### Method 3

• Here every row in A produces the same numbered row in C by multiplying it with the matrix B
• The rows of C are linear combinations of B

### Method 4

• In method 1 we looked at rowA × colB producing a single number in C
• What if we did column × row?
• The size of column of A is mA × 1 and a row of B is of size 1 × nB
• This results in C of size mA × nB
• Let's look at a simple example using python (with sympy)
In [4]
A = Matrix([[2], [3], [4]])
B = Matrix([[1, 6]])
A, B
Out [4]
$$\begin{pmatrix}\left[\begin{matrix}2\\3\\4\end{matrix}\right], & \left[\begin{matrix}1 & 6\end{matrix}\right]\end{pmatrix}$$
In [5]
C = A * B
C
Out [5]
$$\left[\begin{matrix}2 & 12\\3 & 18\\4 & 24\end{matrix}\right]$$
• Notice how the columns of C are linear combinations of the values in the columns of A
• The rows of C are multiples of the rows of B
• So in method 4, C is the sum of the columns of A × the rows of B
$$\begin{bmatrix} { a }{ 11 } & { a }{ 12 } \ { a }{ 21 } & { a }{ 22 } \ { a }{ 31 } & { a }{ 32 } \end{bmatrix}\begin{bmatrix} { b }{ 11 } & { b }{ 12 } \ b{ 21 } & { b }{ 22 } \end{bmatrix}=\begin{bmatrix} { a }{ 11 } \ { a }{ 21 } \ { a }{ 31 } \end{bmatrix}\begin{bmatrix} { b }{ 11 } & { b }{ 12 } \end{bmatrix}+\begin{bmatrix} { a }{ 12 } \ { a }{ 22 } \ { a }{ 32 } \end{bmatrix}\begin{bmatrix} { b }{ 21 } & { b }{ 22 } \end{bmatrix}$$

### Block multiplication

• Combining the above we can do the following
• Both A and B can be broken into block of sizes that allow for multiplication
• Here is an example of two square matrices
$$\begin{bmatrix} { A }{ 1 } & { A }{ 2 } \ { A }{ 3 } & { A }{ 4 } \end{bmatrix}\begin{bmatrix} { B }{ 1 } & { B }{ 2 } \ { B }{ 3 } & { B }{ 4 } \end{bmatrix}=\begin{bmatrix} { A }{ 1 }{ B }{ 1 }+{ A }{ 2 }{ B }{ 3 } & { A }{ 1 }{ B }{ 2 }+{ A }{ 2 }{ B }{ 4 } \ { A }{ 3 }{ B }{ 1 }+{ A }{ 4 }{ B }{ 3 } & { A }{ 3 }{ B }{ 2 }+{ A }{ 4 }{ B }{ 4 } \end{bmatrix}$$

## Inverses

• If the inverse of a matrix A exists then A-1=I, the identity matrix
• Above is a left inverse, but what about a right inverse, AA-1?
• This is also equal to the identity for invertible square inverses
• Invertible matrices are also called non-singular matrices
• Non-invertible matrices are also called singular matrices
• An example would look like this
$$\begin{bmatrix}1&3\2&6\end{bmatrix}$$
• Note how the elements on row two are just two times the elements in row 1 (A linear combination)
• The same go for the columns, the first being a linear combination of the second, multiplying each element by 3
• More profoundly, note that you could find a column vector x such that Ax=0
$$\begin{bmatrix}1&3\2&6\end{bmatrix}\begin{bmatrix}3\-1\end{bmatrix}=\begin{bmatrix}0\0\end{bmatrix}$$
• This says 3 times column 1 in A plus -1 times column 2 gives nothing
• Let construct as example
$$\begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} a & c \ b & d \end{bmatrix}=\begin{bmatrix} 1 & 0 \ 0 & 1 \end{bmatrix}$$
• In essence we have to solve two systems
• A × column j of A-1 = column j of I
• This is the Gauss-Jordan idea of solving two systems at once
$$\begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} a \ b \end{bmatrix}=\begin{bmatrix} 1 \ 0 \end{bmatrix}\ \begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} c \ d \end{bmatrix}=\begin{bmatrix} 0 \ 1 \end{bmatrix}$$
• This gives us the two columns of A-1
• We now create the augmented matrix
$$\begin{bmatrix} 1 & 3 & 1 & 0 \ 2 & 7 & 0 & 1 \end{bmatrix}$$
• Now we use elementary row operations to reduced row-echelon form (leading 1's in the pivot positions, with 0's below and above each)
$$\begin{bmatrix} 1 & 3 & 1 & 0 \ 2 & 7 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 3 & 1 & 0 \ 0 & 1 & -2 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 0 & 7 & -3 \ 0 & 1 & -2 & 1 \end{bmatrix}$$
• We now read off the two columns of A-1
$$\begin{bmatrix}7&-3\-2&1\end{bmatrix}$$
• To do all of the elimination, we created a lot of elimination (elementary) matrices
• If we combine all of them into E we have E[AI]=[IA<sup>-1</sup>], because EA=I tells us E=A-1

## Example problems

### Example problem 1

• Find the conditions on a and b that makes the matrix A invertible and find A-1
$$A=\begin{bmatrix} a & b & b \ a & a & b \ a & a & a \end{bmatrix}$$

#### Solution

• A matrix is singular (non-invertible) if we have a row or column of zeros, so a ≠ 0
• We can also not have similar columns, so a ≠ b
• Using Gauss-Jordan elimination we will have the following
$$\begin{bmatrix} a & b & b & 1 & 0 & 0 \ a & a & b & 0 & 1 & 0 \ a & a & a & 0 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & a-b & 0 & -1 & 1 & 0 \ 0 & a-b & a-b & -1 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & a-b & 0 & -1 & 1 & 0 \ 0 & 0 & a-b & 0 & -1 & 1 \end{bmatrix}\ \rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & \frac { a-b }{ a-b } & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & \frac { a-b }{ a-b } & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ \rightarrow \begin{bmatrix} a & b & 0 & 1 & \frac { 1 }{ a-b } \left( b \right) & -\frac { 1 }{ a-b } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\rightarrow \begin{bmatrix} a & 0 & 0 & 1+\frac { b }{ a-b } & 0 & -\frac { 1 }{ a-b } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ \rightarrow \begin{bmatrix} 1 & 0 & 0 & \frac { 1 }{ a-b } & 0 & -\frac { 1 }{ a\left( a-b \right) } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ { A }^{ -1 }=\frac { 1 }{ a-b } \begin{bmatrix} 1 & 0 & \frac { -b }{ a } \ -1 & 1 & 0 \ 0 & -1 & 1 \end{bmatrix}$$
• Additionally then we note that for the inverse of A to exist a - b ≠ 0, which is the same as ab and again a ≠ 0
In [ ]