Newer
Older
MIT_OCW_Linear_Algebra_18_06 / I_04_Matrix_multiplication_Inverses.ipynb
juanklopper on 13 May 2015 18 KB MIT_OCW_Linear_Algebra_18_06
not connected
In [1]
from IPython.core.display import HTML, Image
css_file = 'style.css'
HTML(open(css_file, 'r').read())
Out [1]
In [2]
from sympy import init_printing, Matrix, symbols, eye, Rational
from warnings import filterwarnings
In [3]
init_printing(use_latex = 'mathjax')
filterwarnings('ignore')

Matrix multiplication, inverse and transpose

Multiplying matrices

Method 1

  • Consider multiply matrices A and B to result in C
  • We have already seen that the column size of the first must equal the row size of the second, nA must equal mB
    $$ {m}{A}\times{n}{A}\cdot{m}{B}\times{n}{B} \ = {m}{A}\cdot{n}{B} $$
  • C will then be of size mA × nB
  • Every position cij, with i as the row position and j as the column position is calculated by taking the dot product (i.e. each element times it's corresponding element, all added), cij = (row i in A ⋅ column j of B)
  • Here we calculate the row 2, column 1 position in C by the dot product of row 2 in A by column 1 in B
    $$ { \begin{bmatrix} \cdots & \cdots & \cdots \ 3 & 2 & -1 \ \cdots & \cdots & \cdots \ \cdots & \cdots & \cdots \end{bmatrix} }{ 4\times 3 }{ \begin{bmatrix} 1 & \vdots \ 2 & \vdots \ 1 & \vdots \end{bmatrix} }{ 3\times 2 }={ \begin{bmatrix} { c }{ 11 } & { c }{ 12 } \ \left( 3\times 1 \right) +\left( 2\times 2 \right) +\left( -1\times 1 \right) & { c }{ 22 } \ { c }{ 31 } & { c }{ 32 } \ { c }{ 41 } & { c }{ 42 } \end{bmatrix} }{ 4\times 2 } $$
  • Notice how this multiplication is only possible because the row size of A equals the column size of B
    $$ { \begin{bmatrix} \cdots & \cdots & \cdots \ { a }{ 21 } & { a }{ 22 } & { a }{ 23 } \ \cdots & \cdots & \cdots \ \cdots & \cdots & \cdots \end{bmatrix} }{ 4\times 3 }{ \begin{bmatrix} { b }{ 11 } & \vdots \ { b }{ 21 } & \vdots \ { b }{ 31 } & \vdots \end{bmatrix} }{ 3\times 2 }={ \begin{bmatrix} { c }{ 11 } & { c }{ 12 } \ \left( { a }{ 21 }{ b }{ 11 } \right) +\left( { a }{ 22 }{ b }{ 21 } \right) +\left( { a }{ 23 }{ b }{ 31 } \right) & { c }{ 22 } \ { c }{ 31 } & { c }{ 32 } \ { c }{ 41 } & { c }{ 42 } \end{bmatrix} }{ 4\times 2 }\ =\sum { k=1 }^{ n }{ { a }{ 2k }{ b }_{ k1 } } $$

Method 2

  • In this method we note that each column in C is the result of the matrix A times the corresponding column in B
  • This is akin to a matrix multiplied by a vector Ax=b
  • We see B as made up of vector columns
  • The columns of C are thus combinations of columns of A
    • The numbers in the corresponding columns in B is this combination

Method 3

  • Here every row in A produces the same numbered row in C by multiplying it with the matrix B
  • The rows of C are linear combinations of B

Method 4

  • In method 1 we looked at rowA × colB producing a single number in C
  • What if we did column × row?
  • The size of column of A is mA × 1 and a row of B is of size 1 × nB
  • This results in C of size mA × nB
  • Let's look at a simple example using python (with sympy)
In [4]
A = Matrix([[2], [3], [4]])
B = Matrix([[1, 6]])
A, B
Out [4]
$$\begin{pmatrix}\left[\begin{matrix}2\\3\\4\end{matrix}\right], & \left[\begin{matrix}1 & 6\end{matrix}\right]\end{pmatrix}$$
In [5]
C = A * B
C
Out [5]
$$\left[\begin{matrix}2 & 12\\3 & 18\\4 & 24\end{matrix}\right]$$
  • Notice how the columns of C are linear combinations of the values in the columns of A
  • The rows of C are multiples of the rows of B
  • So in method 4, C is the sum of the columns of A × the rows of B
    $$ \begin{bmatrix} { a }{ 11 } & { a }{ 12 } \ { a }{ 21 } & { a }{ 22 } \ { a }{ 31 } & { a }{ 32 } \end{bmatrix}\begin{bmatrix} { b }{ 11 } & { b }{ 12 } \ b{ 21 } & { b }{ 22 } \end{bmatrix}=\begin{bmatrix} { a }{ 11 } \ { a }{ 21 } \ { a }{ 31 } \end{bmatrix}\begin{bmatrix} { b }{ 11 } & { b }{ 12 } \end{bmatrix}+\begin{bmatrix} { a }{ 12 } \ { a }{ 22 } \ { a }{ 32 } \end{bmatrix}\begin{bmatrix} { b }{ 21 } & { b }{ 22 } \end{bmatrix} $$

Block multiplication

  • Combining the above we can do the following
    • Both A and B can be broken into block of sizes that allow for multiplication
    • Here is an example of two square matrices
      $$ \begin{bmatrix} { A }{ 1 } & { A }{ 2 } \ { A }{ 3 } & { A }{ 4 } \end{bmatrix}\begin{bmatrix} { B }{ 1 } & { B }{ 2 } \ { B }{ 3 } & { B }{ 4 } \end{bmatrix}=\begin{bmatrix} { A }{ 1 }{ B }{ 1 }+{ A }{ 2 }{ B }{ 3 } & { A }{ 1 }{ B }{ 2 }+{ A }{ 2 }{ B }{ 4 } \ { A }{ 3 }{ B }{ 1 }+{ A }{ 4 }{ B }{ 3 } & { A }{ 3 }{ B }{ 2 }+{ A }{ 4 }{ B }{ 4 } \end{bmatrix} $$

Inverses

  • If the inverse of a matrix A exists then A-1=I, the identity matrix
  • Above is a left inverse, but what about a right inverse, AA-1?
    • This is also equal to the identity for invertible square inverses
  • Invertible matrices are also called non-singular matrices
  • Non-invertible matrices are also called singular matrices
  • An example would look like this
    $$ \begin{bmatrix}1&3\2&6\end{bmatrix} $$
  • Note how the elements on row two are just two times the elements in row 1 (A linear combination)
  • The same go for the columns, the first being a linear combination of the second, multiplying each element by 3
  • More profoundly, note that you could find a column vector x such that Ax=0
    $$ \begin{bmatrix}1&3\2&6\end{bmatrix}\begin{bmatrix}3\-1\end{bmatrix}=\begin{bmatrix}0\0\end{bmatrix} $$
  • This says 3 times column 1 in A plus -1 times column 2 gives nothing
  • Let construct as example
    $$ \begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} a & c \ b & d \end{bmatrix}=\begin{bmatrix} 1 & 0 \ 0 & 1 \end{bmatrix} $$
  • In essence we have to solve two systems
    • A × column j of A-1 = column j of I
    • This is the Gauss-Jordan idea of solving two systems at once
      $$ \begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} a \ b \end{bmatrix}=\begin{bmatrix} 1 \ 0 \end{bmatrix}\ \begin{bmatrix} 1 & 3 \ 2 & 7 \end{bmatrix}\begin{bmatrix} c \ d \end{bmatrix}=\begin{bmatrix} 0 \ 1 \end{bmatrix} $$
    • This gives us the two columns of A-1
    • We now create the augmented matrix
      $$ \begin{bmatrix} 1 & 3 & 1 & 0 \ 2 & 7 & 0 & 1 \end{bmatrix} $$
    • Now we use elementary row operations to reduced row-echelon form (leading 1's in the pivot positions, with 0's below and above each)
      $$ \begin{bmatrix} 1 & 3 & 1 & 0 \ 2 & 7 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 3 & 1 & 0 \ 0 & 1 & -2 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} 1 & 0 & 7 & -3 \ 0 & 1 & -2 & 1 \end{bmatrix} $$
    • We now read off the two columns of A-1
      $$ \begin{bmatrix}7&-3\-2&1\end{bmatrix} $$
  • To do all of the elimination, we created a lot of elimination (elementary) matrices
  • If we combine all of them into E we have E[AI]=[IA<sup>-1</sup>], because EA=I tells us E=A-1

Example problems

Example problem 1

  • Find the conditions on a and b that makes the matrix A invertible and find A-1
    $$ A=\begin{bmatrix} a & b & b \ a & a & b \ a & a & a \end{bmatrix} $$

Solution

  • A matrix is singular (non-invertible) if we have a row or column of zeros, so a ≠ 0
  • We can also not have similar columns, so a ≠ b
  • Using Gauss-Jordan elimination we will have the following
    $$ \begin{bmatrix} a & b & b & 1 & 0 & 0 \ a & a & b & 0 & 1 & 0 \ a & a & a & 0 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & a-b & 0 & -1 & 1 & 0 \ 0 & a-b & a-b & -1 & 0 & 1 \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & a-b & 0 & -1 & 1 & 0 \ 0 & 0 & a-b & 0 & -1 & 1 \end{bmatrix}\ \rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & \frac { a-b }{ a-b } & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & \frac { a-b }{ a-b } & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\rightarrow \begin{bmatrix} a & b & b & 1 & 0 & 0 \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ \rightarrow \begin{bmatrix} a & b & 0 & 1 & \frac { 1 }{ a-b } \left( b \right) & -\frac { 1 }{ a-b } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\rightarrow \begin{bmatrix} a & 0 & 0 & 1+\frac { b }{ a-b } & 0 & -\frac { 1 }{ a-b } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ \rightarrow \begin{bmatrix} 1 & 0 & 0 & \frac { 1 }{ a-b } & 0 & -\frac { 1 }{ a\left( a-b \right) } \left( b \right) \ 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } & 0 \ 0 & 0 & 1 & 0 & \frac { -1 }{ a-b } & \frac { 1 }{ a-b } \end{bmatrix}\ { A }^{ -1 }=\frac { 1 }{ a-b } \begin{bmatrix} 1 & 0 & \frac { -b }{ a } \ -1 & 1 & 0 \ 0 & -1 & 1 \end{bmatrix} $$
  • Additionally then we note that for the inverse of A to exist a - b ≠ 0, which is the same as ab and again a ≠ 0
In [ ]