Difference between revisions of "Linear Algebra"
m (→Vector Dot Product) |
(→Matrix-Vector Product) |
||
Line 89: | Line 89: | ||
=== Matrix-Vector Product === | === Matrix-Vector Product === | ||
+ | |||
+ | The matrix-vector product is frequently encountered in solving [[LinearAlgebra#Linear_Systems_of_Equations|linear systems of equations]], although there are many other uses for it. | ||
+ | |||
+ | Consider a matrix '''A''' with ''m'' rows and ''n'' columns and vector '''b''' with ''n'' rows: | ||
+ | <center><math> | ||
+ | A=\left[\begin{array}{cccc} | ||
+ | a_{11} & a_{12} & \cdots & a_{1n}\\ | ||
+ | a_{21} & a_{22} & \cdots & a_{2n}\\ | ||
+ | \vdots & \vdots & \ddots & \vdots\\ | ||
+ | a_{m1} & a_{m2} & \cdots & a_{mn}\end{array}\right], | ||
+ | \quad | ||
+ | b= | ||
+ | \left(\begin{array}{c} | ||
+ | b_{1}\\ b_{2}\\ \vdots\\ b_{n} | ||
+ | \end{array}\right) | ||
+ | </math></center> | ||
+ | |||
+ | We may multiply '''A''' and '''b''' if the number of columns in '''A''' is equal to the number of rows in '''b'''. The product, '''c''' is a vector with ''m'' rows. The <var>i<sup>th</sup></var> entry in '''c''' is given as | ||
+ | <center><math> | ||
+ | c_i=\sum_{j=1}^n A_{i,j}b_j | ||
+ | </math></center> | ||
+ | |||
+ | For example, let's consider the following case: | ||
+ | <center><math> | ||
+ | A = \left[\begin{array}{ccc} | ||
+ | 1 & 3 & 2 \\ | ||
+ | 2 & 5 & 4 | ||
+ | \end{array}\right], | ||
+ | \quad | ||
+ | b = \left(\begin{array}{c} | ||
+ | 10 \\ 4 \\ 6 | ||
+ | \end{array} \right) | ||
+ | </math></center> | ||
+ | Here we have <var>m=2</var> and <var>n=3</var>. We define the product, c=Ab, as follows: | ||
+ | # Be sure that '''A''' and '''b''' are compatible for multiplication. | ||
+ | #* Since '''A''' has three columns and '''b''' has three rows, they are compatible. | ||
+ | # Determine how many entries are in '''c'''. | ||
+ | #* The '''c''' vector will have as many rows as '''A''' has. | ||
+ | #* Since '''A''' has two rows, '''c''' will have two rows. | ||
+ | # Determine the elements in '''c''' | ||
+ | #* Applying the above formula for <var>i=1</var> we find <center><math> | ||
+ | \begin{align} | ||
+ | c_1 &= \sum_{j=1}^{3} A_{1,j} b_j \\ | ||
+ | &= A_{1,1} b_1 + A_{1,2} b_2 + A_{1,3} b_3 \\ | ||
+ | &= 1 \cdot 10 + 3\cdot 4 + 2 \cdot 6 \\ | ||
+ | &= 34. | ||
+ | \end{align} | ||
+ | </math></center> | ||
+ | #* Applying the above formula for <var>i=2</var> we find <center><math> | ||
+ | \begin{align} | ||
+ | c_2 &= \sum_{j=1}^{3} A_{2,j} b_j \\ | ||
+ | &= A_{2,1} b_1 + A_{2,2} b_2 + A_{2,3} b_3 \\ | ||
+ | &= 2 \cdot 10 + 4\cdot 4 + 4 \cdot 6 \\ | ||
+ | &= 64. | ||
+ | \end{align} | ||
+ | </math></center> | ||
+ | Therefore, we have | ||
+ | <center><math> | ||
+ | c=\left(\begin{array}{c} 34 \\ 64 \end{array}\right). | ||
+ | </math></center> | ||
=== Matrix-Matrix Product === | === Matrix-Matrix Product === |
Revision as of 11:40, 25 August 2008
Contents
Basics
Let's first make some definitions:
- Vector
- A one-dimensional collection of numbers. Examples:
- Matrix
- A two-dimensional collection of numbers. Examples:
- Array
- An n-dimensional collection of numbers. A vector is a 1-dimensional array while a matrix is a 2-dimensional array.
- Transpose
- An operation that involves interchanging the rows and columns of an array. It is indicated by a superscript. For example:
- Vector Magnitude,
- A measure of the length of a vector,
- Example:
- Example:
Matrix & Vector Algebra
There are many common operations involving matrices and vectors including:
These are each discussed in the following sub-sections.
Vector Dot Product
The dot product of two vectors produces a scalar. Physically, the dot product of a and b represents the projection of a onto b.
Given two vectors a and b, their dot product is formed as
where and are the components in vectors a and b respectively. This is most useful when we know the components of the vectors a and b.
Occasionally we know the magnitude of the two vectors and the angle between them. In this case, we can calculate the dot product as
where θ is the angle between the vectors a and b.
For more information on the dot product, see wikipedia's article.
Thought examples
Consider the cartoon shown to the right.- At noon, when the sun is directly overhead, a rocket is launched directly toward the sun at 1000 mph. How fast is its shadow moving?
- Define the rocket's velocity as .
- Define the unit normal on the ground (i.e. the direction of the rocket's shadow on the ground) as , where .
- Intuition tells us that if the it is moving directly toward the sun, then its shadow does not appear to move at all. : The dot product since cos(90°)=0.
- Consider the same rocket going parallel to the earth's surface. How fast is its shadow moving?
- If the rocket is going parallel to the earth's surface, our intuition tells us that is shadow is moving at the same speed. This is confirmed by the mathematics, since the angle between the rocket's path and the ground is 0. Therefore, since cos(0°)=1.
- What if the rocket were going at a 45° angle?
- Our intuition tells us that the shadow will appear to move, but it will not be moving as fast as the rocket is moving. Mathematically, we have .
Vector Cross Product
The cross product of two vectors produces a vector perpendicular to the original two. The common right-hand rule is used to determine the direction of the resulting vector.
Assume that we have vectors a and b defined as
where , , and represent the unit-normal vectors in the x, y, and z directions, respectively. The cross product of a and b is then defined as
If we know the magnitude of a and b and the angle between them (θ), then the cross-product is given as
where n is the unit-normal vector perpendicular to the plane defined by the vectors a and b.
For more information, see wikipedia's article on the cross product.
Matrix-Vector Product
The matrix-vector product is frequently encountered in solving linear systems of equations, although there are many other uses for it.
Consider a matrix A with m rows and n columns and vector b with n rows:
We may multiply A and b if the number of columns in A is equal to the number of rows in b. The product, c is a vector with m rows. The ith entry in c is given as
For example, let's consider the following case:
Here we have m=2 and n=3. We define the product, c=Ab, as follows:
- Be sure that A and b are compatible for multiplication.
- Since A has three columns and b has three rows, they are compatible.
- Determine how many entries are in c.
- The c vector will have as many rows as A has.
- Since A has two rows, c will have two rows.
- Determine the elements in c
- Applying the above formula for i=1 we find
- Applying the above formula for i=2 we find
- Applying the above formula for i=1 we find
Therefore, we have
Matrix-Matrix Product
Linear Systems of Equations
Solving Linear Systems of Equations
|
Direct Solvers
Gaussian Elimination
The Thomas Algorithm (Tridiagonal Systems)
Iterative Solvers
Jacobi Method
Gauss-Seidel Method
Other Methods
- Conjugate-Gradient
- GMRES