0:00

>> In this module we'll review vectors. I will go over definitions of vectors,

what are row vectors, column vectors. We're going to define linear independence,

bases, transposes, inner products, lengths of vectors or norms.

All of these concepts are going to be needed for doing some of the elementary

linear algebra that's needed in this course.

What's a vector? It's just a collection of real numbers.

You can collect them, and put them in a row or you can put them in a column.

So here is an example of a row vector. Where I've put everything as a row here's

an example of a column vector and if you notice carefully both of these vectors

have n components. So we will call a vector to have n

components if it consists of n real numbers.

A row vector or column vector and we will denote any of these vectors by the symbol

r to the n. R to the n means that every component is a

real number and they, there are n of them in the place.

To just fix ideas it might be easier to think in terms of R2 which means these are

vectors with just two components and it's easiest to think of these vectors as

belonging to a plane. And we're used to calling this plane by

the x axis and the y axis. So think of x axis being one of the

components of the vector. Y axis being another component of the

vector. So, let me just give you some examples of

vectors that we going to use later on in this module.

So, here's one vector. It's x component or the first component is

going to be one and the second component is say going to be equal to two.

So that the vector will be this vector. At the end of the dot the dot corresponds

to a y axis of two and x axis of one. So was labeled this as V.

This will simply be the vector one two. The first component equal to one in the

second component equal to two. Here's another example.

The x axis, or the first component equal to 4 and y axis equal to 1.

So that's that point, and I'm going to be connecting it with this vector.

And in the rest of this module, I'm going to referring to this vector as w.

It has 4 in the first component, and 1 in the second component.

By default in this course unless otherwise specified we will assume that all vectors

are column vectors, that is all the components have been arranged as a column.

So we now know what a vector is, they are collections of real numbers without, by

default it's going to be a column vector. Now we want to understand what can we

represent using these vectors, what happens with them and so on So the first

thing that we want to do is multiply these vectors with real numbers and add them up.

So v1 and v2 are vectors. And I want to multiply them by a real

number, alpha 1. And a real number, alpha 2.

And then get a final vector w. So, to give you an example, here is my

vector, v1. One, here is my vector V two.

And just to fix ideas, each of these V one and V two actually lives in R to the

three. Why R to the three?

Every component is a real number, and there are three components.

So the three refers to the fact that there are three components.

I'm going to multiply them by two real numbers, so two is going to play the role

of alpha one. 4 is going to play the role of alpha 2.

And what do I do? I take these real numbers, multiply them

component by component and add them up. So, in order to get the first component, I

take the 2, multiply it to the 1, take the 4, multiply it to the 0.

I get 2 times 1 plus 4 times 0 equals 2. That gives me the first component.

If I want to go to the second component then I take the 2 and multiply it to the

second component 1. Take the 4 multiply it to the second

component which is also 1, 2 times 1 is 2, 4 times 1 is 4 add it up together you get

6. Want to go to the third component the same

thing, 2 times 0. 4 times z, 1, 2 times 0 is 0.

4 times 1 i 4. You get a total component 4.

So when you write a vector w as a combination of vectors v1 and v2, we're

going to say that this Vector w is linearly dependent on v1 and v2.

Why the linear? Because I'm multiplying by a real number,

and adding them up. Other words are, w's a linear combination

of v1 and v2. Get another word.

W belongs to the linear span of v1 and v2. All of these 3 things mean the same.

Linear dependency, linear combination, linear span.

Now, we'll, in the next set of ideas, we'll need a concept of linear

independence. When can we say that we cannot write a

vector as a linear combination of other vectors.

To fix ideas, let's again go back to r2, and I'm going to use my favorite two

vectors, v and w. So here's my vector v and here's my vector

w. For now we don't really need what the

components are so I'm not going to bother with that.

Now we want to understand what does a linear span mean, what can I, what kind of

vectors can I generate by scaling the w? What can I do by multiplying w by areal

number? And you can convince yourself very easily

That all the vector that you can get, are going to be on that straight line.

When you scale it up you get, by a positive number you get up the line if you

multiply by a negative number you go down the line.

Similarly all the vector that can be gotten as linear combinations of v rely on

this straight line. Now we want to ask ourself can I represent

v as A multiple of w. Clearly that's not true because I can't, I

can't, when I scale the w I get points on this line, v doesn't belong to that line

so I can't do anything about that. Similarly, w is not written as a linear

combination of v and therefore we'll say that v and w are linearly independent.

Let's throw in another vector now, x. It's again a vector in r 2 and I want to

ask myself, is this linearly independent of v and w?

Is it possible that x can not be written as a linear combination of v and w?

Again it's very easy to convince yourself that if you just draw a line parallel to

v, you can write, your vector x as a combination of vector that starts from the

origin, it's aligned to the vector w and comes up to this point.

So therefore, this vector is some alpha 1 times w.

This vector here is, is parallel to the vector v.

So I can write it at sum scale multiple alpha 2 times v.

And because, now, x is the sum of this vector and that vector, you end up getting

that x is actually equal to alpha 1 times w, plus alpha 2 times v which means that x

is linearly dependent, LD just for short, linearly dependent on v and w.

In fact we'll see in the next slide that in R2 if I give you any 2 linearly

independent vectors you can write any other vector as a linear combination of

these 2. And that set of vectors, say these two, v

and w, would actually be called a basis. So that's the next concept that I want to

learn about vectors. A basis is a linearly independent set of

vectors that spans the entire space. Any basis for Rn, meaning a vector which

has n components, has exactly n elements. So basis for Rn has exactly n elements.

In the last page I showed you an example of R2, and this should have just two

elements. V and W are linearly independent, there

were two of them, and therefore I know, that this must be a basis.

Now it will turn out that its much easier to think in terms of a standard basis.

What's a standard basis is shown here. Its a collection of vectors such that they

have only one in one of the components and all of the other components are equal to

zero. So the vector E1 will have 1 in the first

component, the vector E2 is going to have a 1 in the second component, and the

vector En is going to have a 1 in the last, or the nth, component.

Now if you give me any vector W which belongs to Rn meaning that it has n

components, I can very easily write it as a linear combination of E1 through En.

Why, because in each of these vectors is exactly one element that's not zero.

So if I want to get the first element of w, right?

I have to, say, take it to be w1 times e1, because everything else has zero

contribution. If I want the second component, right, I

have to take w2 times e2, nnd so on up to wn times en.

And therefore, this basis. I can split any vector w as a linear

combination very easily. And we will see that, in practice, these

terms ought to be very convenient. Again, to put it in perspective.

Here's r2. Here is my x asix.

Here is my y axis. And when I showed you in the first slide,

x axis refers to the first component. And y axis refers to the second component.

So the vector e1 is just this one. It has one component one in the x

direction and zero in the y direction. E2 is this one, it has a component one in

the y direction and zero in every other direction and it's very easy if I take a

vector x, all I have to do is drop it down here and This, this length down here on

the x-axis is x1, on the y-axis is x2. And it's a very easy way to combine.

But any other set of linearly independent vectors, two of them will make sure that

this is going to be a basis. So, again, this was v, and this was w from

the last page. And v and w is also a basis.

This is also a basis. And E1, E2, which is are these 2 special

ones, are also a basis. The second basis is special.

And we'll just call it the standard basis, because it turns out it's very convienent

to work in terms of this basis. Alright.

So, so far what do we know? We know what are vectors.

We know linear dependence, we know linear independence.

We know that if I am a vector in Rn, meaning it has n components, I can find a

basis of n linearly independent vectors, such that every vector can be written as a

combination of these. So what's the next concept?

The next concept that we want to learn, is that of a length of a vector.

Vector. So let's start with the basics.

And then we'll generalize is to what I want here.

So, if I give you r2. And let's take a vector which has the x

component equal to, let's say, 4. And the y component equal to, let's say.

So this vector, has a representation four, three.

Then our high school trigonometry tells us that the length of this vector is nothing

but 4 squared plus 3 squared. So I'm taking the x-axis and squaring it.

I'm taking the y-axis and squaring it. Square root.

So that gives me 16. Plus 9 square root which gives me 25

square root equaled to 5. So that's nice.

I have a, I have a definition of length. What does this definition of length

satisfy? It satisfies several properties.

It satisfies that the length of any vector is always great than equal to 0.

If the length of a vector is equal to zero, then that vector itself must be

zero. So you cannot have a vector which is not

equal to zero who's length is equal to zero.

And the intuition follows from this triangular expression as well.

If I scale a vector by an amount alpha it doesn't matter whether I scale it

positively or negatively. The length just gets scaled by the

absolute value. So, here's the idea here's the value of

vector v if I double it. >> I get this vector if I multiply it by

minus 2 I get that vector, the length of this vector and the length of that vector

is the same. The direction has changed, but the length

remains the same. That's what this absolute value does.

The third one here. Tells you the relationship between lengths

of additional vectors and the original lengths, and this is known as a triangle

inequality. The way you remember this geometrically is

that if you've got one vector here, you've got another vector there, the sum of the

vectors is this one, the length, this length is always going to be, let's call

it l. Three is always going to be lessthan equal

to L1 plus L2. This is the basic geometic fact L3 has to

be less than equal to L1 plus L2. And this particular fact is encapsulated

in this little >> Statement over here, and that's why it's also called the tirangle

inequality. Now mathematicians have looked at this

concept of length, this particular concept of length that I've been talking about

which is take each of the components, square it, add it up, take the square root

and now they're calling it the I2 norm. The 2 stands for the fact that I'm

squaring it and take the square root and you'll notice that the properties that I

want. The fact that the length is greater or

equal to zero. The fact that triangle inequality holds.

The fact that if I scale things remain the same is true for other definitions of

length. Now, since I am taping this in New York,

I'm going to try to tell you about a particular notion of language, which is

called the L1 norm, otherwise known as the Manhattan distance.

I'm sitting at a particular point here, I want to travel to another point, I can

only go In blocks north and south or east and west, and I want to figure out how

much do I need to walk in order to get from this point to that point.

So I have to walk from this point to this point and then walk from that point to

that point. So this distance is 4, that distance is 3,

so the L1 distance, or the L1 length, or the Manhattan distance is 4 plus 37.

The L two distance is five because I can sort of cut across.

The L one distance is seven. L one also satisfies all the properties of

a norm or a length and therefor it's sometimes convenient to encapsulate all of

them as just norms and lengths. For the purposes of this course we will

mostly be focusing on L two. But I just wanted you to know, that there

are other definitions of length, that are interesting, and sometimes become

important in applications. Alright, now we know vectors, we know

linear combination, we know lengths. Now we want to go to the next concept,

which is that of an angle. And in order to get to an angle, I have to

introduce this idea of an inner product or a dot product.

So the dot product of any two vectors v and w, so v and w are vectors in Rn.

They have n components. So the dot product between v and w is

simply going to be taking the ith component of v, taking the ith component

of w, multiplying them together, and adding them up.

For those of you who are experts in Excel, this is nothing but sum product, take the

product of the components and add them up. The sum product function in excel is

nothing but an inner product or dot product.

If you review back, into the last slide we had defined the L 2 norm there to be every

component's squared square root and that can now be written that length L 2 norm of

the vector is simply take the vector v, take its dot product.

So v, dot product with v, and take the square root.

So now we want to understand an angle between these two vectors, so to do that,

here's a picture. Here's my w, here's my v, here's an angle

theta between them. So in order to understand how the inner

product relates to the angle, the length of this vector is The length we, the

length of this vector is w. So that inner product of v and w, simply

is, take a, take the component of v along w and multiply to the length of w.

So if you drop down an orthogonal point over here, this component.

Is exactly equal to the norm, or the length, of v cosine of theta, and that I'm

going to multiply with the length of w, and that should give me v dot w, which

ends up giving me the cosine of theta Is exactly equal to vw divided by the norm of

v, and the norm of w, or the length of v, and the length of w.

And, to emphasize the fact that these are all 2 norms.

I'm just going to put a 2 there. All of that is encapsulated in this slide.

Cosine of theta is v.w divided by v and w. And this is true, not just in our 2, but

in our n. You an define angles in rn.

I'm going to show you in the next module that v dot w is actually a combination of

two operations, the transpose operation and the matrix multiplication operation.

But that has to wait until we get to the module of matrixes.

So that pretty much brings us to the end of the introduction to vectors.

This is all we need to do in terms of vectors, and this is all we're going to be

using in the course. In the next prerequisite module, the next,

the module that comes up next, we're going to review concepts about matrixing