0:01

Hi, guys.

Rich here, one of your friendly Coursera TA's at the University of Washington.

And today I'm here to talk to you about math.

As the course progresses, I'll be going through a couple little miniature

mathematical tutorials regarding the topics that come up during the lectures

for those of you who could maybe use a refresher on the mathematics.

I won't be going through detailed proofs and derivations,

which I'm sure some of you are very thankful for.

Instead, I'll be focusing more on the intuition and how we can start to think

about different mathematical objects in the terms of computational neuroscience.

1:14

So, some of you may have heard about vectors in a physics or engineering class,

for example.

Things that have a direction and a magnitude or whatever, whatever.

But today, while we won't be inconsistent with that understanding,

we are going to start to think about vectors in a more abstract way

that will be useful for us as computational neuroscientists.

2:00

Nothing more, nothing less.

Just a list of numbers.

What's one example?

What are several examples?

Here's an example, (3,1).

That's a list of two numbers.

Here's another example (-10,8), another list of two numbers.

want to get crazy?

Let's add another number.

Let's say (3, 6, -5).

That's a list of three numbers.

Let's make a list of five numbers,

(-1, 2, 8, 6, -5).

3:29

This one is a 5D vector.

Now, there's something very special about 2 and

3D vectors that is not quite the case with higher dimensional vectors.

And that is that, for 2 and 3D vectors you can draw them as arrows or

points in the plane or in some three-dimensional space.

How would we draw the vector 3,1?

First we would start with our 2D plane.

And let's label the axis x1 and x2, and

we want to plot the vector 3,1 in our plane.

How do we do that?

4:11

First we look up the x1 coordinates.

Well, that's just the first number, that's 3.

So we going to go over 1, 2, 3.

Now we look up the x2 coordinates.

That's 1.

So we've been over 3,

since the x2 coordinate is 1 we go up 1 and we put a point there.

Pretty crazy, right?

I know, I thought so too.

Sometimes, maybe in physics and engineering,

you'll see the vectors drawn as arrows.

But often in neuroscience you'll just see them as dots.

So for now we'll just leave it as a dot.

4:49

Sound good?

So that's 3,1.

What if we wanted to plot the vector?

-2, 4.

Well we go over to over -2, and up 4.

1, 2, 3, 4.

So we can plot multiple vectors in the plane as points.

You can do the same thing with 3D vectors if you're good at drawing 3D.

I am not, unfortunately.

So, I wont give you an example there.

However, when you get to four dimensional and

five dimensional vectors it becomes awfully hard to draw pictures.

Yet, all the mathematical wonders we can do with 2 and

3D vectors we can do with 5D vectors as well.

So now we're going to talk about the basic operations you can do with vectors.

And the reason we introduce them with vectors

is because we want to understand their applications to functions as well.

What is a very simple operation?

This first simple operation is just called the sum operation.

Let's say we have the vector x equals, you can put numbers, but I'm just

going to leave them as variables for now so that they can mean anything you want.

And I'm going to have them be arbitrarily dimensioned.

So, what is the dimensionality of this vector?

This is just ND vector, a list of N numbers.

How do we take the sum?

Well, it's very simple.

All you do is add all of the elements together.

Not too hard, right?

So, with the sum operation.

What goes in?

A vector goes in, that was a list of numbers.

And what comes out?

Another vector?

No, a scalar, which is just a single number.

And the reason the output is a scalar is because each of the elements

in the vector x was a scalar.

So when we add a bunch of scalars together, we get another scalar.

All right, next and most important operation is the dot product.

This will come up over and over again in your life.

And if you understand it, you will be a better person.

So how does the dot product work?

First of all, how do we write the dot product?

So let's say vector x = (x1,

x2,..., xn,

and the vector y = y1,

y2, ..., yn.

Then the dot product is just written as x.y.

Pretty crazy.

In order to compute the dot product, all we do is multiply together

the elements of x and y, element ys and then add the whole thing up.

We'll start with x1 and y1, multiply those together.

And to that we add the product of x2y2, and to that we add the product of x3y3.

And we keep going and going and

going until we get to the end, where we add the product xnyn.

8:15

So let's do an example.

Let's say x, we'll just use four d vectors for now.

Let's say x is minus 1,0, 5 and

2 and y is equal to 0, 2 minus 3, 6.

What is x.y equal for?

How do we do it?

Let's start with the first element,

is minus 1 times 0, 0 times 2,

then 5 times minus 3, added to 2 times 6.

Did I do that right?

I think so.

So what's that look like?

0 plus 0 minus 15 plus 12 equals minus 3.

Not too hard, right?

9:33

The input is two vectors.

And what's the output?

The output, here let's draw arrows on these things so they make more sense.

So the input to the dot product is two vectors and the output is a scalar.

An important thing to not is that you can take the sum and

dot products regardless of the dimensionality

of the vector or vectors in the case of the dot product.

This will come in handy later.

Okay, but returning to the dot product, why is the dot product useful?

Some of you may enjoy multiplying pairs of numbers together and

adding them up on a Friday night.

But I promise there's actually a reason to do this too, besides mere boredom.

Let's return to our geometric understanding of vectors.

So this is x1, that's

x2 and let's take the dot product between two vectors,

where we draw them out and see what things look like geometrically.

11:17

And what's x2?

So that's x1, x2 you go over 1 up 5, that's x2.

Oops, let's stay consistent and leave the 1 up there.

What are we actually doing when we are computing the dot product?

What the dot product does is it tells us how aligned two vectors are.

So, how it works, and this is actually one of the reasons I sometimes call it

a dot product projection, a term you will be hearing a lot throughout our lectures.

Is that we imagine a light shining.

So we are going to draw an axis perpendicular to x1 and

we are going to imagine a light, a big bright light,

maybe the sun, shining down on x2 and on x1.

12:13

And this light will cast a shadow of x2 over x1,

or if you like it projects x2 onto x1.

Crazy, right?

Now we can ask, what is the product of the magnitude of x1 and

the component of x2 that is lined up with x1.

Or what is the product of the magnitude of this blue vector and

the magnitude of this red vector that was x1?

Well, it is the dot product.

12:50

All we do is calculate the dot product.

So x1.x2 equals 3 times

minus 1 plus 3 times 5

which is equal to 12.

So, that means that the length of the blue vector, or the magnitude of the blue

vector, multiplied by the magnitude of the red vector x1 is equal to 12.

I'm not going to prove to you why that is the case, but

if you're interested it's a fun exercise for the reader.

But, anyhow,

that's a good way of thinking about what the dot product does geometrically.

It's kind of hard to picture for things like five or ten or

20 dimensional vectors, but it works in generally the same.

It calculates the amount that the two vectors are lined up.

So, just as a quick thing, let's introduce, what color can we use?

Cyan.

Vector x3 which is equal to 4 minus 4.

Where would we draw that?

1, 2, 3, 4, up 4 so that's x3.

15:12

Now we are going to introduce functions.

And the idea is that what you have learned about vectors

will help you understand what you can do with functions.

So what's a function?

Just a refresher.

Here's an example of a function f of x equals 3x minus 2.

A function takes an input or argument x and

it spits out a function value f of x.

So if x were equal to 2, we would say f of 2 is equal to 3 times 2 is 6 minus 2 is 4.

Not too hard, right?

15:50

And what does this function look like?

Well, first of all it's just a line and it's got a slope of three and

y-intercept of, -2.

So we start at -2 and for every one we go over we go up 3, about like that and

it extends forever in to the infinite beyond in both directions.

Okay, so that's one way to think of a function and

a way that a lot of you have probably encountered before.

Today, we are going to try to think of a function as a vector.

That might sound crazy, but

I promise there are crazier things in math than thinking of a function as a vector.

16:31

So let's pick a new example and

I'll try to tell you what the heck I'm talking about.

Let's pick the example f(x), we'll call it f1(x),

just in case we need to introduce more functions, is equal to x/2.

And we're going to say that x is between 0 and 1, we'll make that inclusive.

17:32

And how does that work?

Well, well, let's do this.

Let's pick a few points on our function.

Let's say we'll pick 0 because everybody loves 0.

We'll pick 0.1, 0.2, 0.3, 0.4, 0.5, 0.6 and so forth.

And now what we're going to do is we're going to write down the function

values at each of those numbers.

So this was 0, 0.1, 0.2, 0.3,

these are the x values, 0.4, and so on and so forth.

So what is at f(0)?

f1(0)?

That's just 0.

What is f1(0.1)?

That's 0.1 over 2, which is point 0.05.

What is half of 0.02?

Well that's just 0.02 over 2, 0.1.

So we can continue to fill this out and

hopefully this is starting to look a little familiar to you.

18:43

Now we didn't have to choose 0.1 as our increment.

We could have chose 0.01 in which case

our vector would look like 0.005,

0.01, 0.015 and so on and so forth.

And if we wanted to not only 0.01 as our increment for

x, but 0.001 or 0.0001.

And in fact, you can keep making your increments smaller and smaller and

smaller until you have an infinite list of numbers.

But, the key thing to remember is that

the function is still just a list of numbers.

And I claim this is true because if you had all the numbers for

every increment that would specify your function completely.

19:39

Therefore, the function is a list of numbers or,

as we like to call it, a vector.

Is that crazy?

Yeah, it's pretty crazy.

And I'm not going to go into the math to actually prove that, but

hopefully the intuition makes some sense.

Okay.

Now that we can think of functions as vectors.

And if that doesn't quite sit comfortably with you spend a few minutes.

Pause the video and just really let yourself that this is true,

that functions are vectors.

Because they are nothing more than a really,

really, really detailed list of numbers.

20:21

Okay, that makes sense.

So let's move on to operations, but

now performed not with vectors, but with functions.

And we actually only talked about two operations of vectors, and

those will be the two operations we talk about with functions.

So the first one in the vector world it was the sum.

And for those of you who have taken calculus, what happens to sums when we

move our elements infinitesimally closer together and

let there be an infinite number of them?

Well, that becomes an integral.

And the sum becomes an integral because that is how you would most naturally add

up an infinite list of numbers, which was our function.

21:11

The integral, if we have some arbitrary function, half of x versus x,

and we'll again have it justified under the domain from 0 to 1.

The integral is just the area under the curve.

That's not too hard really.

There are lots of good calculus textbooks that tell you exactly how to compute

the area under the curve.

But what we're interested in is just the fact that it is this integral.

So here's another reason why the sum and the integral are very similar.

23:03

Well, when we deal with infinities we go to calculus.

So that sum over all of the pair-wise products

becomes an integral over the pair-wise products.

So what does that look like?

The projection of one function onto another.

Let's say the projection of, and

sometimes we write this in the same way as a dot product.

We say f1 of x dotted with f2

of x is equal to the integral.

And let's just work from 0 to 1 now.

You can pick other bounds if you like, but for now since we've been using 0 and

1 as our bounds, we'll stick with those.

The dot product of the two functions is the integral of the products

of the functions.

Crazy.

Does that make sense?

I hope so.

And the intuition is that this is just the natural generalization

of dot products to the world of functions.

Sums become integrals.

So for our sum operation, we were adding together a bunch of elements.

That becomes an integral.

In the dot products operation, we were adding together a bunch of these products.

But remember each product was just a number.

So it was a sum of a bunch of numbers, therefore,

in the function world we're just doing an integral of these products.

Hopefully that makes sense.

Okay, so let's do a quick example of how we would compute the dot product of two

functions, or the projection of one function onto another.

So, remember that is the integral.

Let's make it general, let's just say from a to b,

f1(x) times f2(x), dx.

So let's say that f1(x), we'll draw that in green,

is something like a line, so f1(x).

Always label your axes.

And let's say f2(x) is something that looks like that.

25:55

That's not too bad, right?

I think that's okay.

So that's the product of f1(x) and f2(x).

And then the integral is just the area under that curve.

So the integral of f1(x) and f2(x) from

a to b, dx is just the shaded area.

Cool?

Cool.

So what is the dot product between two functions actually telling you?

27:40

So let's say f1(x) is this parabola like thing,

and f2(x), some sort of sinusoid like that.

That's f2(x), and so this is a and b.

What is the dot product of f1(x) and f2(x)?

How do we figure this out?

Well, we multiply f1(x) and

f2(x) together.

So in this part, it's negative because f1(x) is positive, but f2(x) is negative.

And in this part, it's positive because f1(x) and f2(x) are both positive.

And now we take the integral.

So, if you were to take the integral of that blue curve,

which is the product of f1(x) and f2(x) what would you get?

And the answer is 0.

Why is the answer zero?

Well, the first lobe is under the x-axis, so that's a negative area.

And the second lobe is above the x-axis, so that's a positive area.

And since the functions were at least supposed to be kind of symmetric,

the area underneath the x-axis cancels out the area above the x-axis,

which means that the integral is 0.

So you would say that these two functions are orthogonal.

So we would write that f1(x) and

f2(x) are orthogonal.

And so all orthogonal means in this case is that they don't overlap at all.

Or, rather, their overlaps cancel each other out.

That's about it for right now.

I hope that made some sense.

I hope that made things not more confusing, but

at least a little less confusing.

The idea Idea of projecting one function on to another, will come up over and

over again, especially in convolutions in Linux systems.

So it's a good thing to make sure you understand at least basic idea of.

Okay, that's it, see you next time.