So the first thing we want to take into account are user to movie interactions, so, we'll look at each user, each movie in isolation and we won't take into account similarities amongst them, right now, we'll just look at each one in isolation. and see what we can do about making the prediction a little better. And so the first thing you'll notice let's just take a look at this chart over here and we've highlighted, user D and movie three. So one thing you'll notice about user D is that seems kind of harsh in his reviews right? Because he gave a two when everyone else is giving a four or a five. He gave a three to this movie which everyone else gave a four or five, he gave a one where we have two other fours over here. He gave a two. Where again, there's a lot of values higher. So he seems, he seems kind of harsh, like a harsh reviewer. And then on the other hand, if you look at movie three, you know, movie three seems to get pretty good ratings, except, with the exception of the harsh critic D who gave it a three, which is still average. it's getting all fours and fives, whereas other movies are a bit lower, so you could say movie three might be a really good movie, right. so, that's the idea here behind, biasing, right, some, some just tend to have higher mean or average values then others, that's why we can't just average the entire table. We have to take, it's better to take the columns into account as well and the rows and the columns separately. So, this whole idea again, first of easy or harsh reviewers, you can Also imagine that some reviewers, maybe reviewer A over here might just be much easier than the others. you know, whereas D is much harsher and he tends to rate much lower. again people rate differently and we, we pointed out user D for being that one harsh reviewer. at the same time on the movie side there's some movies are good. Some movies are bad, you know, some movies are just higher quality than others and are better perceived by the public. so movie three seems to be a really, really good movie whereas two might be a little lower quality or at least perceived less well by the overall general opinion. And so you know, again movie three is what we pointed out as well here. And this really forms the basis for what we call the baseline predictor. And the baseline predictor. And what we do is we take each of these rating values or we, we start with the average actually and we take we take that raw average, remember last, from last time that was just the 3.5 then we add in a bias for the user and we add in a bias for the movie. So the biases are going to be positive if it's better received or if you tend to be a more lenient critic. And they're going to be negative if it's a movie that's, tends to be worse or if the critic tends to be more harsh. And so we have to find the bias for each of these users. And that's, that's the whole idea here, is that we really need to to find those those bias values. And so finding the bias is, is somewhat of an art actually. there, there is a quote unquote right way to do it probably which would be that we have to solve a complicated optimization problem. [SOUND] Optimization right over here and but we're not dealing with any calculus or linear algebra in this course. We won't go that far, we won't deal with that. We will just look at a simpler approach to doing this. And so we need to find the bias values so we'll try to do something intuitive. Why don't we just compare it to the mean value? Right, so let's just compare it to what the mean was. Again, remember the mean at the overall data set was 3.5 so let's see how much higher or lower the mean is than 3.5, and that's how we find the bias. So we take the average, of the lower column. We'll start with, maybe we'll start with the harsh critic in D, so we'll take. 2 plus 3 plus 1 plus 2 is the average of these values 2, 3, 1 and 2. Again remember we're not including anything that is in the test set when we do these biases out. Divide that by 4, just for those values, and that is 8 divided by 4, which is 2. But then that we actually have to subtract out the mean because we want to do it relative to the overall mean, so we do 2 minus 3.5. So we've just can write up here this is really minus 3.5, this is minus 3.5. So we get 2 minus 3.5 which is negative 1.5. So that's the bias and that should make sense because D is a harsh critic so he's well below the average rating of minus 1.5. He was below the average of 3.5. And now let's try the good movie, or movie three, so we can take the average of the values again. So this is for D. Now we'll try three. So, you, we add up 4 plus 5 plus 3 plus 5 and again, there's five of those, there's four, sorry, there's four of those values. this we don't have and this is in the test sets, we don't use it. So this comes out to be 17 over 4, which is, 4.25. And then remember we subtract 3.5 from all of these values, minus 3.5, minus 3.5. And this is 4.25 minus 3.5, which gives us positive 0.75. So, for D we have a negative value of negative 1.5 and for 3 we have positive of 0.75 which is significantly above zero, or it's above zero reasonably enough, which is what we expected, because we expected that 3 was a much better movie than the others. And so the key idea here, again though, is that you can't use the test set. So the test set has to go. And because we're not using that in these prediction schemes because we're, that's what we're going to test the RMSE on. And so we can do the rest of the values out and I've do note, I've given the values at the end of the columns or rows here. So the bias for A is positive 0.83, for B is 0.5, for C is negative 1. really easy to see we could do C is, for instance, right now there's only two values, there's a two and a three here, so we just do 2 plus 3 divided by 2 minus 3.5, which is 5 divided by 2, which is 2.5 minus 3.5 which is negative 1.0. And you can verify the rest of these also. and then on the movie side, again, two is pretty negative movie actually. 3 plus 2 plus 2 divided, divided by 3, and then, subtract out, the mean of 3.5 from that.
