[MUSIC] Linear regression is used when we have a continuous response variable, and several explanatory variables, or co-variance. But what if the response variable is binary, zero or one? We could still fit a linear regression model. That might look something like this. Here I've written the x axis for the explanatory variable. And the response variable is here y, on the y axis. It can only take values of 0 and 1. So if we were to draw a linear regression through these points it might look something like that. As you can see, only two very specific values of x will cause a prediction of 0 or 1. Not to mention, if we were to draw a residual plot from this regression right here, it would certainly not appear to be very normally distributed. Instead of using a normal likelihood, it is more appropriate to use a Bernoulli likelihood here. That is, the response yi, given phi i would be independent from a Bernoulli distribution. Phi i, where phi i is the probability of success for observation i. In this case, phi i is also the expected value of yi. But we still need to relate this phi i to our explanatory variable x. Remember, in linear regression, We had the expected value of yi was equal to, directly to the linear form in the x's. Like this. One option in this model, would be to model phi directly with this linear form. But there's one problem, because phi represents a probability, it needs to lie between 0 and 1. To keep this linear form right here between 0 and 1, we would need to place some pretty odd restrictions on the beta coefficients. Instead, we'll use what is called a link function that relates the linear form of the restricted parameter, in our case phi. Without any restrictions on the beta's. To do this, we could use the odds which is defined as phi over 1 minus phi. Odds can take on any value greater than 0. So, we're less restricted but, not totally unrestricted. So now let's take the log of the odds. This quantity is called the logit link, or logistic link. It can now take on any real number, positive or negative, so it can be related to our linear form without any restrictions. That is, the logit of phi i, which is equal to the log of the odds, will be equal to this linear form here. So this is the piece that relates the x to the success probability. If we do a little bit of algebra, we can recover the expression for the expected value of yi. That is, from this we can get the expected value of yi, which is phi. So we just need to solve this equation here for phi, which will be the e to the beta not plus beta 1 x1i, over 1 plus the same thing. If we divide the numerator and the denominator by the numerator, we get this form. This form is a little bit more computationally convenient. This process right here is called inverting the link function to get an expression for the expected value of y. Now, if we were to plot expected value of y as a function of x, over here. It might look something like this. Where the expected value now stays between 0 and 1. This is logistic regression using the logistic link. If the beta coefficient for x, for the covariant x, is positive, it means that an increase in the value of x corresponds to an increase in the probability that y is 1. So the curve would look like this. If the coefficient is negative, that is, beta is negative, that means that an increase in x would correspond to a decrease in the probability that y is equal to 1. So it's just a negative version of that dotted curve. There are other possible link functions, but we're just going to focus on this one. Of course we can add more x variables like we did with multiple regression. And some, or all of those x variables, could be categorical, like we had with ANOVA. [MUSIC]