Explore
For Enterprise
Join for Free
Log In

艺术与人文
商务
计算机科学
数据科学
信息技术
健康
数学和逻辑
个人发展
物理科学与工程
社会科学
语言学习
學位
證書

瀏覽 Coursera 的全部課程

瀏覽
搜索
企業版
登錄
免費加入

Regularized Linear Regression

Loading...

机器学习

斯坦福大学

4.9（107,939 個評分） | 2.4M 名學生已註冊

机器学习是一门研究在非特定编程条件下让计算机采取行动的学科。最近二十年，机器学习为我们带来了自动驾驶汽车、实用的语音识别、高效的网络搜索，让我们对人类基因的解读能力大大提高。当今机器学习技术已经非常普遍，您很可能在毫无察觉情况下每天使用几十次。许多研究者还认为机器学习是人工智能（AI）取得进展的最有效途径。在本课程中，您将学习最高效的机器学习技术，了解如何使用这些技术，并自己动手实践这些技术。更重要的是，您将不仅将学习理论知识，还将学习如何实践，如何快速使用强大的技术来解决新问题。最后，您将了解在硅谷企业如何在机器学习和AI领域进行创新。本课程将广泛介绍机器学习、数据挖掘和统计模式识别。相关主题包括：(i) 监督式学习（参数和非参数算法、支持向量机、核函数和神经网络）。(ii) 无监督学习（集群、降维、推荐系统和深度学习）。(iii) 机器学习实例（偏见/方差理论；机器学习和AI领域的创新）。课程将引用很多案例和应用，您还需要学习如何在不同领域应用学习算法，例如智能机器人（感知和控制）、文本理解（网络搜索和垃圾邮件过滤）、计算机视觉、医学信息学、音频、数据库挖掘等领域。

查看授課大綱

您將學習的技能

Logistic Regression, Artificial Neural Network, Machine Learning (ML) Algorithms, Machine Learning

審閱

4.9（107,939 個評分）

5 stars
100,023 ratings
4 stars
7,303 ratings
3 stars
453 ratings
2 stars
77 ratings
1 star
83 ratings

PT

Sep 01, 2018

Sub title should be corrected. Since I'm not that good in English but I know when there're mis-traslated or wrong sub title. If you fix this problems , I thin it helps many students a lot. Thanks!!!!!

OK

Apr 18, 2018

You need to know, what do you want to get out of this course. It gives you a lot of information, but be prepared to work hard with linear algeabra and make efforts to compute things in Mathlab/Octave.

從本節課中

Regularization

Machine learning models need to generalize well to new examples that the model has not seen in practice. In this module, we introduce regularization, which helps prevent models from overfitting the training data.

The Problem of Overfitting9:42

Cost Function10:10

Regularized Linear Regression10:40

Regularized Logistic Regression8:33

教學方

Andrew Ng
CEO/Founder Landing AI; Co-founder, Coursera; Adjunct Professor, Stanford University; formerly Chief Scientist,Baidu and founding lead of Google Brain

腳本

For linear regression, we have previously worked out two learning algorithms.

One based on gradient descent and one based on the normal equation.

In this video, we'll take those two algorithms and

generalize them to the case of regularized linear regression.

Here's the optimization objective that we came up with last time for

regularized linear regression.

This first part is our usual objective for linear regression.

And we now have this additional regularization term, where lambda

is our regularization parameter, and we like to find parameters theta

that minimizes this cost function, this regularized cost function, J of theta.

Previously, we were using gradient descent for

the original cost function without the regularization term.

And we had the following algorithm, for regular linear regression, without

regularization, we would repeatedly update the parameters theta J as follows for

J equals 0, 1, 2, up through n.

Let me take this and just write the case for theta 0 separately.

So I'm just going to write the update for theta 0 separately than for

the update for the parameters 1, 2, 3, and so on up to n.

And so this is, I haven't changed anything yet, right.

This is just writing the update for theta 0 separately from the updates for theta 1,

theta 2, theta 3, up to theta n.

And the reason I want to do this is you may remember that for

our regularized linear regression, we penalize the parameters theta 1, theta 2,

and so on up to theta n.

But we don't penalize theta 0.

So, when we modify this algorithm for regularized linear regression,

we're going to end up treating theta zero slightly differently.

Concretely, if we want to take this algorithm and

modify it to use the regular rise objective,

all we need to do is take this term at the bottom and modify it as follows.

We'll take this term and

add minus lambda over m times theta j.

And if you implement this, then you have gradient descent for

trying to minimize the regularized cost function, j of theta.

And concretely, I'm not gonna do the calculus to prove it, but

concretely if you look at this term, this term hat I've written in square brackets,

if you know calculus it's possible to prove that that term

is the partial derivative with respect to J of theta

using the new definition of J of theta with the regularization term.

And similarly, this term up on top which I'm drawing the cyan box,

that's still the partial derivative respect of theta zero of J of theta.

If you look at the update for theta j,

it's possible to show something very interesting.

Concretely, theta j gets updated as theta j minus alpha times,

and then you have this other term here that depends on theta J.

So if you group all the terms together that depend on theta j,

you can show that this update can be written equivalently as follows.

And all I did was add theta j here is, so theta j times 1.

And this term is, right, lambda over m, there's also an alpha here,

so you end up with alpha lambda over m multiplied into theta j.

And this term here, 1 minus alpha times lambda m, is a pretty interesting term.

It has a pretty interesting effect.

Concretely this term, 1 minus alpha times lambda over m,

is going to be a number that is usually a little bit less than one,

because alpha times lambda over m is going to be positive, and usually

if your learning rate is small and if m is large, this is usually pretty small.

So this term here is gonna be a number that's usually a little bit less than 1,

so think of it as a number like 0.99, let's say.

And so the effect of our update to theta j is,

we're going to say that theta j gets replaced by theta j times 0.99, right?

So theta j times 0.99 has the effect of

shrinking theta j a little bit towards zero.

So this makes theta j a bit smaller.

And more formally, this makes the square norm of theta j a little bit smaller.

And then after that, the second term here, that's actually

exactly the same as the original gradient descent update that we had,

before we added all this regularization stuff.

So, hopefully this gradient descent, hopefully this update makes sense.

When we're using a regularized linear regression and

what we're doing is on every iteration we're multiplying theta j by a number

that's a little bit less then one, so its shrinking the parameter a little bit, and

then we're performing a similar update as before.

Of course that's just the intuition behind what this particular update is doing.

Mathematically what it's doing is it's exactly gradient descent

on the cost function J of theta that we defined on the previous slide that uses

the regularization term.

Gradient descent was just one of our two algorithms for

fitting a linear regression model.

The second algorithm was the one based on the normal equation,

where what we did was we created the design matrix X where

each row corresponded to a separate training example.

And we created a vector y, so this is a vector, that's an m dimensional vector.

And that contained the labels from my training set.

So whereas X is an m by (n+1) dimensional matrix, y is an m dimensional vector.

And in order to minimize the cost function J, we found that one

way to do so is to set theta to be equal to this.

Right, you have X transpose X, inverse, X transpose Y.

I'm leaving room here to fill in stuff of course.

And what this value for theta does is this

minimizes the cost function J of theta, when we were not using regularization.

Now that we are using regularization, if you were to derive what the minimum is,

and just to give you a sense of how to derive the minimum, the way you derive it

is you take partial derivatives with respect to each parameter.

Set this to zero, and then do a bunch of math and

you can then show that it's a formula like this that minimizes the cost function.

And concretely, if you are using regularization,

then this formula changes as follows.

Inside this parenthesis, you end up with a matrix like this.

0, 1, 1, 1, and so on, 1, until the bottom.

So this thing over here is a matrix whose upper left-most entry is 0.

There are ones on the diagonals, and then zeros everywhere else in this matrix.

Because I'm drawing this rather sloppily.

But as a example, if n = 2,

then this matrix is going to be a three by three matrix.

More generally, this matrix is an (n+1) by (n+1) dimensional matrix.

So if n = 2, then that matrix becomes something that looks like this.

It would be 0, and then 1s on the diagonals, and

then 0s on the rest of the diagonals.

And once again, I'm not going to show this derivation, which is frankly somewhat long

and involved, but it is possible to prove that if you are using the new definition

of J of theta, with the regularization objective, then this new formula for

theta is the one that we give you, the global minimum of J of theta.

So finally I want to just quickly describe the issue of non-invertibility.

This is relatively advanced material, so you should consider this as optional.

And feel free to skip it, or if you listen to it and

positive it doesn't really make sense, don't worry about it either.

But earlier when I talked about the normal equation method,

we also had an optional video on the non-invertibility issue.

So this is another optional part to this,

sort of an add-on to that earlier optional video on non-invertibility.

Now, consider a setting where m, the number of examples, is less than or

equal to n, the number of features.

If you have fewer examples than features, than this matrix,

X transpose X will be non-invertible, or singular.

Or the other term for this is the matrix will be degenerate.

And if you implement this in Octave anyway and you use the pinv

function to take the pseudo inverse, it will kind of do the right thing, but

it's not clear that it would give you a very good hypothesis, even though

numerically the Octave pinv function will give you a result that kinda makes sense.

But if you were doing this in a different language, and

if you were taking just the regular inverse, which in Octave denoted with

the function inv, we're trying to take the regular inverse of X transpose X.

Then in this setting, you find that X transpose X is singular,

is non-invertible, and if you're doing this in different program language and

using some linear algebra library to try to take the inverse of this matrix,

it just might not work because that matrix is non-invertible or singular.

Fortunately, regularization also takes care of this for us.

And concretely, so long as the regularization parameter lambda

is strictly greater than 0, it is actually possible to prove that this matrix,

X transpose X plus lambda times this funny matrix here, it is possible to prove

that this matrix will not be singular and that this matrix will be invertible.

So using regularization also takes care of any non-invertibility

issues of the X transpose X matrix as well.

So you now know how to implement regularized linear regression.

Using this you'll be able to avoid overfitting

even if you have lots of features in a relatively small training set.

And this should let you get linear regression to work much better for

many problems.

In the next video we'll take this regularization idea and

apply it to logistic regression.

So that you'd be able to get logistic regression to avoid overfitting and

perform much better as well.

探索我們的目錄

免費加入並獲得個性化推薦、更新和優惠。

Coursera 致力於普及全世界最好的教育，它與全球一流大學和機構合作提供在線課程。

© 2019 Coursera Inc. 保留所有權利。

通過 App Store 下載

通過 Google Play 獲取

Coursera
關於
管理團隊
工作機會
目錄
證書
MasterTrack™ 證書
學位
企業版
政府版

社區
Learners
Partners
Developers
Beta Testers
Translators

連接
博客
Facebook
領英
Twitter
Instagram
YouTube
技術博客

更多
條款
隱私
幫助
內容訪問
媒體
聯繫我們
目錄
附屬公司