Explore
For Enterprise
Join for Free
Log In

艺术与人文
商务
计算机科学
数据科学
信息技术
健康
数学和逻辑
个人发展
物理科学与工程
社会科学
语言学习
學位
證書

瀏覽 Coursera 的全部課程

瀏覽
搜索
企業版
登錄
免費加入

Other interpretations of PCA (optional)

Loading...

Mathematics for Machine Learning: PCA

伦敦帝国学院

4（860 個評分） | 21K 名學生已註冊

課程 1（共 3 門，Mathematics for Machine Learning 專項課程）

This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction. At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge. The lectures, examples and exercises require: 1. Some ability of abstract thinking 2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis) 3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization) 4. Basic knowledge in python programming and numpy Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms.

查看授課大綱

您將學習的技能

Python Programming, Principal Component Analysis (PCA), Projection Matrix, Mathematical Optimization

審閱

4（860 個評分）

5 stars
428 ratings
4 stars
198 ratings
3 stars
122 ratings
2 stars
57 ratings
1 star
55 ratings

AN

Nov 25, 2018

This course demystifies the Principal Components Analysis through practical implementation. It gives me solid foundations for learning further data science techniques.

CD

Apr 14, 2018

I found this course really excellent. Very clear explanations with very hepful illustrations.\n\nI was looking for course on PCA, thank you for this one

從本節課中

Principal Component Analysis

We can think of dimensionality reduction as a way of compressing data with some loss, similar to jpg or mp3. Principal Component Analysis (PCA) is one of the most fundamental dimensionality reduction techniques that are used in machine learning. In this module, we use the results from the first three modules of this course and derive PCA from a geometric point of view. Within this course, this module is the most challenging one, and we will go through an explicit derivation of PCA plus some coding exercises that will make us a proficient user of PCA.

Steps of PCA4:07

PCA in high dimensions5:48

Other interpretations of PCA (optional)7:42

Summary of this module0:42

This was the course on PCA0:56

教學方

Marc P. Deisenroth
Lecturer in Statistical Machine Learning

腳本

We derive PCA from the perspective of

minimising the average squared reconstruction error.

However, PCA can also be interpreted from different perspectives.

In this video, we'll have a brief look at some of these interpretations.

Let's start with a recap,

of what we have done so far.

We took a high dimensional vector x and we projected it onto

a lower dimensional representation z using the matrix B transpose.

The columns of this matrix B are the eigenvectors of

the data covariance matrix that are associated with the largest eigenvalues.

The z values are the coordinates of

our data point with respect to the basis vectors which span the principal subspace,

and that is also called the code of our data point.

Once we have that lower dimensional representation z,

we can get a higher dimensional version of it by using the matrix B again,

so multiplying B onto z to get

a higher dimensional version of the z in the original data space.

We found the PCA parameters such that the reconstruction error between x and

the reconstruction x tilde is minimised.

We can also think of PCA as a linear autoencoder.

An autoencoder encodes a data point x and

tries to decode it to something similar to the same data point.

The mapping from the data to the code is called an encoder. Let's write this down.

So this part here is called an encoder,

the mapping from the code to the original data space is called the decoder.

If the encoder and decoder are linear mappings,

then we get the PCA solution when we minimise the squared autoencoding loss.

If we replace the linear mapping of PCA with a nonlinear mapping,

we get a nonlinear autoencoder.

A prominent example of this is a deep autoencoder with

the linear functions of the encoder and decoder are replaced with deep neural networks.

Another interpretation of PCA is related to information theory.

We can think of the code as a smaller compressed version of the original data point.

When we reconstruct our original data using the code,

we don't get the exact data point back,

but a slightly distorted or noisy version of it.

This means that our compression is lossy.

Intuitively, we want to maximise

the correlation between the original data and the lower dimensional code.

More formally, this would be related to the mutual information.

We would then get the same solution to PCA we discussed

earlier in this course by maximising the mutual information,

a core concept in information theory.

When we derived PCA using projections,

we reformulated the average reconstruction error loss as minimising

the variance of the data that is projected onto

the orthogonal complement of the principle subspace.

minimising that variance is equivalent to mazimising

the variance of the data when projected onto the principle subspace.

If we think of variance in the data as information contained in the data,

this means that PCA can also be

interpreted as a method that retains as much information as possible.

We can also look at PCA from the perspective of a latent variable model.

We assume that an unknown lower dimensional code z generates data

x and we assume that we have a linear relationship between z and x.

So, generally, we can then write that x is B times z plus mu and maybe some noise.

We assume that the noise is

isotropic with mean zero and covariance matrix sigma squared times I.

We further assume that the distribution of the z is a standard normal so P of z is

Gaussian with mean zero and covariance matrix the identity matrix.

We can now write down the likelihood of this model.

So, the likelihood is P of x given z and that is a Gaussian distribution in x

with mean Bz plus

mu and covariance matrix sigma squared I.

And we can also compute the marginal likelihood as P

of x is the integral of P of x given z.

So, that is the likelihood times the distribution on z, dz,

and that turns out to be a Gaussian distribution in x with

mean mu and with covariance matrix B times B transpose plus sigma squared I.

The parameters of this model

are mu, B, and sigma squared.

And we can write them explicitly down in our model up here.

So model parameters are B and mu and sigma squared.

We can now determine the parameters of this model

using maximum likelihood estimation,

and we will find that mu is the mean of the data and B is

a matrix that contains the eigenvectors that correspond to the largest eigenvalues.

To get the low dimension code of a data point,

we can apply Bayes' theorem to invert the linear relationship between z and x.

In particular, we are going to get P of z

given x as P of x given z.

So, that is the likelihood which comes from here times P of z.

So, that's our distribution that we have here.

Divided by the marginal likelihood P of x which comes from here.

In this video, we looked at five different perspectives of

PCA that lead to different objectives: minimising the squared reconstruction error,

minimising the autoencoder loss,

mazimising the mutual information,

mazimising the variance of the projected data,

and mazimising the likelihood in a latent variable model.

All these different perspectives give us the same solution to the PCA problem.

The strengths and weaknesses of individual perspectives become

more clear and important when we consider properties of real data.

探索我們的目錄

免費加入並獲得個性化推薦、更新和優惠。

Coursera provides universal access to the world’s best education, partnering with top universities and organizations to offer courses online.

© 2019 Coursera Inc. 保留所有權利。

通過 App Store 下載

通過 Google Play 獲取

Coursera
關於
管理團隊
工作機會
目錄
證書
MasterTrack™ 證書
學位
企業版
政府版

社區
Learners
Partners
Developers
Beta Testers
Translators

連接
博客
Facebook
領英
Twitter
Instagram
YouTube
技術博客

更多
條款
隱私
幫助
內容訪問
媒體
聯繫我們
目錄
附屬公司