## 主成分分析與奇異值分解

$X=\begin{bmatrix} (\mathbf{x}_1-\mathbf{m})^T\\ \vdots\\ (\mathbf{x}_n-\mathbf{m})^T \end{bmatrix}$

$\tilde{X}=\begin{bmatrix} (\mathbf{x}_1-\mathbf{m})^T\\ \vdots\\ (\mathbf{x}_n-\mathbf{m})^T \end{bmatrix}\begin{bmatrix} 1/s_1&&\\ &\ddots&\\ &&1/s_p \end{bmatrix}$

$\displaystyle S=\frac{1}{n-1}X^TX$

$\displaystyle R=\frac{1}{n-1}\tilde{X}^T\tilde{X}=\frac{1}{n-1}D^{-1}X^TXD^{-1}=D^{-1}SD^{-1}$

$S=W\Lambda W^T$

$z_{kj}=(\mathbf{x}_k-\mathbf{m})^T\mathbf{w}_j,~~1\le k\le n,~~1\le j\le p$

$Z=\begin{bmatrix} (\mathbf{x}_1-\mathbf{m})^T\\ \vdots\\ (\mathbf{x}_n-\mathbf{m})^T \end{bmatrix}\begin{bmatrix} \mathbf{w}_1&\cdots&\mathbf{w}_p \end{bmatrix}=XW$

$A=\begin{bmatrix} 1&1&1\\ \epsilon&0&0\\ 0&\epsilon&0\\ 0&0&\epsilon \end{bmatrix}$

$A^TA=\begin{bmatrix} 1+\epsilon^2&1&1\\ 1&1+\epsilon^2&1\\ 1&1&1+\epsilon^2 \end{bmatrix}$

$X=U\Sigma V^T$

$X^TX=(U\Sigma V^T)^T(U\Sigma V^T)=V\Sigma^TU^TU\Sigma V^T=V\Sigma^2V^T$

$\displaystyle S=\frac{1}{n-1}X^TX=\frac{1}{n-1}V\Sigma^2V^T=V\left(\frac{1}{n-1}\Sigma^2\right)V^T$

$\displaystyle \Lambda=\frac{1}{n-1}\Sigma^2$

$W=V$

$Z=XV=(U\Sigma V^T)V=U\Sigma$

\displaystyle\begin{aligned} U&=Z\Sigma^{-1}=\frac{1}{\sqrt{n-1}}Z\Lambda^{-1/2}\\ &=\frac{1}{\sqrt{n-1}}\begin{bmatrix} z_{11}&z_{12}&\cdots&z_{1p}\\ z_{21}&z_{22}&\cdots&z_{2P}\\ \vdots&\vdots&\ddots&\vdots\\ z_{n-1}&z_{n2}&\cdots&z_{np} \end{bmatrix}\begin{bmatrix} \frac{1}{\sqrt{\lambda_1}}&&&\\ &\frac{1}{\sqrt{\lambda_2}}&&\\ &&\ddots&\\ &&&\frac{1}{\sqrt{\lambda_p}} \end{bmatrix}\\ &=\frac{1}{\sqrt{n-1}}\begin{bmatrix} \frac{z_{11}}{\sqrt{\lambda_1}}&\frac{z_{12}}{\sqrt{\lambda_2}}&\cdots&\frac{z_{1p}}{\sqrt{\lambda_p}}\\ \frac{z_{21}}{\sqrt{\lambda_1}}&\frac{z_{22}}{\sqrt{\lambda_2}}&\cdots&\frac{z_{2p}}{\sqrt{\lambda_p}}\\ \vdots&\vdots&\ddots&\vdots\\ \frac{z_{n1}}{\sqrt{\lambda_1}}&\frac{z_{n2}}{\sqrt{\lambda_2}}&\cdots&\frac{z_{np}}{\sqrt{\lambda_p}} \end{bmatrix}.\end{aligned}

$\tilde{Z}=Z\Lambda^{-1/2}=\sqrt{n-1}U$

$\displaystyle F=\frac{1}{n-1}\tilde{X}^T\tilde{Z}$

$\tilde{X}=XD^{-1}$$\tilde{Z}=XV\Lambda^{-1/2}$$S=\frac{1}{n-1}X^TX=V\Lambda V^T$$\Lambda^{1/2}=\frac{1}{\sqrt{n-1}}\Sigma$ 代入上式，可得

\displaystyle\begin{aligned} F&=\frac{1}{n-1}(D^{-1}X^T)(XV\Lambda^{-1/2})=D^{-1}SV\Lambda^{-1/2}\\ &=D^{-1}(V\Lambda V^T)V\Lambda^{-1/2}=D^{-1}V\Lambda\Lambda^{-1/2}\\ &=D^{-1}V\Lambda^{1/2}=\frac{1}{\sqrt{n-1}}D^{-1}V\Sigma. \end{aligned}

$\displaystyle X=U\Sigma V^T=U\Sigma V^TD^{-1}D=\sqrt{n-1}U\left(\frac{1}{\sqrt{n-1}}D^{-1}V\Sigma\right)^TD=\tilde{Z}F^TD$

$\displaystyle X^T=\begin{bmatrix} \mathbf{x}_1-\mathbf{m}&\cdots&\mathbf{x}_n-\mathbf{m} \end{bmatrix}=DF\tilde{Z}^T=DF\begin{bmatrix} \tilde{\mathbf{z}}_1&\cdots&\tilde{\mathbf{z}}_n \end{bmatrix}$

This entry was posted in 線性代數專欄, 二次型 and tagged , , , , , , , , , , . Bookmark the permalink.

### 4 Responses to 主成分分析與奇異值分解

1. w says:

老师您好，请问 UU^T 不一定等於 I_n

• ccjou says:

舉個例子可以幫助理解，比如
$U=\begin{bmatrix} 0&1\\ 1&0\\ 0&0 \end{bmatrix}$

• w says:

SVD分解中的U不是m*m的吗？

• w says:

哦 谢谢老师 没看仔细 一个是In 一个是Ip