2维

PCA即最大方差线性投影

3维

PCA is just the SVD!

$$A=U\Sigma V^T$$

SVD is nothing more than decomposing vectors onto orthogonal axes!

恭喜，您已经掌握了神奇的SVD的核心!!!

向量投影

矩阵相乘

• n = no. of points, d = no. of dimensions
• A = matrix containing points
• V = matrix containing the decomposition axes
• S = matrix containing lengths of projection

Vectors dot TO matrix multiply

$$A=SV^{-1}=SV^{T}$$

But we need this:$$A=U\Sigma V^{T}$$

令$$\sigma_1^2=s_{a1}^2+s_{b1}^2$$
$$\sigma_2^2=s_{a2}^2+s_{b2}^2$$

最后，$$A=U\Sigma V^{T}$$

ICA vs PCA

• 最小化互信息
• 最大化非高斯统计特性

ICA试图寻找：

• 各成分之间统计独立
• 每个成分具备非高斯分布特性

主要算法:

1. FastICA
2. InfoMax

• $$\max_{{\bf a}: \| {\bf a}\| =1} \frac1n \sum_i \bigl[ {\bf a}'({\bf x}_i-\bar {\bf x})\bigr]^4$$
• 最小化$E[X_iX_j]-E[X_i]E[X_j]$

• 类间方差：$S_b=\sum_{i=1}^g N_i(\bar{x_i}-\bar{x})(\bar{x_i}-\bar{x})^T$


• 类内方差：$S_w=\sum_{1=1}^g \sum_{j=1}^{N_i}(x_{i,j}-\bar{x_i})(x_{i,j}-\bar{x_i})^T$


• $$P_{lda}=argmax\frac{|P^TS_bP|}{|P^TS_wP|}$$