矩阵分解相关知识点总结（四）

书接上上文矩阵分解相关知识点总结（二）

四、矩阵的满秩分解

  设$A\in C_r^{m\times n}(r>0)$，存在矩阵$F\in C_r^{m\times r}$和$G\in C_r^{r\times n}$，使得
$$A=FG\text{\hspace{1em}(7)}$$

其中$r$为矩阵的秩，$F$是列满秩矩阵，$G$是行满秩矩阵，上式即为矩阵的满秩分解。当$A$是满秩（列满秩或行满秩）矩阵时，$A$可分解为一个因子是单位矩阵，另一个因子是$A$本身，称此满秩分解为平凡分解。

五、矩阵的奇异值分解

  设$A\in C_r^{m\times n}(r>0)$，则存在$m$阶酉矩阵$U$和$n$阶酉矩阵$V$，使得
$$A=UD{V}^{\mathrm{H}}=U\left[\begin{array}{cc}\Sigma & O\\ O& O\end{array}\right]V^{\mathrm{H}}\text{\hspace{1em}(8)}$$

其中，$V^{\mathrm{H}}$代表酉矩阵$V$的共轭转置，$\Sigma =\mathrm{diag}({\sigma }_1,{\sigma }_2,\cdots ,{\sigma }_r)$，${\sigma }_i(i=1,2,\cdots ,r)$为矩阵$A$的全部非零奇异值。上式即为矩阵的奇异值分解（Singular Value Decomposition，简称SVD分解），当$A$为实对称矩阵时，也称正交对角分解。

  矩阵的奇异值分解在最优化问题、特征值问题、最小二乘问题、广义逆矩阵问题及统计学等方面都有重要应用。下面通过一个具体实例，将矩阵$A=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right]$进行SVD分解。

• 首先计算$A^{\mathrm{T}}A=\left[\begin{array}{ccc}1& 0& 1\\ 0& 1& 1\end{array}\right]\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right]=\left[\begin{array}{cc}2& 1\\ 1& 2\end{array}\right]$

• 对矩阵$A^{\mathrm{T}}A$，由$|\lambda I-A^{\mathrm{T}}A|=\left|\begin{array}{cc}\lambda -2& -1\\ -1& \lambda -2\end{array}\right|=(\lambda -1)(\lambda -3)=0$，得特征值${\lambda }_1=1,{\lambda }_2=3$，对应的特征向量分别为：

$${\mathbf{\xi }}_1=k_1\left[\begin{array}{c}1\\ -1\end{array}\right],{\mathbf{\xi }}_2=k_2\left[\begin{array}{c}1\\ 1\end{array}\right]$$

• 于是有

$$\begin{array}{l}\Sigma =\mathrm{diag}(\sqrt{{\lambda }_1},\sqrt{{\lambda }_2})=\left[\begin{array}{cc}1& 0\\ 0& \sqrt{3}\end{array}\right]\\ D=\left[\begin{array}{cc}\Sigma & O\\ O& O\end{array}\right]=\left[\begin{array}{cc}1& 0\\ 0& \sqrt{3}\\ 0& 0\end{array}\right]\\ V=\left[\begin{array}{cc}\frac{{\mathbf{\xi }}_1}{\Vert {\mathbf{\xi }}_1\Vert }& \frac{{\mathbf{\xi }}_2}{\Vert {\mathbf{\xi }}_2\Vert }\end{array}\right]=\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\end{array}\right],\text{这里V可以有四种解}\end{array}$$

• 计算

$$U_1=AV{\Sigma }^{-1}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right]\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\end{array}\right]\left[\begin{array}{cc}1& 0\\ 0& \frac{1}{\sqrt{3}}\end{array}\right]=\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}\\ 0& \frac{2}{\sqrt{6}}\end{array}\right]$$

• 构造一个列向量$U_2=[{\alpha }_1,{\alpha }_2,{\alpha }_3{]}^{\mathrm{T}}$，使得$U=\left[\begin{array}{cc}{U}_1& U_2\end{array}\right]=\left[\begin{array}{ccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& {\alpha }_1\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& {\alpha }_2\\ 0& \frac{2}{\sqrt{6}}& {\alpha }_3\end{array}\right]$为酉矩阵，即有：

$$\left[\begin{array}{ccc}\frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}& 0\\ \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{6}}& \frac{2}{\sqrt{6}}\end{array}\right]\left[\begin{array}{c}{\alpha }_1\\ {\alpha }_2\\ {\alpha }_3\end{array}\right]=0,且\sqrt{{\alpha }_1^2+{\alpha }_2^2+{\alpha }_3^2}=1$$

得$$U_2=\pm \left[\begin{array}{ccc}\frac{1}{\sqrt{3}}& \frac{1}{\sqrt{3}}& -\frac{1}{\sqrt{3}}\end{array}\right]$$

取$$U=\left[\begin{array}{ccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ 0& \frac{2}{\sqrt{6}}& -\frac{1}{\sqrt{3}}\end{array}\right],\text{这里U可以有两种解}$$

• 最后，$A$的SVD分解为：

$$A=UD{V}^{\mathrm{T}}=\left[\begin{array}{ccc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{6}}& \frac{1}{\sqrt{3}}\\ 0& \frac{2}{\sqrt{6}}& -\frac{1}{\sqrt{3}}\end{array}\right]\left[\begin{array}{cc}1& 0\\ 0& \sqrt{3}\\ 0& 0\end{array}\right]{\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\end{array}\right]}^{\mathrm{T}}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right]$$

当调整奇异值的顺序，使$\Sigma =\left[\begin{array}{cc}\sqrt{3}& 0\\ 0& 1\end{array}\right]$时，可得
$$\begin{array}{l}D=\left[\begin{array}{cc}\sqrt{3}& 0\\ 0& 1\\ 0& 0\end{array}\right]\\ V=\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right],\text{这里V可以有四种解}\\ U=\left[\begin{array}{ccc}\frac{1}{\sqrt{6}}& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{3}}\\ \frac{2}{\sqrt{6}}& 0& -\frac{1}{\sqrt{3}}\end{array}\right],\text{这里U可以有两种解}\end{array}$$

$$A=UD{V}^{\mathrm{T}}=\left[\begin{array}{ccc}\frac{1}{\sqrt{6}}& \frac{1}{\sqrt{2}}& \frac{1}{\sqrt{3}}\\ \frac{1}{\sqrt{6}}& -\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{3}}\\ \frac{2}{\sqrt{6}}& 0& -\frac{1}{\sqrt{3}}\end{array}\right]\left[\begin{array}{cc}\sqrt{3}& 0\\ 0& 1\\ 0& 0\end{array}\right]{\left[\begin{array}{cc}\frac{1}{\sqrt{2}}& \frac{1}{\sqrt{2}}\\ \frac{1}{\sqrt{2}}& -\frac{1}{\sqrt{2}}\end{array}\right]}^{\mathrm{T}}=\left[\begin{array}{cc}1& 0\\ 0& 1\\ 1& 1\end{array}\right]$$

可以看到，矩阵的奇异值分解通常不是唯一的。