滨海新网站建设模板网站也需要服务器吗

当前位置： 首页 > news >正文

滨海新网站建设,模板网站也需要服务器吗,宠物网站建设内容,网站制作怎么自己做​ 大纲 矩阵微积分#xff1a;多元微积分的一种特殊表达#xff0c;尤其是在矩阵空间上进行讨论的时候逆矩阵(inverse matrix)矩阵分解#xff1a;特征分解#xff08;Eigendecomposition#xff09;#xff0c;又称谱分解#xff08;Spectral decomposition#xf…​ 大纲 矩阵微积分多元微积分的一种特殊表达尤其是在矩阵空间上进行讨论的时候逆矩阵(inverse matrix)矩阵分解特征分解Eigendecomposition又称谱分解Spectral decompositionLU分解奇异值分解singular value decompositionQR分解科列斯基分解矩阵行列式Determinant在欧几里得空间中行列式描述的是一个线性变换对“体积”所造成的影响特征向量eigenvector A v λ v Av\lambda v Avλv其中 λ \lambda λ为特征值 v v v为 A A A的特征向量 A A A的所有特征值的全体叫 A A A的谱记为 λ ( A ) \lambda(A) λ(A)迹trance tr ⁡ ( A ) A 1 , 1 ⋯ A n , n \operatorname{tr}(\mathbf{A}) \mathbf{A}{1, 1} \cdots \mathbf{A}{n, n} tr(A)A1,1​⋯An,n​一个矩阵的迹是其特征值的总和正交矩阵orthogonal matrix是一个方阵其行向量與列向量皆為正交的单位向量使得該矩陣的转置矩阵為其逆矩阵。 Q Q T I QQ^TI QQTI正定矩阵和半正定矩阵positive semi-definite matrix一个 n × n n\times n n×n 的实对称矩阵 M M M 是正定的当且仅当对于所有的非零实系数向量 z \mathbf {z} z都有 z T M z 0 \mathbf {z} ^{T}M\mathbf {z} 0 zTMz0。其中 z T \mathbf {z} ^{T} zT表示 z \mathbf {z} z 的转置伴随矩阵adjugate matrix如果矩阵可逆那么它的逆矩阵和它的伴随矩阵之间只差一个系数共轭矩阵又叫Hermite矩阵矩阵本身先转置再把矩阵中每个元素取共轭(虚部变号的运算)得到的矩阵共轭转置conjugate transpose or Hermitian transpose A ∗ ( A ‾ ) T A T ‾ A^* (\overline{A})^\mathrm{T} \overline{A^\mathrm{T}} A∗(A)TAT, A ‾ \overline{A} A表示对矩阵A元素取复共轭酉矩阵又叫幺正矩阵unitary matrix指其共轭转置恰为其逆矩阵的复数方阵 U ∗ U U U ∗ I n U^{}UUU^{}I{n} U∗UUU∗In​实对称矩阵元素都为实数的对称矩阵对角矩阵(diagonal matrix)一个主对角线之外的元素皆为0的矩阵常写为diaga1a2,…,an)雅可比矩阵Jacobian matrix J [ ∂ f ∂ x 1 ⋯ ∂ f ∂ x n ] [ ∂ f 1 ∂ x 1 ⋯ ∂ f 1 ∂ x n ⋮ ⋱ ⋮ ∂ f m ∂ x 1 ⋯ ∂ f m ∂ x n ] \mathbf {J} {\begin{bmatrix}{\dfrac {\partial \mathbf {f} }{\partial x{1}}}\cdots {\dfrac {\partial \mathbf {f} }{\partial x{n}}}\end{bmatrix}}{\begin{bmatrix}{\dfrac {\partial f{1}}{\partial x{1}}}\cdots {\dfrac {\partial f{1}}{\partial x{n}}}\\vdots \ddots \vdots \{\dfrac {\partial f{m}}{\partial x{1}}}\cdots {\dfrac {\partial f{m}}{\partial x_{n}}}\end{bmatrix}} J[∂x1​∂f​​⋯​∂xn​∂f​​] ​∂x1​∂f1​​⋮∂x1​∂fm​​​⋯⋱⋯​∂xn​∂f1​​⋮∂xn​∂fm​​​ ​黑塞矩阵又叫海森矩阵Hessian matrix)由多变量实值函数的所有二阶偏导数组成的方阵 H i j ∂ 2 f ∂ x i ∂ x j \mathbf {H} {ij}{\frac {\partial ^{2}f}{\partial x{i}\partial x{j}}} Hij​∂xi​∂xj​∂2f​矩阵范数matrix norm 一、矩阵微积分 向量对向量的偏导称 Jacobian Matrix: J ∂ y ( n ) ∂ x ( m ) ( ∂ y 1 ∂ x 1 ⋯ ∂ y 1 ∂ x m ⋮ ⋱ ⋮ ∂ y n ∂ x 1 ⋯ ∂ y n ∂ x m ) n × m J \frac{\partial{y{(n)}}}{\partial{x_{(m)}}} \begin{pmatrix} \frac{\partial{y_1}}{\partial{x_1}} \cdots \frac{\partial{y_1}}{\partial{x_m}} \ \vdots \ddots \vdots \ \frac{\partial{y_n}}{\partial{x_1}} \cdots \frac{\partial{y_n}}{\partial{xm}} \end{pmatrix}{n \times m} J∂x(m)​∂y(n)​​ ​∂x1​∂y1​​⋮∂x1​∂yn​​​⋯⋱⋯​∂xm​∂y1​​⋮∂xm​∂yn​​​ ​n×m​ 标量对向量的偏导、向量对标量的偏导都是相应向量为一维的情况。 这里采用了称为分子布局的表示方法另外还有将矩阵向量微积分表示为这里这种形式的转置的称为分母布局。但用分母布局表示时下面的运算法则没有这么好记的形式。 与标量微积分对比 加法法则不变 ∂ y z ∂ x ∂ y ∂ x ∂ z ∂ x \frac{\partial{y z}}{\partial{x}} \frac{\partial{y}}{\partial{x}} \frac{\partial{z}}{\partial{x}} ∂x∂yz​∂x∂y​∂x∂z​ 链式法则不变 ∂ z ∂ x ∂ z ∂ y ⋅ ∂ y ∂ x \frac{\partial{z}}{\partial{x}} \frac{\partial{z}}{\partial{y}} \cdot \frac{\partial{y}}{\partial{x}} ∂x∂z​∂y∂z​⋅∂x∂y​ 乘法法则形式不变 ∂ y ⊗ z ∂ x y ⊗ ∂ z ∂ x z ⊗ ∂ y ∂ x \frac{\partial{y \otimes z}}{\partial{x}} y \otimes \frac{\partial{z}}{\partial{x}} z \otimes \frac{\partial{y}}{\partial{x}} ∂x∂y⊗z​y⊗∂x∂z​z⊗∂x∂y​ 向量内积 ∂ y T z ∂ x y T ⋅ ∂ z ∂ x z T ⋅ ∂ y ∂ x \frac{\partial{y^Tz}}{\partial{x}} y^T \cdot \frac{\partial{z}}{\partial{x}} z^T \cdot \frac{\partial{y}}{\partial{x}} ∂x∂yTz​yT⋅∂x∂z​zT⋅∂x∂y​矩阵乘积A 与 x 无关 ∂ A y ∂ x A ⋅ ∂ y ∂ x \frac{\partial{Ay}}{\partial{x}} A \cdot \frac{\partial{y}}{\partial{x}} ∂x∂Ay​A⋅∂x∂y​向量数乘y 或 z 为标量 ∂ y z ∂ x y ⋅ ∂ z ∂ x z ⋅ ∂ y ∂ x \frac{\partial{yz}}{\partial{x}} y \cdot \frac{\partial{z}}{\partial{x}} z \cdot \frac{\partial{y}}{\partial{x}} ∂x∂yz​y⋅∂x∂z​z⋅∂x∂y​ ∑ i 1 n i 2 n ( n 1 ) ( 2 n 1 ) 6 \sum_{i1}^n i^2 \frac{n(n1)(2n1)}{6} ∑i1n​i26n(n1)(2n1)​

1. 表示法 A , X , Y \mathbf{A}, \mathbf{X}, \mathbf{Y} A,X,Y 等粗体的大写字母表示一个矩阵 a , x , y \mathbf a, \mathbf x, \mathbf y a,x,y 等粗体的小写字母表示一个向量 a , x , y a, x, y a,x,y 等斜体的小写字母表示一个标量 X T \mathbf X^T XT表示矩阵 X \mathbf X X 的转置 X H \mathbf X^H XH表示矩阵 X \mathbf X X 的共轭转置 ∣ X ∣ | \mathbf X | ∣X∣表示方阵 X \mathbf X X 的行列式 ∣ ∣ x ∣ ∣ || \mathbf x || ∣∣x∣∣表示向量 x \mathbf x x 的范数 I \mathbf I I表示单位矩阵。
2. 向量微分 2.1 向量-标量 列向量函数 y [ y 1 y 2 ⋯ y m ] T \mathbf y \begin{bmatrix} y_1 y_2 \cdots y_m \end{bmatrix}^T y[y1​​y2​​⋯​ym​​]T 对标量 x x x 的导数称为 y \mathbf y y 的切向量可以以 分子记法 表示为 ∂ y ∂ x [ ∂ y 1 ∂ x ∂ y 2 ∂ x ⋮ ∂ y m ∂ x ] m × 1 \frac{\partial \mathbf y}{\partial x} \begin{bmatrix} \frac{\partial y_1}{\partial x} \newline \frac{\partial y_2}{\partial x} \newline \vdots \newline \frac{\partial ym}{\partial x}\end{bmatrix}{m \times 1} ∂x∂y​ ​∂x∂y1​​∂x∂y2​​⋮∂x∂ym​​​ ​m×1​ 若以 分母记法 则可以表示为 ∂ y ∂ x [ ∂ y 1 ∂ x ∂ y 2 ∂ x ⋯ ∂ y m ∂ x ] 1 × m \frac{\partial \mathbf y}{\partial x} \begin{bmatrix} \frac{\partial y_1}{\partial x} \frac{\partial y_2}{\partial x} \cdots \frac{\partial ym}{\partial x}\end{bmatrix}{1 \times m} ∂x∂y​[∂x∂y1​​​∂x∂y2​​​⋯​∂x∂ym​​​]1×m​ 2.2 标量-向量 标量函数 y y y 对列向量 x [ x 1 x 2 ⋯ x n ] T \mathbf x \begin{bmatrix} x_1 x_2 \cdots x_n \end{bmatrix}^T x[x1​​x2​​⋯​xn​​]T 的导数可以以 分子记法 表示为 ∂ y ∂ x [ ∂ y ∂ x 1 ∂ y ∂ x 2 ⋯ ∂ y ∂ x n ] 1 × n \frac{\partial y}{\partial \mathbf x} \begin{bmatrix} \frac{\partial y}{\partial x_1} \frac{\partial y}{\partial x_2} \cdots \frac{\partial y}{\partial xn}\end{bmatrix}{1 \times n} ∂x∂y​[∂x1​∂y​​∂x2​∂y​​⋯​∂xn​∂y​​]1×n​ 若以 分母记法 则可以表示为 ∂ y ∂ x [ ∂ y ∂ x 1 ∂ y ∂ x 2 ⋮ ∂ y ∂ x n ] n × 1 \frac{\partial y}{\partial \mathbf x} \begin{bmatrix} \frac{\partial y}{\partial x_1} \newline \frac{\partial y}{\partial x_2} \newline \vdots \newline \frac{\partial y}{\partial xn}\end{bmatrix}{n \times 1} ∂x∂y​ ​∂x1​∂y​∂x2​∂y​⋮∂xn​∂y​​ ​n×1​ 2.3 向量-向量 列向量函数 y [ y 1 y 2 ⋯ y m ] T \mathbf y \begin{bmatrix} y_1 y_2 \cdots y_m \end{bmatrix}^T y[y1​​y2​​⋯​ym​​]T 对列向量 x [ x 1 x 2 ⋯ x n ] T \mathbf x \begin{bmatrix} x_1 x_2 \cdots x_n \end{bmatrix}^T x[x1​​x2​​⋯​xn​​]T 的导数可以以 分子记法 表示为 ∂ y ∂ x [ ∂ y 1 ∂ x 1 ∂ y 1 ∂ x 2 ⋯ ∂ y 1 ∂ x n ∂ y 2 ∂ x 1 ∂ y 2 ∂ x 2 ⋯ ∂ y 2 ∂ x n ⋮ ⋮ ⋱ ⋮ ∂ y m ∂ x 1 ∂ y m ∂ x 2 ⋯ ∂ y m ∂ x n ] m × n \frac{\partial \mathbf y}{\partial \mathbf x} \begin{bmatrix} \frac{\partial y_1}{\partial x_1} \frac{\partial y_1}{\partial x_2} \cdots \frac{\partial y_1}{\partial x_n} \newline \frac{\partial y_2}{\partial x_1} \frac{\partial y_2}{\partial x_2} \cdots \frac{\partial y_2}{\partial x_n} \newline \vdots \vdots \ddots \vdots \newline \frac{\partial y_m}{\partial x_1} \frac{\partial y_m}{\partial x_2} \cdots \frac{\partial y_m}{\partial xn} \newline\end{bmatrix}{m \times n} ∂x∂y​ ​∂x1​∂y1​​∂x1​∂y2​​⋮∂x1​∂ym​​​∂x2​∂y1​​∂x2​∂y2​​⋮∂x2​∂ym​​​⋯⋯⋱⋯​∂xn​∂y1​​∂xn​∂y2​​⋮∂xn​∂ym​​​ ​m×n​ 若以 分母记法 则可以表示为 ∂ y ∂ x [ ∂ y 1 ∂ x 1 ∂ y 2 ∂ x 1 ⋯ ∂ y m ∂ x 1 ∂ y 1 ∂ x 1 ∂ y 2 ∂ x 1 ⋯ ∂ y m ∂ x 1 ⋮ ⋮ ⋱ ⋮ ∂ y 1 ∂ x 1 ∂ y 2 ∂ x 1 ⋯ ∂ y m ∂ x 1 ] n × m \frac{\partial \mathbf y}{\partial \mathbf x} \begin{bmatrix} \frac{\partial y_1}{\partial x_1} \frac{\partial y_2}{\partial x_1} \cdots \frac{\partial y_m}{\partial x_1} \newline \frac{\partial y_1}{\partial x_1} \frac{\partial y_2}{\partial x_1} \cdots \frac{\partial y_m}{\partial x_1} \newline \vdots \vdots \ddots \vdots \newline \frac{\partial y_1}{\partial x_1} \frac{\partial y_2}{\partial x_1} \cdots \frac{\partial y_m}{\partial x1} \newline\end{bmatrix}{n \times m} ∂x∂y​ ​∂x1​∂y1​​∂x1​∂y1​​⋮∂x1​∂y1​​​∂x1​∂y2​​∂x1​∂y2​​⋮∂x1​∂y2​​​⋯⋯⋱⋯​∂x1​∂ym​​∂x1​∂ym​​⋮∂x1​∂ym​​​ ​n×m​
3. 矩阵微分
4. 矩阵-标量 形状为 m × n m \times n m×n 的矩阵函数 Y \mathbf Y Y 对标量 x x x 的导数称为 Y \mathbf Y Y 的切矩阵可以以 分子记法 表示为 ∂ Y ∂ x [ ∂ y 11 ∂ x ∂ y 12 ∂ x ⋯ ∂ y 1 n ∂ x ∂ y 21 ∂ x ∂ y 22 ∂ x ⋯ ∂ y 2 n ∂ x ⋮ ⋮ ⋱ ⋮ ∂ y m 1 ∂ x ∂ y m 2 ∂ x ⋯ ∂ y m n ∂ x ] m × n \frac{\partial \mathbf Y}{\partial x} \begin{bmatrix} \frac{\partial y{11}}{\partial x} \frac{\partial y{12}}{\partial x} \cdots \frac{\partial y{1n}}{\partial x} \newline \frac{\partial y{21}}{\partial x} \frac{\partial y{22}}{\partial x} \cdots \frac{\partial y{2n}}{\partial x} \newline \vdots \vdots \ddots \vdots \newline \frac{\partial y{m1}}{\partial x} \frac{\partial y{m2}}{\partial x} \cdots \frac{\partial y{mn}}{\partial x} \newline\end{bmatrix}{m \times n} ∂x∂Y​ ​∂x∂y11​​∂x∂y21​​⋮∂x∂ym1​​​∂x∂y12​​∂x∂y22​​⋮∂x∂ym2​​​⋯⋯⋱⋯​∂x∂y1n​​∂x∂y2n​​⋮∂x∂ymn​​​ ​m×n​
5. 标量-矩阵 标量函数 y y y 对形状为 p × q p \times q p×q 的矩阵 X \mathbf X X 的导数可以 分子记法 表示为 ∂ y ∂ X [ ∂ y ∂ x 11 ∂ y ∂ x 21 ⋯ ∂ y ∂ x p 1 ∂ y ∂ x 12 ∂ y ∂ x 22 ⋯ ∂ y ∂ x p 2 ⋮ ⋮ ⋱ ⋮ ∂ y ∂ x 1 q ∂ y ∂ x 2 q ⋯ ∂ y ∂ x p q ] q × p \frac{\partial y}{\partial \mathbf X} \begin{bmatrix} \frac{\partial y}{\partial x{11}} \frac{\partial y}{\partial x{21}} \cdots \frac{\partial y}{\partial x{p1}} \newline \frac{\partial y}{\partial x{12}} \frac{\partial y}{\partial x{22}} \cdots \frac{\partial y}{\partial x{p2}} \newline \vdots \vdots \ddots \vdots \newline \frac{\partial y}{\partial x{1q}} \frac{\partial y}{\partial x{2q}} \cdots \frac{\partial y}{\partial x{pq}} \newline\end{bmatrix}{q \times p} ∂X∂y​ ​∂x11​∂y​∂x12​∂y​⋮∂x1q​∂y​​∂x21​∂y​∂x22​∂y​⋮∂x2q​∂y​​⋯⋯⋱⋯​∂xp1​∂y​∂xp2​∂y​⋮∂xpq​∂y​​ ​q×p​ 若以 分母记法 则可以表示为 ∂ y ∂ X [ ∂ y ∂ x 11 ∂ y ∂ x 12 ⋯ ∂ y ∂ x 1 q ∂ y ∂ x 21 ∂ y ∂ x 22 ⋯ ∂ y ∂ x 2 q ⋮ ⋮ ⋱ ⋮ ∂ y ∂ x p 1 ∂ y ∂ x p 2 ⋯ ∂ y ∂ x p q ] p × q \frac{\partial y}{\partial \mathbf X} \begin{bmatrix} \frac{\partial y}{\partial x{11}} \frac{\partial y}{\partial x{12}} \cdots \frac{\partial y}{\partial x{1q}} \newline \frac{\partial y}{\partial x{21}} \frac{\partial y}{\partial x{22}} \cdots \frac{\partial y}{\partial x{2q}} \newline \vdots \vdots \ddots \vdots \newline \frac{\partial y}{\partial x{p1}} \frac{\partial y}{\partial x{p2}} \cdots \frac{\partial y}{\partial x{pq}} \newline\end{bmatrix}{p \times q} ∂X∂y​ ​∂x11​∂y​∂x21​∂y​⋮∂xp1​∂y​​∂x12​∂y​∂x22​∂y​⋮∂xp2​∂y​​⋯⋯⋱⋯​∂x1q​∂y​∂x2q​∂y​⋮∂xpq​∂y​​ ​p×q​
6. 恒等式 以下各式中无特别备注默认被求导的复合函数的各因式皆不是求导变量的函数。 4.1. 向量-向量 表达式分子记法分母记法备注 ∂ a ∂ x \frac{\partial \mathbf a}{\partial \mathbf x} ∂x∂a​ 0 \mathbf 0 0 0 \mathbf 0 0 ∂ x ∂ x \frac{\partial \mathbf x}{\partial \mathbf x} ∂x∂x​ I \mathbf I I I \mathbf I I ∂ A x ∂ x \frac{\partial \mathbf A \mathbf x}{\partial \mathbf x} ∂x∂Ax​ A \mathbf A A A T \mathbf A^T AT ∂ x T A ∂ x \frac{\partial \mathbf x^T \mathbf A}{\partial \mathbf x} ∂x∂xTA​ A T \mathbf A^T AT A \mathbf A A ∂ a u ∂ x \frac{\partial a \mathbf u}{\partial \mathbf x} ∂x∂au​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial x} a∂x∂u​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial x} a∂x∂u​ u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ v u ∂ x \frac{\partial v \mathbf u}{\partial \mathbf x} ∂x∂vu​ v ∂ u ∂ x u ∂ v ∂ x v \frac{\partial \mathbf u}{\partial \mathbf x} \mathbf u \frac{\partial v}{\partial \mathbf x} v∂x∂u​u∂x∂v​ v ∂ u ∂ x ∂ v ∂ x u T v \frac{\partial \mathbf u}{\partial \mathbf x} \frac{\partial v}{\partial \mathbf x} \mathbf u^T v∂x∂u​∂x∂v​uT v v ( x ) , u u ( x ) v v(\mathbf x), \mathbf u \mathbf u(\mathbf x) vv(x),uu(x) ∂ A u ∂ x \frac{\partial \mathbf A \mathbf u}{\partial \mathbf x} ∂x∂Au​ A ∂ u ∂ x \mathbf A \frac{\partial \mathbf u}{\partial \mathbf x} A∂x∂u​ ∂ u ∂ x A T \frac{\partial \mathbf u}{\partial \mathbf x} \mathbf A^T ∂x∂u​AT u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ ( u v ) ∂ x \frac{\partial (\mathbf u \mathbf v)}{\partial \mathbf x} ∂x∂(uv)​ ∂ u ∂ x ∂ v ∂ x \frac{\partial \mathbf u}{\partial \mathbf x} \frac{\partial \mathbf v}{\partial \mathbf x} ∂x∂u​∂x∂v​ ∂ u ∂ x ∂ v ∂ x \frac{\partial \mathbf u}{\partial \mathbf x} \frac{\partial \mathbf v}{\partial \mathbf x} ∂x∂u​∂x∂v​ u u ( x ) , v v ( x ) \mathbf u \mathbf u(\mathbf x), \mathbf v \mathbf v(\mathbf x) uu(x),vv(x) ∂ f ( g ( u ) ) ∂ x \frac{\partial \mathbf f(\mathbf g(\mathbf u))}{\partial \mathbf x} ∂x∂f(g(u))​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ x \frac{\partial \mathbf f(\mathbf g)}{\partial \mathbf g} \frac{\partial \mathbf g(\mathbf u)}{\partial \mathbf u} \frac{\partial \mathbf u}{\partial \mathbf x} ∂g∂f(g)​∂u∂g(u)​∂x∂u​ ∂ u ∂ x ∂ g ( u ) ∂ u ∂ f ( g ) ∂ g \frac{\partial \mathbf u}{\partial \mathbf x} \frac{\partial \mathbf g(\mathbf u)}{\partial \mathbf u} \frac{\partial \mathbf f(\mathbf g)}{\partial \mathbf g} ∂x∂u​∂u∂g(u)​∂g∂f(g)​ u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) 4.2. 标量-向量 表达式分子记法分母记法备注 ∂ a ∂ x \frac{\partial a}{\partial \mathbf x} ∂x∂a​ 0 T \mathbf 0^T 0T 0 \mathbf 0 0 ∂ a u ∂ x \frac{\partial a u}{\partial \mathbf x} ∂x∂au​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial \mathbf x} a∂x∂u​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial \mathbf x} a∂x∂u​ u u ( x ) u u(\mathbf x) uu(x) ∂ ( u v ) ∂ x \frac{\partial (u v)}{\partial \mathbf x} ∂x∂(uv)​ ∂ u ∂ x ∂ v ∂ x \frac{\partial u}{\partial \mathbf x} \frac{\partial v}{\partial \mathbf x} ∂x∂u​∂x∂v​ ∂ u ∂ x ∂ v ∂ x \frac{\partial u}{\partial \mathbf x} \frac{\partial v}{\partial \mathbf x} ∂x∂u​∂x∂v​ u u ( x ) , v v ( x ) u u(\mathbf x), v v(\mathbf x) uu(x),vv(x) ∂ u v ∂ x \frac{\partial u v}{\partial \mathbf x} ∂x∂uv​ u ∂ v ∂ x v ∂ u ∂ x u \frac{\partial v}{\partial \mathbf x} v \frac{\partial u}{\partial \mathbf x} u∂x∂v​v∂x∂u​ u ∂ v ∂ x v ∂ u ∂ x u \frac{\partial v}{\partial \mathbf x} v \frac{\partial u}{\partial \mathbf x} u∂x∂v​v∂x∂u​ u u ( x ) , v v ( x ) u u(\mathbf x), v v(\mathbf x) uu(x),vv(x) ∂ f ( g ( u ) ) ∂ x \frac{\partial f(g(u))}{\partial \mathbf x} ∂x∂f(g(u))​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ x \frac{\partial f(g)}{\partial g} \frac{\partial g(u)}{\partial u} \frac{\partial u}{\partial \mathbf x} ∂g∂f(g)​∂u∂g(u)​∂x∂u​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ x \frac{\partial f(g)}{\partial g} \frac{\partial g(u)}{\partial u} \frac{\partial u}{\partial \mathbf x} ∂g∂f(g)​∂u∂g(u)​∂x∂u​ u u ( x ) u u(\mathbf x) uu(x) ∂ ( u ⋅ v ) ∂ x ∂ u T v ∂ x \frac{\partial (\mathbf u \cdot \mathbf v)}{\partial \mathbf x} \frac{\partial \mathbf u^T \mathbf v}{\partial \mathbf x} ∂x∂(u⋅v)​∂x∂uTv​ u T ∂ v ∂ x v T ∂ u ∂ x \mathbf u^T \frac{\partial \mathbf v}{\partial \mathbf x} \mathbf v^T \frac{\partial \mathbf u}{\partial \mathbf x} uT∂x∂v​vT∂x∂u​ ∂ v ∂ x u ∂ u ∂ x v \frac{\partial \mathbf v}{\partial \mathbf x} \mathbf u \frac{\partial \mathbf u}{\partial \mathbf x} \mathbf v ∂x∂v​u∂x∂u​v u u ( x ) , v v ( x ) \mathbf u \mathbf u(\mathbf x), \mathbf v \mathbf v(\mathbf x) uu(x),vv(x) ∂ ( u ⋅ A v ) ∂ x ∂ u T A v ∂ x \frac{\partial (\mathbf u \cdot \mathbf A \mathbf v)}{\partial \mathbf x} \frac{\partial \mathbf u^T \mathbf A \mathbf v}{\partial \mathbf x} ∂x∂(u⋅Av)​∂x∂uTAv​ u T A ∂ v ∂ x v T A T ∂ u ∂ x \mathbf u^T \mathbf A \frac{\partial \mathbf v}{\partial \mathbf x} \mathbf v^T \mathbf A^T \frac{\partial \mathbf u}{\partial \mathbf x} uTA∂x∂v​vTAT∂x∂u​ ∂ v ∂ x A T u ∂ u ∂ x A v \frac{\partial \mathbf v}{\partial \mathbf x} \mathbf A^T \mathbf u \frac{\partial \mathbf u}{\partial \mathbf x} \mathbf A \mathbf v ∂x∂v​ATu∂x∂u​Av u u ( x ) , v v ( x ) \mathbf u \mathbf u(\mathbf x), \mathbf v \mathbf v(\mathbf x) uu(x),vv(x) ∂ ( a ⋅ u ) ∂ x ∂ a T u ∂ x \frac{\partial (\mathbf a \cdot \mathbf u)}{\partial \mathbf x} \frac{\partial \mathbf a^T \mathbf u}{\partial \mathbf x} ∂x∂(a⋅u)​∂x∂aTu​ a T ∂ u ∂ x \mathbf a^T \frac{\partial \mathbf u}{\partial \mathbf x} aT∂x∂u​ ∂ u ∂ x a \frac{\partial \mathbf u}{\partial \mathbf x} \mathbf a ∂x∂u​a u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ b T A x ∂ x \frac{\partial \mathbf b^T \mathbf A \mathbf x}{\partial \mathbf x} ∂x∂bTAx​ b T A \mathbf b^T \mathbf A bTA A T b \mathbf A^T \mathbf b ATb ∂ x T A x ∂ x \frac{\partial \mathbf x^T \mathbf A \mathbf x}{\partial \mathbf x} ∂x∂xTAx​ x T ( A A T ) \mathbf x^T (\mathbf A \mathbf A^T) xT(AAT) ( A A T ) x (\mathbf A \mathbf A^T) \mathbf x (AAT)x ∂ 2 x T A x ∂ x ∂ x T \frac{\partial^2 \mathbf x^T \mathbf A \mathbf x}{\partial \mathbf x \partial \mathbf x^T} ∂x∂xT∂2xTAx​ A A T \mathbf A \mathbf A^T AAT A A T \mathbf A \mathbf A^T AAT ∂ a T x x T b ∂ x \frac{\partial \mathbf a^T \mathbf x \mathbf x^T \mathbf b}{\partial \mathbf x} ∂x∂aTxxTb​ x T ( a b T b a T ) \mathbf x^T (\mathbf a \mathbf b^T \mathbf b \mathbf a^T) xT(abTbaT) ( a b T b a T ) x (\mathbf a \mathbf b^T \mathbf b \mathbf a^T) \mathbf x (abTbaT)x ∂ ( A x b ) T C ( D x e ) ∂ x \frac{\partial (\mathbf A \mathbf x \mathbf b)^T \mathbf C (\mathbf D \mathbf x \mathbf e)}{\partial \mathbf x} ∂x∂(Axb)TC(Dxe)​ ( A x b ) T C D ( D x e ) T C T A (\mathbf A \mathbf x \mathbf b)^T \mathbf C \mathbf D (\mathbf D \mathbf x \mathbf e)^T \mathbf C^T \mathbf A (Axb)TCD(Dxe)TCTA D T C T ( A x b ) A T C ( D x e ) T \mathbf D^T \mathbf C^T(\mathbf A \mathbf x \mathbf b) \mathbf A^T \mathbf C (\mathbf D \mathbf x \mathbf e)^T DTCT(Axb)ATC(Dxe)T ∂ ∣ ∣ x ∣ ∣ 2 ∂ x ∂ ( x ⋅ x ) ∂ x \frac{\partial || \mathbf x ||^2}{\partial \mathbf x} \frac{\partial (\mathbf x \cdot \mathbf x)}{\partial \mathbf x} ∂x∂∣∣x∣∣2​∂x∂(x⋅x)​ 2 x T 2 \mathbf x^T 2xT 2 x 2 \mathbf x 2x ∂ ∣ ∣ x − a ∣ ∣ ∂ x \frac{\partial || \mathbf x - \mathbf a || }{\partial \mathbf x} ∂x∂∣∣x−a∣∣​ ( x − a ) T ∣ ∣ x − a ∣ ∣ \frac{(\mathbf x - \mathbf a)^T}{ || \mathbf x - \mathbf a || } ∣∣x−a∣∣(x−a)T​ ( x − a ) ∣ ∣ x − a ∣ ∣ \frac{(\mathbf x - \mathbf a)}{ || \mathbf x - \mathbf a || } ∣∣x−a∣∣(x−a)​ 4.3. 向量-标量 表达式分子记法分母记法备注 ∂ a ∂ x \frac{\partial \mathbf a}{\partial x} ∂x∂a​ 0 \mathbf 0 0 0 \mathbf 0 0 ∂ a u ∂ x \frac{\partial a \mathbf u}{\partial x} ∂x∂au​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial x} a∂x∂u​ a ∂ u ∂ x a \frac{\partial \mathbf u}{\partial x} a∂x∂u​ u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ A u ∂ x \frac{\partial \mathbf A \mathbf u}{\partial x} ∂x∂Au​ A ∂ u ∂ x \mathbf A \frac{\partial \mathbf u}{\partial x} A∂x∂u​ ∂ u ∂ x A T \frac{\partial \mathbf u}{\partial x} \mathbf A^T ∂x∂u​AT u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ u T ∂ x \frac{\partial \mathbf u^T}{\partial x} ∂x∂uT​ ( ∂ u ∂ x ) T \left( \frac{\partial \mathbf u}{\partial x} \right)^T (∂x∂u​)T ( ∂ u ∂ x ) T \left( \frac{\partial \mathbf u}{\partial x} \right)^T (∂x∂u​)T u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ ( u v ) ∂ x \frac{\partial (\mathbf u \mathbf v)}{\partial x} ∂x∂(uv)​ ∂ u ∂ x ∂ v ∂ x \frac{\partial \mathbf u}{\partial x} \frac{\partial \mathbf v}{\partial x} ∂x∂u​∂x∂v​ ∂ u ∂ x ∂ v ∂ x \frac{\partial \mathbf u}{\partial x} \frac{\partial \mathbf v}{\partial x} ∂x∂u​∂x∂v​ u u ( x ) , v v ( x ) \mathbf u \mathbf u(\mathbf x), \mathbf v \mathbf v(\mathbf x) uu(x),vv(x) ∂ ( u T × v ) ∂ x \frac{\partial (\mathbf u^T \times \mathbf v)}{\partial x} ∂x∂(uT×v)​ ( ∂ u ∂ x ) T × v u T × ∂ v ∂ x \left( \frac{\partial \mathbf u}{\partial x} \right)^T \times \mathbf v \mathbf u^T \times \frac{\partial \mathbf v}{\partial x} (∂x∂u​)T×vuT×∂x∂v​ ∂ u ∂ x × v u T × ( ∂ v ∂ x ) T \frac{\partial \mathbf u}{\partial x} \times \mathbf v \mathbf u^T \times \left( \frac{\partial \mathbf v}{\partial x} \right)^T ∂x∂u​×vuT×(∂x∂v​)T u u ( x ) , v v ( x ) \mathbf u \mathbf u(\mathbf x), \mathbf v \mathbf v(\mathbf x) uu(x),vv(x) ∂ f ( g ( u ) ) ∂ x \frac{\partial \mathbf f(\mathbf g(\mathbf u))}{\partial x} ∂x∂f(g(u))​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ x \frac{\partial \mathbf f(\mathbf g)}{\partial \mathbf g} \frac{\partial \mathbf g(\mathbf u)}{\partial \mathbf u} \frac{\partial \mathbf u}{\partial x} ∂g∂f(g)​∂u∂g(u)​∂x∂u​ ∂ u ∂ x ∂ g ( u ) ∂ u ∂ f ( g ) ∂ g \frac{\partial \mathbf u}{\partial x}\frac{\partial \mathbf g(\mathbf u)}{\partial \mathbf u} \frac{\partial \mathbf f(\mathbf g)}{\partial \mathbf g} ∂x∂u​∂u∂g(u)​∂g∂f(g)​ u u ( x ) \mathbf u \mathbf u(\mathbf x) uu(x) ∂ ( U × v ) ∂ x \frac{\partial (\mathbf U \times \mathbf v)}{\partial x} ∂x∂(U×v)​ ∂ U ∂ x × v U × ∂ v ∂ x \frac{\partial \mathbf U}{\partial x} \times \mathbf v \mathbf U \times \frac{\partial \mathbf v}{\partial x} ∂x∂U​×vU×∂x∂v​ v T × ∂ U ∂ x ∂ v ∂ x × U T \mathbf v^T \times \frac{\partial \mathbf U}{\partial x} \frac{\partial \mathbf v}{\partial x} \times \mathbf U^T vT×∂x∂U​∂x∂v​×UT U U ( x ) , v v ( x ) \mathbf U \mathbf U(\mathbf x), \mathbf v \mathbf v(\mathbf x) UU(x),vv(x) 4.4. 标量-矩阵 表达式分子记法分母记法备注 ∂ a ∂ X \frac{\partial a}{\partial \mathbf X} ∂X∂a​ 0 T \mathbf 0^T 0T 0 \mathbf 0 0 ∂ a u ∂ X \frac{\partial a u}{\partial \mathbf X} ∂X∂au​ a ∂ u ∂ X a \frac{\partial u}{\partial \mathbf X} a∂X∂u​ a ∂ u ∂ X a \frac{\partial u}{\partial \mathbf X} a∂X∂u​ u u ( X ) u u(\mathbf X) uu(X) ∂ ( u v ) ∂ X \frac{\partial (u v)}{\partial \mathbf X} ∂X∂(uv)​ ∂ u ∂ X ∂ v ∂ X \frac{\partial u}{\partial \mathbf X} \frac{\partial v}{\partial \mathbf X} ∂X∂u​∂X∂v​ ∂ u ∂ X ∂ v ∂ X \frac{\partial u}{\partial \mathbf X} \frac{\partial v}{\partial \mathbf X} ∂X∂u​∂X∂v​ u u ( X ) , v v ( X ) u u(\mathbf X), v v(\mathbf X) uu(X),vv(X) ∂ u v ∂ X \frac{\partial u v}{\partial \mathbf X} ∂X∂uv​ u ∂ v ∂ X v ∂ u ∂ X u \frac{\partial v}{\partial \mathbf X} v \frac{\partial u}{\partial \mathbf X} u∂X∂v​v∂X∂u​ u ∂ v ∂ X v ∂ u ∂ X u \frac{\partial v}{\partial \mathbf X} v \frac{\partial u}{\partial \mathbf X} u∂X∂v​v∂X∂u​ u u ( X ) , v v ( X ) u u(\mathbf X), v v(\mathbf X) uu(X),vv(X) ∂ f ( g ( u ) ) ∂ X \frac{\partial f(g(u))}{\partial \mathbf X} ∂X∂f(g(u))​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ X \frac{\partial f(g)}{\partial g} \frac{\partial g(u)}{\partial u} \frac{\partial u}{\partial \mathbf X} ∂g∂f(g)​∂u∂g(u)​∂X∂u​ ∂ f ( g ) ∂ g ∂ g ( u ) ∂ u ∂ u ∂ X \frac{\partial f(g)}{\partial g} \frac{\partial g(u)}{\partial u} \frac{\partial u}{\partial \mathbf X} ∂g∂f(g)​∂u∂g(u)​∂X∂u​ u u ( X ) u u(\mathbf X) uu(X) ∂ a T X b ∂ X \frac{\partial \mathbf a^T \mathbf X \mathbf b}{\partial \mathbf X} ∂X∂aTXb​ b a T \mathbf b \mathbf a^T baT a b T \mathbf a \mathbf b^T abT ∂ a T X T b ∂ X \frac{\partial \mathbf a^T \mathbf X^T \mathbf b}{\partial \mathbf X} ∂X∂aTXTb​ a b T \mathbf a \mathbf b^T abT b a T \mathbf b \mathbf a^T baT ∂ ( X a b ) T C ( X a b ) ∂ X \frac{\partial (\mathbf X \mathbf a \mathbf b)^T \mathbf C (\mathbf X \mathbf a \mathbf b)}{\partial \mathbf X} ∂X∂(Xab)TC(Xab)​ [ ( C C T ) ( X a b ) a T ] T [ (\mathbf C \mathbf C^T) (\mathbf X \mathbf a \mathbf b) \mathbf a^T ]^T [(CCT)(Xab)aT]T ( C C T ) ( X a b ) a T (\mathbf C \mathbf C^T) (\mathbf X \mathbf a \mathbf b) \mathbf a^T (CCT)(Xab)aT ∂ ( X a ) T C ( X b ) ∂ X \frac{\partial (\mathbf X \mathbf a)^T \mathbf C (\mathbf X \mathbf b)}{\partial \mathbf X} ∂X∂(Xa)TC(Xb)​ ( C X b a T C T X a b T ) T ( \mathbf C \mathbf X \mathbf b \mathbf a^T \mathbf C^T \mathbf X \mathbf a \mathbf b^T )^T (CXbaTCTXabT)T C X b a T C T X a b T \mathbf C \mathbf X \mathbf b \mathbf a^T \mathbf C^T \mathbf X \mathbf a \mathbf b^T CXbaTCTXabT ∂ ∣ X ∣ ∂ X \frac{\partial | \mathbf X | }{\partial \mathbf X} ∂X∂∣X∣​ ∣ X ∣ X − 1 | \mathbf X | \mathbf X^{ - 1} ∣X∣X−1 ∣ X ∣ ( X − 1 ) T | \mathbf X | (\mathbf X^{ - 1})^T ∣X∣(X−1)T ∂ ln ⁡ ∣ a X ∣ ∂ X \frac{\partial \ln | a \mathbf X | }{\partial \mathbf X} ∂X∂ln∣aX∣​ X − 1 \mathbf X^{ - 1} X−1 ( X − 1 ) T (\mathbf X^{ - 1})^T (X−1)T ∂ ∣ A X B ∣ ∂ X \frac{ \partial | \mathbf A \mathbf X \mathbf B | }{\partial \mathbf X} ∂X∂∣AXB∣​ ∣ A X B ∣ X − 1 | \mathbf A \mathbf X \mathbf B | \mathbf X^{ - 1} ∣AXB∣X−1 ∣ A X B ∣ ( X − 1 ) T | \mathbf A \mathbf X \mathbf B | (\mathbf X^{ - 1})^T ∣AXB∣(X−1)T ∂ ∣ X n ∣ ∂ X \frac{ \partial | \mathbf X^n | }{\partial \mathbf X} ∂X∂∣Xn∣​ n ∣ X n ∣ X − 1 n | \mathbf X^n | \mathbf X^{ - 1} n∣Xn∣X−1 n ∣ X n ∣ ( X − 1 ) T n | \mathbf X^n | (\mathbf X^{ - 1})^T n∣Xn∣(X−1)T ∂ ln ⁡ ∣ X T X ∣ ∂ X \frac{ \partial \ln | \mathbf X^T \mathbf X | }{\partial \mathbf X} ∂X∂ln∣XTX∣​ 2 X 2 \mathbf X^ 2X 2 ( X ) T 2 (\mathbf X^)^T 2(X)T X \mathbf X^ X 为 X \mathbf X X 的广义逆 ∂ ln ⁡ ∣ X T X ∣ ∂ X \frac{\partial \ln | \mathbf X^T \mathbf X | }{\partial \mathbf X^} ∂X∂ln∣XTX∣​ − 2 X - 2 \mathbf X −2X − 2 X T - 2 \mathbf X^T −2XT X \mathbf X^ X 为 X \mathbf X X 的广义逆 ∂ ∣ X T A X ∣ ∂ X \frac{\partial | \mathbf X^T \mathbf A \mathbf X | }{\partial \mathbf X} ∂X∂∣XTAX∣​ 2 ∣ X T A X ∣ X − 1 2 ∣ X T ∣ ∣ A ∣ ∣ X ∣ X − 1 2 | \mathbf X^T \mathbf A \mathbf X | \mathbf X^{ - 1} 2 | \mathbf X^T | | \mathbf A | | \mathbf X | \mathbf X^{ - 1} 2∣XTAX∣X−12∣XT∣∣A∣∣X∣X−1 2 ∣ X T A X ∣ ( X − 1 ) T 2 | \mathbf X^T \mathbf A \mathbf X | (\mathbf X^{ - 1})^T 2∣XTAX∣(X−1)T X \mathbf X X 为方阵且可逆 ∂ ∣ X T A X ∣ ∂ X \frac{\partial | \mathbf X^T \mathbf A \mathbf X | }{\partial \mathbf X} ∂X∂∣XTAX∣​ 2 ∣ X T A X ∣ ( X T A T X ) − 1 X T A T 2 | \mathbf X^T \mathbf A \mathbf X | ( \mathbf X^T \mathbf A^T \mathbf X )^{ - 1} \mathbf X^T \mathbf A^T 2∣XTAX∣(XTATX)−1XTAT 2 ∣ X T A X ∣ A X ( X T A X ) − 1 2 | \mathbf X^T \mathbf A \mathbf X | \mathbf A \mathbf X ( \mathbf X^T \mathbf A \mathbf X )^{ - 1} 2∣XTAX∣AX(XTAX)−1 A \mathbf A A 对称 ∂ ∣ X T A X ∣ ∂ X \frac{\partial | \mathbf X^T \mathbf A \mathbf X | }{\partial \mathbf X} ∂X∂∣XTAX∣​ ∣ X T A X ∣ [ ( X T A X ) − 1 X T A ( X T A T X ) − 1 X T A T ] | \mathbf X^T \mathbf A \mathbf X | [ ( \mathbf X^T \mathbf A \mathbf X)^{ - 1} \mathbf X^T \mathbf A ( \mathbf X^T \mathbf A^T \mathbf X )^{ - 1} \mathbf X^T \mathbf A^T ] ∣XTAX∣[(XTAX)−1XTA(XTATX)−1XTAT] ∣ X T A X ∣ [ A X ( X T A X ) − 1 A T X ( X T A T X ) − 1 ] | \mathbf X^T \mathbf A \mathbf X | [ \mathbf A \mathbf X ( \mathbf X^T \mathbf A \mathbf X )^{ - 1} \mathbf A^T \mathbf X ( \mathbf X^T \mathbf A^T \mathbf X )^{ - 1} ] ∣XTAX∣[AX(XTAX)−1ATX(XTATX)−1] 4.5. 矩阵-标量 表达式分子记法备注 ∂ a U ∂ x \frac{\partial a \mathbf U}{\partial x} ∂x∂aU​ a ∂ U ∂ x a \frac{\partial \mathbf U}{\partial x} a∂x∂U​ U U ( x ) \mathbf U \mathbf U(x) UU(x) ∂ A U B ∂ x \frac{\partial \mathbf A \mathbf U \mathbf B}{\partial x} ∂x∂AUB​ A ∂ U ∂ x B \mathbf A \frac{\partial \mathbf U}{\partial x} \mathbf B A∂x∂U​B U U ( x ) \mathbf U \mathbf U(x) UU(x) ∂ ( U V ) ∂ x \frac{\partial (\mathbf U \mathbf V)}{\partial x} ∂x∂(UV)​ ∂ U ∂ x ∂ V ∂ x \frac{\partial \mathbf U}{\partial x} \frac{\partial \mathbf V}{\partial x} ∂x∂U​∂x∂V​ U U ( x ) , V V ( x ) \mathbf U \mathbf U(x), \mathbf V \mathbf V(x) UU(x),VV(x) ∂ ( U V ) ∂ x \frac{\partial (\mathbf U \mathbf V)}{\partial x} ∂x∂(UV)​ U ∂ V ∂ x ∂ U ∂ x V \mathbf U \frac{\partial \mathbf V}{\partial x} \frac{\partial \mathbf U}{\partial x} \mathbf V U∂x∂V​∂x∂U​V U U ( x ) , V V ( x ) \mathbf U \mathbf U(x), \mathbf V \mathbf V(x) UU(x),VV(x) ∂ ( U ⊗ V ) ∂ x \frac{\partial (\mathbf U \otimes \mathbf V)}{\partial x} ∂x∂(U⊗V)​ U ⊗ ∂ V ∂ x ∂ U ∂ x ⊗ V \mathbf U \otimes \frac{\partial \mathbf V}{\partial x} \frac{\partial \mathbf U}{\partial x} \otimes \mathbf V U⊗∂x∂V​∂x∂U​⊗V U U ( x ) , V V ( x ) \mathbf U \mathbf U(x), \mathbf V \mathbf V(x) UU(x),VV(x) ⊗ \otimes ⊗ 表示 Kronecker 乘积 ∂ ( U ∘ V ) ∂ x \frac{\partial (\mathbf U \circ \mathbf V)}{\partial x} ∂x∂(U∘V)​ U ∘ ∂ V ∂ x ∂ U ∂ x ∘ V \mathbf U \circ \frac{\partial \mathbf V}{\partial x} \frac{\mathbf \partial U}{\partial x} \circ \mathbf V U∘∂x∂V​∂x∂U​∘V U U ( x ) , V V ( x ) \mathbf U \mathbf U(x), \mathbf V \mathbf V(x) UU(x),VV(x) ∘ \circ ∘ 表示 Hadamard 乘积 ∂ U − 1 ∂ x \frac{\partial \mathbf U^{ - 1}}{\partial x} ∂x∂U−1​ − U − 1 ∂ U ∂ x U − 1 -\mathbf U^{ - 1} \frac{\partial \mathbf U}{\partial x} \mathbf U^{ - 1} −U−1∂x∂U​U−1 U U ( x ) \mathbf U \mathbf U(x) UU(x) ∂ 2 U − 1 ∂ x ∂ y \frac{\partial^2 \mathbf U^{ - 1}}{\partial x \partial y} ∂x∂y∂2U−1​ U − 1 ( ∂ U ∂ x U − 1 ∂ U ∂ y − ∂ 2 U ∂ x ∂ y ∂ U ∂ y U − 1 ∂ U ∂ x ) U − 1 \mathbf U^{ - 1} \left( \frac{\partial \mathbf U}{\partial x} \mathbf U^{ - 1} \frac{\partial \mathbf U}{\partial y} - \frac{\partial^2 \mathbf U}{\partial x \partial y} \frac{\partial \mathbf U}{\partial y} \mathbf U^{ - 1} \frac{\partial \mathbf U}{\partial x} \right) \mathbf U^{ - 1} U−1(∂x∂U​U−1∂y∂U​−∂x∂y∂2U​∂y∂U​U−1∂x∂U​)U−1 U U ( x , y ) \mathbf U \mathbf U(x, y) UU(x,y) ∂ g ( x A ) ∂ x \frac{\partial g (x \mathbf A)}{\partial x} ∂x∂g(xA)​ A g ′ ( x A ) g ′ ( x A ) A \mathbf A g (x \mathbf A) g (x \mathbf A) \mathbf A Ag′(xA)g′(xA)A应为 Hadamard 乘积 g ( ⋅ ) g (\cdot) g(⋅) 为逐元函数如下例 ∂ e x A ∂ x \frac{\partial e^{x \mathbf A}}{\partial x} ∂x∂exA​ A e x A e x A A \mathbf A e^{x \mathbf A} e^{x \mathbf A} \mathbf A AexAexAA 二、矩阵分解 QR分解 M Q R M QR MQR, Q正交R上三角。奇异值分解Singular Value DecompositionSVD M U Σ V T M UΣV^T MUΣVT, U和V正交Σ非负对角。特征分解Eigendecomposition,又叫谱分解(Spectral decomposition) S Q Λ Q T S QΛQ^T SQΛQT, S对称Q正交Λ对角。极分解 M Q S M QS MQS, Q正交S对称半正定。科列斯基分解Cholesky decomposition A L L ∗ \mathbf {A} \mathbf {LL} ^{} ALL∗ L \mathbf{L} L 下三角矩阵且所有对角元素均为正实数 L ∗ \mathbf {L} ^{} L∗表示 L \mathbf {L} L 的共轭转置。每一个正定埃尔米特矩阵都有一个唯一的科列斯基分解LU分解 A L U ALU ALUL下三角, U上三角
7. 科列斯基分解
   科列斯基分解主要被用于线性方程组 A x b \mathbf {Ax} \mathbf {b} Axb 的求解。如果 A A A 是对称正定的我们可以先求出 A L L T \mathbf {A} \mathbf {LL} ^{\mathbf {T} } ALLT随后借向后替换法对 y y y 求解 L y b \mathbf {Ly} \mathbf {b} Lyb再以向前替换法对 x x x 求解 L T x y \mathbf {L} ^{\mathbf {T} }\mathbf {x} \mathbf {y} LTxy即得最终解。 另一种可避免在计算 L L T \mathbf {LL} ^{\mathbf {T} } LLT时需要解平方根的方法就是计算 A L D L T \mathbf {A} \mathbf {LDL} ^{\mathrm {T} } ALDLT然后对 y y y 求解 L y b \mathbf {Ly} \mathbf {b} Lyb最后求解 D L T x y \mathbf {DL} ^{\mathrm {T} }\mathbf {x} \mathbf {y} DLTxy 对于可以被改写成对称矩阵的线性方程组科列斯基分解及其LDL变形是一个较高效率及较高数值稳定性的求解方法。相比之下其效率几近为LU分解的两倍
8. SGD分解 三、矩阵种类 1.「正定矩阵」和「半正定矩阵」 案例多元正态分布的协方差矩阵要求是半正定的 【定义1】 给定一个大小为 n × n n\times n n×n 的实对称矩阵 A A A若对于任意长度为 n n n 的非零向量 x \boldsymbol{x} x有 x T A x 0 \boldsymbol{x}^TA\boldsymbol{x}0 xTAx0 恒成立则矩阵 A A A是一个正定矩阵。 【定义2】 给定一个大小为 n × n n\times n n×n 的实对称矩阵 A A A 若对于任意长度为 n n n 的向量 x \boldsymbol{x} x 有 x T A x ≥ 0 \boldsymbol{x}^TA\boldsymbol{x}\geq0 xTAx≥0 恒成立则矩阵 A A A 是一个半正定矩阵。 直观解释 若给定任意一个正定矩阵 A ∈ R n × n A\in\mathbb{R}^{n\times n} A∈Rn×n 和一个非零向量 x ∈ R n \boldsymbol{x}\in\mathbb{R}^{n} x∈Rn 则两者相乘得到的向量 y A x ∈ R n \boldsymbol{y}A\boldsymbol{x}\in\mathbb{R}^{n} yAx∈Rn 与向量 x \boldsymbol{x} x 的夹角恒小于 π 2 \frac{\pi}{2} 2π​ . (等价于 x T A x 0 \boldsymbol{x}^TA\boldsymbol{x}0 xTAx0 .) 若给定任意一个半正定矩阵 A ∈ R n × n A\in\mathbb{R}^{n\times n} A∈Rn×n 和一个向量 x ∈ R n \boldsymbol{x}\in\mathbb{R}^{n} x∈Rn 则两者相乘得到的向量 y A x ∈ R n \boldsymbol{y}A\boldsymbol{x}\in\mathbb{R}^{n} yAx∈Rn 与向量 x \boldsymbol{x} x 的夹角恒小于或等于 π 2 \frac{\pi}{2} 2π​ . (等价于 x T A x ≥ 0 \boldsymbol{x}^TA\boldsymbol{x}\geq0 xTAx≥0 .) 1.1 为什么协方差矩阵是半正定的 对于任意多元随机变量 t \boldsymbol{t} t 协方差矩阵为 C E [ ( t − t ˉ ) ( t − t ˉ ) T ] C\mathbb{E}\left[(\boldsymbol{t}-\bar{\boldsymbol{t}})(\boldsymbol{t}-\bar{\boldsymbol{t}})^T\right] CE[(t−tˉ)(t−tˉ)T] 现给定任意一个向量 x \boldsymbol{x} x 则 x T C x x T E [ ( t − t ˉ ) ( t − t ˉ ) T ] x E [ x T ( t − t ˉ ) ( t − t ˉ ) T x ] E ( s 2 ) σ s 2 \boldsymbol{x}^TC\boldsymbol{x}\boldsymbol{x}^T\mathbb{E}\left[(\boldsymbol{t}-\bar{\boldsymbol{t}})(\boldsymbol{t}-\bar{\boldsymbol{t}})^T\right]\boldsymbol{x} \mathbb{E}\left[\boldsymbol{x}^T(\boldsymbol{t}-\bar{\boldsymbol{t}})(\boldsymbol{t}-\bar{\boldsymbol{t}})^T\boldsymbol{x}\right]\mathbb{E}(s^2)\sigma_{s}^2 xTCxxTE[(t−tˉ)(t−tˉ)T]xE[xT(t−tˉ)(t−tˉ)Tx]E(s2)σs2​ 其中 σ s x T ( t − t ˉ ) ( t − t ˉ ) T x \sigma_s\boldsymbol{x}^T(\boldsymbol{t}-\bar{\boldsymbol{t}})(\boldsymbol{t}-\bar{\boldsymbol{t}})^T\boldsymbol{x} σs​xT(t−tˉ)(t−tˉ)Tx。由于 σ s 2 ≥ 0 \sigma_s^2\geq0 σs2​≥0 因此 x T C x ≥ 0 \boldsymbol{x}^TC\boldsymbol{x}\geq0 xTCx≥0 协方差矩阵 C C C 是半正定的。
9. 逆矩阵 分块矩阵Block matrix 的逆矩阵恒等式 ( A B C D ) − 1 ( M − M B D − 1 − D − 1 C M D − 1 D − 1 C M B D − 1 ) \begin{pmatrix}AB\CD\end{pmatrix}^{-1}\begin{pmatrix}M-MBD^{-1}\-D^{-1}CMD^{-1}{D^{-1}CMBD^{-1}}\end{pmatrix} (AC​BD​)−1(M−D−1CM​−MBD−1D−1D−1CMBD−1​) 其中 M ( A − B D − 1 C ) − 1 M(A-BD^{-1}C)^{-1} M(A−BD−1C)−1 若A,C为可逆方阵则有 ( A B C D ) − 1 A − 1 − A − 1 B ( D A − 1 B C − 1 ) − 1 D A − 1 (ABCD)^{-1}A^{-1}-A^{-1}B(DA^{-1}BC^{-1})^{-1}DA^{-1} (ABCD)−1A−1−A−1B(DA−1BC−1)−1DA−1 工具网站 Matrix Calculus在线计算矩阵导数 References 矩阵微积分 | Here4U