有限實矩陣乘法法則的證明
設 m , n , p , q m,n,p,q m , n , p , q A ∈ R m × n A\in\mathbb R^{m\times n} A ∈ R m × n B ∈ R n × p B\in\mathbb R^{n\times p} B ∈ R n × p C ∈ R p × q C\in\mathbb R^{p\times q} C ∈ R p × q ( i , ℓ ) (i,\ell) ( i , ℓ )
( ( A B ) C ) i ℓ = ∑ k = 1 p ( ∑ j = 1 n a i j b j k ) c k ℓ = ∑ j = 1 n a i j ( ∑ k = 1 p b j k c k ℓ ) = ( A ( B C ) ) i ℓ . \begin{aligned}
((AB)C)_{i\ell}
&=\sum_{k=1}^{p}\left(\sum_{j=1}^{n}a_{ij}b_{jk}\right)c_{k\ell}\\
&=\sum_{j=1}^{n}a_{ij}\left(\sum_{k=1}^{p}b_{jk}c_{k\ell}\right)
=(A(BC))_{i\ell}.
\end{aligned} (( A B ) C ) i ℓ = k = 1 ∑ p ( j = 1 ∑ n a ij b j k ) c k ℓ = j = 1 ∑ n a ij ( k = 1 ∑ p b j k c k ℓ ) = ( A ( B C ) ) i ℓ . 中間等號只交換兩個有限和的求和次序，並使用實數分配律。兩邊同為 m × q m\times q m × q
對 D ∈ R n × p D\in\mathbb R^{n\times p} D ∈ R n × p
[ A ( B + D ) ] i k = ∑ j = 1 n a i j ( b j k + d j k ) = ( A B ) i k + ( A D ) i k . [A(B+D)]_{ik}=\sum_{j=1}^{n}a_{ij}(b_{jk}+d_{jk})
=(AB)_{ik}+(AD)_{ik}. [ A ( B + D ) ] ik = j = 1 ∑ n a ij ( b j k + d j k ) = ( A B ) ik + ( A D ) ik . 對 E ∈ R m × n E\in\mathbb R^{m\times n} E ∈ R m × n
[ ( A + E ) B ] i k = ∑ j = 1 n ( a i j + e i j ) b j k = ( A B ) i k + ( E B ) i k . [(A+E)B]_{ik}=\sum_{j=1}^{n}(a_{ij}+e_{ij})b_{jk}
=(AB)_{ik}+(EB)_{ik}. [( A + E ) B ] ik = j = 1 ∑ n ( a ij + e ij ) b j k = ( A B ) ik + ( E B ) ik . 所以在所寫的相容大小下，兩條分配律均成立。對任意實純量 c c c
[ ( c A ) B ] i k = ∑ j ( c a i j ) b j k = c ∑ j a i j b j k = ∑ j a i j ( c b j k ) = [ A ( c B ) ] i k , [(cA)B]_{ik}=\sum_j(ca_{ij})b_{jk}
=c\sum_j a_{ij}b_{jk}
=\sum_j a_{ij}(cb_{jk})=[A(cB)]_{ik}, [( c A ) B ] ik = j ∑ ( c a ij ) b j k = c j ∑ a ij b j k = j ∑ a ij ( c b j k ) = [ A ( c B ) ] ik , 逐項相等便證明 ( c A ) B = c ( A B ) = A ( c B ) (cA)B=c(AB)=A(cB) ( c A ) B = c ( A B ) = A ( c B ) x , y ∈ R n x,y\in\mathbb R^n x , y ∈ R n s , t s,t s , t
[ A ( s x + t y ) ] i = ∑ j a i j ( s x j + t y j ) = s ( A x ) i + t ( A y ) i . [A(sx+ty)]_i=\sum_j a_{ij}(sx_j+ty_j)
=s(Ax)_i+t(Ay)_i. [ A ( s x + t y ) ] i = j ∑ a ij ( s x j + t y j ) = s ( A x ) i + t ( A y ) i . 這證明了比較同一線性系統的解時所用的矩陣乘向量線性性質。所有恆等式都只針對大小相容的有限實矩陣，不聲稱任意矩陣因子可交換或可消去。
實內積、誘導範數與所需不等式
實向量空間上的內積是取值於實數的映射 ⟨ u , v ⟩ \langle u,v\rangle ⟨ u , v ⟩ v ≠ 0 v\ne0 v = 0 ⟨ v , v ⟩ > 0 \langle v,v\rangle>0 ⟨ v , v ⟩ > 0 ⟨ 0 , v ⟩ = 0 \langle0,v\rangle=0 ⟨ 0 , v ⟩ = 0 ∥ v ∥ = ⟨ v , v ⟩ \|v\|=\sqrt{\langle v,v\rangle} ∥ v ∥ = ⟨ v , v ⟩ ∥ c v ∥ = ∣ c ∣ ∥ v ∥ \|cv\|=|c|\|v\| ∥ c v ∥ = ∣ c ∣∥ v ∥ R m \mathbb R^m R m ⟨ u , v ⟩ = ∑ i = 1 m u i v i \langle u,v\rangle=\sum_{i=1}^mu_iv_i ⟨ u , v ⟩ = ∑ i = 1 m u i v i
若 v ≠ 0 v\ne0 v = 0 t = ⟨ u , v ⟩ / ∥ v ∥ 2 t=\langle u,v\rangle/\|v\|^2 t = ⟨ u , v ⟩ /∥ v ∥ 2 ∥ u − t v ∥ 2 \|u-tv\|^2 ∥ u − t v ∥ 2 ∥ u ∥ 2 − ⟨ u , v ⟩ 2 / ∥ v ∥ 2 ≥ 0 \|u\|^2-\langle u,v\rangle^2/\|v\|^2\ge0 ∥ u ∥ 2 − ⟨ u , v ⟩ 2 /∥ v ∥ 2 ≥ 0 ∣ ⟨ u , v ⟩ ∣ ≤ ∥ u ∥ ∥ v ∥ |\langle u,v\rangle|\le\|u\|\|v\| ∣ ⟨ u , v ⟩ ∣ ≤ ∥ u ∥∥ v ∥ v = 0 v=0 v = 0 v ≠ 0 v\ne0 v = 0 ∥ u − t v ∥ = 0 \|u-tv\|=0 ∥ u − t v ∥ = 0 u = t v u=tv u = t v ∥ u + v ∥ 2 \|u+v\|^2 ∥ u + v ∥ 2 ∥ u + v ∥ 2 ≤ ( ∥ u ∥ + ∥ v ∥ ) 2 \|u+v\|^2\le(\|u\|+\|v\|)^2 ∥ u + v ∥ 2 ≤ ( ∥ u ∥ + ∥ v ∥ ) 2
不預設計數結論的有限基存在與擴充
設實向量空間 V V V t 1 , … , t N t_1,\ldots,t_N t 1 , … , t N u 1 , … , u k u_1,\ldots,u_k u 1 , … , u k N N N j j j u 1 , … , u j u_1,\ldots,u_j u 1 , … , u j N − j N-j N − j u j + 1 u_{j+1} u j + 1 u i u_i u i u j + 1 u_{j+1} u j + 1 k > N k>N k > N N N N N N N V V V u N + 1 u_{N+1} u N + 1 N = 0 N=0 N = 0
每個子空間 W W W W W W W W W N N N W W W W = { 0 } W=\{0\} W = { 0 } W W W W W W p , q p,q p , q p ≤ q p\le q p ≤ q q ≤ p q\le p q ≤ p
正交規範基，以及保留指定向量的擴充
再設 V V V v 1 , … , v s v_1,\ldots,v_s v 1 , … , v s
w j = v j − ∑ i < j ⟨ v j , q i ⟩ q i , q j = w j / ∥ w j ∥ . w_j=v_j-\sum_{i<j}\langle v_j,q_i\rangle q_i,\qquad q_j=w_j/\|w_j\|. w j = v j − i < j ∑ ⟨ v j , q i ⟩ q i , q j = w j /∥ w j ∥. 歸納假設先前 q i q_i q i l < j l<j l < j q l q_l q l ⟨ w j , q l ⟩ = ⟨ v j , q l ⟩ − ⟨ v j , q l ⟩ = 0 \langle w_j,q_l\rangle=\langle v_j,q_l\rangle-\langle v_j,q_l\rangle=0 ⟨ w j , q l ⟩ = ⟨ v j , q l ⟩ − ⟨ v j , q l ⟩ = 0 w j = 0 w_j=0 w j = 0 v j v_j v j j j j
QR 唯一性所用的上三角逆矩陣
設整數 n ≥ 1 n\geq1 n ≥ 1 R = ( r i j ) R=(r_{ij}) R = ( r ij ) n × n n\times n n × n r i i ≠ 0 r_{ii}\ne0 r ii = 0 b ∈ R n b\in\mathbb R^n b ∈ R n R x = b Rx=b R x = b
x i = b i − ∑ k = i + 1 n r i k x k r i i , i = n , n − 1 , … , 1. x_i=\frac{b_i-\sum_{k=i+1}^n r_{ik}x_k}{r_{ii}},\qquad i=n,n-1,\ldots,1. x i = r ii b i − ∑ k = i + 1 n r ik x k , i = n , n − 1 , … , 1. 每步右邊的項都已經確定，所以該公式給出一個解；同一個向下歸納也證明解唯一。因此 R R R R − 1 R^{-1} R − 1 j j j b = e j b=e_j b = e j i > j i>j i > j x i = 0 x_i=0 x i = 0 x j = 1 / r j j x_j=1/r_{jj} x j = 1/ r j j R − 1 R^{-1} R − 1 1 / r j j 1/r_{jj} 1/ r j j R R R
若 U , V U,V U , V i > j i>j i > j ∑ k u i k v k j \sum_k u_{ik}v_{kj} ∑ k u ik v k j k ≥ i k\geq i k ≥ i k ≤ j k\leq j k ≤ j i = j i=j i = j k = i k=i k = i ( U V ) i i = u i i v i i (UV)_{ii}=u_{ii}v_{ii} ( U V ) ii = u ii v ii R 1 , R 2 R_1,R_2 R 1 , R 2 R 1 R 2 − 1 R_1R_2^{-1} R 1 R 2 − 1
預備知識與學習目標
本章假設讀者熟悉實向量、內積、歐氏範數、矩陣乘法與轉置、消元、欄空間、零空間和基。對 u , v ∈ R m u,v\in\mathbb R^m u , v ∈ R m u T v u^Tv u T v ∥ u ∥ = u T u \|u\|=\sqrt{u^Tu} ∥ u ∥ = u T u q 1 , … , q r q_1,\ldots,q_r q 1 , … , q r 標準正交組 ，如果 i = j i=j i = j q i T q j q_i^Tq_j q i T q j
Gram–Schmidt 正交化不只是把兩個向量變成一對互相垂直的向量：它在保持逐步生成的張成空間時構造基，其係數組成矩陣分解，而該分解又把最小二乘問題分解為垂直殘差與三角方程組。本章完整展開這一過程，包括某欄線性相關或所有欄均為零時的情形。
學習目標是構造並檢驗薄 QR 分解、由矩陣識別正交投影、求解最小二乘問題而不混淆擬合向量與係數，以及判斷模型的欄線性相關時哪些信息仍可識別。
1. 保持張成空間的 Gram–Schmidt 方法與線性相關情形
設 a 1 , … , a n ∈ R m a_1,\ldots,a_n\in\mathbb R^m a 1 , … , a n ∈ R m A A A q 1 , … , q r q_1,\ldots,q_r q 1 , … , q r a j a_j a j
w = a j − ∑ i = 1 r ( q i T a j ) q i . w=a_j-\sum_{i=1}^r(q_i^Ta_j)q_i. w = a j − i = 1 ∑ r ( q i T a j ) q i . 與先前的 q ℓ q_\ell q ℓ
q ℓ T w = q ℓ T a j − ∑ i = 1 r ( q i T a j ) ( q ℓ T q i ) = 0. q_\ell^Tw=q_\ell^Ta_j-\sum_{i=1}^r(q_i^Ta_j)(q_\ell^Tq_i)=0. q ℓ T w = q ℓ T a j − i = 1 ∑ r ( q i T a j ) ( q ℓ T q i ) = 0. 有兩種情況：若 w ≠ 0 w\ne0 w = 0 q r + 1 = w / ∥ w ∥ q_{r+1}=w/\|w\| q r + 1 = w /∥ w ∥ w = 0 w=0 w = 0
用歸納法可證明該過程保持所需張成空間。非零情形下，w w w a j a_j a j q r + 1 q_{r+1} q r + 1
a j = ∑ i = 1 r ( q i T a j ) q i + ∥ w ∥ q r + 1 , a_j=\sum_{i=1}^r(q_i^Ta_j)q_i+\|w\|q_{r+1}, a j = i = 1 ∑ r ( q i T a j ) q i + ∥ w ∥ q r + 1 , 所以新欄也屬於新基向量組的張成空間。零情形下，展示的投影和已經等於 a j a_j a j
處理完所有欄後，最終向量組的長度為 r = rank A r=\operatorname{rank}A r = rank A Q Q Q Q T Q = I r Q^TQ=I_r Q T Q = I r
A = Q B , B = Q T A . A=QB,\qquad B=Q^TA. A = QB , B = Q T A . 事實上，A A A Q Q Q q i q_i q i B B B r × n r\times n r × n
若 A = 0 A=0 A = 0 Q Q Q
精確線性相關與數值判斷
對精確的有理數或符號數據，w = 0 w=0 w = 0
2. 從線性無關的欄到薄 QR 分解
現假設 n n n n ≤ m n\le m n ≤ m n n n
r i j = q i T a j ( i < j ) , r j j = ∥ w j ∥ > 0 , r i j = 0 ( i > j ) . r_{ij}=q_i^Ta_j\quad(i<j),\qquad r_{jj}=\|w_j\|>0,\qquad r_{ij}=0\quad(i>j). r ij = q i T a j ( i < j ) , r j j = ∥ w j ∥ > 0 , r ij = 0 ( i > j ) . 第 j j j a j = ∑ i = 1 j r i j q i a_j=\sum_{i=1}^j r_{ij}q_i a j = ∑ i = 1 j r ij q i
A = Q R , Q T Q = I n , A=QR,\qquad Q^TQ=I_n, A = QR , Q T Q = I n , 其中 Q Q Q m × n m\times n m × n R R R n × n n\times n n × n 薄 QR 分解 。“薄”指僅保留欄空間所需的 n n n n < m n<m n < m Q Q Q Q T Q^T Q T R = Q T A R=Q^TA R = Q T A
反過來，若這種分解中的 R R R A x = 0 Ax=0 A x = 0 R x = Q T A x = 0 Rx=Q^TAx=0 R x = Q T A x = 0 x = 0 x=0 x = 0 A A A R R R
正對角線還消除了符號歧義。事實上，對固定次序的線性無關欄組，這種薄 QR 分解唯一。假設 Q 1 R 1 = Q 2 R 2 = A Q_1R_1=Q_2R_2=A Q 1 R 1 = Q 2 R 2 = A
Q 2 = Q 1 M , M = R 1 R 2 − 1 . Q_2=Q_1M,\qquad M=R_1R_2^{-1}. Q 2 = Q 1 M , M = R 1 R 2 − 1 . 矩陣 M M M I = Q 2 T Q 2 = M T M I=Q_2^TQ_2=M^TM I = Q 2 T Q 2 = M T M I I I M = I M=I M = I Q 1 = Q 2 Q_1=Q_2 Q 1 = Q 2 R 1 = R 2 R_1=R_2 R 1 = R 2
3. 完整的三欄計算
例 1。 設
A = ( 1 1 1 1 0 0 0 1 0 0 0 1 ) , a 1 = ( 1 , 1 , 0 , 0 ) T , a 2 = ( 1 , 0 , 1 , 0 ) T , a 3 = ( 1 , 0 , 0 , 1 ) T . A=\begin{pmatrix}
1&1&1\\1&0&0\\0&1&0\\0&0&1
\end{pmatrix},\qquad
a_1=(1,1,0,0)^T,\ a_2=(1,0,1,0)^T,\ a_3=(1,0,0,1)^T. A = 1 1 0 0 1 0 1 0 1 0 0 1 , a 1 = ( 1 , 1 , 0 , 0 ) T , a 2 = ( 1 , 0 , 1 , 0 ) T , a 3 = ( 1 , 0 , 0 , 1 ) T . 最後三行直接說明各欄線性無關。對第一欄，
q 1 = ( 1 , 1 , 0 , 0 ) T / 2 , r 11 = 2 . q_1=(1,1,0,0)^T/\sqrt2,\qquad r_{11}=\sqrt2. q 1 = ( 1 , 1 , 0 , 0 ) T / 2 , r 11 = 2 . 對第二欄，r 12 = q 1 T a 2 = 1 / 2 r_{12}=q_1^Ta_2=1/\sqrt2 r 12 = q 1 T a 2 = 1/ 2
w 2 = a 2 − 1 2 ( 1 , 1 , 0 , 0 ) T = 1 2 ( 1 , − 1 , 2 , 0 ) T . w_2=a_2-\tfrac12(1,1,0,0)^T=\tfrac12(1,-1,2,0)^T. w 2 = a 2 − 2 1 ( 1 , 1 , 0 , 0 ) T = 2 1 ( 1 , − 1 , 2 , 0 ) T . 其範數為 6 / 2 \sqrt6/2 6 /2 q 2 = ( 1 , − 1 , 2 , 0 ) T / 6 q_2=(1,-1,2,0)^T/\sqrt6 q 2 = ( 1 , − 1 , 2 , 0 ) T / 6 r 22 = 6 / 2 r_{22}=\sqrt6/2 r 22 = 6 /2
對第三欄，r 13 = 1 / 2 r_{13}=1/\sqrt2 r 13 = 1/ 2 r 23 = 1 / 6 r_{23}=1/\sqrt6 r 23 = 1/ 6
w 3 = ( 1 , 0 , 0 , 1 ) T − 1 2 ( 1 , 1 , 0 , 0 ) T − 1 6 ( 1 , − 1 , 2 , 0 ) T = 1 3 ( 1 , − 1 , − 1 , 3 ) T . \begin{aligned}
w_3&=(1,0,0,1)^T-\tfrac12(1,1,0,0)^T-\tfrac16(1,-1,2,0)^T\\
&=\tfrac13(1,-1,-1,3)^T.
\end{aligned} w 3 = ( 1 , 0 , 0 , 1 ) T − 2 1 ( 1 , 1 , 0 , 0 ) T − 6 1 ( 1 , − 1 , 2 , 0 ) T = 3 1 ( 1 , − 1 , − 1 , 3 ) T . 因此 ∥ w 3 ∥ = 2 / 3 \|w_3\|=2/\sqrt3 ∥ w 3 ∥ = 2/ 3
q 3 = 1 2 3 ( 1 , − 1 , − 1 , 3 ) T . q_3=\frac1{2\sqrt3}(1,-1,-1,3)^T. q 3 = 2 3 1 ( 1 , − 1 , − 1 , 3 ) T . 得到的三角因子為
R = ( 2 1 / 2 1 / 2 0 6 / 2 1 / 6 0 0 2 / 3 ) . R=\begin{pmatrix}
\sqrt2&1/\sqrt2&1/\sqrt2\\
0&\sqrt6/2&1/\sqrt6\\
0&0&2/\sqrt3
\end{pmatrix}. R = 2 0 0 1/ 2 6 /2 0 1/ 2 1/ 6 2/ 3 . 獨立核對時，未歸一化方向 ( 1 , 1 , 0 , 0 ) (1,1,0,0) ( 1 , 1 , 0 , 0 ) ( 1 , − 1 , 2 , 0 ) (1,-1,2,0) ( 1 , − 1 , 2 , 0 ) ( 1 , − 1 , − 1 , 3 ) (1,-1,-1,3) ( 1 , − 1 , − 1 , 3 ) 2 , 6 , 12 2,6,12 2 , 6 , 12 Q T Q = I 3 Q^TQ=I_3 Q T Q = I 3 Q R = A QR=A QR = A
4. 作為線性映射的投影
對任意具有標準正交欄的矩陣 Q Q Q
P = Q Q T . P=QQ^T. P = Q Q T . 對 b ∈ R m b\in\mathbb R^m b ∈ R m p = P b = ∑ i ( q i T b ) q i p=Pb=\sum_i(q_i^Tb)q_i p = P b = ∑ i ( q i T b ) q i r = b − p r=b-p r = b − p Q T r = 0 Q^Tr=0 Q T r = 0 z z z p − z p-z p − z r r r
∥ b − z ∥ 2 = ∥ r + ( p − z ) ∥ 2 = ∥ r ∥ 2 + ∥ p − z ∥ 2 . \|b-z\|^2=\|r+(p-z)\|^2=\|r\|^2+\|p-z\|^2. ∥ b − z ∥ 2 = ∥ r + ( p − z ) ∥ 2 = ∥ r ∥ 2 + ∥ p − z ∥ 2 . 唯一的最近向量是 z = p z=p z = p z = p z=p z = p
該矩陣滿足兩個恆等式：
P T = P , P 2 = Q ( Q T Q ) Q T = P . P^T=P,\qquad P^2=Q(Q^TQ)Q^T=P. P T = P , P 2 = Q ( Q T Q ) Q T = P . 其像為 col Q \operatorname{col}Q col Q Q c Qc Q c ( col Q ) ⊥ (\operatorname{col}Q)^\perp ( col Q ) ⊥ P b = 0 Pb=0 P b = 0 Q T b = 0 Q^Tb=0 Q T b = 0 R m \mathbb R^m R m I − P I-P I − P
這些恆等式也刻畫正交投影。若實方陣滿足 P T = P P^T=P P T = P P 2 = P P^2=P P 2 = P P b Pb P b P y Py P y
( P y ) T ( b − P b ) = y T P ( I − P ) b = 0. (Py)^T(b-Pb)=y^TP(I-P)b=0. ( P y ) T ( b − P b ) = y T P ( I − P ) b = 0. 所以殘差垂直於像空間，上述距離恆等式適用。僅有冪等性並不充分：J = ( 1 1 0 0 ) J=\begin{pmatrix}1&1\\0&0\end{pmatrix} J = ( 1 0 1 0 ) J 2 = J J^2=J J 2 = J ( 0 , 1 ) T (0,1)^T ( 0 , 1 ) T ( 1 , 0 ) T (1,0)^T ( 1 , 0 ) T ( 0 , 1 ) T (0,1)^T ( 0 , 1 ) T
對 §3 的四維例子，n = ( 1 , − 1 , − 1 , − 1 ) T n=(1,-1,-1,-1)^T n = ( 1 , − 1 , − 1 , − 1 ) T A A A n T z = 0 n^Tz=0 n T z = 0
P = Q Q T = I 4 − n n T n T n = I 4 − 1 4 n n T . P=QQ^T=I_4-\frac{nn^T}{n^Tn}=I_4-\frac14nn^T. P = Q Q T = I 4 − n T n n n T = I 4 − 4 1 n n T . 這個等式成立，因為兩個映射都保留該超平面內的分量並去掉沿法向量的分量。也可直接相乘驗證。
5. 最小二乘：不預設唯一性的證明
給定 A ∈ R m × n A\in\mathbb R^{m\times n} A ∈ R m × n b ∈ R m b\in\mathbb R^m b ∈ R m 最小二乘係數向量 是在所有 x ∈ R n x\in\mathbb R^n x ∈ R n ∥ b − A x ∥ 2 \|b-Ax\|^2 ∥ b − A x ∥ 2 A x = b Ax=b A x = b
係數向量 x ^ \widehat x x r = b − A x ^ r=b-A\widehat x r = b − A x h h h x ^ + t h \widehat x+th x + t h
∥ r − t A h ∥ 2 = ∥ r ∥ 2 − 2 t r T A h + t 2 ∥ A h ∥ 2 . \|r-tAh\|^2=\|r\|^2-2t\,r^TAh+t^2\|Ah\|^2. ∥ r − t A h ∥ 2 = ∥ r ∥ 2 − 2 t r T A h + t 2 ∥ A h ∥ 2 . 若 r T A h ≠ 0 r^TAh\ne0 r T A h = 0 A h ≠ 0 Ah\ne0 A h = 0 t = ( r T A h ) / ∥ A h ∥ 2 t=(r^TAh)/\|Ah\|^2 t = ( r T A h ) /∥ A h ∥ 2 h h h r T A h = 0 r^TAh=0 r T A h = 0 h h h
∥ b − A ( x ^ + h ) ∥ 2 = ∥ r ∥ 2 + ∥ A h ∥ 2 ≥ ∥ r ∥ 2 . \|b-A(\widehat x+h)\|^2=\|r\|^2+\|Ah\|^2\ge\|r\|^2. ∥ b − A ( x + h ) ∥ 2 = ∥ r ∥ 2 + ∥ A h ∥ 2 ≥ ∥ r ∥ 2 . 用矩陣記號，垂直條件為
A T ( b − A x ^ ) = 0 , A T A x ^ = A T b . A^T(b-A\widehat x)=0,\qquad A^TA\widehat x=A^Tb. A T ( b − A x ) = 0 , A T A x = A T b . 這些稱為正規方程 。無論欄是否線性無關，它們都是充要條件。線性無關性是係數唯一所需的條件，而不是這一刻畫本身所需的條件。
存在性由投影得到。用考慮秩的過程構造欄空間的標準正交基 Q Q Q p = Q Q T b p=QQ^Tb p = Q Q T b x ^ \widehat x x A x ^ = p A\widehat x=p A x = p p p p x 0 x_0 x 0
x 0 + N ( A ) = { x 0 + h : A h = 0 } . x_0+N(A)=\{x_0+h:Ah=0\}. x 0 + N ( A ) = { x 0 + h : A h = 0 } . 事實上，加入零空間向量不會改變擬合結果；反過來，所有最小化係數都給出同一個唯一的最近擬合向量，所以它們的差位於零空間中。係數唯一當且僅當 N ( A ) = { 0 } N(A)=\{0\} N ( A ) = { 0 } ∥ b ∥ 2 \|b\|^2 ∥ b ∥ 2
6. 用三角方程組求最小二乘解
當欄線性無關時，使用 §2 的 A = Q R A=QR A = QR b b b Q Q T b + ( I − Q Q T ) b QQ^Tb+(I-QQ^T)b Q Q T b + ( I − Q Q T ) b Q Q Q Q T b − R x Q^Tb-Rx Q T b − R x
∥ b − A x ∥ 2 = ∥ ( I − Q Q T ) b ∥ 2 + ∥ Q T b − R x ∥ 2 . \|b-Ax\|^2
=\|(I-QQ^T)b\|^2+\|Q^Tb-Rx\|^2. ∥ b − A x ∥ 2 = ∥ ( I − Q Q T ) b ∥ 2 + ∥ Q T b − R x ∥ 2 . 對第二項使用了 ∥ Q z ∥ 2 = z T Q T Q z = ∥ z ∥ 2 \|Qz\|^2=z^TQ^TQz=\|z\|^2 ∥ Q z ∥ 2 = z T Q T Q z = ∥ z ∥ 2 x x x R R R
R x ^ = Q T b . R\widehat x=Q^Tb. R x = Q T b . 用回代求解這個上三角方程組。擬合向量是 Q Q T b QQ^Tb Q Q T b A = Q R A=QR A = QR R T R x ^ = R T Q T b R^TR\widehat x=R^TQ^Tb R T R x = R T Q T b R T R^T R T A T A A^TA A T A
例 2。 使用 §3 的矩陣並取 b = ( 2 , 0 , 1 , 3 ) T b=(2,0,1,3)^T b = ( 2 , 0 , 1 , 3 ) T
Q T b = ( 2 , 4 / 6 , 5 / 3 ) T . Q^Tb=(\sqrt2,4/\sqrt6,5/\sqrt3)^T. Q T b = ( 2 , 4/ 6 , 5/ 3 ) T . 最後一個三角方程給出 x ^ 3 = 5 / 2 \widehat x_3=5/2 x 3 = 5/2
6 2 x ^ 2 + 5 2 6 = 4 6 , \frac{\sqrt6}{2}\widehat x_2+\frac{5}{2\sqrt6}=\frac4{\sqrt6}, 2 6 x 2 + 2 6 5 = 6 4 , 故 x ^ 2 = 1 / 2 \widehat x_2=1/2 x 2 = 1/2 x ^ 1 = − 1 / 2 \widehat x_1=-1/2 x 1 = − 1/2
x ^ = ( − 1 / 2 , 1 / 2 , 5 / 2 ) T , p = A x ^ = ( 5 / 2 , − 1 / 2 , 1 / 2 , 5 / 2 ) T . \widehat x=(-1/2,1/2,5/2)^T,\quad
p=A\widehat x=(5/2,-1/2,1/2,5/2)^T. x = ( − 1/2 , 1/2 , 5/2 ) T , p = A x = ( 5/2 , − 1/2 , 1/2 , 5/2 ) T . 殘差為 r = ( − 1 / 2 , 1 / 2 , 1 / 2 , 1 / 2 ) T r=(-1/2,1/2,1/2,1/2)^T r = ( − 1/2 , 1/2 , 1/2 , 1/2 ) T ∥ r ∥ 2 = 1 \|r\|^2=1 ∥ r ∥ 2 = 1 n n n n T b = − 2 n^Tb=-2 n T b = − 2 r = n ( − 2 ) / 4 r=n(-2)/4 r = n ( − 2 ) /4
秩虧情況下的係數
例 3。 設
A = ( 1 2 0 1 2 1 0 0 1 ) , b = ( 1 , 0 , 2 ) T . A=\begin{pmatrix}1&2&0\\1&2&1\\0&0&1\end{pmatrix},\qquad b=(1,0,2)^T. A = 1 1 0 2 2 0 0 1 1 , b = ( 1 , 0 , 2 ) T . 第二欄是第一欄的兩倍，所以 Gram–Schmidt 跳過它。由第一欄與第三欄構造
q 1 = ( 1 , 1 , 0 ) T / 2 , q 2 = ( − 1 , 1 , 2 ) T / 6 . q_1=(1,1,0)^T/\sqrt2,\qquad q_2=(-1,1,2)^T/\sqrt6. q 1 = ( 1 , 1 , 0 ) T / 2 , q 2 = ( − 1 , 1 , 2 ) T / 6 . 這裏
B = Q T A = ( 2 2 2 1 / 2 0 0 6 / 2 ) , Q T b = ( 1 / 2 , 6 / 2 ) T . B=Q^TA=\begin{pmatrix}\sqrt2&2\sqrt2&1/\sqrt2\\0&0&\sqrt6/2\end{pmatrix},\qquad
Q^Tb=(1/\sqrt2,\sqrt6/2)^T. B = Q T A = ( 2 0 2 2 0 1/ 2 6 /2 ) , Q T b = ( 1/ 2 , 6 /2 ) T . 為了得到投影向量，求解 B x = Q T b Bx=Q^Tb B x = Q T b x 3 = 1 x_3=1 x 3 = 1 x 1 + 2 x 2 = 0 x_1+2x_2=0 x 1 + 2 x 2 = 0
x ^ = ( − 2 t , t , 1 ) T , t ∈ R . \widehat x=(-2t,t,1)^T,\qquad t\in\mathbb R. x = ( − 2 t , t , 1 ) T , t ∈ R . 它們都給出 p = ( 0 , 1 , 1 ) T p=(0,1,1)^T p = ( 0 , 1 , 1 ) T r = ( 1 , − 1 , 1 ) T r=(1,-1,1)^T r = ( 1 , − 1 , 1 ) T ( 1 , − 1 , 1 ) T (1,-1,1)^T ( 1 , − 1 , 1 ) T 5 t 2 + 1 \sqrt{5t^2+1} 5 t 2 + 1 t = 0 t=0 t = 0
7. 直線擬合中的中心化就是 Gram–Schmidt 正交化
對 m ≥ 1 m\ge1 m ≥ 1 ( t i , y i ) (t_i,y_i) ( t i , y i ) α + β t \alpha+\beta t α + β t 1 = ( 1 , … , 1 ) T \mathbf1=(1,\ldots,1)^T 1 = ( 1 , … , 1 ) T t = ( t 1 , … , t m ) T t=(t_1,\ldots,t_m)^T t = ( t 1 , … , t m ) T A = ( 1 t ) A=(\mathbf1\ t) A = ( 1 t )
t ‾ = 1 m ∑ i t i , y ‾ = 1 m ∑ i y i , S t t = ∑ i ( t i − t ‾ ) 2 . \overline t=\frac1m\sum_i t_i,\qquad \overline y=\frac1m\sum_i y_i,\qquad
S_{tt}=\sum_i(t_i-\overline t)^2. t = m 1 i ∑ t i , y = m 1 i ∑ y i , S tt = i ∑ ( t i − t ) 2 . Gram–Schmidt 從 q 1 = 1 / m q_1=\mathbf1/\sqrt m q 1 = 1 / m t t t 1 \mathbf1 1 w 2 = t − t ‾ 1 w_2=t-\overline t\mathbf1 w 2 = t − t 1 S t t > 0 S_{tt}>0 S tt > 0
q 2 = t − t ‾ 1 S t t , R = ( m m t ‾ 0 S t t ) . q_2=\frac{t-\overline t\mathbf1}{\sqrt{S_{tt}}},\qquad
R=\begin{pmatrix}\sqrt m&\sqrt m\,\overline t\\0&\sqrt{S_{tt}}\end{pmatrix}. q 2 = S tt t − t 1 , R = ( m 0 m t S tt ) . 在 R ( α , β ) T = Q T y R(\alpha,\beta)^T=Q^Ty R ( α , β ) T = Q T y
β ^ = ∑ i ( t i − t ‾ ) ( y i − y ‾ ) S t t , α ^ = y ‾ − β ^ t ‾ . \widehat\beta=\frac{\sum_i(t_i-\overline t)(y_i-\overline y)}{S_{tt}},\qquad
\widehat\alpha=\overline y-\widehat\beta\,\overline t. β = S tt ∑ i ( t i − t ) ( y i − y ) , α = y − β t . 分子中減去 y ‾ \overline y y ∑ i ( t i − t ‾ ) = 0 \sum_i(t_i-\overline t)=0 ∑ i ( t i − t ) = 0 t t t
若 S t t = 0 S_{tt}=0 S tt = 0 t i t_i t i c c c α \alpha α β \beta β y ‾ \overline y y α + c β = y ‾ \alpha+c\beta=\overline y α + c β = y S t t S_{tt} S tt
例 4。 對 ( t i , y i ) = ( − 1 , 0 ) , ( 0 , 1 ) , ( 1 , 1 ) , ( 2 , 4 ) (t_i,y_i)=(-1,0),(0,1),(1,1),(2,4) ( t i , y i ) = ( − 1 , 0 ) , ( 0 , 1 ) , ( 1 , 1 ) , ( 2 , 4 ) t ‾ = 1 / 2 \overline t=1/2 t = 1/2 y ‾ = 3 / 2 \overline y=3/2 y = 3/2 S t t = 5 S_{tt}=5 S tt = 5 β ^ = 6 / 5 \widehat\beta=6/5 β = 6/5 α ^ = 9 / 10 \widehat\alpha=9/10 α = 9/10 ( − 3 / 10 , 9 / 10 , 21 / 10 , 33 / 10 ) (-3/10,9/10,21/10,33/10) ( − 3/10 , 9/10 , 21/10 , 33/10 ) ( 3 / 10 , 1 / 10 , − 11 / 10 , 7 / 10 ) (3/10,1/10,-11/10,7/10) ( 3/10 , 1/10 , − 11/10 , 7/10 ) ( − 1 , 0 , 1 , 2 ) (-1,0,1,2) ( − 1 , 0 , 1 , 2 ) ( 9 + 1 + 121 + 49 ) / 100 = 9 / 5 (9+1+121+49)/100=9/5 ( 9 + 1 + 121 + 49 ) /100 = 9/5
等權平方誤差是此模型的一部分。對某個觀測誤差重新縮放或賦予權重會改變優化準則。推導這些精確的代數最小二乘公式不需要關於測量噪聲的概率假設。
8. 練習
對 a 1 = ( 1 , 0 , 1 ) T a_1=(1,0,1)^T a 1 = ( 1 , 0 , 1 ) T a 2 = ( 0 , 1 , 1 ) T a_2=(0,1,1)^T a 2 = ( 0 , 1 , 1 ) T Q T Q = I Q^TQ=I Q T Q = I Q R = A QR=A QR = A
按所列順序，對 ( 1 , 0 , 1 ) T , ( 2 , 0 , 2 ) T , ( 0 , 1 , 0 ) T (1,0,1)^T,(2,0,2)^T,(0,1,0)^T ( 1 , 0 , 1 ) T , ( 2 , 0 , 2 ) T , ( 0 , 1 , 0 ) T Q Q Q B = Q T A B=Q^TA B = Q T A
解釋為什麼 n > m n>m n > m m × n m\times n m × n n n n
求 b = ( 3 , − 1 , 2 ) T b=(3,-1,2)^T b = ( 3 , − 1 , 2 ) T x + y + z = 0 x+y+z=0 x + y + z = 0
證明 P = 1 3 ( 2 − 1 − 1 − 1 2 − 1 − 1 − 1 2 ) P=\frac13\begin{pmatrix}2&-1&-1\\-1&2&-1\\-1&-1&2\end{pmatrix} P = 3 1 2 − 1 − 1 − 1 2 − 1 − 1 − 1 2
對練習 1 的矩陣和 b = ( 1 , 2 , 0 ) T b=(1,2,0)^T b = ( 1 , 2 , 0 ) T
對 A = ( 1 2 2 4 0 0 ) A=\begin{pmatrix}1&2\\2&4\\0&0\end{pmatrix} A = 1 2 0 2 4 0 b = ( 2 , 0 , 1 ) T b=(2,0,1)^T b = ( 2 , 0 , 1 ) T
用中心化和對 ( 0 , 0 ) , ( 1 , 1 ) , ( 2 , 1 ) , ( 3 , 3 ) (0,0),(1,1),(2,1),(3,3) ( 0 , 0 ) , ( 1 , 1 ) , ( 2 , 1 ) , ( 3 , 3 )
三個觀測輸入全為三，輸出為 1 , 4 , 7 1,4,7 1 , 4 , 7
對 J = ( 1 1 0 0 ) J=\begin{pmatrix}1&1\\0&0\end{pmatrix} J = ( 1 0 1 0 ) J 2 = J J^2=J J 2 = J J b Jb J b im J \operatorname{im}J im J
假設 Q Q Q Q ~ \widetilde Q Q Q Q T = Q ~ Q ~ T QQ^T=\widetilde Q\widetilde Q^T Q Q T = Q Q T
設 A A A 3 × 2 3\times2 3 × 2 b = ( 1 , − 2 , 2 ) T b=(1,-2,2)^T b = ( 1 , − 2 , 2 ) T
9. 完整解答
1. 第一個向量的範數為 2 \sqrt2 2 q 1 = ( 1 , 0 , 1 ) T / 2 q_1=(1,0,1)^T/\sqrt2 q 1 = ( 1 , 0 , 1 ) T / 2 a 2 a_2 a 2 1 / 2 1/\sqrt2 1/ 2
w 2 = ( 0 , 1 , 1 ) T − 1 2 ( 1 , 0 , 1 ) T = 1 2 ( − 1 , 2 , 1 ) T . w_2=(0,1,1)^T-\tfrac12(1,0,1)^T=\tfrac12(-1,2,1)^T. w 2 = ( 0 , 1 , 1 ) T − 2 1 ( 1 , 0 , 1 ) T = 2 1 ( − 1 , 2 , 1 ) T . 所以 q 2 = ( − 1 , 2 , 1 ) T / 6 q_2=(-1,2,1)^T/\sqrt6 q 2 = ( − 1 , 2 , 1 ) T / 6
R = ( 2 1 / 2 0 6 / 2 ) . R=\begin{pmatrix}\sqrt2&1/\sqrt2\\0&\sqrt6/2\end{pmatrix}. R = ( 2 0 1/ 2 6 /2 ) . 未歸一化的兩個方向內積為 − 1 + 1 = 0 -1+1=0 − 1 + 1 = 0 Q T Q = I 2 Q^TQ=I_2 Q T Q = I 2 2 q 1 = a 1 \sqrt2q_1=a_1 2 q 1 = a 1 ( 1 / 2 ) q 1 + ( 6 / 2 ) q 2 = ( 1 / 2 , 0 , 1 / 2 ) T + ( − 1 / 2 , 1 , 1 / 2 ) T = a 2 (1/\sqrt2)q_1+(\sqrt6/2)q_2=(1/2,0,1/2)^T+(-1/2,1,1/2)^T=a_2 ( 1/ 2 ) q 1 + ( 6 /2 ) q 2 = ( 1/2 , 0 , 1/2 ) T + ( − 1/2 , 1 , 1/2 ) T = a 2 Q R = A QR=A QR = A
2. 從 q 1 = ( 1 , 0 , 1 ) T / 2 q_1=(1,0,1)^T/\sqrt2 q 1 = ( 1 , 0 , 1 ) T / 2 q 1 q_1 q 1 2 2 2\sqrt2 2 2 q 1 q_1 q 1 q 2 = e 2 q_2=e_2 q 2 = e 2
Q = ( 1 / 2 0 0 1 1 / 2 0 ) , B = ( 2 2 2 0 0 0 1 ) . Q=\begin{pmatrix}1/\sqrt2&0\\0&1\\1/\sqrt2&0\end{pmatrix},\qquad
B=\begin{pmatrix}\sqrt2&2\sqrt2&0\\0&0&1\end{pmatrix}. Q = 1/ 2 0 1/ 2 0 1 0 , B = ( 2 0 2 2 0 0 1 ) . 直接相乘得到原來的三欄。秩為二；跳過某欄會把它的信息保存在 B B B
3. 標準正交向量線性無關：對一個線性關係分別與每個向量取內積，就得到各系數為零。R m \mathbb R^m R m m m m n > m n>m n > m n n n rank A ≤ m \operatorname{rank}A\le m rank A ≤ m
4. 法向量為 n = ( 1 , 1 , 1 ) T n=(1,1,1)^T n = ( 1 , 1 , 1 ) T n T b = 4 n^Tb=4 n T b = 4 n T n = 3 n^Tn=3 n T n = 3 r = ( 4 / 3 ) n r=(4/3)n r = ( 4/3 ) n
p = b − r = ( 5 / 3 , − 7 / 3 , 2 / 3 ) T . p=b-r=(5/3,-7/3,2/3)^T. p = b − r = ( 5/3 , − 7/3 , 2/3 ) T . 它的座標和為零，故位於平面內。殘差平行於法向量，因而垂直於平面，證明這是正確的投影。距離為 ∥ r ∥ = 4 / 3 \|r\|=4/\sqrt3 ∥ r ∥ = 4/ 3
5. 令 n = ( 1 , 1 , 1 ) T n=(1,1,1)^T n = ( 1 , 1 , 1 ) T P = I − n n T / 3 P=I-nn^T/3 P = I − n n T /3 N = n n T / 3 N=nn^T/3 N = n n T /3 N 2 = n ( n T n ) n T / 9 = N N^2=n(n^Tn)n^T/9=N N 2 = n ( n T n ) n T /9 = N P T = P P^T=P P T = P P 2 = I − 2 N + N 2 = P P^2=I-2N+N^2=P P 2 = I − 2 N + N 2 = P x + y + z = 0 x+y+z=0 x + y + z = 0 span { n } \operatorname{span}\{n\} span { n }
6. 由練習 1，
Q T b = ( 1 / 2 , 3 / 6 ) T . Q^Tb=(1/\sqrt2,3/\sqrt6)^T. Q T b = ( 1/ 2 , 3/ 6 ) T . 最後一個三角方程 ( 6 / 2 ) x 2 = 3 / 6 (\sqrt6/2)x_2=3/\sqrt6 ( 6 /2 ) x 2 = 3/ 6 x 2 = 1 x_2=1 x 2 = 1 2 x 1 + 1 / 2 = 1 / 2 \sqrt2x_1+1/\sqrt2=1/\sqrt2 2 x 1 + 1/ 2 = 1/ 2 x 1 = 0 x_1=0 x 1 = 0 ( 0 , 1 , 1 ) T (0,1,1)^T ( 0 , 1 , 1 ) T ( 1 , 1 , − 1 ) T (1,1,-1)^T ( 1 , 1 , − 1 ) T ( 1 , 0 , 1 ) T (1,0,1)^T ( 1 , 0 , 1 ) T ( 0 , 1 , 1 ) T (0,1,1)^T ( 0 , 1 , 1 ) T
7. 每個擬合向量都形如 s ( 1 , 2 , 0 ) T s(1,2,0)^T s ( 1 , 2 , 0 ) T s = x 1 + 2 x 2 s=x_1+2x_2 s = x 1 + 2 x 2
s = ( 1 , 2 , 0 ) T b 1 2 + 2 2 = 2 5 . s=\frac{(1,2,0)^Tb}{1^2+2^2}=\frac25. s = 1 2 + 2 2 ( 1 , 2 , 0 ) T b = 5 2 . 因此所有係數為 ( 2 / 5 − 2 t , t ) T (2/5-2t,t)^T ( 2/5 − 2 t , t ) T ( 2 / 5 , 4 / 5 , 0 ) T (2/5,4/5,0)^T ( 2/5 , 4/5 , 0 ) T ( 8 / 5 , − 4 / 5 , 1 ) T (8/5,-4/5,1)^T ( 8/5 , − 4/5 , 1 ) T 64 / 25 + 16 / 25 + 1 = 21 / 5 64/25+16/25+1=21/5 64/25 + 16/25 + 1 = 21/5
∥ ( 2 / 5 − 2 t , t ) T ∥ 2 = 5 ( t − 4 / 25 ) 2 + 4 / 125. \|(2/5-2t,t)^T\|^2=5(t-4/25)^2+4/125. ∥ ( 2/5 − 2 t , t ) T ∥ 2 = 5 ( t − 4/25 ) 2 + 4/125. 唯一的最小範數選擇是 t = 4 / 25 t=4/25 t = 4/25 ( x 1 , x 2 ) = ( 2 / 25 , 4 / 25 ) (x_1,x_2)=(2/25,4/25) ( x 1 , x 2 ) = ( 2/25 , 4/25 )
8. 這裏 t ‾ = 3 / 2 \overline t=3/2 t = 3/2 y ‾ = 5 / 4 \overline y=5/4 y = 5/4 S t t = 9 / 4 + 1 / 4 + 1 / 4 + 9 / 4 = 5 S_{tt}=9/4+1/4+1/4+9/4=5 S tt = 9/4 + 1/4 + 1/4 + 9/4 = 5 ∑ i ( t i − t ‾ ) y i = 0 − 1 / 2 + 1 / 2 + 9 / 2 = 9 / 2 \sum_i(t_i-\overline t)y_i=0-1/2+1/2+9/2=9/2 ∑ i ( t i − t ) y i = 0 − 1/2 + 1/2 + 9/2 = 9/2 β ^ = 9 / 10 \widehat\beta=9/10 β = 9/10 α ^ = 5 / 4 − ( 9 / 10 ) ( 3 / 2 ) = − 1 / 10 \widehat\alpha=5/4-(9/10)(3/2)=-1/10 α = 5/4 − ( 9/10 ) ( 3/2 ) = − 1/10 ( − 1 / 10 , 8 / 10 , 17 / 10 , 26 / 10 ) (-1/10,8/10,17/10,26/10) ( − 1/10 , 8/10 , 17/10 , 26/10 ) ( 1 / 10 , 2 / 10 , − 7 / 10 , 4 / 10 ) (1/10,2/10,-7/10,4/10) ( 1/10 , 2/10 , − 7/10 , 4/10 ) ( 0 , 1 , 2 , 3 ) (0,1,2,3) ( 0 , 1 , 2 , 3 ) ( 2 − 14 + 12 ) / 10 = 0 (2-14+12)/10=0 ( 2 − 14 + 12 ) /10 = 0 ( 1 + 4 + 49 + 16 ) / 100 = 7 / 10 (1+4+49+16)/100=7/10 ( 1 + 4 + 49 + 16 ) /100 = 7/10
9. 該模型對每個觀測預測相同值 s = α + 3 β s=\alpha+3\beta s = α + 3 β
( 1 − s ) 2 + ( 4 − s ) 2 + ( 7 − s ) 2 = 18 + 3 ( s − 4 ) 2 . (1-s)^2+(4-s)^2+(7-s)^2=18+3(s-4)^2. ( 1 − s ) 2 + ( 4 − s ) 2 + ( 7 − s ) 2 = 18 + 3 ( s − 4 ) 2 . 它恰在 s = 4 s=4 s = 4 ( α , β ) = ( 4 − 3 t , t ) (\alpha,\beta)=(4-3t,t) ( α , β ) = ( 4 − 3 t , t ) ( 4 , 4 , 4 ) (4,4,4) ( 4 , 4 , 4 ) S t t = 0 S_{tt}=0 S tt = 0
10. 相乘得 J 2 = ( 1 1 0 0 ) = J J^2=\begin{pmatrix}1&1\\0&0\end{pmatrix}=J J 2 = ( 1 0 1 0 ) = J b = ( 0 , 1 ) T b=(0,1)^T b = ( 0 , 1 ) T J b = ( 1 , 0 ) T Jb=(1,0)^T J b = ( 1 , 0 ) T b b b J T = ( 1 0 1 0 ) ≠ J J^T=\begin{pmatrix}1&0\\1&0\end{pmatrix}\ne J J T = ( 1 1 0 0 ) = J
11. 對任意 b b b p = Q Q T b p=QQ^Tb p = Q Q T b p ~ = Q ~ Q ~ T b \widetilde p=\widetilde Q\widetilde Q^Tb p = Q Q T b p − p ~ p-\widetilde p p − p p = p ~ p=\widetilde p p = p b b b
12. 每個 x ∈ R 2 x\in\mathbb R^2 x ∈ R 2 A x = 0 Ax=0 A x = 0 0 2 × 2 x = 0 2 × 1 0_{2\times2}x=0_{2\times1} 0 2 × 2 x = 0 2 × 1 b b b 1 + 4 + 4 = 9 1+4+4=9 1 + 4 + 4 = 9
總結
Gram–Schmidt 方法逐欄保持張成空間，並通過零殘差揭示精確線性相關。線性無關的欄給出帶可逆三角因子的薄 QR 分解。標準正交座標產生正交投影，其對稱和冪等恆等式表達了投影的幾何性質。最小二乘始終選出唯一的擬合向量，而只有欄映射為單射時係數才唯一。直線擬合的中心化就是同一構造的一個具體例子。
