每週問題 September 5, 2016

推導 Tikhonov 正則化 (regularization) 的最佳解。

Let A be an m\times n real matrix and \mathbf{b}\in\mathbb{R}^m. Solve the Tikhonov regularization problem:

\displaystyle \min_{\mathbf{x}}\Vert A\mathbf{x}-\mathbf{b}\Vert_2^2+\lambda\Vert \mathbf{x}\Vert_2^2,

where \lambda>0.

 
參考解答:

寫出目標函數

\begin{aligned} J(\mathbf{x})&=\Vert A\mathbf{x}-\mathbf{b}\Vert_2^2+\lambda\Vert \mathbf{x}\Vert_2^2\\ &=(A\mathbf{x}-\mathbf{b})^T(A\mathbf{x}-\mathbf{b})+\lambda\mathbf{x}^T\mathbf{x}\\ &=\mathbf{x}^TA^TA\mathbf{x}-2\mathbf{x}^TA^T\mathbf{b}+\mathbf{b}^T\mathbf{b}+\lambda\mathbf{x}^T\mathbf{x}. \end{aligned}

目標函數 J\mathbf{x} 求導,如下:

\displaystyle \frac{\partial J}{\partial \mathbf{x}}=2A^TA\mathbf{x}-2A^T\mathbf{b}+2\lambda\mathbf{x}

設上式等於零向量,可得優化條件式:

\displaystyle (A^TA+\lambda I)\mathbf{x}=A^T\mathbf{b}

\lambda>0A^TA+\lambda I 是一個正定矩陣 (即可逆),因此最佳解為

\displaystyle \mathbf{x}^\ast=(A^TA+\lambda I)^{-1}A^T\mathbf{b}

Advertisements
This entry was posted in pow 內積空間, 每週問題 and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s