每週問題 September 5, 2016

推導 Tikhonov 正則化 (regularization) 的最佳解。

Let A be an m\times n real matrix and \mathbf{b}\in\mathbb{R}^m. Solve the Tikhonov regularization problem:

\displaystyle  \min_{\mathbf{x}}\Vert A\mathbf{x}-\mathbf{b}\Vert_2^2+\lambda\Vert \mathbf{x}\Vert_2^2,

where \lambda>0.

 
參考解答:

寫出目標函數

\begin{aligned}  J(\mathbf{x})&=\Vert A\mathbf{x}-\mathbf{b}\Vert_2^2+\lambda\Vert \mathbf{x}\Vert_2^2\\  &=(A\mathbf{x}-\mathbf{b})^T(A\mathbf{x}-\mathbf{b})+\lambda\mathbf{x}^T\mathbf{x}\\  &=\mathbf{x}^TA^TA\mathbf{x}-2\mathbf{x}^TA^T\mathbf{b}+\mathbf{b}^T\mathbf{b}+\lambda\mathbf{x}^T\mathbf{x}.  \end{aligned}

目標函數 J\mathbf{x} 求導,如下:

\displaystyle  \frac{\partial J}{\partial \mathbf{x}}=2A^TA\mathbf{x}-2A^T\mathbf{b}+2\lambda\mathbf{x}

設上式等於零向量,可得優化條件式:

\displaystyle  (A^TA+\lambda I)\mathbf{x}=A^T\mathbf{b}

\lambda>0A^TA+\lambda I 是一個正定矩陣 (即可逆),因此最佳解為

\displaystyle  \mathbf{x}^\ast=(A^TA+\lambda I)^{-1}A^T\mathbf{b}

This entry was posted in pow 內積空間, 每週問題 and tagged . Bookmark the permalink.

發表迴響

在下方填入你的資料或按右方圖示以社群網站登入:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / 變更 )

Twitter picture

You are commenting using your Twitter account. Log Out / 變更 )

Facebook照片

You are commenting using your Facebook account. Log Out / 變更 )

Google+ photo

You are commenting using your Google+ account. Log Out / 變更 )

連結到 %s