Motivation:

We wish to choose x to balance the two objective values

$f_1(x)=\lVert Ax-b\rVert^2$ and $f_2(x)=\lVert Cx-d\rVert^2$.

Generally, we can make $f_1(x)$ or $f_2(x)$ small, but not both.

b is observation (noisy)

Naive Least-Sq formation:

true signal $x_0$ recovered via $\underset{x}{\min} \lVert x-b\rVert^2$

$b=x_0+noise$ ⇒ $x_0-b=noise$

Need to incorporate “side” information

“side” ↔ ”prior” ⇒ signal $x_0$ is smooth

Combining two models

$\underset{x}{\min}f_1(x)+f_2(x)$

Balance competing objectives $f_1$ and $f_2$

1. $\underset{x}{\min}f_1(x)+\lambda f_2(x)$, $\lambda ≥ 0$

   $\lambda$: “regularization” parameter

   $f_2(x)$: regularization function

2. $\underset{x}{\min}\{f_1(x)\mid f_2(x)≤\tau\}$, $\tau ≥ 0$ $\min\{f_1(x)\mid \forall x \text{ st. } f_2(x) \le \tau$

   $\min\{f_1(x)\mid x \in lev_\tau f_2\}$

3. $\min\{f_2(x)\mid f_1(x)≤ \Delta\}$, $\Delta≥ 0$