$\underset{x \in \R^n}{\lim} f(x)$ subject to $\underbrace{Ax=b}_{\text{undetermined system}}$

• effectively an “unconstrained problem”
• geometric introduction Lagrange Multipliers
• Connection to QR decomposition

Matrices $A=m \times n$, $b=m \times 1$

• $m$ constants; $n$ variables
• $rank(A)=m$ (full-rank)

  • $f:\R^n \to \R$ continuous differentiable once or twice

  • Feasible Set

    • $F=\{x \mid Ax=b\}=\phi \Leftrightarrow b\in Range(A)$

    

Equivalent representation of feasible set

$F=\{x \mid Ax=b\}=\{\bar{x}+Zp \mid p \in \R^{n-m}\}$,

$\begin{aligned}\text{where } & \bar{x}\text{ is any “particular” solution for }A\bar{x}=b,\\ & Z \text{ is any basis for }Null(A) (Z \text{ is }n\times (n-m) \text{ s.t. }AZ=0),\\& p\text{ along the null basis}\end{aligned}$

Equivalent Reduced Problem

$\underset{p \in \R^{n-m}}{\lim} f(\bar{x}+Zp) \equiv \underset{x \in \R^{n}}{\lim}\{f(x)\mid Ax=b\}$

• $\bar{x}+Zp^* \equiv x^*$

Example:

$$ \lim \frac{1}{2}(x_1^2+x_2^2) \text{ s.t. } x_1+x_2=1 $$