$f: \mathbb{R}^n \to \mathbb{R}$

Assume $i^{th}$-partial derivative exists:

$\frac{\delta f(x)}{\delta x_i} = \underset{\varepsilon→0}{lim}\frac{f(x+\varepsilon e_i)-f(x)}{\varepsilon}$

$e_i=\begin{bmatrix} 0 \\ \vdots \\ 1 \\ 0\\ \vdots \\ 0\end{bmatrix}$i$^{th}$ position

The gradient of $f$ is the collection of partials:

$$ \nabla f(x)= \begin{bmatrix}\delta f(x)/\delta x_1\\ \delta f(x)/\delta x_2 \\ \vdots\\ \delta f(x)/\delta x_n\end{bmatrix} \in \mathbb{R}^n $$

Directional derivative of $f$ at $x\in \mathbb{R}^n$

in the direction $d \in \mathbb{R}^n$

$f’(x,d)=\underset{\varepsilon → 0}{lim}\frac{f(x+\varepsilon d)-f(x)}{\varepsilon}$

The function $f$ is differentiable of $x \in \mathbb{R}^n$ if $f’(x,d)$ exists for all of $\mathbb{R}^n$, and $f’(x,d)=\nabla f(x)^Td$

Fact: the direction derivative is positively homogeneous of degree 1, ie,

$f’(x,\alpha d)=\alpha f’(x,d)$ $\forall \alpha≥0$

Special Directions

$d=\begin{bmatrix}d_1 \\ d_2 \\ \vdots \\ d_n \end{bmatrix}$→ $d=e_i$

$f’(x,e_i)=\nabla f(x)^Te_i=[\nabla f(x)]_i=\frac{\delta f(x)}{\delta x_i}$

Calculus Rules for $\mathbb{R}^n$

$f:\mathbb{R}^n→\mathbb{R}$

Linear function: $f(x)=a^Tx=\sum^{n}_{i=1}a_ix_i$

$\nabla f(x)=\begin{bmatrix}a_1\\a_2\\ \vdots\\ a_n \end{bmatrix}=a$