Differentiation Properties

$\newenvironment {prompt}{}{} \newcommand {\DeclareMathOperator }[2]{\@OldDeclareMathOperator {##1}{##2}\immediate \write \myfile {\unexpanded {\DeclareMathOperator }{\unexpanded {##1}}{\unexpanded {##2}}}} \newcommand {\ungraded }[0]{} \newcommand {\reserved@a }[2]{} \newcommand {\HyperFirstAtBeginDocument }[0]{\AtBeginDocument } \newcommand {\reserved@a }[1]{} \newcommand {\reserved@a }[2]{} \newcommand {\vnameref }[1]{\unskip ~\nameref {##1}\@vpageref [\unskip ]{##1}} \newcommand {\ref }[0]{\@ifstar \@refstar \T@ref } \newcommand {\pageref }[0]{\@ifstar \@pagerefstar \T@pageref } \newcommand {\nameref }[0]{\@ifstar \@namerefstar \T@nameref } \newcommand {\reserved@a }[2]{} \newcommand {\reserved@a }[0]{\AtBeginDocument } \newcommand {\reserved@a }[1]{} \newcommand {\reserved@a }[2]{}$

In single variable calculus, you learned many differentiate rules and properties in order to more easily compute derivatives without the limit definition. These rules included the product rule, the quotient rule, the chain rule, and more, along with memorizing the derivatives of common functions.

Since partial derivatives obey the same rules as single variable derivatives, and the derivative matrix and gradient are comprised of partial derivatives, many of the rules listed above have analogous results in multivariable calculus. We’ll now start to explore some differentiation results for multivariable functions.

Linearity of the derivative

As you may recall from single variable calculus, “the derivative of the sum is the sum of the derivatives.” That is, if we have differentiable functions $f(x)$ and $g(x)$ , we compute the derivative of the sum $f(x)+g(x)$ by taking the sum of the derivatives $f'(x)$ and $g'(x)$ . For example,

$\begin{align*} \frac{d}{dx}\left(\sin(x)+x^2\right) &= \frac{d}{dx}\left(\sin(x)\right) + \frac{d}{dx}\left(x^2\right)\\ &= \cos(x)+2x. \end{align*}$

An analogous result holds in multivariable calculus, for the derivative matrix.

Suppose $\vec {f}:X\subset \mathbb {R}^n\rightarrow \mathbb {R}^m$ and $\vec {g}:Y\subset \mathbb {R}^n\rightarrow \mathbb {R}^m$ are differentiable functions on subsets $X$ and $Y$ of $\mathbb {R}^n$ , respectively. Then $\vec {f}+\vec {g}$ is differentiable on $X\cap Y$ , and $D(\vec {f}+\vec {g})=D\vec {f}+D\vec {g}$ on $X\cap Y$ .

Proof

We will begin by showing that $D(\vec {f}+\vec {g}) = D\vec {f}+D\vec {g}$ on $X\cap Y$ . Since $X\cap Y$ is contained in the domains of both $\vec {f}$ and $\vec {g}$ , $D\vec {f}$ and $D\vec {g}$ both exist on $X\cap Y$ .

Write $\vec {f}(\vec {x}) = (f_1(\vec {x}),...,f_m(\vec {x}))$ and $\vec {g}(\vec {x}) = (g_1(\vec {x}),...,g_m(\vec {x}))$ in terms of their component functions. Then, we have

$\begin{align*} (\vec{f}+\vec{g})(\vec{x}) &= \vec{f}(\vec{x})+\vec{g}(\vec{x})\\ &= (f_1(\vec{x}),...,f_m(\vec{x})) + (g_1(\vec{x}),...,g_m(\vec{x}))\\ &= (f_1(\vec{x})+g_1(\vec{x}),...,f_m(\vec{x})+g_m(\vec{x}))\\ &= ((f_1+g_1)(\vec{x}),...,(f_m+g_m)(\vec{x})), \end{align*}$

giving us the component functions of $\vec {f}+\vec {g}$ .

Using the component functions found above, we take the derivative matrix of $\vec {f}+\vec {g}$ , using the linearity of partial derivatives.

$\begin{align*} D(\vec{f}+\vec{g}) &=\left(\begin{array}{cccc} \frac{\partial (f_1+g_1)}{\partial x_1} & \frac{\partial (f_1+g_1)}{\partial x_2} & \cdots &\frac{\partial (f_1+g_1)}{\partial x_n}\\ \frac{\partial (f_2+g_2)}{\partial x_1} & \frac{\partial (f_2+g_2)}{\partial x_2} & \cdots &\frac{\partial (f_2+g_2)}{\partial x_n}\\ \vdots & \vdots && \vdots\\ \frac{\partial (f_m+g_m)}{\partial x_1} & \frac{\partial (f_m+g_m)}{\partial x_2} & \cdots &\frac{\partial (f_m+g_m)}{\partial x_n}\\ \end{array}\right)\\ &=\left(\begin{array}{cccc} \frac{\partial f_1}{\partial x_1}+\frac{\partial g_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2}+\frac{\partial g_1}{\partial x_2} & \cdots &\frac{\partial f_1}{\partial x_n}+\frac{\partial g_1}{\partial x_n}\\ \frac{\partial f_2}{\partial x_1}+\frac{\partial g_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2}+\frac{\partial g_2}{\partial x_2} & \cdots &\frac{\partial f_2}{\partial x_n}+\frac{\partial g_2}{\partial x_n}\\ \vdots & \vdots && \vdots\\ \frac{\partial f_m}{\partial x_1}+\frac{\partial g_m}{\partial x_1} & \frac{\partial f_m}{\partial x_2}+\frac{\partial g_m}{\partial x_2} & \cdots &\frac{\partial f_m}{\partial x_n}+\frac{\partial g_m}{\partial x_n}\\ \end{array}\right)\\ &=\left(\begin{array}{cccc} \frac{\partial f_1}{\partial x_1} & \frac{\partial f_1}{\partial x_2} & \cdots &\frac{\partial f_1}{\partial x_n}\\ \frac{\partial f_2}{\partial x_1} & \frac{\partial f_2}{\partial x_2} & \cdots &\frac{\partial f_2}{\partial x_n}\\ \vdots & \vdots && \vdots\\ \frac{\partial f_m}{\partial x_1} & \frac{\partial f_m}{\partial x_2} & \cdots &\frac{\partial f_m}{\partial x_n}\\ \end{array}\right) +\left(\begin{array}{cccc} \frac{\partial g_1}{\partial x_1} & \frac{\partial g_1}{\partial x_2} & \cdots &\frac{\partial g_1}{\partial x_n}\\ \frac{\partial g_2}{\partial x_1} & \frac{\partial g_2}{\partial x_2} & \cdots &\frac{\partial g_2}{\partial x_n}\\ \vdots & \vdots && \vdots\\ \frac{\partial g_m}{\partial x_1} & \frac{\partial g_m}{\partial x_2} & \cdots &\frac{\partial g_m}{\partial x_n}\\ \end{array}\right)\\ &= D\vec{f}+D\vec{g}. \end{align*}$

Thus, we have $D(\vec {f}+\vec {g}) = D\vec {f}+D\vec {g}$ on $X\cap Y$ .

Next, we need to show that $\vec {f}+\vec {g}$ is differentiable on $X\cap Y$ . In order to show this, we will show that the following limit evaluates to $0$ , for $\vec {a}\in X\cap Y$ . This is done by separating the $\vec {f}$ and $\vec {g}$ terms, using the triangle inequality, and using the fact that $\vec {f}$ and $\vec {g}$ are both differentiable on $X\cap Y$ .

$\begin{align*} \lim_{\vec{x}\rightarrow\vec{a}}&\frac{\|(\vec{f}+\vec{g})(\vec{x}) - ((\vec{f}+\vec{g})(\vec{a})+D(\vec{f}+\vec{g})(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &= \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x})+\vec{g}(\vec{x}) - \vec{f}(\vec{a})-\vec{g}(\vec{a})-(D\vec{f}(\vec{a})+D\vec{g}(\vec{a}))(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &= \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x})+\vec{g}(\vec{x}) - \vec{f}(\vec{a})-\vec{g}(\vec{a})-D\vec{f}(\vec{a})(\vec{x}-\vec{a})-D\vec{g}(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &= \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x}) - \vec{f}(\vec{a})-D\vec{f}(\vec{a})(\vec{x}-\vec{a})+\vec{g}(\vec{x}) - \vec{g}(\vec{a})-D\vec{g}(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &\leq \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x}) - \vec{f}(\vec{a})-D\vec{f}(\vec{a})(\vec{x}-\vec{a})\|+\|\vec{g}(\vec{x}) - \vec{g}(\vec{a})-D\vec{g}(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &\leq \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x}) - \vec{f}(\vec{a})-D\vec{f}(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}+\lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{g}(\vec{x}) - \vec{g}(\vec{a})-D\vec{g}(\vec{a})(\vec{x}-\vec{a})\|}{\|\vec{x}-\vec{a}\|}\\ &= \lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{f}(\vec{x}) - (\vec{f}(\vec{a})+D\vec{f}(\vec{a})(\vec{x}-\vec{a}))\|}{\|\vec{x}-\vec{a}\|}+\lim_{\vec{x}\rightarrow\vec{a}}\frac{\|\vec{g}(\vec{x}) - (\vec{g}(\vec{a})+D\vec{g}(\vec{a})(\vec{x}-\vec{a}))\|}{\|\vec{x}-\vec{a}\|}\\ &= 0 + 0\\ &= 0 \end{align*}$ $\blacksquare$

You may also recall the constant multiple rule from single variable calculus. For example,

$\begin{align*} \frac{d}{dx}(4x^2) &= 4\left(\frac{d}{dx}(x^2)\right)\\ &= 4(2x)\\ &=8x. \end{align*}$

We have an analogous result in multivariable calculus.

Suppose $\vec {f}:X\subset \mathbb {R}^n\rightarrow \mathbb {R}^m$ is a differentiable function on a subset $X$ of $\mathbb {R}^n$ , and let $k$ be any constant. Then $k\vec {f}$ is also differentiable on $X$ , and $D(k\vec {f})=kD\vec {f}$ on $X$ .

The proof of this proposition is somewhat similar to the previous theorem, and it is left as an exercise.

Although these are important results, they actually aren’t particularly useful for computing derivative matrices. In practice, you’d compute the derivative matrix of a sum of functions by first adding the components, then differentiating. We include an example to demonstrate this.

Consider the functions $\vec {f},\vec {g}:\mathbb {R}^2\rightarrow \mathbb {R}^2$ given by $\vec {f}(x,y) = (x^2, xy)$ and $\vec {g}(x,y) = (xy^2, y)$ .

We will compute the derivative matrix $D(\vec {f}+\vec {g})$ in two ways: first directly, and then by using the sum rule proved above.

The function $\vec {f}+\vec {g}$ is given by $(\vec {f}+\vec {g})(x,y) = \answer {(x^2+xy^2, xy+y)}.$ We can then compute the derivative matrix $D(\vec {f}+\vec {g})$ directly as $D(\vec {f}+\vec {g})=\left (\begin {array}{cc} \answer {2x+y^2} & \answer {2xy}\\\answer {y} & \answer {x+1}\end {array}\right ).$

We will now use our multivariable sum rule. We have $D\vec {f}(x,y) = \left (\begin {array}{cc} \answer {2x} & \answer {0}\\\answer {y} & \answer {x}\end {array}\right ),$ and $D\vec {g}(x,y) = \left (\begin {array}{cc} \answer {y^2} & \answer {2xy}\\\answer {0} & \answer {1}\end {array}\right ).$ Adding these matrices, we then have that

$\begin{align*} D(\vec{f}+\vec{g})(x,y) &= D\vec{f}(x,y)+D\vec{g}(x,y)\\ &= \left(\begin{array}{cc} \answer{2x+y^2} & \answer{2xy}\\\answer{y} & \answer{x+1}\end{array}\right). \end{align*}$

We get the correct answer using either method, but using the sum rule doesn’t seem to provide much (if any) of a computational advantage over computing the derivative matrix directly. Nonetheless, it is mathematically important that the derivative seems to “distribute” over addition.

Product and Quotient laws

We will now try to find multi-variable analogs to the product and quotient rules from single variable calculus. Let’s start by considering when these rules might make sense.

Suppose we have functions $\vec {f}(x,y) = (x^2,xy)$ and $\vec {g}(x,y) = (xy^2, y)$ . If we wanted to define the product of these functions, what would that mean? The outputs of $\vec {f}$ and $\vec {g}$ are vectors, so we’d be trying to multiply two vectors - but there isn’t really a clear multiplication on vectors! We could try multiplying component-wise, taking the dot product, or taking the cross product, and in different settings, these all might be reasonable things to do. We could work on finding product rules for all of the different ways we could “multiply” two vectors (these can be found in the exercises), but we’ll save some time, and focus on a case where we do have one clear choice for multiplication: scalar-valued functions.

Suppose $\vec {f}:X\subset \mathbb {R}^n\rightarrow \mathbb {R}$ and $\vec {g}:Y\subset \mathbb {R}^n\rightarrow \mathbb {R}$ are differentiable scalar-valued functions on subsets $X$ and $Y$ of $\mathbb {R}^n$ , respectively. Then $fg$ is differentiable on $X\cap Y$ , and $D(fg)(\vec {a})=g(\vec {a})Df(\vec {a})+f(\vec {a})Dg(\vec {a})$ for $\vec {a}$ in $X\cap Y$ .

Similarly, we have a multivariable quotient rule for scalar valued functions.

Suppose $\vec {f}:X\subset \mathbb {R}^n\rightarrow \mathbb {R}$ and $\vec {g}:Y\subset \mathbb {R}^n\rightarrow \mathbb {R}$ are differentiable scalar-valued functions on subsets $X$ and $Y$ of $\mathbb {R}^n$ , respectively. Then $\frac {f}{g}$ is differentiable on $X\cap Y$ , and $D\left (\frac {f}{g}\right )(\vec {a})=\frac {g(\vec {a})Df(\vec {a})-f(\vec {a})Dg(\vec {a})}{(g(\vec {a}))^2}$ for $\vec {a}$ in $X\cap Y$ .

We’ll leave the proofs of these results as exercises, as they are similar to the single variable proofs.