*Note: 2 lectures*

We would like an analogue of Bolzano–Weierstrass. Something to the tune of “every bounded sequence of functions (with some property) has a convergent subsequence.” Matters are not as simple even for continuous functions. Not every bounded sequence in the metric space \(C\bigl([0,1],\R\bigr)\) has a convergent subsequence.

###
Definition 11.6.1.

Let \(X\) be a set. Let \(f_n \colon X \to \C\) be functions in a sequence. We say that \(\{ f_n \}_{n=1}^\infty\) is *pointwise bounded* if for every \(x \in X\text{,}\) there is an \(M_x \in \R\) such that

\begin{equation*}
\sabs{f_n(x)} \leq M_x \qquad \text{for all } n \in \N .
\end{equation*}

We say that \(\{ f_n \}_{n=1}^\infty\) is *uniformly bounded* if there is an \(M \in \R\) such that

\begin{equation*}
\sabs{f_n(x)} \leq M \qquad \text{for all } n \in \N \text{ and all } x \in X.
\end{equation*}

If \(X\) is a compact metric space, then a sequence in \(C(X,\C)\) is uniformly bounded if it is bounded as a set in the metric space \(C(X,\C)\) using the uniform norm.

###
Example 11.6.2.

There exist sequences of continuous functions on \([0,1]\) that are uniformly bounded but contain no subsequence converging even pointwise. Let us state without proof that \(f_n(x) \coloneqq \sin (2\pi n x)\) is one such sequence. Below we will show that there must always exist a subsequence converging at countably many points, but \([0,1]\) is uncountable.

###
Example 11.6.3.

The sequence \(f_n(x) \coloneqq x^n\) of continuous functions on \([0,1]\) is uniformly bounded, but contains no subsequence that converges uniformly, although the sequence converges pointwise (to a discontinuous function).

###
Example 11.6.4.

The sequence \(\{ f_n \}_{n=1}^\infty\) of functions in \(C\bigl([0,1],\R\bigr)\) given by \(f_n(x) \coloneqq \frac{n^3x}{1+n^4x^2}\) converges pointwise to the zero function (obvious at \(x=0\text{,}\) and for \(x > 0\text{,}\) we have \(\frac{n^3x}{1+n^4x^2} \leq \frac{1}{nx}\)). As for each \(x\text{,}\) \(\{f_n(x)\}_{n=1}^\infty\) converges to 0, it is bounded so \(\{ f_n \}_{n=1}^\infty\) is pointwise bounded.

Via calculus, we find that the maximum of \(f_n\) on \([0,1]\) occurs at the critical point \(x=\nicefrac{1}{n^2}\text{:}\)

\begin{equation*}
\snorm{f_n}_{[0,1]}
=
f_n\left(\nicefrac{1}{n^2}\right)
= \nicefrac{n}{2} .
\end{equation*}

So \(\lim_{n\to\infty} \snorm{f_n}_{[0,1]} = \infty\text{,}\) and this sequence is not uniformly bounded.

When the domain is countable, we can locate a subsequence converging at least pointwise. The proof uses a very common and useful diagonal argument.

###
Proposition 11.6.5.

Let \(X\) be a countable set and \(f_n \colon X \to \C\) give a pointwise bounded sequence of functions. Then \(\{ f_n \}_{n=1}^\infty\) has a subsequence that converges pointwise.

### Proof.

Let \(x_1,x_2,x_3,\ldots\) be an enumeration of the elements of \(X\text{.}\) The sequence \(\{ f_n(x_1) \}_{n=1}^\infty\) is bounded and hence we have a subsequence of \(\{ f_n \}_{n=1}^{\infty}\text{,}\) which we denote by \(\{ f_{1,k} \}_{k=1}^\infty\text{,}\) such that \(\{ f_{1,k}(x_1) \}_{k=1}^\infty\) converges. Next \(\{ f_{1,k}(x_2) \}_{k=1}^\infty\) is bounded and so \(\{ f_{1,k} \}_{k=1}^\infty\) has a subsequence \(\{ f_{2,k} \}_{k=1}^\infty\) such that \(\{ f_{2,k}(x_2) \}_{k=1}^\infty\) converges. Note that \(\{ f_{2,k}(x_1) \}_{k=1}^\infty\) is still convergent.

In general, we have a sequence \(\{ f_{m,k} \}_{k=1}^\infty\text{,}\) which is a subsequence of \(\{ f_{m-1,k} \}_{k=1}^\infty\text{,}\) such that \(\{ f_{m,k}(x_j) \}_{k=1}^\infty\) converges for \(j=1,2,\ldots, m\text{.}\) We let \(\{ f_{m+1,k} \}_{k=1}^\infty\) be a subsequence of \(\{ f_{m,k} \}_{k=1}^\infty\) such that \(\{ f_{m+1,k}(x_{m+1}) \}_{k=1}^\infty\) converges (and hence it converges for all \(x_j\) for \(j=1,2,\ldots,m+1\)). Rinse and repeat.

If \(X\) is finite, we are done as the process stops at some point. If \(X\) is countably infinite, we pick the sequence \(\{ f_{k,k} \}_{k=1}^\infty\text{.}\) This is a subsequence of the original sequence \(\{ f_n \}_{n=1}^\infty\text{.}\) For every \(m\text{,}\) the tail \(\{ f_{k,k} \}_{k=m}^\infty\) is a subsequence of \(\{ f_{m,k} \}_{k=1}^\infty\) and hence for any \(m\) the sequence \(\{ f_{k,k}(x_m) \}_{k=1}^\infty\) converges.

For larger than countable sets, we need the functions of the sequence to be related. When we look at continuous functions, the concept we need is equicontinuity.

###
Definition 11.6.6.

Let \((X,d)\) be a metric space. A set \(S\) of functions \(f \colon X \to \C\) is *uniformly equicontinuous* if for every \(\epsilon > 0\text{,}\) there is a \(\delta > 0\) such that if \(x,y \in X\) with \(d(x,y) < \delta\text{,}\) we have

\begin{equation*}
\sabs{f(x)-f(y)} < \epsilon \qquad \text{for all } f \in S .
\end{equation*}

Notice that functions in a uniformly equicontinuous sequence are all uniformly continuous. It is not hard to show that a finite set of uniformly continuous functions is uniformly equicontinuous. The definition is really interesting if \(S\) is infinite.

Just as for continuity, one can define equicontinuity at a point. That is, \(S\) is *equicontinuous* at \(x \in X\) if for every \(\epsilon > 0\text{,}\) there is a \(\delta > 0\) such that for \(y \in X\) with \(d(x,y) < \delta\text{,}\) we have \(\sabs{f(x)-f(y)} < \epsilon\) for all \(f \in S\text{.}\) We will only deal with compact \(X\) here, and one can prove (exercise) that for a compact metric space \(X\text{,}\) if \(S\) is equicontinuous at every \(x \in X\text{,}\) then it is uniformly equicontinuous. For simplicity we stick to uniform equicontinuity.

###
Proposition 11.6.7.

Suppose \((X,d)\) is a compact metric space, \(f_n \in C(X,\C)\text{,}\) and \(\{ f_n \}_{n=1}^\infty\) converges uniformly, then \(\{ f_n \}_{n=1}^\infty\) is uniformly equicontinuous.

### Proof.

Let \(\epsilon > 0\) be given. As \(\{ f_n \}_{n=1}^\infty\) converges uniformly, there is an \(N \in \N\) such that for all \(n \geq N\)

\begin{equation*}
\sabs{f_n(x)-f_N(x)} < \nicefrac{\epsilon}{3} \qquad \text{for all } x \in X.
\end{equation*}

As \(X\) is compact, every continuous function is uniformly continuous. So \(\{ f_1,f_2,\ldots,f_N \}\) is a finite set of uniformly continuous functions. And so, as we mentioned above, the set is uniformly equicontinuous. Hence there is a \(\delta > 0\) such that

\begin{equation*}
\sabs{f_j(x)-f_j(y)} < \nicefrac{\epsilon}{3} < \epsilon
\end{equation*}

whenever \(d(x,y) < \delta\) and \(1 \leq j \leq N\text{.}\)

Take \(n > N\text{.}\) For \(d(x,y) < \delta\text{,}\) we have

\begin{equation*}
\sabs{f_n(x)-f_n(y)}
\leq
\sabs{f_n(x)-f_N(x)}
+
\sabs{f_N(x)-f_N(y)}
+
\sabs{f_N(y)-f_n(y)}
<
\nicefrac{\epsilon}{3}
+
\nicefrac{\epsilon}{3}
+
\nicefrac{\epsilon}{3}
=\epsilon . \qedhere
\end{equation*}

###
Proposition 11.6.8.

A compact metric space \((X,d)\) contains a countable dense subset, that is, there exists a countable \(D \subset X\) such that \(\widebar{D} = X\text{.}\)

### Proof.

For each \(n \in \N\) there are finitely many balls of radius \(\nicefrac{1}{n}\) that cover \(X\) (as \(X\) is compact). That is, for every \(n\text{,}\) there exists a finite set of points \(x_{n,1},x_{n,2},\ldots,x_{n,k_n}\) such that

\begin{equation*}
X= \bigcup_{j=1}^{k_n} B(x_{n,j},\nicefrac{1}{n}) .
\end{equation*}

Let \(D \coloneqq \bigcup_{n=1}^\infty \{ x_{n,1},x_{n,2},\ldots,x_{n,k_n} \}\text{.}\) The set \(D\) is countable as it is a countable union of finite sets. For every \(x \in X\) and every \(\epsilon > 0\text{,}\) there exists an \(n\) such that \(\nicefrac{1}{n} < \epsilon\) and an \(x_{n,j} \in D\) such that

\begin{equation*}
x \in B(x_{n,j},\nicefrac{1}{n}) \subset B(x_{n,j},\epsilon) .
\end{equation*}

Hence \(x \in \widebar{D}\text{,}\) so \(\widebar{D} = X\text{,}\) and \(D\) is dense.

We are now ready for the main result of this section, the Arzelà–Ascoli theorem

^{ 1 } about existence of convergent subsequences.

###
Theorem 11.6.9. Arzelà–Ascoli.

Let \((X,d)\) be a compact metric space, and let \(\{ f_n \}_{n=1}^\infty\) be pointwise bounded and uniformly equicontinuous sequence of functions \(f_n \in C(X,\C)\text{.}\) Then \(\{f_n\}_{n=1}^\infty\) is uniformly bounded and \(\{ f_n \}_{n=1}^\infty\) contains a uniformly convergent subsequence.

Basically, a uniformly equicontinuous sequence in the metric space \(C(X,\C)\) that is pointwise bounded is bounded (in \(C(X,\C)\)) and furthermore contains a convergent subsequence in \(C(X,\C)\text{.}\)

As we mentioned before, as \(X\) is compact, it is enough to just assume that \(\{ f_n \}_{n=1}^\infty\) is equicontinuous as uniform equicontinuity is automatic via an exercise.

### Proof.

We first show that the sequence is uniformly bounded. By uniform equicontinuity, there is a \(\delta > 0\) such that for all \(x \in X\) and all \(n \in \N\text{,}\)

\begin{equation*}
B(x,\delta) \subset f_n^{-1}\bigl(B(f_n(x),1)\bigr) .
\end{equation*}

The space \(X\) is compact, so there exist \(x_1,x_2,\ldots,x_k\) such that

\begin{equation*}
X = \bigcup_{j=1}^k B(x_j,\delta) .
\end{equation*}

As \(\{ f_n \}_{n=1}^\infty\) is pointwise bounded there exist \(M_1,M_2,\ldots,M_k\) such that for \(j=1,2,\ldots,k\text{,}\)

\begin{equation*}
\sabs{f_n(x_j)} \leq M_j \qquad \text{for all } n.
\end{equation*}

Let \(M \coloneqq 1+ \max \{ M_1,M_2,\ldots,M_k \}\text{.}\) Given any \(x \in X\text{,}\) there is a \(j\) such that \(x \in B(x_j,\delta)\text{.}\) Therefore, for all \(n\text{,}\) we have \(x \in f_n^{-1}\bigl(B(f_n(x_j),1)\bigr)\text{,}\) or in other words

\begin{equation*}
\sabs{f_n(x)-f_n(x_j)} < 1 .
\end{equation*}

By the reverse triangle inequality,

\begin{equation*}
\sabs{f_n(x)} < 1+ \sabs{f_n(x_j)} \leq 1+M_j \leq M .
\end{equation*}

As \(x\) was arbitrary, \(\{f_n\}_{n=1}^\infty\) is uniformly bounded.

Next, pick a countable dense subset

\(D \subset X\text{.}\) By

Proposition 11.6.5, we find a subsequence

\(\{ f_{n_j} \}_{j=1}^\infty\) that converges pointwise on

\(D\text{.}\) Write

\(g_j \coloneqq f_{n_j}\) for simplicity. The sequence

\(\{ g_n \}_{n=1}^\infty\) is uniformly equicontinuous. Let

\(\epsilon > 0\) be given, then there exists a

\(\delta > 0\) such that for all

\(x \in X\) and all

\(n \in \N\text{,}\)
\begin{equation*}
B(x,\delta) \subset g_n^{-1}\bigl(B(g_n(x),\nicefrac{\epsilon}{3})\bigr).
\end{equation*}

By density of \(D\) and because \(\delta\) is fixed, every \(x \in X\) is in \(B(y,\delta)\) for some \(y \in D\text{.}\) By compactness of \(X\text{,}\) there is a finite subset \(\{ x_1,x_2,\ldots,x_k \} \subset D\) such that

\begin{equation*}
X = \bigcup_{j=1}^k B(x_j,\delta) .
\end{equation*}

As \(\{ x_1,x_2,\ldots,x_k \}\) is a finite set and \(\{ g_n \}_{n=1}^\infty\) converges pointwise on \(D\text{,}\) there exists a single \(N\) such that for all \(n,m \geq N\text{,}\)

\begin{equation*}
\sabs{g_n(x_j)-g_m(x_j)} < \nicefrac{\epsilon}{3}
\qquad \text{for all } j=1,2,\ldots,k.
\end{equation*}

Let \(x \in X\) be arbitrary. There is some \(j\) such that \(x \in B(x_j,\delta)\) and so for all \(\ell \in \N\text{,}\)

\begin{equation*}
\sabs{g_\ell(x)-g_\ell(x_j)} < \nicefrac{\epsilon}{3}.
\end{equation*}

So for \(n,m \geq N\text{,}\)

\begin{equation*}
\begin{split}
\sabs{g_n(x)-g_m(x)} & \leq
\sabs{g_n(x)-g_n(x_j)} +
\sabs{g_n(x_j)-g_m(x_j)} +
\sabs{g_m(x_j)-g_m(x)}
\\
& <
\nicefrac{\epsilon}{3} +
\nicefrac{\epsilon}{3} +
\nicefrac{\epsilon}{3} = \epsilon .
\end{split}
\end{equation*}

Hence, \(\{ g_n \}_{n=1}^\infty\) is uniformly Cauchy. By completeness of \(\C\text{,}\) it is uniformly convergent.

###
Corollary 11.6.10.

Let \((X,d)\) be a compact metric space. Let \(S \subset C(X,\C)\) be a closed, bounded and uniformly equicontinuous set. Then \(S\) is compact.

The theorem says that \(S\) is sequentially compact and that means compact in a metric space. Recall that the closed unit ball in \(C\bigl([0,1],\R\bigr)\text{,}\) and therefore also in \(C\bigl([0,1],\C\bigr)\text{,}\) is not compact. Hence it cannot be a uniformly equicontinuous set.

###
Corollary 11.6.11.

Suppose \(\{ f_n \}_{n=1}^\infty\) is a sequence of differentiable functions on \([a,b]\text{,}\) \(\{ f_n' \}_{n=1}^\infty\) is uniformly bounded, and there is an \(x_0 \in [a,b]\) such that \(\{ f_n(x_0) \}_{n=1}^\infty\) is bounded. Then there exists a uniformly convergent subsequence \(\{ f_{n_j} \}_{j=1}^\infty\text{.}\)

### Proof.

The trick is to use the mean value theorem. If \(M\) is the uniform bound on \(\{ f_n' \}_{n=1}^\infty\text{,}\) then by the mean value theorem for every \(n\)

\begin{equation*}
\sabs{f_n(x)-f_n(y)} \leq M \sabs{x-y} \qquad \text{for all } x,y \in X.
\end{equation*}

All the \(f_n\) are Lipschitz with the same constant and hence the sequence is uniformly equicontinuous.

Suppose \(\sabs{f_n(x_0)} \leq M_0\) for all \(n\text{.}\) For all \(x \in [a,b]\text{,}\)

\begin{equation*}
\sabs{f_n(x)} \leq \sabs{f_n(x_0)}+ \sabs{f_n(x)-f_n(x_0)} \leq M_0+ M \sabs{x-x_0}
\leq M_0 + M(b-a) .
\end{equation*}

So

\(\{ f_n \}_{n=1}^\infty\) is uniformly bounded. We apply

Arzelà–Ascoli to find the subsequence.

A classic application of the corollary above to Arzelà–Ascoli in the theory of differential equations is to prove the Peano existence theorem, that is, the existence of solutions to ordinary differential equations. See

Exercise 11.6.11 below.

Another application of Arzelà–Ascoli using the same idea as the corollary above is the following. Take a continuous \(k \colon [0,1] \times [0,1] \to \C\text{.}\) For every \(f \in C\bigl([0,1],\C\bigr)\) define

\begin{equation*}
T\bigl(f\bigr)(x) \coloneqq \int_0^1 f(t) \, k(x,t)\,dt .
\end{equation*}

In exercises to earlier sections you have shown that \(T\) is a linear operator on \(C\bigl([0,1],\C\bigr)\text{.}\) Via Arzelà–Ascoli, we also find (exercise) that the image of the unit ball of functions

\begin{equation*}
T\bigl( B(0,1) \bigr) =
\bigl\{
Tf \in C\bigl([0,1],\C\bigr) :
\snorm{f}_{[0,1]} < 1
\bigr\}
\end{equation*}

has compact closure, usually called *relatively compact*. Such an operator is called a *compact operator*. And they are very useful. Generally operators defined by integration tend to be compact.