Section 11.6 Equicontinuity and the Arzelà–Ascoli theorem
Note: 2 lectures
We would like an analogue of Bolzano–Weierstrass. Something to the tune of “every bounded sequence of functions (with some property) has a convergent subsequence.” Matters are not as simple even for continuous functions. Not every bounded sequence in the metric space \(C([0,1],\R)\) has a convergent subsequence.
Definition 11.6.1.
Let \(X\) be a set. Let \(f_n \colon X \to \C\) be functions in a sequence. We say that \(\{ f_n \}\) is pointwise bounded if for every \(x \in X\text{,}\) there is an \(M_x \in \R\) such that
We say that \(\{ f_n \}\) is uniformly bounded if there is an \(M \in \R\) such that
If \(X\) is a compact metric space, then a sequence in \(C(X,\C)\) is uniformly bounded if it is bounded as a set in the metric space \(C(X,\C)\) using the uniform norm.
Example 11.6.2.
There exist sequences of continuous functions on \([0,1]\) that are uniformly bounded but contain no subsequence converging even pointwise. Let us state without proof that \(f_n(x) := \sin (2\pi n x)\) is one such sequence. Below we will show that there must always exist a subsequence converging at countably many points, but \([0,1]\) is uncountable.
Example 11.6.3.
The sequence \(f_n(x) := x^n\) of continuous functions on \([0,1]\) is uniformly bounded, but contains no subsequence that converges uniformly, although the sequence converges pointwise (to a discontinuous function).
Example 11.6.4.
The sequence \(\{ f_n \}\) of functions in \(C([0,1],\R)\) given by \(f_n(x) := \frac{n^3x}{1+n^4x^2}\) converges pointwise to the zero function (obvious at \(x=0\text{,}\) and for \(x > 0\text{,}\) we have \(\frac{n^3x}{1+n^4x^2} \leq \frac{1}{nx}\)). As for each \(x\text{,}\) \(\{f_n(x)\}\) converges to 0, it is bounded so \(\{ f_n \}\) is pointwise bounded.
By calculus, we maximize \(f_n\) on \([0,1]\text{,}\) and we find the maximum occurs at the critical point \(x=\nicefrac{1}{n^2}\text{:}\)
So \(\lim\, \snorm{f_n}_u = \infty\text{,}\) and this sequence is not uniformly bounded.
When the domain is countable, we can locate a subsequence converging at least pointwise. The proof uses a very common and useful diagonal argument.
Proposition 11.6.5.
Let \(X\) be a countable set and \(f_n \colon X \to \C\) give a pointwise bounded sequence of functions. Then \(\{ f_n \}\) has a subsequence that converges pointwise.
Proof.
Let \(x_1,x_2,x_3,\ldots\) be an enumeration of the elements of \(X\text{.}\) The sequence \(\{ f_n(x_1) \}_{n=1}^\infty\) is bounded and hence we have a subsequence of \(\{ f_n \}_{n=1}^{\infty}\text{,}\) which we denote by \(\{ f_{1,k} \}_{k=1}^\infty\text{,}\) such that \(\{ f_{1,k}(x_1) \}_{k=1}^\infty\) converges. Next \(\{ f_{1,k}(x_2) \}_{k=1}^\infty\) is bounded and so \(\{ f_{1,k} \}_{k=1}^\infty\) has a subsequence \(\{ f_{2,k} \}_{k=1}^\infty\) such that \(\{ f_{2,k}(x_2) \}_{k=1}^\infty\) converges. Note that \(\{ f_{2,k}(x_1) \}_{k=1}^\infty\) is still convergent.
In general, we have a sequence \(\{ f_{m,k} \}_{k=1}^\infty\text{,}\) which is a subsequence of \(\{ f_{m1,k} \}_{k=1}^\infty\text{,}\) such that \(\{ f_{m,k}(x_j) \}_{k=1}^\infty\) converges for \(j=1,2,\ldots, m\text{.}\) We let \(\{ f_{m+1,k} \}_{k=1}^\infty\) be a subsequence of \(\{ f_{m,k} \}_{k=1}^\infty\) such that \(\{ f_{m+1,k}(x_{m+1}) \}_{k=1}^\infty\) converges (and hence it converges for all \(x_j\) for \(j=1,2,\ldots,m+1\)). Rinse and repeat.
If \(X\) is finite, we are done as the process stops at some point. If \(X\) is countably infinite, we pick the sequence \(\{ f_{k,k} \}_{k=1}^\infty\text{.}\) This is a subsequence of the original sequence \(\{ f_n \}_{n=1}^\infty\text{.}\) For every \(m\text{,}\) the tail \(\{ f_{k,k} \}_{k=m}^\infty\) is a subsequence of \(\{ f_{m,k} \}_{k=1}^\infty\) and hence for any \(m\) the sequence \(\{ f_{k,k}(x_m) \}_{k=1}^\infty\) converges.
For larger than countable sets, we need the functions of the sequence to be related. When we look at continuous functions, the concept we need is equicontinuity.
Definition 11.6.6.
Let \((X,d)\) be a metric space. A set \(S\) of functions \(f \colon X \to \C\) is uniformly equicontinuous if for every \(\epsilon > 0\text{,}\) there is a \(\delta > 0\) such that if \(x,y \in X\) with \(d(x,y) < \delta\text{,}\) we have
Notice that functions in a uniformly equicontinuous sequence are all uniformly continuous. It is not hard to show that a finite set of uniformly continuous functions is uniformly equicontinuous. The definition is really interesting if \(S\) is infinite.
Just as for continuity, one can define equicontinuity at a point. That is, \(S\) is equicontinuous at \(x \in X\) if for every \(\epsilon > 0\text{,}\) there is a \(\delta > 0\) such that for \(y \in X\) with \(d(x,y) < \delta\text{,}\) we have \(\sabs{f(x)f(y)} < \epsilon\) for all \(f \in S\text{.}\) We will only deal with compact \(X\) here, and one can prove (exercise) that for a compact metric space \(X\text{,}\) if \(S\) is equicontinuous at every \(x \in X\text{,}\) then it is uniformly equicontinuous. For simplicity we stick to uniform equicontinuity.
Proposition 11.6.7.
Suppose \((X,d)\) is a compact metric space, \(f_n \in C(X,\C)\text{,}\) and \(\{ f_n \}\) converges uniformly, then \(\{ f_n \}\) is uniformly equicontinuous.
Proof.
Let \(\epsilon > 0\) be given. As \(\{ f_n \}\) converges uniformly, there is an \(N \in \N\) such that for all \(n \geq N\)
As \(X\) is compact, every continuous function is uniformly continuous. So \(\{ f_1,f_2,\ldots,f_N \}\) is a finite set of uniformly continuous functions. And so, as we mentioned above, the set is uniformly equicontinuous. Hence there is a \(\delta > 0\) such that
whenever \(d(x,y) < \delta\) and \(1 \leq j \leq N\text{.}\)
Take \(n > N\text{.}\) For \(d(x,y) < \delta\text{,}\) we have
Proposition 11.6.8.
A compact metric space \((X,d)\) contains a countable dense subset, that is, there exists a countable \(D \subset X\) such that \(\widebar{D} = X\text{.}\)
Proof.
For each \(n \in \N\) there are finitely many balls of radius \(\nicefrac{1}{n}\) that cover \(X\) (as \(X\) is compact). That is, for every \(n\text{,}\) there exists a finite set of points \(x_{n,1},x_{n,2},\ldots,x_{n,k_n}\) such that
Let \(D := \bigcup_{n=1}^\infty \{ x_{n,1},x_{n,2},\ldots,x_{n,k_n} \}\text{.}\) The set \(D\) is countable as it is a countable union of finite sets. For every \(x \in X\) and every \(\epsilon > 0\text{,}\) there exists an \(n\) such that \(\nicefrac{1}{n} < \epsilon\) and an \(x_{n,j} \in D\) such that
Hence \(x \in \widebar{D}\text{,}\) so \(\widebar{D} = X\text{,}\) and \(D\) is dense.
We are now ready for the main result of this section, the Arzelà–Ascoli theorem^{ 1 } about existence of convergent subsequences.
Theorem 11.6.9. Arzelà–Ascoli.
Let \((X,d)\) be a compact metric space, and let \(\{ f_n \}\) be pointwise bounded and uniformly equicontinuous sequence of functions \(f_n \in C(X,\C)\text{.}\) Then \(\{f_n\}\) is uniformly bounded and \(\{ f_n \}\) contains a uniformly convergent subsequence.
Basically, a uniformly equicontinuous sequence in the metric space \(C(X,\C)\) that is pointwise bounded is bounded (in \(C(X,\C)\)) and furthermore contains a convergent subsequence in \(C(X,\C)\text{.}\)
As we mentioned before, as \(X\) is compact, it is enough to just assume that \(\{ f_n \}\) is equicontinuous as uniform equicontinuity is automatic via an exercise.
Proof.
Let us first show that the sequence is uniformly bounded.
By uniform equicontinuity, there is a \(\delta > 0\) such that for all \(x \in X\) and all \(n \in \N\text{,}\)
The space \(X\) is compact, so there exist \(x_1,x_2,\ldots,x_k\) such that
As \(\{ f_n \}\) is pointwise bounded there exist \(M_1,M_2,\ldots,M_k\) such that for \(j=1,2,\ldots,k\text{,}\) we have
Let \(M := 1+ \max \{ M_1,M_2,\ldots,M_k \}\text{.}\) Given any \(x \in X\text{,}\) there is a \(j\) such that \(x \in B(x_j,\delta)\text{.}\) Therefore, for all \(n\text{,}\) we have \(x \in f_n^{1}\bigl(B(f_n(x_j),1)\bigr)\text{,}\) or in other words
By reverse triangle inequality,
And as \(x\) was arbitrary, \(\{f_n\}\) is uniformly bounded.
Next, pick a countable dense subset \(D \subset X\text{.}\) By Proposition 11.6.5, we find a subsequence \(\{ f_{n_j} \}\) that converges pointwise on \(D\text{.}\) Write \(g_j := f_{n_j}\) for simplicity. The sequence \(\{ g_n \}\) is uniformly equicontinuous. Let \(\epsilon > 0\) be given, then there exists a \(\delta > 0\) such that for all \(x \in X\) and all \(n \in \N\)
By density of \(D\) and because \(\delta\) is fixed, every \(x \in X\) is in some \(B(y,\delta)\) for some \(y \in D\text{.}\) By compactness of \(X\text{,}\) there is a finite subset \(\{ x_1,x_2,\ldots,x_k \} \subset D\) such that
As there are finitely many points and \(\{ g_n \}\) converges pointwise on \(D\text{,}\) there exists a single \(N\) such that for all \(n,m \geq N\text{,}\) we have
Let \(x \in X\) be arbitrary. There is some \(j\) such that \(x \in B(x_j,\delta)\) and so for all \(\ell \in \N\text{,}\)
So for \(n,m \geq N\text{,}\)
Hence, the sequence is uniformly Cauchy. By completeness of \(\C\text{,}\) it is uniformly convergent.
Corollary 11.6.10.
Let \((X,d)\) be a compact metric space. Let \(S \subset C(X,\C)\) be a closed, bounded and uniformly equicontinuous set. Then \(S\) is compact.
The theorem says that \(S\) is sequentially compact and that means compact in a metric space. Recall that the closed unit ball in \(C([0,1],\R)\) (and therefore also in \(C([0,1],\C)\)) is not compact. Hence it cannot be a uniformly equicontinuous set.
Corollary 11.6.11.
Suppose \(\{ f_n \}\) is a sequence of differentiable functions on \([a,b]\text{,}\) \(\{ f_n' \}\) is uniformly bounded, and there is an \(x_0 \in [a,b]\) such that \(\{ f_n(x_0) \}\) is bounded. Then there exists a uniformly convergent subsequence \(\{ f_{n_j} \}\text{.}\)
Proof.
The trick is to use the mean value theorem. If \(M\) is the uniform bound on \(\{ f_n' \}\text{,}\) then by the mean value theorem for every \(n\)
All the \(f_n\) are Lipschitz with the same constant and hence the sequence is uniformly equicontinuous.
Suppose \(\sabs{f_n(x_0)} \leq M_0\) for all \(n\text{.}\) For all \(x \in [a,b]\text{,}\)
So \(\{ f_n \}\) is uniformly bounded. We apply Arzelà–Ascoli to find the subsequence.
A classic application of the corollary above to Arzelà–Ascoli in the theory of differential equations is to prove the Peano existence theorem, that is, the existence of solutions to ordinary differential equations. See Exercise 11.6.11 below.
Another application of Arzelà–Ascoli using the same idea as the corollary above is the following. Take a continuous \(k \colon [0,1] \times [0,1] \to \C\text{.}\) For every \(f \in C([0,1],\C)\) define
In exercises to earlier sections you have shown that \(T\) is a linear operator on \(C([0,1],\C)\text{.}\) Via Arzelà–Ascoli, we also find (exercise) that the image of the unit ball of functions
has compact closure, usually called relatively compact. Such an operator is called a compact operator. And they are very useful. Generally operators defined by integration tend to be compact.
Subsection 11.6.1 Exercises
Exercise 11.6.1.
Let \(f_n \colon [1,1] \to \R\) be given by \(f_n(x) := \frac{nx}{1+{(nx)}^2}\text{.}\) Prove that the sequence is uniformly bounded, converges pointwise to 0, yet there is no subsequence that converges uniformly. Which hypothesis of Arzelà–Ascoli is not satisfied? Prove your assertion.
Exercise 11.6.2.
Define \(f_n \colon \R \to \R\) by \(f_n(x) := \frac{1}{{(xn)}^2+1}\text{.}\) Prove that this sequence is uniformly bounded, uniformly equicontinuous, the sequence converges pointwise to zero, yet there is no subsequence that converges uniformly. Which hypothesis of Arzelà–Ascoli is not satisfied? Prove your assertion.
Exercise 11.6.3.
Let \((X,d)\) be a compact metric space, \(C > 0\text{,}\) \(0 < \alpha \leq 1\text{,}\) and suppose \(f_n \colon X \to \C\) are functions such as \(\abs{f_n(x)f_n(y)} \leq C {d(x,y)}^\alpha\) for all \(x,y \in X\) and \(n \in \N\text{.}\) Suppose also that there is a point \(p \in X\) such that \(f_n(p) = 0\) for all \(n\text{.}\) Show that there exists a uniformly convergent subsequence converging to an \(f \colon X \to \C\) that also satisfies \(f(p) = 0\) and \(\abs{f(x)f(y)} \leq C {d(x,y)}^\alpha\text{.}\)
Exercise 11.6.4.
Let \(T \colon C([0,1],\C) \to C([0,1],\C)\) be the operator given by
(That \(T\) is linear and that \(Tf\) is continuous follows from linearity of the integral and the fundamental theorem of calculus.)
Show that \(T\) takes the unit ball centered at 0 in \(C([0,1],\C)\) into a relatively compact set (a set with compact closure). That is, \(T\) is a compact operator.
Hint: See Exercise 7.4.20.Let \(C \subset C([0,1],\C)\) the closed unit ball, prove that the image \(T(C)\) is not closed (though it is relatively compact).
Exercise 11.6.5.
Given \(k \in C([0,1]\times [0,1],\C)\text{,}\) let \(T \colon C([0,1],\C) \to C([0,1],\C)\) be the operator defined by
Show that \(T\) takes the unit ball centered at 0 in \(C([0,1],\C)\) into a relatively compact set (a set with compact closure). That is, \(T\) is a compact operator.
Hint: See Exercise 7.4.20.
Note: That \(T\) is a welldefined linear operator was proved in Exercise 8.1.6.
Exercise 11.6.6.
Suppose \(S^1 \subset \C\) is the unit circle, that is the set where \(\sabs{z}=1\text{.}\) Suppose the continuous functions \(f_n \colon S^1 \to \C\) are uniformly bounded. Let \(\gamma \colon [0,1] \to S^1\) be a parametrization of \(S^1\text{,}\) and \(g(z,w)\) a continuous function on \(C(0,1) \times S^1\) (here \(C(0,1) \subset \C\) is the closed unit ball). Define the functions \(F_n \colon C(0,1) \to \C\) by the path integral (see Section 9.2)
Show that \(\{ F_n \}\) has a uniformly convergent subsequence.
Exercise 11.6.7.
Suppose \((X,d)\) is a compact metric space, \(\{ f_n \}\) a uniformly equicontinuous sequence of functions in \(C(X,\C)\text{.}\) Suppose \(\{ f_n \}\) converges pointwise. Show that it converges uniformly.
Exercise 11.6.8.
Suppose that \(\{ f_n \}\) is a uniformly equicontinuous uniformly bounded sequence of \(2\pi\)periodic functions \(f_n \colon \R \to \R\text{.}\) Show that there is a uniformly convergent subsequence.
Exercise 11.6.9.
Show that for a compact metric space \(X\text{,}\) a sequence \(\{ f_n \}\) that is equicontinuous at every \(x \in X\) is uniformly equicontinuous.
Exercise 11.6.10.
Define \(f_n \colon [0,1] \to \C\) by \(f_n(t) := e^{i(2\pi t + n)}\text{.}\) This is a uniformly equicontinuous uniformly bounded sequence. Prove more than just the conclusion of Arzelà–Ascoli for this sequence. Let \(\gamma \in \R\) be given, and define \(g(t) := e^{i(2\pi t + \gamma)}\text{.}\) Show that there exists a subsequence of \(\{ f_n \}\) converging uniformly to \(g\text{.}\)
Hint: Feel free to use the Kronecker density theorem^{ 4 }: The sequence \(\{ e^{in} \}_{n=1}^\infty\) is dense in the unit circle.
Exercise 11.6.11.
Prove the Peano existence theorem (note the weaker hypotheses than Picard, but also the lack of uniqueness in this theorem):
Theorem: Suppose \(F \colon I \times J \to \R\) is a continuous function where \(I, J \subset \R\) are closed bounded intervals, let \(I^\circ\) and \(J^\circ\) be their interiors, and let \((x_0,y_0) \in I^\circ \times J^\circ\text{.}\) Then there exists an \(h > 0\) and a differentiable function \(f \colon [x_0  h, x_0 + h] \to J \subset \R\text{,}\) such that
Use the following outline:

We wish to define the Picard iterates, that is, set \(f_0(x) := y_0\text{,}\) and
\begin{equation*} f_{n+1}(x) := y_0 + \int_{x_0}^x F\bigl(t,f_n(t)\bigr)\,dt . \end{equation*}Prove that there exists an \(h > 0\) such that \(f_n \colon [x_0  h, x_0 + h] \to \C\) is welldefined for all \(n\text{.}\) Hint: \(F\) is bounded (why?).
Show that \(\{ f_n \}\) is equicontinuous and bounded, in fact it is Lipschitz with a uniform Lipschitz constant. Arzelà–Ascoli then says that there exists a uniformly convergent subsequence \(\{ f_{n_k} \}\text{.}\)
Prove \(\bigl\{ F\bigl(x,f_{n_k}(x)\bigr) \bigr\}_{k=1}^\infty\) converges uniformly on \([x_0h,x_0+h]\text{.}\) Hint: \(F\) is uniformly continuous (why?).
Finish the proof of the theorem by taking the limit under the integral and applying the fundamental theorem of calculus.
https://en.wikipedia.org/wiki/Cesare_Arzel\%C3\%A0
https://en.wikipedia.org/wiki/Giulio_Ascoli
https://en.wikipedia.org/wiki/Leopold_Kronecker