by Richard Palais
What is Hilbert’s Fifth Problem?
As Hilbert stated it in his lecture delivered before the International Congress of Mathematicians in Paris in 1900 [e2], the Fifth Problem is linked to Sophus Lie’s theory of transformation groups [e1], i.e., Lie groups acting as groups of transformations on manifolds. The “groups” that Lie dealt with were really just neighborhoods of the identity in what we now call a Lie group, and his group actions were defined only locally, but we will ignore such local versus global considerations in what follows. However, it was crucial to the techniques that Lie used that his manifolds should be analytic and that both the group law and the functions defining the action of the group on the manifold should be analytic, that is, given by convergent power series. For Lie, who applied his theory to such things as studying the symmetries of differential equations, the analyticity assumptions were natural enough. But Hilbert wanted to use Lie’s theory as part of his logical foundations of geometry, and for this purpose Hilbert felt that analyticity was unnatural and perhaps superfluous. So Hilbert asked if analyticity could be dropped in favor of mere continuity. More precisely, if one only assumed a priori that the group \( G \) was a locally Euclidean topological group, that the manifold \( M \) was a topological manifold, and that the action of \( G \) on \( M \) was continuous, could one nevertheless always choose local coordinates in \( G \) and \( M \) so that both the group operations and the action became analytic when expressed in these coordinates? We shall speak of the problem in this generality as the unrestricted Hilbert Fifth Problem. The restricted problem is the important special case in which \( G = M \) and the action is left translation. Asking whether we can always find analytic coordinates in the restricted problem is clearly the same as asking whether a locally Euclidean group is necessarily a Lie group.
Counterexamples
It turned out that there are many — and in fact many different kinds of — counterexamples to the unrestricted Hilbert Fifth Problem. Perhaps the first published counterexample, due to R. H. Bing [e18], is an action of \( \mathbf{Z}^2 \) on \( \mathbf{S}^3 \) whose fixed-point set is the Alexander Horned Sphere \( \Sigma \). Now \( \Sigma \) is not “tamely embedded” in \( \mathbf{S}^3 \), meaning that there are points where it is impossible to choose coordinates so that locally \( \Sigma \) looks like the usual embedding of \( \mathbf{R}^2 \) in \( \mathbf{R}^3 \). If the action were even differentiable in some suitable coordinates, then it is easy to see that the fixed-point set would in fact be tamely embedded. (For an even more bizarre type of counterexample, recall that in 1960 M. Kervaire [e24] constructed a topological manifold that did not admit any differentiable structure, providing what can be considered a counterexample even for the case when \( G \) is the trivial group.)
One could make a case that these examples are “monsters” that could have been ruled out if Hilbert had phrased his statement of the Fifth Problem more carefully. But there is a more serious kind of counterexample that is so elementary that it makes one wonder how much thought Hilbert had given to the Fifth Problem before proposing it. Here is a particularly elementary example, due to Montgomery and Zippin [e22], with \( G = \mathbf{R} \), the additive group of the real numbers, and \( M = \mathbf{C} \), the complex plane. Let \( f \) be a continuous real-valued function defined on the positive real axis, and define the action \( \phi :\mathbf{R} \times \mathbf{C} \to \mathbf{C} \) by \( \phi(t , r e^{i \theta} ) := r e^{i (\theta +f (r )t)} \) . (In words, \( \phi \) is a one-parameter group of homeomorphisms of the plane that rotates each circle centered at the origin into itself, the circle of radius \( r \) being rotated with angular velocity \( f (r ) \).) If we choose \( f (r ) \) to equal 1 for \( r \leq 1 \) and 0 for \( r \geq 2 \), the action is the standard one-parameter group of rotations of \( \mathbf{C} \) inside the unit disk and is trivial outside the disk of radius 2, so by the Principle of Analytic Continuation, this action cannot be made analytic in any coordinate system. What is worse, we can choose \( f \) to have these properties and also be smooth (meaning \( C^{\infty} \)), so we see that even if we assume a priori that the action of a Lie Group on a manifold is smooth, it does not follow that it can be made analytic!
After these counterexamples to the unrestricted Hilbert Fifth Problem became known, a tacit understanding grew up to interpret “the Fifth Problem” as referring to the restricted version: Is every locally Euclidean group a Lie group? and we shall follow this convention below.
Early history of the Fifth Problem
It was fairly easy to settle the one-dimensional case. The only (paracompact) connected manifolds of dimension one are the real line, \( \mathbf{R} \), and the circle, \( \mathbf{S}^1 \), and both of course are Lie groups. In 1909 L. E. J. Brouwer [e3] showed that a topological group that is homeomorphic to either of these is in fact isomorphic to it as a topological group. Using results from Brouwer’s paper, B. Kerékjártó [e6] settled the two-dimensional case in 1931. There seems to have been little if any published work on the Fifth Problem between the papers of Brouwer and Kerékjártó, but that is not too surprising considering that much of the modern mathematical infrastructure required for a rigorous discussion of topological groups and the Fifth Problem became available only after a 1926 paper by O. Schreier [e4]. The three-dimensional and four-dimensional cases of the Fifth Problem were settled much later, by Montgomery [e12] in 1948 and by Montgomery and Zippin [e16] in 1952.
The first major breakthrough in the general theory came in 1933, when J. von Neumann [e8], using the recently discovered Haar [e7] measure, extended the Peter–Weyl Theorem [e5] to general compact groups and used it to settle the Fifth Problem in the affirmative for compact groups. We will sketch a proof of von Neumann’s theorem below. Several years later, building on von Neumann’s work, Pontryagin [e9] settled the abelian case, and Chevalley [e10] the solvable case.
The no small subgroups (NSS) condition
The first time I encountered the phrase “group without small subgroups” I wondered what kind of subgroup a “small” one could possibly be. Of course, what the phrase means is a topological group without arbitrarily small subgroups, i.e., one having a neighborhood of the identity that includes no subgroup except the trivial group. We shall follow Kaplansky [e25] and call this the NSS Condition and a group satisfying it an NSS group. Since NSS may seem a little contrived, here is a brief discussion of the “why and how” of its use in solving the Fifth Problem.
It turns out to be difficult to draw useful conclusions about a topological group from the assumption that it is locally Euclidean. So the strategy used for settling the Fifth Problem was to look for a more group-oriented “bridge condition” and use it in a two-pronged attack: on the one hand show that a topological group that satisfies this condition is a Lie group, and on the other show that a locally Euclidean group satisfies the condition. If these two propositions can be proved, then the positive solution of the Fifth Problem follows — and even a little more.
As you may have guessed, NSS turned out to be ideally suited to play the role of the bridge. In retrospect this is not entirely surprising. A powerful but relatively elementary property of Lie groups is the existence of so-called canonical coordinates, or equivalently the fact that the exponential map is a diffeomorphism of a neighborhood of zero in the Lie algebra onto a neighborhood \( U \) of the identity in the group (see below). Since a line through the origin in the Lie algebra maps to a one-parameter subgroup of the group, it follows that such a \( U \) contains no nontrivial subgroup and hence that Lie groups satisfy NSS.
Starting in the late 1940s Gleason [1],1 Montgomery [e15], and Iwasawa [e13] made several solid advances related to the Fifth Problem. This led in 1952 to a satisfying denouement to the story of the Fifth Problem, with Gleason and Montgomery–Zippin carrying out the above two-pronged attack. First Gleason [3] proved that a locally compact group satisfying NSS is a Lie group, and then immediately afterwards Montgomery and Zippin [e16] used Gleason’s result to prove inductively that locally Euclidean groups of any dimension satisfy NSS. Their two papers appeared together in the same issue of the Annals of Mathematics, and at that point one knew that for locally compact topological groups: \[ \text{Locally Euclidean } \Longleftrightarrow \text{NSS } \Longleftrightarrow \text{Lie}. \] (Actually, the above is not quite the full story; Gleason assumed a weak form of finite dimensionality in his original argument that NSS implies Lie, but shortly thereafter Yamabe [e20] showed that finite dimensionality was not needed in the proof.)
Cartan’s theorem
Here is a quick sketch of how the proof of the Fifth Problem for a compact NSS group \( G \) follows. Let \( \mathcal{H} \) denote the Hilbert space \( L^2 (G) \) of square-integrable functions on \( G \) with respect to Haar measure. Left translation induces an orthogonal representation of \( G \) on \( \mathcal{H} \), the so-called regular representation, and, according to the Peter–Weyl Theorem, \( \mathcal{H} \) is the orthogonal direct sum of finite-dimensional subrepresentations, \( \mathcal{H}_i \), i.e., \( \mathcal{H} =\bigoplus^{\infty}_{i=1} \mathcal{H}_i \). Define \( W_N := \bigoplus^N_{i=1}\mathcal{H}_i \). We will show that for \( N \) sufficiently large, the finite-dimensional representation of \( G \) on \( W_N \) is faithful or, equivalently, that for some \( N \) the kernel \( K_N \) of the regular representation restricted to \( W_N \) is the trivial group \( \{e\} \). Since the regular representation itself is clearly faithful, \( K_N \) is a decreasing sequence of compact subgroups of \( G \) whose intersection is \( \{e\} \). Thus if \( U \) is an open neighborhood of \( e \) that contains no nontrivial subgroup, \( K_N \backslash U \) is a decreasing sequence of compact sets with empty intersection and, by the definition of compactness in terms of closed sets, some \( K_N \backslash U \) must be empty. Hence \( K_N \subseteq U \), and since \( K_N \) is a subgroup of \( G \), \( K_N = \{e\} \).
Following in Gleason’s footsteps
Let’s start with a Lie group \( G \), and let \( \mathfrak{g} \) denote its Lie algebra. There are (at least!) three equivalent ways to think of an element of the vector space \( \mathfrak{g} \). First as a vector \( v \) in \( T G_e \), the tangent space to \( G \) at \( e \); second as the left-invariant vector field \( X \) on \( G \) obtained by left translating \( v \) over the group; and third as the one-parameter subgroup \( \phi \) of \( G \) obtained as the integral curve of \( X \) starting at the identity. The exponential map \( \exp : \mathfrak{g} \to G \) is defined by \( \exp(v ) = \phi(1) \). It follows immediately from this definition that \( \exp(0) = e \) and that the differential of \( \exp \) at 0 is the identity map of \( T G_e \), so by the inverse function theorem, \( \exp \) maps a neighborhood of 0 in \( \mathfrak{g} \) diffeomorphically onto a neighborhood of \( e \) in \( G \). Such a chart for \( G \) is called a canonical coordinate system (of the first kind).
Now, suppose we somehow “lost” the differentiable structure of \( G \) but retained our knowledge of \( G \) as a topological group. Is there some way we could use the latter knowledge to recover the differentiable structure? That is, can we reconstruct \( \mathfrak{g} \) and the exponential map? If so, then we are clearly close to a solution of the Fifth Problem. Let’s listen in as Andy ponders this question.
“Well, if I think of \( \mathfrak{g} \) as being the one-parameter groups, that’s a group theoretic concept. Let’s see — is there some way I can invert \( \exp \)? That is, given \( g \) in \( G \) close to \( e \), can I find the one-parameter group \( \phi \) such that \( \phi (1) = \exp(\phi) = g \)? Now I know square roots are unique near \( e \) and in fact \( \phi(1/2) \) is the square root of \( g \). By induction, I can find \( \phi(1/2^n) \) by starting with \( g \) and taking the square root \( n \) times. And once I have \( \phi(1/2^n) \), by simply taking its \( m \)-th power I can find \( \phi(m/2^n ) \) for all \( m \). So, if I know how to take square roots near \( e \), then I can compute \( \phi \) at all the dyadic rationals \( m /2^n \), and since they are dense in \( \mathbf{R} \), I can extend \( \phi \) by continuity to find it on all of \( \mathbf{R}! \)”
This was the stated motivation for Gleason’s paper “Square roots in locally Euclidean groups” [1], and in it he goes on to take the first step and show that in any NSS locally Euclidean group \( G \), there are neighborhoods \( U \) and \( V \) of the identity such that every element in \( U \) has a unique square root in \( V \). Almost immediately after this article appeared, in a paper called “On a theorem of Gleason”, Chevalley [e14] went on to complete the program Andy outlined. That is, Chevalley used Gleason’s existence of unique square roots to construct a neighborhood \( U \) of the identity in \( G \) and a continuous mapping \( (g , t ) \mapsto \phi^g (t ) \) of \( U \times \mathbf{R} \) into \( G \) such that each \( \phi^g \) is a one-parameter subgroup of \( G , \phi^g (t) \in U \) for \( |t| \leq 1 \), and \( \phi^g (1) = g \).
In his key 1952 Annals paper “Groups without small subgroups” [3], Gleason decided not to follow up this approach to the solution of the Fifth Problem and instead used a variant of von Neumann’s method. His approach was based on the construction of one-parameter subgroups, but these were used as a tool to find a certain finite-dimensional invariant linear subspace \( Z \) of the regular representation of \( G \) on which \( G \) acted faithfully and appealed to Cartan’s Theorem to complete the proof. The construction of \( Z \) is a technical tour de force, but it is too complicated to outline here, and we refer instead to the original paper [2] or the review by Iwasawa.
Andy Gleason as mentor
Looking back at how it happened, it seems almost accidental that I became Andy Gleason’s first Ph.D. student — and David Hilbert was partly responsible.
As an undergraduate at Harvard I had developed a very close mentoring relationship with George Mackey, then a resident tutor in my dorm, Kirkland House. We had meals together several times each week, and I took many of his courses. So, when I returned in 1953 as a graduate student, it was natural for me to ask Mackey to be my thesis advisor. When he inquired what I would like to work on for my thesis research, my first suggestion turned out to be something he had thought about himself, and he was able to convince me quickly that it was unsuitably difficult for a thesis topic. A few days later I came back and told him I would like to work on reformulating the classical Lie theory of germs of Lie groups acting locally on manifolds as a rigorous modern theory of full Lie groups acting globally. Fine, he said, but then explained that the local expert on such matters was a brilliant young former Junior Fellow named Andy Gleason who had just joined the Harvard math department. Only a year before he had played a major role in solving Hilbert’s Fifth Problem, which was closely related to what I wanted to work on, so he would be an ideal person to direct my research.
I felt a little unhappy at being cast off like that by Mackey, but of course I knew perfectly well who Gleason was and I had to admit that George had a point. Andy was already famous for being able to think complicated problems through to a solution incredibly fast. “Johnny” von Neumann had a similar reputation, and since this was the year that High Noon came out, I recall jokes about having a mathematical shootout — Andy vs. Johnny solving math problems with blazing speed at the OK Corral. In any case, it was with considerable trepidation that I went to see Andy for the first time.
Totally unnecessary! In our sessions together I never felt put down. It is true that occasionally when I was telling him about some progress I had made since our previous discussion, partway through my explanation Andy would see the crux of what I had done and say something like, “Oh! I see. Very nice! and then…,” and in a matter of minutes he would reconstruct (often with improvements) what had taken me hours to figure out. But it never felt like he was acting superior. On the contrary, he always made me feel that we were colleagues, collaborating to discover the way forward. It was just that when he saw his way to a solution of one problem, he liked to work quickly through it and then go on to the next problem. Working together with such a mathematical powerhouse put pressure on me to perform at top level — and it was sure a good way to learn humility!
My apprenticeship wasn’t over when my thesis was done. I remember that shortly after I had finished, Andy said to me, “You know, some of the ideas in your thesis are related to some ideas I had a few years back. Let me tell you about them, and perhaps we can write a joint paper.” The ideas in that paper were in large part his, but on the other hand, I did most of the writing, and in the process of correcting my attempts he taught me a lot about how to write a good journal article.
But it was only years later that I fully appreciated just how much I had taken away from those years working under Andy. I was very fortunate to have many excellent students do their graduate research with me over the years, and often as I worked together with them I could see myself behaving in some way that I had learned to admire from my own experience working together with Andy.
Let me finish with one more anecdote. It concerns my favorite of all Andy’s theorems, his elegant classification of the measures on the lattice of subspaces of a Hilbert space. Andy was writing up his results during the 1955–56 academic year, as I was writing up my thesis, and he gave me a draft copy of his paper to read. I found the result fascinating, and even contributed a minor improvement to the proof, as Andy was kind enough to footnote in the published article. When I arrived at the University of Chicago for my first position the next year, Andy’s paper was not yet published, but word of it had gotten around, and there was a lot of interest in hearing the details. So when I let on that I was familiar with the proof, Kaplansky asked me to give a talk on it in his analysis seminar. I’ll never forget walking into the room where I was to lecture and seeing Ed Spanier, Marshall Stone, Saunders Mac Lane, André Weil, Kaplansky, and Chern all looking up at me. It was pretty intimidating, and I was suitably nervous!
But the paper was so elegant and clear that it was an absolute breeze to lecture on it, so all went well, and this “inaugural lecture” helped me get off to a good start in my academic career.
