Simulation and Understanding

To put it simply, we value theory. And simulation, at least the caricature of simulation we describe, gives us problems. So we put our prejudices up front.

They place theory, numerical simulation, and understanding at the vertices of a triangle. You may wonder where experimental data are in that setup, but as the paper (correctly) notes, the data themselves are mute. This essay is about what we do with the experimental results, what we make of them (the authors return to experiment in more detail in the third section). One of their concerns is that the current wave of AI hype can along the way demote theory (a true endeavor of human reason if ever there was one) to “that biased stuff that people fumbled along with before they had cool software”.

Understanding is a state of mind, and one good test for it is whether you have a particular concept or subject in mind well enough, thoroughly enough, that you can bring another person up to the same level you have reached. Can you teach it? Explain it so that it makes sense to someone else? You have to learn it and understand it inside your own head to do that successfully – I can speak from years of experience on this very blog, because I’ve taught myself a number of things in order to be able to write about them.

My own example of what understanding is like goes back to the Euclid’s demonstration that the number of primes is infinite (which was chosen for this purpose by G. H. Hardy in A Mathematician’s Apology). A prime number, of course, is not divisible by any smaller numbers (except the universal factor of 1). So on the flip side, every number that isn’t prime is divisible by at least one prime number (and usually several) – primes are the irreducible building blocks of factors. Do they run out eventually? Is there a largest prime? Euclid says, imagine that there is. Let’s call that number P – it’s a prime, which means that it’s not divisible by any smaller numbers, and we are going to say that it’s the largest one there is.

Now, try this. Let’s take all the primes up to P (the largest, right?) and multiply them together to make a new (and rather large) number, Q. Q is then 2 · 3 · 5 · 7 · 11 · (lots of primes all the way up to) · P. That means that Q is, in fact, divisible by all the primes there are, since P is the last one. But what happens when you have the very slightly larger number Q+1? Now, well. . .that number isn’t divisible by any of the primes, because it’ll leave 1 as a remainder every single time. But that means that Q+1 is either divisible by some prime larger than P, or it’s a new prime itself (and way larger than P) and we just started by saying that there aren’t any such things. The assumption that P is the largest prime has just blown up; there are no other options. There is no largest prime, and there cannot be.

As Hardy says, “two thousand years have not written a wrinkle” on this proof. It is a fundamental result in number theory, and once you work your way through that not-too-complicated chain of reasoning, you can see it, feel it, understand that prime numbers can never run out. The dream is to understand everything that way, but only mathematicians can approach their subject at anything close to that level. Gauss famously said that if you didn’t immediately see why Euler’s identity (e^iπ +1 = 0) had to be true, then you were never going to be a first-rate mathematician. And that’s just the sort of thing he would say, but then again, he indisputably was one and should know.

Now you see where Thom is coming from. And where Hoffmann and Malrieu are coming from, too, because their point is that simulation (broadly defined as the whole world of numerical approximation, machine-learning, modeling, etc.) is not understanding (part two of their essay is on this, and more). It can, perhaps, lead to understanding: if a model uncovers some previously unrealized relationship in its pile of data, we humans can step in and ask if there is something at work here, some new principle to figure out. But no one would say that the software “understands” any such thing. People who are into these topics will immediately make their own prediction, that Searle’s Chinese Room problem will make an appearance in the three essays, and it certainly does.

This is a fine time to mention a recent result in the machine-learning field – a neural-network setup that is forced to try to distill its conclusions down to concise forms and equations. The group at the ETH working on this fed it a big pile of astronomical observations about the positions of the planet Mars in the sky. If you’re not into astronomy and its history like I am, I’ll just say that this puzzled the crap out of people for thousands of years, because Mars does this funny looping sewing-stitch-like motion in the sky, occasionally pausing among the stars and then moving backwards before stopping yet again and resuming its general forward motion across the sky). You can see why people starting added epicycles to make this all come out right if you started by putting the Earth at the center of the picture. This neural network, though, digested all this and came up with equations that, like Copernicus (whose birthday I share), puts the sun at the center and has both Earth and Mars going around it, with us in the inner orbit. But note:

Renner stresses that although the algorithm derived the formulae, a human eye is needed to interpret the equations and understand how they relate to the movement of planets around the Sun.

Exactly. There’s that word “understand” again. This is a very nice result, but nothing was understood until a human looked at the output. When we start to wonder about that will be the day we can really talk about artificial intelligence. And to Hoffman and Marlieu’s point about simulation, it has to be noted that some of those epicycle models did a pretty solid job of predicting the motion of Mars in the sky. Simulations and models can indeed get the right numbers for the wrong reasons, with no way of ever knowing that those reasons were wrong in the first place, or what a “reason” even is, or what is meant by “wrong”. And the humans that came up with the epicycles knew that prediction was not explanation, either – they had no idea why such epicycles should exist (other than perhaps “God wanted it that way”). They just knew that these things made the numbers come out right and match the observations, the early equivalent of David Mermin’s quantum-mechanics advice to “shut up and calculate“.

Well, “God did it that way” is the philosophical equivalent of dividing by 1: works every time, doesn’t tell you a thing. We as scientists are looking for the factors in between, the patterns they make, and trying to work out what those patterns tell us about still larger questions. That’s literally true about the distribution of prime numbers, and it’s true for everything else we’ve built on top of such knowledge. In part three of this series of essays, the authors say