The Onion Test for Scientific Explanations

TLDR: Good scientific explanations follow the Onion Test, and tell a story at some level of abstraction/simplification that points towards the complexities of deeper levels.

Prerequisites: None

There’s a failure in how many schools teach science. It’s probably true in other fields, such as history, but I’m most aware of it in the scientific context. The problem comes in three parts:

1. An authority (i.e. teacher or textbook) presents a story for how the universe works that’s simple and accessible to the student. Thanks to it being selected for understandability, the student easily learns that story.

2. There are complexities of reality that show up a while later that are incompatible with the simple story, and require the student to cary an unnatural model that becomes increasingly bloated with exceptions.

3. To understand why the exceptions exist, the student looks into the underlying physics of the system in question and is bamboozled by “how weird reality is” — or more to the point: how much it disagrees with their first, simple story.

To illustrate, consider the story that most children are told about in elementary school…

The Solar System

You know how the solar system works: The sun is at the center, and there are a bunch of celestial bodies like Earth that go around the sun in orbits. (Questions: What is “an orbit”? Is the pattern of planetary motion circular or elliptical?)

This monstrosity of an image is courtesy of NASA!

Imagine you’re a child looking at that image. If that’s all you have to go on, I suspect you’re going to wind up more confused in the long-run than if you’d been given no picture at all.

Let’s start with the obvious: things in that picture are are wildly not-to-scale, either in size or in position. The planets have their shadows in the wrong place, poorly indicating how night is about being in the planet’s shadow. And don’t get me started on the random galaxy+nebula+comet+lens flares. It’s crazy to me just how frequent pictures like that are, even from high-quality sources. It’s genuinely hard to find good illustrations of the solar system, and if you look at enough of the bad ones it gives the impression of a great conspiracy to undermine public understanding of space.

The reason that realistic depictions of the solar system are rare is because of its most important feature: our solar system is extremely empty. Like, it’s called space for a reason. When traveling at the speed of light it takes minutes to get between inner planets like Earth and Mars, hours to get between outer planets like Neptune and Uranus, and days to get from one side of the heliosphere to another. And it’s thanks to the vast distances between worlds that orbits even make sense as a thing to talk about, as we’ll explore in a moment. I generally think kids (and adults) should experience more simulations and interactive explainers, but if we must use pictures, I’d very much prefer if we used ones like like this:

Combined with this:

Both images courtesy of Michael Richmond

While less flashy, these images are far more real than almost all the pictures we find elsewhere. From them we get a feeling for the great void of empty space. Perhaps more importantly, pictures like these are a good jumping-off point for exploring two surprising properties: the solar system is extremely flat, but not perfectly flat, and the orbits are extremely circular, but not perfect circles (this second part is easier if Pluto is added, or more subtly with Mercury for a diagram of the inner system).

One of the “facts” that people learn about orbits (the trajectory/path of a celestial body) is that they’re not perfect circles, but actually ellipses. To some, this is “extremely obvious” because they see pictures like that first NASA picture. When viewed from an angle, circles appear elliptical, and many people mistakenly believe that the orbits of the planets are as eccentric as they naively appear in bad diagrams if we interpret them as viewed-from-above! But that misunderstanding aside: it should be clear to students that there’s a mystery in why the orbits of planets are almost exactly circular, but are better modeled as ellipses.

Let me be clear about my point: it’s more important for scientific explanations to highlight what’s true, even if it’s less clean and beautiful than a simple story; but the complexity of a good explanation should lurk in the background, provoking curiosity about the underlying generators of the phenomena without overwhelming the audience. Unfortunately, to get ideas across, teachers and other science communicators often gloss over the finicky reality, and thus stop their students from becoming curious for more.

Back to the object-level: to understand why planetary orbits are elliptical, it’s useful to introduce a mathematical model of a two-body system. If we assume Newtonian gravitation, then the paths of the bodies are naturally described by ellipses with one of their foci at the center of mass for the system.

When the two bodies have unequal masses, the center of mass can be inside one of them:

But notably, both bodies still co-orbit! This means it is technically incorrect to say that the Earth goes around the Sun, but rather the Earth goes around the center of mass of the Earth+Sun system.

Or does it…

In exploring these models, a good teacher will nudge students towards the question of what it looks like to model a three-body system, or perhaps an 9-body system, like if we model our planets and sun. Will they still form elliptical orbits?

Animation by Dnttllthmmnm

Nope! Even when we restrict ourselves to a two-dimensional plane, the orbital pattern of any gravitational system with more than two bodies is (in general) provably chaotic.

This means the orbits of the planets aren’t even ellipses! We can model the orbits as circles, or we can model them as ellipses; both models are pretty good, but neither are really right. And this doesn’t even get into the way in which Newtonian gravitation is also merely an approximation!

The point is not to deny the simple story; the planets do in fact go around the sun in paths that are approximately concentric circles. The point is to peel back the layers of simplification in a way such that each revealed layer hints at the complexity of the reality underneath.

(Exercises for the reader: Why don’t the planets fly off in wild trajectories like the three-body-problem suggests they should? Why do orbits (supposedly) only make sense to talk about in the context of vast distances between the planets? What does this imply about planets in binary star systems? Why are the planetary orbits in our star system so flat? What does this imply about the existence of aliens? Why is Newtonian gravitation wrong? (Responses in the comments are welcome. I’ll be happy to share my thoughts.))

The Onion Test

In the Rationality community there is a notion of an “Onion Test” for honesty. (Alas, it’s totally different than the onion test in genetics. 😔) An onion has layers. Each layer, being slightly-translucent, hides information about what’s beneath without hiding all of it.

When people get to know you better, or rise higher in your organization, they may find out new things, but should not be shocked by the types of information that were hidden. If they are, you messed up in creating the outer layers to describe appropriately the kind-of-thing that might be inside.

This is useful in personal/institutional integrity by maintaining a healthy balance between openness and privacy. Being deeply honest isn’t just about what is literally said, it’s about not giving the wrong impression. By trying to follow the onion test, we can become more honest even in the face of pressures against sharing our true beliefs.

It takes a bit of work to put sign-posts on your outer layer about what kinds of information are inside, and it takes more work to present those sign-posts in a socially smooth way that doesn’t raise unnecessary fears or alarms.  However, if you put in that work, you can safely get to know people without them starting to wonder, “What else is this person or institution hiding from me?”  And, if everyone puts in that work, society in general becomes more trustworthy and navigable.

Following the onion test is tricky, and not what happens by default. I think for most people there’s a feeling of “ugh, I don’t want my boss to know that I’m only doing this job for the money, better to simply pretend like I love doing my job” or more commonly still: no reflective awareness of the deceptions that they’re living day-by-day. It becomes even harder to maintain the semi-transparency in the presence of curious minds that want to peel back layers; ability to set and maintain boundaries is probably a prerequisite.

In this essay I’ve been trying to illustrate the way in which the onion test doesn’t just apply to being honest about personal/organizational details — it applies to being honest when communicating our model of the world. There’s a tendency for scientific explanations, particularly those aimed at children, to hide the fact that they’re oversimplified and have known flaws.

Explanations like “the planets move around the sun in elliptical orbits” fail the onion test because they don’t include a flag that says what kinds of complicated/weird stuff is generating that approximation. Instead, a better high-level story is “in our solar system the planets fall around the sun in conveniently circular paths.” By flagging that (1) this is only a claim about our particular system, that (2) the planets are falling, and that (3) the circular paths are convenient¹, this explanation communicates facts without being a curiosity stopper or implying there’s no deeper layer of detail.

Utopian Scientific Explanations

In Utopia there’s a lot of work put into having the right sort of scientific explanations. Particular emphasis is made not just about content, but also how well an explanation guides the listener to ask questions that peel back the narrative and poke at the underlying dynamics. To this end, scientific explanations in Utopia are normally checked for whether they pass the onion test.

This helps make particular jumps in the sciences smoother. In particular, Utopians find relativistic physics and quantum physics significantly less surprising, thanks to interesting phenomena being flagged early and often. But this smoothness extends throughout the sciences and engineering disciplines to help everything from evolutionary genetics to artificial intelligence to the reversibility of time.

Scientific explanations serve as a role-model for explanations in other contexts. As a result, there is less unnecessary confusion and false confidence in Utopia, as deeper complexities are more generally flagged. Combined with a more general willingness to say “I don’t know,” Utopia has a generally healthier epistemic commons.

Black hole visualization by NASA’s Goddard Space Flight Center/Jeremy Schnittman. Some NASA images are great!

1. To understand why the orbits of the planets are “conveniently circular” try modeling what happens in a system of planets with eccentric orbits.