Transcription
Is there a problem that just haunts you? It sits there in the dark corners. You know, twin prime conjecture, Riemann hypothesis, Goldbach conjecture. Twin prime—that sounds again. So, I mean, the problem is like, agreement hypothesis; those are so far out of reach.
You think so?
Yeah. There's no even viable strate—like, even if I activate all my all all the cheats that I know of in this like it, there's just still no way to get me to be um, like it's um, I think it needs a breakthrough in another area of mathematics to happen first and for someone to recognize that it that would be a useful thing to transport into this problem. So we we should maybe step back for a little bit and just talk about prime numbers.
Okay. So they're often referred to as the atoms of mathematics. Can you just speak to the structure that these uh atoms, the natural numbers, have? Two basic operations are attached to them: addition and multiplication. Um, so if you want to generate the natural numbers, you can do one of two things. You can just start with one and add one to itself over and over again, and that generates you the natural numbers. So additively they're very easy to generate: 1, 2, 3, 4, 5. Or you can take the prime—if you want to generate multiplicatively, you can take all the prime numbers: 2, 3, 5, 7, and multiply them all together. Um, and together that gives you all the the natural numbers except maybe for one. So there these two separate ways of thinking about the natural numbers from an additive point of view and a multiplicative point of view. Um, and separately they're not so bad. Um, so like any question about that natural number that only used addition is relatively easy to solve, and any question that only used multiplication is relatively easy to solve. Um, but what has been frustrating is that you combine the two together. Um, and suddenly you get this extremely rich—I mean, we know that there are statements in number theory that are actually undecidable. There are certain polynomials in some number of variables—you know, is there a solution in the natural numbers?—and the answer depends on on an undecidable statement um, like like whether um the axioms of of mathematics are consistent or not. Um, but um, yeah, but even the simplest problems that combine something multiplicative such as the primes with something additive such as shifting by two—separately we understand both of them well—but if you ask, when you shift the prime by two, do you can you get a—how often can you get another prime?—we—it's been amazingly hard to relate the two, and we should say that the twin prime conjecture is just that—it posits that there are infinitely many pairs of prime numbers that differ by two.
Yes.
Now the interesting thing is that you have been very successful at pushing forward the field in answering these complicated questions uh of this variety—like you mentioned the Green-Tao theorem. It proves that prime numbers contain arithmetic progressions of any length, right? Which is mind-blowing that you can prove something like that, right?
Yeah.
So, what we've realized because of this this this type of of research is that there's different patterns have different levels of uh indestructibility. Um, so, so what makes the twin prime problem hard is that if you take all the primes in the world, you know, 3, 5, 7, 11, so forth, there are some twins in there. 11 and 13 is a twin prime—a pair of twin primes—and so forth. But you could easily, if you wanted to, um redact the primes to get rid of to get rid of the um these twins—like the twins—they show up, and there are infinitely many of them, but they're actually reasonably sparse. Um, not—there's there's not—I mean, initially there's quite a few, but once you got to the millions, trillions, they become rarer and rarer, and you could actually just, you know, if if someone was given access to the database of primes, just edit out a a few primes here and there—they could make the conjecture false by just removing like 0.1% of the primes or something—just well chosen to to um to do this—and so you could present a censored database of the primes which passes all the statistical tests of the primes. You know that it obeys things like the prime number theorem and other facts about the primes but doesn't contain any twin primes anymore. Um, and this is a real obstacle for the twin prime conjecture. It means that any proof strategy to actually find twin primes in the actual primes must fail when applied to these slightly edited primes. And so it must be some very um subtle, delicate feature of the primes that you can't just get from like like aggregate statistical analysis.
Okay. So that's all.
Yeah.
On the other hand, arithmetic progressions has turned out to be much more robust. Um, like you can take the primes and you can eliminate 99% of the primes, actually, you know, and you can take take any 99% you want, and it turns out—and another thing we prove is that you still get arithmetic progressions. Um, arithmetic progressions are much, you know, they're like cockroaches of arbitrary length.
Yes.
That's crazy. I mean, so, so this for for people who don't know, arithmetic progressions is a sequence of numbers that differ by some fixed amount.
Yeah.
But it's again like it's an infinite monkey type phenomenon—for any fixed length of your set, you don't get arbitrary progressions. You only get quite short progressions. But you're saying twin prime is not an infinite monkey phenomenon. I mean, it's a very subtle monkey.
It's still an infinite monkey phenomenon. Yeah. If the primes were really genuinely random, if the primes were generated by monkeys, um then yes, in fact, the infinite monkey theorem would—
Oh, but you're saying that twin prime is—it doesn't—you can't use the same tools—like the—it doesn't appear random almost.
Well, we don't know. Yeah, we we we we believe the primes behave like a random set. And so the reason why we care about the twin prime conjecture is it's a test case for whether we can genuinely confidently say with with 0% chance of error that the primes behave like a random set.
Okay.
Random—yeah, random versions of the primes we know contain twins um at least with with 100% probability or probability tending to 100% as you go out further and further. Um, yeah, so the primes—we believe that they're random. Um, the reason why arithmetic progressions are indestructible is that regardless of whether your set looks random or looks um structured—like periodic—in both cases um arithmetic progressions appear, but for different reasons. Um, and this is basically all the ways in which the—there are many proofs of of these sort of arithmetic progression theorems, and they're all proven by some sort of dichotomy where your set is either structured or random, and in both cases you can say something, and then you put the two together. Um, but in twin primes, if if the primes are random, then you're happy, you win. But if your primes are structured, they can be structured in in a specific way that eliminates the the twins. Uh, and we can't rule out that one conspiracy—and yet you were able to make a, as I understand, progress on the bounded version, right?
Yeah.
So, um, the one funny thing about conspiracies is that any one conspiracy theory is really hard to disprove—that you know, if you believe the world is run by lizards, you say, "Here's some evidence that that it's not run by lizards," well, but that that evidence was planted by the lizards.
Yeah. You may have encountered this kind of phenomenon.
Yeah. So so like like um a pure like there's there's almost no way to um definitively rule out a conspiracy—and the same is true in mathematics—that a conspiracy is to solely devoted to eliminating twin primes, you know, like it would—you would have to also infiltrate other areas of mathematics to sort of—but but like it could be made consistent, at least as far as we know. But there's a weird phenomenon that you can make one um one conspiracy rule out other conspiracies. So you know, if the if the world is is run by—this can also be run by aliens, right?
Right.
So one unreasonable thing is is is is hard to dispute, but but more than one—there are there are tools. Um, so yeah, so for example, we we know there's infinitely many primes that are um no two which are um—so there are infinite pairs of primes which differ by at most um 246, actually, is is the code.
Oh, so there's like a bound, yes, on the difference.
So like there's twin primes—there's a thing called cousin primes that differ by by four. Um, there's called sexy primes that differ by six.
What are sexy primes?
Primes that differ by six. The name is much less—the concept is much less exciting than the name suggests.
Got it.
Um, so you can make a conspiracy rule out one of these, but like once you have like 50 of them, it turns out that you can't rule out all of them at once. It just—it requires too much energy somehow in this conspiracy space.
How do you do the bound part? How do you how do you develop a bound for the difference between the primes that—okay, so—um—that there's an infinite number of—so it's ultimately based on what's called the pigeonhole principle. Um, so the pigeonhole principle, uh, it's the same—that if you have a number of pigeons and they all have to go into into pigeonholes, and you have more pigeons than pigeonholes, then one of the pigeonholes has to have at least two pigeons there. So there has to be two pigeons that that are close together. So for instance, if you have 100 numbers and they all range from one to 1,000, and um, two of them have to be at most 10 apart.
Mhm.
Because you can divide up the numbers from one to 100 into 10 pigeonholes. Let's let's say you have 100—if you have 101 numbers, 101 numbers, then two of them have to be distance less than 10 apart because two of them have to belong to the same original. So it's a basic um basic principle in mathematics. Um, so it doesn't quite work with the primes regularly because the primes get sparser and sparser as you go out—that that few and fewer numbers are prime. But it turns out that there's a way to assign weights to the to to numbers—like um, so there are numbers that are kind of almost prime, but they're not—they they don't have no factors at all other than themselves and one, but they have very few factors. Um, and it turns out that we understand almost primes a lot better than we understand primes. Um, and so for example, it was known for a long time that there were twin almost primes. This has been worked out. So almost primes are something we can understand. So you can actually restrict attention to a suitable set of almost primes and u—whereas the primes are very sparse overall—relative to the almost primes, actually, are much less sparse. They may—you can set up a set of almost primes where the primes have density like say 1%. Um, and that gives you a shot at proving by applying some pigeonhole principle that that there's pairs of primes that are just only 100 100 apart. But in order to prove the twin conjecture, you need to get the density of primes inside the almost primes up to up to a threshold of 50%. Um, once you get up to 50%, you would get twin primes. But uh, unfortunately, there are barriers. Um, we know that that no matter what kind of good set of almost primes you pick, the density of primes can never get above 50%. It's called the parity barrier. Um, and I would love to find—yes, so one of my long-term dreams is to find a way to breach that barrier because it would open up not only the twin prime conjecture but the Goldbach conjecture—and many other problems in number theory are currently blocked because our current techniques would require improvements going beyond this theoretical um parity barrier—it's like it's like pulling past the speed of light.
Yeah.
So we should say—a twin prime conjecture—one of the biggest problems in the history of mathematics—Goldbach conjecture also—they feel like next-door neighbors. Uh, is there been days when you felt you saw the path?
Oh, yeah. Um, um, yeah, uh, sometimes you try something and it works super well. Um, you you—again—again the sense of mathematical smell uh we talked about earlier—uh, you learn from experience when things are going too well because there are certain difficulties that you sort of have to encounter—um—I think the way a colleague might put it is that um, you know, like if if you are on the streets of New York and you put in a blindfold and you put in a car and and um after some hours um you—the blindfold is off and you're in Beijing—you know, I mean, that was too easy somehow—like like there was no ocean being crossed. Even if you don't know exactly what how what what was done, you're suspecting that that something wasn't right.
But is that still in the back of your head—to do—you return to these—to the prime—do you return to the prime numbers every once in a while to see?
Yeah. Yeah, when I have nothing better to do, which is less and less than 10 times—which is—I get busy with so many things these days. But yeah, when I have free time and I'm not and I'm too frustrated to to work on my sort of real research projects and I also don't want to do my administrative stuff. I don't want to do some errands with my family. Um, I can play with these these things um for fun. Uh, and usually you get nowhere. Yeah, you have you have to learn to just say, "Okay, fine—once again, nothing happened. I will I will move on." Um, yeah, very occasionally one of these problems I actually solved—or sometimes, as you say, you think you solved it, and then you're euphoric for uh maybe 15 minutes, and then you think, "I should check this because this is too easy—too good to be true," and it usually is.
What's your gut say about when these problems would be uh solved—twin prime and Goldbach prime?
I think we'll keep getting keep getting more partial results. um, it does need at least one—this parity barrier is is the biggest remaining obstacle. Um, there are simpler versions of the conjecture where we are getting really close. Um, so I think we will—in 10 years we will have many more much closer results—may not have the whole thing. Um, yeah, so twin prime is somewhat close.
Yeah.
Riemann hypothesis—I have no—I mean, it has to happen by accident, I think.
So the Riemann hypothesis is a kind of more general conjecture about the distribution of prime numbers.
Right.
Right. Yeah. It's it's states are sort of viewed multiplicatively—like for questions only involving multiplication—no addition—the primes really do behave as randomly as as you could hope. So there's a phenomenon in probability called square root cancellation that um, you know, like if you want to poll say America on on some issue um and you you ask one or two voters and you may have sampled a bad sample and you get you get a really imprecise um measurement of of the full average, but if you sample more and more people, the accuracy gets better and better, and it actually improves like the square root of the number of people you you sample. So yeah, if you sample a thousand people, you can get like a two or three percent margin of error. So in the same sense, if you measure the primes in a certain multiplicative sense, there's a certain type of statistic you can measure. It's called the Riemann zeta function, and it fluctuates up and down. But in some sense, um, as you keep averaging more and more, if you sample more and more, the fluctuation should go down as if they were random. And there's a very precise way to quantify that. And the Riemann hypothesis is a very elegant way that captures this. But um, as with many other ways in mathematics, we have very few tools to show that something really genuinely behaves like really random. And this is actually not just a little bit random, but it's it's asking that it behaves as random as a actually random set. This this square root cancellation—and we know because of things related to the parity problem, actually—that most of our usual techniques cannot hope to settle this question. Um, the proof has to come out of left field. Um, yeah, but uh what that is—yeah, no one has any serious proposal—um, yeah, and and there's there's various ways to sort of—as I said, you can modify the primes a little bit, and you can destroy the Riemann hypothesis—so like it has to be very delicate—you can't apply something that has huge margins of error—it has to just barely work—and like um there's like all these pitfalls that you have to like dodge very adeptly—the prime numbers are just fascinating.
Yeah.
Yeah.
Yeah. What—what to you is um most mysterious about the prime numbers?
So that's a good question. So like conjecturally we have a good model of them. I mean, like as I said, I mean, they have certain patterns—like the primes are usually odd, for instance—but apart from the sorts of obvious patterns, they behave very randomly, and just assuming that they behave—so there's something called the Cramér random model of the primes that that after a certain point primes just behave like a random set. Um, and there's various slight modifications of this model, but this has been a very good model. It matches the numerics. It tells us what to predict. Like I can tell you with complete certainty the twin prime conjecture is true. Uh, the random model gives overwhelming odds that it's true. I just can't prove it. Most of our mathematics is optimized for solving things with patterns in them. Um, and the primes have this anti-pattern—as do almost everything, really. But we can't prove that.
Yeah.
Yeah, I guess it's not mysterious that the primes be kind of random because there's no reason for them to be um uh to have any kind of secret pattern. But what is mysterious is what is the mechanism that really forces the randomness to happen. Uh, and this is just absent.