Transcription
Let’s get started. So, what are we looking at? Anyone? Mice, right? Rats?
These mice were in a trial that was looking at tumor suppression for multiple myeloma. This was done at the Dana-Farber Cancer Institute, which is here in Boston. It’s the Harvard sort of preeminent cancer institute—maybe in the world.
What’s really interesting is that this top line is the control group, and the bottom line is the intervention group. You can see as the cancer progresses, it lights up in the mice. But what’s great, theoretically, is that by day 16 of the intervention group, the mice look a lot like day one, as though the tumors hadn’t grown.
They published this in Blood, which is the high-impact journal where you’d want to publish this kind of research. Great, right? Maybe not.
Do these images look similar? Now, if you notice in the bottom left corner, there’s a little fragment that’s identical in both images. I don’t know if you’d be able to get mice to position themselves perfectly, but that’s the real tell in this.
Just to be clear, this animation helps to show what was done—it’s the same image. They took the image from day one of the control group, copied and pasted it into day 16 of the intervention group, and said, “Look, we have tumor suppression!”
Now, there’s an internet SLO—a group at Broken Science—who are all sort of obsessed with looking for image manipulation, duplication, or any kind of fraudulent thing that can be done with an image. They don’t get paid to do this; they’re just really good at looking at images and using technology. Sometimes they use AI, but with something like this, you don’t even need AI. You can see it with the human eye.
This guy, Sholto David, who we’ve been in touch with, started looking at these images from Dana-Farber and flagged 58 papers. That’s just one that he said had image manipulation.
Now, I should pause for a second—my slides are really detailed. You don’t have to read them. I include a lot of information because so much of what I present is incredible to people and hard to believe. I’ll make my slides available to anyone, so you can fact-check this on your own. There’s a lot of information, but it’s not necessarily important to the gist of what we’ll talk about.
Sholto David tried really hard to sound the alarm. He did all the things you’d want to do: he contacted the authors, Dana-Farber, and the medical journals, saying, “You guys, somebody needs to look at this—this doesn’t look quite right; it looks like there was manipulation.” He got no response from anybody.
So, he took matters into his own hands and decided to publish on For Better Science, a blog. It’s a great blog—if you’re into this kind of content, I highly recommend it. This got some attention.
Interestingly, it didn’t get attention from the mainstream media until The Harvard Crimson—the student newspaper—broke the story. I don’t know if you remember, but Stanford’s president was accused of scientific misconduct, and that was also broken by their student newspaper. So, this happens.
The Crimson publishes it, other news outlets pick it up, and Dana-Farber says, “Oh, well, we’re doing a big investigation; we take this very seriously.” Let’s look at the investigation. The people being investigated are the heads of the hospital.
Lori Glimcher is the CEO of Dana-Farber. She’s the author of four of those papers. William Hahn is the COO, and he’s on 19. Griel is on 14. And then there’s Ken Anderson, who’s on 30 and is a professor at Harvard Medical School, teaching students how to do research. My favorite is Barrett Rollins, who is named on three papers. He’s the advisor to the president and their Research Integrity Officer. So, we have the Research Integrity Officer as the author of three flagged papers.
I include this because there are obviously many authors on these papers, but it’s important to see that many of them are people in the most powerful positions. Oftentimes, if it’s your lab, you’re named on a paper, and that becomes important later in this investigation.
The mainstream media starts picking up the story, saying the investigation is happening. But people start pushing back, saying, “How can Rollins be leading this investigation? He’s on three of the papers!” Rollins says, “You’re right. I’ll recuse myself from the three papers—not all of them, just the ones I wrote. I probably can’t evaluate myself for fraud.” These are the three papers.
However, this is an org chart at Dana-Farber. Now, we have Rollins with direct relationships with the other executives, including his boss. Is he not compromised in a way that would allow him to lead a thorough, unbiased investigation?
We started digging into this because we love this kind of stuff. Bob and Russell Berger have been doing great work looking at the papers. We’ve been in touch with Sholto David. As we uncovered this, I thought, “What does the federal government say?” We identified about $30 million of funding that went to some of these studies—our money. Why isn’t the government saying, “You committed fraud; we want our money back or a real investigation?”
Sure enough, the federal research policy is clear—you can’t have even the illusion of a connection between investigators and those being investigated. But then we found a clip of Lori Glimcher and Barrett Rollins. She basically says, “He’s my bestie. He’s been my loyal friend for 30 years.” That’s who will lead your investigation into whether fraud happened?
There’s also this issue of intellectual honesty. Science—and especially medicine—is at a critical point where nobody has complete confidence. There’s not a lot of credibility right now in the medical field. You’d think, just for your own reputation, you’d want to outsource this investigation and ensure it’s done thoroughly so people aren’t questioning whether it’s legitimate or not. But they’re not doing that.
In fact, Rollins did this interesting sleight of hand with the media. He started couching everything with intent, saying things like, “We don’t know if there was any intent. Maybe someone in the lab made a mistake. You can’t crucify people for that.” Intent matters in a criminal investigation—but not here. The federal government is clear: they don’t care about intent. If it’s reckless—meaning you’re running a lab and don’t know what’s going on, or you’re an author on a paper and don’t know what your co-authors are doing—that counts too. But nobody in the mainstream media bothered to check the policy, so the narrative became, “Maybe it was an accident,” and the story went away.
We didn’t drop it. Russell has been emailing them, asking if the investigation is still happening. They keep responding with, “We can’t give you any information about the investigation.” In his most recent correspondence—well, it might not be the most recent because knowing Russell, he’s still working on it—but in April, he emailed again, asking if they’re still investigating and who’s heading it. Their response? “No, sorry, we can’t tell you anything. We can’t even tell you if we’re still doing it.”
Meanwhile, Sholto David has identified even more of these papers. So, the question becomes: Is this an outlier? Because this is crazy, right? How can this happen at a Harvard-affiliated, preeminent teaching hospital? I hate to say it, but it’s not an outlier.
A couple of years ago, Science magazine did an incredible investigation into amyloid-beta, foundational work done in Alzheimer’s research. They brought in their own image experts after internet sleuths flagged issues. This is a great story—it’s on brokenscience.org. If you like true crime, you’ll love it. It’s the same thing: manipulated images. This work was done in 2006, foundational research everyone else has been building upon as though it was true—but it was fake. The damage isn’t just in that one experiment; it’s in all the subsequent work based on it.
When I was first putting this talk together in June, the story in Science had already been out for two years. The original journal hadn’t retracted the paper yet. Instead, there was a note saying, “We’re looking into it.” Two years—how much more looking do you need? Around that time, the lead author came forward, saying, “We did mess with those images, actually. But here’s the thing: now we’ve been able to replicate it, so we were right! Even though our experiment didn’t show it, we knew it was the right thing, so we shouldn’t retract the paper.” That level of cognitive dissonance is mind-blowing. You should never be allowed to do research again after admitting that. Under pressure, the journal eventually retracted the paper, but it took years.
This kind of thing happens all the time. There’s a website called PubPeer where people crowdsource the analysis of research papers, often exposing duplicated or photoshopped images. Elizabeth Bik is a personal hero of mine. She’s broken many stories herself by identifying image manipulation. What’s funny is that, decades ago, people didn’t anticipate we’d develop AI that could mathematically match images. People who thought they were safe are now getting busted as technology catches up.
Here’s an example Greg and I love: Amgen, Begley, and Ellis tried to replicate 53 hallmark studies in hematology and cancer. They could only replicate 11. And they went to great lengths—they recreated the labs, recruited the original scientists, and accounted for every variable they could. Despite that, they couldn’t replicate most of the studies. To get the scientists’ buy-in, they promised to anonymize the findings and never reveal whose studies failed. So, those unreplicable studies remain in the literature as though they’re valid. Not one scientist came forward to say, “Hey, my study can’t be replicated; we should retract it.”
Meanwhile, another high-profile scandal out of Dana-Farber involves six new papers. Expect Mirnezami resigned. In other news, an investigation revealed that over 17,000 Chinese papers contained fabricated data, and the president of a university stepped down in the wake of a fraud scandal. This isn’t uncommon—it’s way too common.
Let’s talk about peer review because it’s supposed to be a gatekeeper. Someone submits research, and peer review is supposed to catch issues like duplicated images. But as a former journalist, I can tell you: peer review has none of the standards good journalism has. In journalism, if I write a story and submit it to The New York Times, they pay me. If I pay them, it’s called advertising. In peer review, researchers often have to pay to have their manuscripts published. There’s an inherent bias in that system.
There are no fact-checkers. When I did investigative journalism, fact-checkers spent months verifying everything—calling sources, rerunning figures. That doesn’t happen in peer review. No one reruns the figures. The peer reviewers are anonymous. Imagine a magazine or newspaper with no masthead, no accountability. That’s peer review.
Then, there’s the almighty p-value. The p-value is a statistical metric that’s supposed to measure significance. It’s a requirement for most studies. But the p-value doesn’t tell you whether results are valid, replicable, or even if the hypothesis was tested correctly. It’s just telling you something about the data and the comparison between groups. It’s flawed as a metric for good research.
Over time, the problems with peer review have worsened, and we now have things like “paper mills.” These are foreign companies mass-producing research papers—some of which were never even conducted—and getting them published in high-impact journals. These fake papers are getting cited and used, and we have no idea if the results are valid.
But you don’t have to take my word for it. Listen to what the editors of the world’s top medical journals have to say: Richard Smith, former editor of the British Medical Journal, said, “Medical journals are an extension of the marketing arm of pharmaceutical companies.” John Ioannidis from The Journal of the American Medical Association said, “Most published research findings are false or exaggerated, and an estimated 85% of research resources are wasted.” Marcia Angell, former editor of the New England Journal of Medicine, said, “It is simply no longer possible to believe much of the clinical research that is published or to rely on the judgment of trusted physicians or authoritative medical guidelines. I take no pleasure in this conclusion, which I reached slowly and reluctantly over my two decades as editor.” Richard Horton, editor-in-chief of The Lancet, said, “Much of the scientific literature, perhaps half, may simply be untrue. Afflicted by studies with small sample sizes, tiny effects, invalid exploratory analyses, and flagrant conflicts of interest, together with an obsession for pursuing fashionable trends of dubious importance, science has taken a turn toward the dark side.”
These aren’t random people—they’re the editors of the most prestigious journals in the world. They’re sounding the alarm, and yet nobody is reforming the system. If they can’t fix it, who will?
So, how did we get here? Let’s talk about money. If you look at how much money the pharmaceutical industry is pumping into Congress, you’ll start to understand. It’s more than any other industry. Over time, their spending has only increased. The revolving door between government and pharma is also a big problem.
And it’s not just government—pharma is using the same playbook with the media. Pharma is the biggest advertiser in the U.S., by far. The American Medical Association and others have raised concerns that we’re one of only two countries in the world where pharmaceutical companies can advertise directly to consumers. But I don’t think the ads themselves are the real problem. I don’t think most people see a drug ad on TV and immediately run to their doctor asking for it. The bigger issue is the influence pharma has on the media through their massive advertising budgets. That’s where the bias comes in.
Before Broken Science, I spent a lot of time looking at women’s health—which is essentially a dumpster fire of science. If you want to see a branch of medicine riddled with bad science and poor validation, look at women’s health. Women have different hearts, lungs, brains, and bodies, all managed by a different endocrine system than men. Yet women are grossly underrepresented in clinical trials. Two-thirds of Alzheimer’s patients are women, yet 90% of subjects in Alzheimer’s drug trials are men. It’s only been a couple of decades since medical schools started using female cadavers. That’s how archaic this is. Why? Because of hormones. Researchers have long said women are “too complicated” to study because of hormonal fluctuations. So, they just studied men and assumed the findings would apply to women. Think about this: when you take your dog to the vet, they prescribe medication based on the dog’s weight. Yet a 100-pound woman with a completely different biological system gets prescribed the same dosage as her 200-pound husband. That’s how oversimplified it is.
One tragic example is the drug thalidomide, prescribed to pregnant women for morning sickness. It caused severe birth defects. In response, Congress essentially banned women of childbearing age (13 to 60) from clinical trials from 1977 to 1993. That’s decades where women weren’t studied.
I like this chart because it shows the prevalence of diseases and the gender breakdown of clinical trials. You can see women are underrepresented in almost every category, despite their unique biology. This is a classic example of broken science—failing to account for basic biological differences. These are conditions where women present symptoms differently, yet they’re treated the same as men. This is systemic, and it’s why we say science is broken.
We need to wake up and accept that there is pervasive scientific misconduct. The incentives are wrong, and much of the research lacks rigor—if the research was even conducted at all. Interestingly, most of these issues are in the realm of chronic disease. But chronic diseases can often be prevented or reversed through lifestyle choices. We don’t necessarily need to rely on an academic system that has gotten so much wrong.
This map may look familiar to some of you. It’s a map of CrossFit affiliates worldwide when Greg was running CrossFit. These were little hubs where people were preventing and reversing chronic disease—not in the medical system, but on their own. Greg recognized he had about 20,000 doctors working out in CrossFit gyms. They knew they had a solution, so he mobilized them to spread his methodology. What we’re doing now is carrying that message forward—empowering people to take control of their health, think critically, and feel better in their bodies.
This is Greg’s methodology pyramid, and at the base is nutrition. That’s how important it is. Metabolic dysfunction starts with what you’re eating—sugar and carbs. Anyone trying to tell you otherwise either doesn’t understand or is trying to sell you something. Verta Health has been reversing type 2 diabetes in nearly 80% of participants for years—not for just three months, but for years—using dietary intervention.
The evolution of this revolution has led Greg and me to take a deep dive into the philosophy of science. How did science break? How did we get here? The philosophy of science had an undue influence on the practice of science. Philosophers—who weren’t scientists—were dictating what science should be. But there’s a pantheon of scientists who recognized these problems and wrote about them. We’re promoting their works because we think they got it right.
Greg came up with one of our taglines: “When consensus replaces predictive value, science becomes nonsense.” That’s exactly where we are now—a consensus-driven system with no predictive value. CrossFit revolutionized the fitness space by defining terms and requiring measurements. These concepts represent the best of science in any field. We’re now using that same rubric to help people in healthcare—teaching them what research should look like and how to read it critically.
We have Bob running a Journal Club where, every six weeks, he rips apart studies. It’s a fantastic education on how to spot statistical manipulation and inaccuracies. We’re also getting into the fitness space by updating the nutrition protocol to focus on metabolic health and derangement. This will help coaches deliver meaningful information to their clients and truly change lives. Additionally, we’re focusing on education. It’s become clear that the school system isn’t teaching kids basic skills like logic and plausible reasoning—tools that are critical for deconstructing problems and thinking independently. Greg will talk more about this later. We need to arm ourselves with knowledge to avoid falling into the "disease economy."
What would it feel like if doctors could actually heal again? I think most doctors would say they don’t feel like they’re healing—they’re just treating. For patients, it’s the same. At the heart of this issue is predictive power. Are you making a guess about outcomes and then testing to see if you’re less wrong? Science isn’t about certainty; it’s about being uncertain and narrowing that uncertainty. But instead, we have a system focused on symptom-matching rather than treatment. You come into a doctor’s office, present a symptom, and the doctor thinks, “What can I prescribe for this symptom?” We need to go back to empowering doctors to actually heal.
Part of this problem is the conflation of public health and medicine. Public health is a political apparatus that studies populations, while medicine is about treating the individual in front of you. These are very different missions, but they’ve become blurred.
This next part is both humorous and sobering. Here’s how regulators thought of ivermectin: As dangerous or experimental. Here’s how scientists thought of it: As a medication with potential applications. Here’s how doctors thought of it: As a tool in their arsenal to help patients. And here’s how politicians told us to think of it: As something to ridicule or dismiss.
We see this with statins, too. Statins are a public health intervention, not individualized medicine. Greg often says public health is “herd management,” which encapsulates the issue perfectly. Here’s a staggering statistic: You need to treat 200 people with a statin for one person to benefit. And by “benefit,” they mean that person might live one month longer. Meanwhile, 199 people are exposed to the drug and its side effects, with no benefit.
Doctors are the only ones with any moral authority in this system. They take the Hippocratic Oath. But pharmaceutical companies, hospitals, insurance companies, and even academia are beholden to profit—not patients. If you have a fiduciary responsibility to turn a profit, there’s an incentive to keep people sick. Healing them means you lose a customer. The incentive structures are completely wrong. It used to be that doctors had the power to say, “No, I’m not doing that,” or, “12 minutes a year with my patients isn’t enough.” That power has been taken from them.
Now, let’s listen to the Hippocratic Oath. This was beautifully animated, and I want you to really hear the words: "By all that I hold highest, I promise my patients confidence, integrity, candor, personal commitment to their best interest, compassion, and absolute discretion and confidentiality within the law. I shall do by my patients as I would be done by. I shall obtain consultation whenever I or they desire, include them in all important decisions to the extent they wish, and minimize suffering whenever a cure cannot be obtained, understanding that a dignified death is an important goal in everyone’s life. I shall establish a friendly relationship with my patients, accepting each one in a non-judgmental manner, appreciating the validity and worth of different value systems, and according each person a full measure of human dignity. I shall charge only for my professional services and shall not profit financially in any other way as a result of the advice and care I render. I shall provide advice and encouragement for my patients in their efforts to sustain their own health. I shall work with my profession to improve the quality of medical care and public health, but I shall not let any lesser public or professional considerations interfere with my primary commitment to provide the best care available to each of my patients."
If doctors truly lived by this oath, we wouldn’t have the problems we see today. Right now, the state of medicine is dismal. Doctors are unhappy, burnout rates are high, and job satisfaction is at an all-time low. Why? Because the system is broken.
Polypharmacy is one example. Fifty-seven percent of women over 65 are on five or more medications. Many of these medications are prescribed to treat side effects caused by the first medication, creating a toxic cocktail in their bodies. What’s even more alarming is that there’s no research on what happens when you combine five different medications in one person. It’s a Wild West approach—completely untested. Polypharmacy leads to drug interactions, falls, and a shorter lifespan. Older adults who fall are often on multiple medications, and those falls can be devastating.
Here’s a lighthearted video that perfectly illustrates this problem: "With the arthritis, the ibuprofen wasn’t doing anything, so the doctor put me on prednisone. That gave me heartburn, so they gave me omeprazole. Then, that made me nauseous, so they added Ondansetron. For my stomach issues, they gave me metoprolol for the heart flutter, which made my legs swell. Then, they gave me furosemide, and that’s when I stopped walking much. My blood pressure shot up, so they added amlodipine…" And on and on it goes—a cascade of medications treating symptoms instead of addressing root causes. The irony is that most of these chronic illnesses could be prevented or reversed through diet and exercise. But that’s not the focus—it’s far easier to prescribe another pill.
So, what’s the solution? Education and empowerment. In this room, we have people who are living purposeful lives, exposing these issues, and spreading awareness. We’re lucky to have Greg Glassman here, who has spent three decades critically analyzing these issues. Before starting CrossFit, Greg was already questioning the system. He wrote a column for the Santa Cruz Sentinel, where he explored these problems. He even called Dr. Atkins to discuss nutrition. Greg has always had a deep understanding of predictive value and how to distinguish what works and what doesn’t from his work with clients.
We’re also honored to have Dr. Chris Palmer, a true hero, who’s making groundbreaking changes within the system. At McLean Psychiatric Hospital, he’s pioneering the brain energy model and advancing the field of mitochondrial health. This ties in beautifully with the work of Dr. Tom Seyfried, who views cancer as a metabolic disease, updating Otto Warburg’s theories. Mitochondrial health is becoming increasingly important. It’s not just the power center of the cell—it’s the power center of your health. We’re going to hear more about this today and, hopefully, in the future.
I also want to acknowledge Karen, who’s in the room with us today. She wrote a book two decades ago about sugar as an addictive substance and treated patients for sugar addiction long before it was popular. I’m trying to convince her to write an updated version of the book because her work was so ahead of its time. If you see her later, give her a nudge—it’s a message the world needs.
So, those are just a few of the incredible people here. I’m excited for the second half of this event because those networking connections are so important. Let me share something that really struck me—a powerful ad from a hospital. It emphasizes how small, seemingly insignificant choices can add up, not just for ourselves but for our children and our communities. [VIDEO] That ad always gets me emotional because it shows how our choices today impact our future and the future of those we love.
At Broken Science, we’re working on several projects to empower people and create change. Our redesigned David Stove book, a philosophical exploration of science, is coming out in October. We’re launching a math class, continuing our Journal Club, and introducing a new concept we’ll discuss later. All of this is accessible to anyone who wants to engage and learn. Our goal is simple: we want to arm people with the tools and knowledge to reclaim their health and escape the "disease economy." We say, “Freedom is useless if you’re not healthy.” If you were to create a freedom pyramid, health would be the base. What good are free speech, freedom of assembly, or any other rights if you’re in a hospital bed and unable to exercise them? Reclaiming health is about reclaiming autonomy and freedom.
This is the map of our tour and the other places we’ll be visiting. I hope you all sign up and continue this journey with us. That’s my talk. Thank you. (Applause)