How do scientists know if something is true?
(8.5 minute read)
This is an important question to ask. We should not simply believe everything we hear. We should believe something because we have confidence that the person we hear it from knows what they’re talking about, and that they’re not lying to us. So let’s dig in: how do scientists know if something is true, and how do we know that they’re not lying?
The world of science has overlapping systems in place to make sure that its information is reliable.
From the very beginning, scientists are trained to question their own theories, and to put them through rigorous testing. When a scientist comes up with an idea, her first thought is probably: this is a really good idea! But then her next thought is: how would I know if it’s not true? And what are the things I could to do to prove myself wrong? And if she actively tries to prove herself wrong and she still has confidence in her idea, then it’s a good one. “Rigorous testing” involves attacking the idea from many different angles. When a chemist makes a new molecule, he’ll test the molecule many different ways. He’ll put it in machines that can analyze its structure, he’ll break it down into its elements, he’ll find its melting and boiling points. Then he’ll put all that information together to say with confidence, “This is the molecule that I made.” And at the same time he might make a computer model of this molecule and do some calculations. If the model matches his observations, he can have even more confidence in his result.
Scientific thinking involves questioning: how do I know if I’m right, and how can I test if I’m right in many different ways? But that’s at the individual level. Once a scientist thinks she’s right about something, then she’ll want to publish her work so other scientists can see what she’s done. Before the work can be published, it has to be checked by other experts in the same field. This is the process called peer review, where independent evaluators read the scientist’s description of her methods, her data, and her conclusions. And because they are experts in the same field, they’ll know if the scientist used the proper methods and did the proper testing, or if she needs to go back and do more testing. They will also be able to determine whether the conclusions that person drew from her data were well founded.
After these independent experts have scrutinized the scientist’s work, then it can be published. That means it’s put into a journal (a periodical for scientists in a specific field of study), so that more scientists can read the new research. Now the author’s data are a matter of public record. Any other scientist can read them and draw their own conclusions. And believe me, other scientists are a tough audience, because they are trained to question other people’s theories as well as their own. Maybe the other scientists who read the paper will have a different interpretation of the results, or maybe they’ll connect the author’s ideas to things that they themselves have been studying. And then they can engage each other in discussion and debate, whether through e-mail or by meeting up at scientific conferences. These conversations can only improve the ideas that were published. Good ideas will be developed more sharply, and bad ideas will fall to the wayside. Even scientists who use good methodology can get things wrong. Perhaps they were missing some knowledge outside of their expertise, or maybe the technology they were using hadn’t evolved enough yet. Whatever the reason, conclusions that don’t stand up to other scientists’ scrutiny will eventually be discarded or changed.
All those layers of scientific integrity help us have confidence that something is true. And we can also be confident that the scientific community does not willfully lie on a regular basis because scientists who lie are ostracized. In a graduate class I was taking, the professor once mentioned a chemist who was caught fabricating data. He followed up this story with, “Nobody talks to him at conferences.”[1]
Seriously, this is a community that values truth above all else. That’s why they became scientists! They want to know things about the world beyond the shadow of a doubt. And so they care very much about scientific integrity in themselves and in others. Scientists are sticklers for hard evidence because their reputations depend on it.
There are always a few outliers who are willing to stretch the truth to get grant money or fame. Or maybe they didn’t lie but just made a mistake. But the scientific community will always sort out lies and mistakes by checking each other’s work and debating the results.
. . .
I have a couple examples of this process at work. Here’s one.
In 1989, a two-scientist team, men by the names of Martin Fleischmann and Stanley Pons, announced that they had achieved cold fusion. Now if those words don’t mean anything to you, here’s the short version. Fusion is the nuclear reaction that powers the sun. The sun is really, really hot – we’re talking millions of degrees Fahrenheit – so the atoms inside it are flying around at extremely high speeds. So high that when these atoms collide, their nuclei can smash together and “fuse” into one atom. Typically it’s hydrogen atoms that fuse into helium atoms. When that happens, the newly formed nucleus is unstable, so it will release energy and particles (usually neutrons) until it becomes more stable. If we could do the same reaction on Earth without the 27,000,000° F requirement, we would have a powerful, clean energy source that produces no radioactive waste. So naturally “cold fusion” was (and still is) a subject of interest to many researchers.
These two guys built a special apparatus they hoped would make cold fusion possible, and they ran some experiments. They detected that heat was being released in the reaction vessel, and their calculations suggested that the amount of heat produced was large enough to prove that fusion was happening! But these were preliminary results. They were planning to do more testing, but then they heard that another researcher (Steven E. Jones) was getting ready to publish his own attempts at cold fusion. So Pons and Fleischmann panicked, because the first to publish gets credit for the discovery. They rushed out a manuscript in five days. The editor of the journal decided this paper was exciting and important enough to shorten the review process to one week, a fraction of the usual time. The evaluators didn’t catch several calculation errors that had to be corrected later.
But even this wasn’t fast enough for Fleischmann and Pons. Their paper hadn’t been published yet when they held a press conference and announced: “We have made fusion happen at room temperature! Get ready for a global energy shakeup.” (I’m paraphrasing.)
And so of course this was very exciting, because nobody had thought it was actually possible. When scientists all around the world got the news, they wanted to see for themselves. So they replicated Fleischmann and Pons’ reaction setup as best they could, but the results were patchy and inconclusive. So this claim had to be tested other methods. And it turns out, when scientists looked for neutrons or helium produced by the reaction, both came up negative. So really, it was a dud. No fusion at all.
Within a year after the press conference in 1989, it had been definitively shown that Pons and Fleischmann’s claims were unfounded. Normally the peer review process would do a better job of filtering out these bad results. In this case, the lure of fame and the excitement of such a potentially large breakthrough led to a rushed writing and peer review process. Mistakes were made. Instead, it was up to the scientific community to check the results. And in the end, it was shown that the results were false.[2]
. . .
Here’s another example of sorting truth from untruth.
Scientists have been discussing climate and the forces that shape it for 200 years. They have been recording global temperatures since 1850. And they have been measuring the concentration of carbon dioxide in the atmosphere since 1960. Those are hard data that show a clear trend: an increase in global temperatures, and a corresponding increase in CO2 concentration in the atmosphere.[3]
In addition, scientists have studied ice cores that date back 800,000 years. The crystal structure of the ice, the materials trapped in it, and the relative abundance of its isotopes can tell us a lot about the conditions in which it was formed. Rocks and fossils can give clues about climate that go back hundreds of millions of years. They tell us where there used to be glaciers, rain forests, and lakes where there currently are none. All this information gives us a pretty good understanding of how Earth’s climate has changed over very long periods of time.
Scientists also make computer models of the climate using all of this information. These models allow them to see what major forces shape our climate, and how things might be different if one variable or another were removed.
And the results show the same thing, over and over again. The earth is warming at a much faster rate than usual, mostly due to the rapid increase in carbon dioxide, methane, and nitrous oxide in the atmosphere (the greenhouse gases). The increase in greenhouse gases is due solely to human activities such as deforestation and burning fossil fuels.
There have been so many peer-reviewed papers that have come out on this topic that the evidence is overwhelming. At least 18 scientific organizations (professional associations, government agencies, etc.) have posted in clear terms on their websites a statement that says: the climate is warming, and this warming is caused by humans.[4]
That’s what scientific consensus looks like. And when that kind of consensus is achieved, we can be confident that something is true.
[1] Prof. Patrick J. Walsh, University of Pennsylvania. Fall of 2011. Organometallics lecture.
[2] For a more detailed account of this very interesting story, check out: https://undsci.berkeley.edu/article/cold_fusion_01
[3] See https://www.epa.gov/climatechange-science/causes-climate-change
[4] See a sampling of these organizations at https://climate.nasa.gov/scientific-consensus/
