June 20th, 2012Top Story How to Determine If A Controversial Statement Is Scientifically True By Alan Henry Every day, we're confronted with claims that others present as fact. Some are easily debunked, some are clearly true, and some are particularly difficult to get to the bottom of. So how do you determine if a controversial statement is scientifically true? It can be tricky, but it's not too difficult to get to the truth. Every internet user has developed a healthy dose of skepticism that keeps us from being duped by things that don't pass the smell test, but it's not enough to just think something might not be true. What if you think the statement might be true and you want to learn more? What if you want to respond to the assertion or engage in conversation but you don't know enough to do so? We sat down with experts Dr. Phil Plait, aka The Bad Astronomer, and David McRaney of You Are Not So Smart to figure out a working approach to discovering the truth of any statement, from obvious hoaxes (think Nigerian prince emails) to more difficult topics (think vaccine "controversies".) First, Learn to Avoid Confirmation Bias Before we get into what you should do when confronted with a statement you're curious about, the first thing you have to strip yourself of is confirmation bias. Says Dr. Plait: The biggest problem is one of confirmation bias: finding an answer you already believe. If someone has a question about a belief or opinion—say, that vaccines are dangerous—then when they look it up online they'll tend to be biased toward sites that have information they already agree with! This is a well-known effect, and is one reason some things, like anti-vaccination beliefs, are strong even in well-educated communities. The people are smart enough to look up and understand what they read, but perhaps not experienced enough in critical thinking to evaluate what they're reading without bias. So how do you beat back confirmation bias? "Even with experience, it's incredibly tough to do," Dr. Plait explained. First, be aware that confirmation bias exists, shake yourself of your natural tendency to draw a conclusion before you've researched a topic, and be open to information that falls on either side of a statement. Don't just demand someone else present studies that support their assertion—go looking for them yourself. This must-read by David McRaney tackles the topic of confirmation bias in detail, but in short: Keep an open mind, seek evidence to the contrary for every opinion (especially ones you believe), and don't treat your research like a crusade. Photo by Yi Chen. Your First Line of Defense: Search Google, Snopes, and Other Popular Web Sites Your first instinct when confronted with a statement that seems controversial is probably to hit Google and start looking around for more information on it. That's good: it's the first thing that Phil Plait and David McRaney both suggest you do as well. In fact, you can prove and disprove many statements and research other contentious journal articles just by doing a little legwork and seeing what the web has to say. "If you are skeptical of a claim or a factoid or a quote or whatever, try some Google-fu that includes the word "skeptic" or "hoax" or "bogus" or "rumor" or "urban legend" with your search term. If you are lucky, you might find existing discussions in the forums of skeptical societies or science groups," McRaney suggested. Dr. Plait suggested the same, but noted, "This is becoming less useful, though, as people on the, um, wrong side of reality have taken up the mantle of 'skeptics.' That's irritating. And you can't always trust experts, since sometimes their credentials are inflated, or they're simply wrong (I can name several Nobel laureates who have said probably wrong things about their own beliefs)." So if Google doesn't seal the deal, and you're still awash in more opinions than facts, what do you do? Here are a few good places to visit if you're on the hunt for good, unbiased information: Snopes: Snopes.com has been the rumor debunker of the internet for years, and despite the occassional claim against its own impartiality, it's an excellent resource. "The first place I go when I'm in a hurry is Snopes," McRaney said. "I'm often surprised at how quickly they cover a topic that is bouncing around in the echo chamber." He also noted that Snopes lists its sources, so you can always jump off and do more reading if you want more data. Wikipedia: It's not perfect, but Wikipedia does have a wealth of information, and in most cases, that information is sourced—which means that it's less useful for its own articles, and more useful for the sourced links to any evidence you find. Science Daily: While more news and less rumor-debunking, Science Daily has a huge database of articles to search, most of which link off to the source journal articles and studies that may have started the rumor in the first place. Phys.Org: Again, Phys.Org is primarily a news site, but they've dedicated plenty of articles to debunking popular rumors or addressing trending topics in science and medicine. Plus, they also link to their sources if they can. Finally, when you're reading news articles and sites, even if they link to their supporting studies, try to pry apart the article from the studies. Speaking as someone who's spent a lot of time in the scientific and media communities, both camps almost always mean well when trying to share news or studies with the public—it's just incredibly difficult to get those stories out in a digestible, commonly understood way without losing the science in the message. If you're skeptical, don't dismiss the story or the study—look deeper and draw your own conclusions. The Big Guns: Search Public Journals and Contact Science Advocates So you've done some Google searching, looked at a few skeptical sites, but you want more information. That's great! It's time to go to the source: peer-reviewed journals, and people you can trust about science. One thing I like to do before I write about or share any news article that's based on a study of some kind is to see if I can get to the study itself. Is it mentioned in an article, or did someone say "a study said X?" Fire Google back up and include the journal name and publication date, if you can. "Include "research" or "evidence" or "study" in your search terms. With enough digging, you will often find several scientific papers related to your topic," McRaney suggests. Read the full-text if it's available, or at least the abstract. It'll help you get a picture for what the study really concludes. Of course, you'll probably run into many studies that are locked up behind paywalls. McRaney suggests using Google Scholar, a search engine just for peer-reviewed journals. "Just copy the full name of the paper you want and paste it into Google Scholar's search bar. Look for .pdf versions, as they will most likely be the full paper." If you still can't find it, just email one of the study's authors and ask for a copy. It may sound crazy, but "they are usually more than happy to oblige, and you'll probably put a spring in their step for a full 24 hours." Finally, you can always ask the experts you look up to for their thoughts on a topic. Dr. Plait explained: "I get questions often about topics I'm not all that familiar with. I actually have a long list of people I know who are experts in various topics, and whose opinions I trust, so I will sometimes email them and ask. Not everyone has a stable of experts in their contacts list though!" He went on to explain that there are some people you can trust: researchers, medical professionals, and so on. Seek them out, ask your questions—often they won't be able to resist weighing in on a hot topic, even if you're asking out of the blue. McRaney pointed out that the Ask Science subreddit at Reddit covers a lot of popular, contentious topics from multiple angles. See if your statement has been debated. If not, bring it up yourself. Extra Credit: Visit Your Local Library and Consult Librarians and Reference Materials You've done your homework, looked up some studies, read both sides of an argument, and you're still not sure what to believe, or if there's enough information to believe anything. That's great—you're still hungry for information, and there's one place left to get it: your local library. Photo by Manchester City Library. If you catch yourself unable to download a specific study, or the study is so old (or too new!) that it's not available, or you just want help getting to the bottom of an issue, visit your library's reference desk. Often, public libraries—and especially university libraries—have free access to scholarly journals and their archives so you can download, print, and read full-text articles you wouldn't be able to get at home. Even many university libraries only require student ID if you're going to check something out, so they're a great resource for everyone. "Most university librarians will happily provide you a copy of a paper if you or someone you know is enrolled in the university," McRaney adds. If you are going to chat up your local reference librarian, see what they think of the topic, and if they can do some digging on your behalf. Most often, they can do some research for you and present you with findings to read through, or they can at least help guide you to authoritative sources on the topic. A Last Word: Research Is Great, but Don't Forget to Think Critically All of the research in the world won't help educate you on a controversial issue or statement if you don't look at the evidence and try to draw your own conclusions from it. "What I can say is that if you want to know what's what, the best thing to do is approach the question honestly and openly," Dr. Plait explains. "Read up on opinions for and against. Do the basic arguments make sense? Are the arguments simply ad hominems (attacks against the other side) or is there actual evidence backing up the claims? And what does the other side have to say about those claims?" Image by Image by SMBC. This isn't perfect though, and both McRaney and Dr. Plait warned that here are a few things to watch out for when reading journals and articles that reference journal articles, or when you're talking to science advocates: Watch out for anecdotal evidence. The problem with anecdotal evidence is well documented, and when presented with it, you should make a concerted effort to ignore it, or at least take it into lesser account than rigorous research. It's difficult: our minds are wired to exaggerate the value of anecdotal experience, but if you're thinking and reading critically, it's important to discount them. Watch out for scientists who are anything but. Most scientists know their field well, but you still have to think critically about what you hear. Someone claiming to be knowledgeable may not be at all. "The problem there again is that there are people who claim to be in this group and aren't," Dr. Plait warned. "Or worse, ones who actively promote conspiracy theories against scientists—and those are legion. You've no doubt seen these—people who say scientists are lying to us, or that Big Pharma is paying them, or whatever. I have no easy solution for things like that." Be careful which science advocates you trust. The best science advocates have backgrounds in science themselves, are passionate about science, and are willing to engage openly and non-defensively about a given topic. The worst tend to behave like conversations about research are political or religious ones, and roll out the ad hominem attacks at the first sign of dissention. If may sound cliche, but Dr. Plait and Dave McRaney agree: If most scientists fall on one side of an argument, it's a safe bet that's where the evidence lies. Controversy does not disprove fact, and ongoing research doesn't diminish the research already done. It may require unlearning things you already believe, but doing your own research is worth it. "There's a reason lots and lots of scientists agree on something: usually because it's right," Dr. Plait said. "Sure, science can change its mind, but going with it is the way to bet, because there is usually vast research, evidence, and experience behind it. Go that way and you'll be right the vast majority of the time." Image by XKCD. McRaney sums it up nicely: We invented the scientific method because we are naturally terrible at explaining our own experiences. Without the scientific method, there is no way to know what causes simple, everyday things like thunder. Every explanation is as good as another, and if an explanation becomes culturally bound and passed down, that becomes the official explanation for millennia. Our natural tendency is to confirm our assumptions, but science tries to disconfirm our assumptions one by one until the outline of the truth begins to form. Once we realized that approach generates results, we went from horses and tobacco enemas to mapping DNA and walking on the moon in a few generations. What do you think? How do you avoid confirmation bias when you're researching a new study or article? Which sources do you trust? Share your research tips in the comments below. Phil Plait, Ph.D. is also known as The Bad Astronomer, and writes the Bad Astronomy blog for Discover Magazine. You can follow him on Twitter at @badastronomer. David McRaney is author of the blog You Are Not So Smart and has contributed to Lifehacker many many times. Both gentlemen offered their expertise for this post, and we thank them. Title image remixed from liskus (Shutterstock).

Keep Your Desktop Neat and Tidy With These Built-in Organization Wallpapers How Your Passwords Are Stored on the Internet (and When Your Password Strength Doesn't Matter) How to Determine If A Controversial Statement Is Scientifically True Stop Working All Those Long Hours How to Avoid Impulse Purchases in the Internet Shopping Age Build Your Own Sturdy, Good-Looking Standing Desk for Less Than $25 The SanDisk Extreme SD Card Is the Perfect Balance Between Price and Performance More Stories on Lifehacker »

dwyld.kwu.11muchado2011@blogger.com: Change your e-mail address | Unsubscribe

Gawker Media, 210 Elizabeth Street, Floor 4, New York, NY 10012
Terms of use | Mailing List Policy