When the science is questionable


Even the most popular publications will at times publish research that is later found to be false. Image courtesy of the Manchester City Library.

Hope Brusstar

For a long time, the various scientific disciplines have fought to enlighten all of human society, despite always meeting with numerous obstacles along the way. Sometimes, after all, the most groundbreaking discoveries can become the most controversial.

But as of the last decade, and increasingly still, hundreds of scientific papers have essentially been thrown in the trash. This is due to what was dubbed the “replication crisis.” The scientific method, the accepted procedure amongst all researchers, stipulates that an experiment or study must follow certain guidelines. This includes that the experiment must be repeatable, based upon the description of the experiment’s steps within the published academic paper. This is so that anyone can experience the same results by following the steps.

Scholars began to realize that either the statistical data on many academic papers had been calculated or interpreted incorrectly, or that the experimental results themselves were impossible to reproduce. At times, some research papers even failed to describe their experimentation precisely enough to make the steps imitable for others.

Society depends on science in a countless number of ways in order to make its decisions, and rightfully so. But when the science is undependable, well — it’s like we’ve had the rug pulled out from under our feet. Those who paid attention to where the research has been proven wrong may find themselves needing to rethink political opinions, dietary choices, social debates, daily purchases, major financial decisions and much more. One lesson we’ve learned, then, is to not be so trusting: it is, after all, very easy to find two data-based academic papers with diametrically opposed conclusions. One paper may tell you that a grain-based diet is important to heart health, while another will confirm that the advent of grain is what ruined the health of civilization. We can only conclude, therefore, that at least one of these two papers must be wrong.

But of course there are some things we can watch out for. We must be careful about how we read an article. For example, there is a phenomenon known as “correlation without causation,” which means that simply because two things happened at once does not mean one of those things caused the other. Your mother left the room and you ate a strawberry, but you didn’t eat that strawberry because she left the room.

In his recent email, “Welcome to the 2018-19 Academic Year,” President Nicklow said, “The data shows that, on average, students who are engaged in campus activities such as clubs, social organizations or research opportunities persist at a much higher rate and achieve higher GPAs.” Though encouraging this information may be, it is too presumptuous. Those who like to work hard will often be just as likely to work hard for their grades as they are to work hard for student organizations. Thus the behaviors may go hand-in-hand  — but I’d hesitate to assume that one caused the other.

Another good example of possible correlation without causation lies in a Gallup poll studied by Dean of Students Carolyn Golz for her PhD dissertation research: “Gallup (2014) found that alumni who had a professor who cared about them as students and a mentor who encouraged them to pursue their dreams were more likely to be engaged employees,” said Golz on her website at carolyngolz.com. Here, she is stating that the involved professor gave rise to an engaged employee. However, the person who is innately likely to be an engaged employee is also the person who is innately likely to pique a professor’s interest. Farmers don’t water dead plants, and mentors don’t waste time on uninterested students.

We may also recognize another issue that can make research considerably more difficult within the social sciences: what defines an “engaged employee”? How can this quality be quantified, using numbers? And considering the many practices and policies which vary from employer to employer, how do these merits translate from one company to another? How do we compare employees? Maybe Hannah has been keeping up well with emails and calls but barely reaches out to new clients, while Jennifer makes the entire workspace better but can’t complete certain work assignments. Who is the better employee? This is just one of the many ways in which the social sciences can weaken, although earnestly attempting to uncover truth as much as the rest of the sciences.

Which brings me to another point: sometimes scientist actually aren’t in earnest. Sometimes they just want to push out another paper. This is unfortunate and hopefully rare, but the fact that people fudge their margins of error, lie about their data or make conclusions after studying tiny groups of people, ought to make us all more vigilant about what research we do or don’t believe. It is never enough to just read the abstract; one must email the authors directly and request a copy of the full paper before making any decisions. What we discover about the experiment itself may be just as important.