Research. Research. Research. Always approach with caution until you have looked at the research yourself. With how many ads and false information we see on tv and social media we need to be informed as information spreads quickly. In this blog we will help you read articles like a pro so you can understand, educate, and make better informed decisions.
Common sense tips
Date: The more recent the study was conducted the better just for relevancy.
Source: Articles from scholarly sources such as an accredited university, Journal of Medicine, or government source such as the National Institute of Health are all good. Also, government sources are usually very accessible to the public; you will know you’re on a government source if it ends in “.gov.”
Blind vs Double Blind: single blind studies mean that only the one group is blinded to an intervention, such as the participants. Double blind studies have more relevance over single blind studies. This means the participants are blinded to what the researchers want to analyze such as one intervention over the other. The people administering the intervention don’t know if that is the thing being tested eg. Ibuprofen vs acetaminophen for pain relief. And the rating clinicians that look at the results of the medications don’t know which medication has been administered. Basically nobody knows anything, which limits bias. If there is an added sugar pill for a placebo it holds even more credence. In fact, randomized double blind placebo control (RDBPC) studies are considered the “gold standard” of epidemiologic studies (Misra, 2012). Don’t underestimate the power of the placebo-effect, better to just eliminate it!
Number of participants: In order to remain clinically relevant to the healthcare field, there needs to be a minimum of 60 participants total in the study (Faber & Fonseca, 2014). This total can be split between a control group and a testing group. Another important note is to look at how many in the testing/control group started vs finished. If there were originally 30 people in each group being tested and then 10 participants dropped out it’s no longer enough participants to show significant results.
Peer-review: articles that have been “peer-reviewed” have been looked at by other scholars in the field that did not partake in the study. In fact, they do not know whose research they are even reviewing, further eliminating bias. The Cochrane review is a good example of how another unbiased group overlooks someone else’s research.
Size of the analysis: A meta-analysis is a research study that pools multiple research studies together. This kind of study first compares many different studies testing the same theory. Then they pick only the cream of the crop from that bunch, eliminating studies that showed: increased bias, poor standardized testing, poor subject participation, etc. These are often easier as they compare/contrast for you, condensing the findings from multiple articles. They often use large research studies with participants in the hundreds to thousands for a better sample size. One well-respected organization who does this is the Cochrane organization. They are a world-wide organization that works closely with the World Health Organization and does not accept any commercial or conflicting funding, choosing to function as a charity instead. The Cochrane Organization started critiquing evidence-based research in 1993 in Britain to ensure safety standards were being followed and to encourage drug companies to out-source for testing to eliminate bias from funding.
Groups: randomized groups also help eliminate bias. For instance if you wanted to get more people to buy a product to “lose weight” you would be able to skew results by putting people who are more overweight in the testing group using the product, while you put mostly healthy individuals at an healthy weight into the control group since they will most likely not lose any weight. For this reason you want selection to be random between groups.
Testing: The measurements pre/post have to be done identically. For example, if you tested an athlete’s running speed outside for the initial test you wouldn’t want to post-test on a treadmill, as this will most likely throw off the results. The parameters have to be the same. Some research studies will even control things such as temperature in the lab, water consumption, and clothing. The more strict the better. A cautionary note though, sometimes this can cause issues. A good example of this is the drug-dose gender gap. This happened when studies were looking at the safety and effect of the sleeping pill ambien in relation to alertness and operating a vehicle. It appeared to be both safe and effective for users, with only very small percentage still having decreased alertness while driving 8 hours after ingestion. However, in the original research study it was only being tested on healthy males. It has since come about that women metabolize the drug much more slowly than men. On study states, “Women had on average 35% lower apparent clearance of zolpidem than men.” This difference in physiology may have contributed to more car accidents the next morning in female drivers and has spurred the FDA to reduce the recommended dose by 50% for women(Greenblatt, Harmatz & Roth, 2019). So in this case, women should have also been studied to ensure it was safe for both genders. Common sense right? If you’re selling it to both genders, it needs to be tested on both genders.
Values and tests
Statistical significance: the p value that you will often see in the methods and stats section represents significance when statistics are applied to research. “…the lower the p-value, the less likely the results are due purely to chance” (Gallo, 2016). A p value <.03 is good, a p value of less than .01 is even better, showing that if the test is repeated again, 99% of the results from similarly conducted research would yield the same results.
There you have it. Be skeptical, be informed, and be safe.
Faber, J., & Fonseca, L. M. (2014). How sample size influences research outcomes. Dental press journal of orthodontics, 19(4), 27–29. doi:10.1590/2176-9451.19.4.027-029.ebo
Gallo, A. (2016). A Refresher on Statistical Significance. Retrieved from https://hbr.org/2016/02/a-refresher-on-statistical-significance
Greenblatt, D., Harmatz, J., & Roth, T. (2019). Zolpidem and Gender. Journal Of Clinical Psychopharmacology, 39(3), 189-199. doi: 10.1097/jcp.0000000000001026
Misra S. (2012). Randomized double blind placebo control studies, the “Gold Standard” in intervention based studies. Indian journal of sexually transmitted diseases and AIDS, 33(2), 131–134. doi:10.4103/0253-7184.102130