Issue Navigator

Volume 09 No. 12
Earn CME
Accepted Papers


Trust, Verify and Replicate

Stuart F. Quan, M.D., F.A.A.S.M.
Editor, Journal of Clinical Sleep Medicine; Division of Sleep Medicine, Harvard Medical School, Boston, MA; Arizona Respiratory Center, University of Arizona College of Medicine, Tucson AZ

A number of studies have reported an association between obstructive sleep apnea (OSA) and metabolic syndrome.14 However, until 2 years ago, it was unclear whether metabolic syndrome could be reversed by treatment of OSA with continuous positive airway pressure (CPAP). Studies were uncontrolled, small in size and conflicting in their results.5,6 Thus, the purported well designed and conducted report in a high profile medical journal in 2011 that metabolic syndrome could be reversed with 12 weeks of CPAP treatment was welcomed by some in the medical community as another piece of evidence supporting the practice to aggressively treat moderate to severe OSA.7 Nevertheless, there were some doubts expressed about the validity of the results.8,9 Several weeks ago, the other shoe finally dropped. The authors of the 2011 study retracted their paper writing that they were unable to locate and verify some of their primary data.10 Although they assert that their conclusions remain valid, this claim is difficult to believe given the reasons for the retraction.

What have we learned about this retraction of a high profile paper? One lesson should be that “replication is a necessity of the scientific process.” One analysis of highly cited clinical research papers found that results were not confirmed in 16% of cases.11 Were all of these authors guilty of scientific conduct or sloppy research? Most likely not. In many cases they were likely complicit to our worship of the “p value.” In today's science, results are considered significant and therefore “true” if the p value of a statistical test is less than or equal to 0.05, or less than 1 in 20. Conversely, there is a 1 in 20 chance or less that results are “not true.” This is the most compelling reason for replication. The more times that identical or similar studies find the same results, the greater likelihood that the findings are indeed correct. Other explanations for the failure to replicate include prevailing bias and study design issues.12 Nevertheless, scientific misconduct and fraud do exist and potentially may adversely influence public opinion concerning scientific research.13

The second lesson that should be taken to heart by all of us who participate in research is that we need to take some personal responsibility for papers we co-author. Frequently, co-authors or senior authors are not the primary persons who acquire or analyze the data. As a group, we need to take greater responsibility in verifying and questioning results that do not seem “right,” i.e., difficult to believe, implausible or perhaps too perfect. We also need to take collective responsibility for conducting high quality research including accurate data and analysis.

Finally, for the practitioner, you need to interpret and utilize the results reported in journal articles in the context of your practice and the potential impact on patients. Caution and awaiting replication is reasonable for most patients. For others with serious or soon to be life-threatening conditions, embracing a new technology or treatment approach may be the only option.

Obviously, the process by which scientific and medical advances are made is imperfect. When asked about nuclear disarmament with the former Soviet Union, President Reagan quoted an old Russian proverb “Trust, but verify.”14 In the context of medical sciences, we should trust, verify and replicate.


Dr. Quan is the Editor-in-Chief of the Journal of Clinical Sleep Medicine.


Quan SF. Trust, verify and replicate. J Clin Sleep Med 2013;9(12):1233.



Basoglu OK, Sarac F, Sarac S, Uluer H, Yilmaz C, authors. Metabolic syndrome, insulin resistance, fibrinogen, homocysteine, leptin, and C-reactive protein in obese patients with obstructive sleep apnea syndrome. Ann Thorac Med. 2011;6:120–5. [PubMed Central][PubMed]


Coughlin SR, Mawdsley L, Mugarza JA, Calverley PM, Wilding JP, authors. Obstructive sleep apnoea is independently associated with an increased prevalence of metabolic syndrome. Eur Heart J. 2004;25:735–41. [PubMed]


Drager LF, Lopes HF, Maki-Nunes C, et al., authors. The impact of obstructive sleep apnea on metabolic and inflammatory markers in consecutive patients with metabolic syndrome. PLoS One. 2010;5:e12065. [PubMed Central][PubMed]


Parish JM, Adam T, Facchiano L, authors. Relationship of metabolic syndrome and obstructive sleep apnea. J Clin Sleep Med. 2007;3:467–72. [PubMed Central][PubMed]


Dorkova Z, Petrasova D, Molcanyiova A, Popovnakova M, Tkacova R, authors. Effects of continuous positive airway pressure on cardiovascular risk profile in patients with severe obstructive sleep apnea and metabolic syndrome. Chest. 2008;134:686–92. [PubMed]


Coughlin SR, Mawdsley L, Mugarza JA, Wilding JP, Calverley PM, authors. Cardiovascular and metabolic effects of CPAP in obese males with OSA. Eur Respir J. 2007;29:720–7. [PubMed]


Sharma SK, Agrawal S, Damodaran D, et al., authors. CPAP for the metabolic syndrome in patients with obstructive sleep apnea. N Engl J Med. 2011;365:2277–86. [PubMed]


Bertisch S, Patel SR, authors. CPAP for obstructive sleep apnea and the metabolic syndrome. N Engl J Med. 2012;366:963–4. [PubMed]


Drager LF, Lorenzi-Filho G, authors. CPAP for obstructive sleep apnea and the metabolic syndrome. N Engl J Med. 2012;366:964. [PubMed]


Sharma SK, Agrawal S, Damodaran D, et al., authors. Retraction: CPAP for the metabolic syndrome in patients with obstructive sleep apnea. N Engl J Med. 2011;365:2277–86. N Engl J Med 2013;369:1770. [PubMed]


Ioannidis JP, author. Contradicted and initially stronger effects in highly cited clinical research. JAMA. 2005;294:218–28. [PubMed]


Ioannidis JP, author. Why most published research findings are false. PLoS Med. 2005;2:e124. [PubMed Central][PubMed]


Jha A, author. False positives: fraud and misconduct are threatening scientific research: High-profile cases and modern technology are putting scientific deceit under the microscope. The Guardian. September 13, 2012.

14 Accessed November 3,2013.