The letter by Owens et al.1 regarding our systematic review and meta-analysis of the effect of high school start times on sleep and various outcomes2 is thoughtful and highlights an important issue regarding the role of scientific publications. Their concerns were chiefly (1) that the literature review excluded education-oriented journals, and (2) that describing the quality of evidence as weak or very weak, “has the potential to dissuade decision- and policy-makers from even initiating a dialogue about start time change.”
The first concern regarding the breadth of the literature search can be quickly addressed. We did not intentionally exclude any “major educational journals” from our search. Our search was performed in both PubMed and the Scopus databases, of peer-reviewed journals published prior to April 2016. Scopus includes not only biomedical literature, but also a very broad range of other scientific and health policy literature. Specifically, its database contains more than 1,500 educational journals. The exemplar cited, Educational Researcher, is included in both the PubMed and Scopus databases. Our search yielded only one article from that particular journal, which was excluded because it was a review, not because it was from an educational journal.3 It is possible that searching even more databases might yield yet more articles but we do not believe we were negligent in our search.
Second, our colleagues are concerned that stating that the strength of the published evidence is weak does not provide “strong public statements from the sleep community about healthy school start times” and that, thereby, we sabotage efforts to change public policies. We respectfully disagree. Our stated purpose was to provide as unbiased and accurate an assessment of the currently available literature as possible, and to point out gaps in the evidence, not to make strong public statements in support or refutation of any particular policy or political agenda.
Owens et al. emphasized the importance of ensuring a multidimensional appraisal of scientific evidence.4 We agree and thought that, given limitations in study design, indirectness, lack of contextual description, and other important aspects, the evidence was not “high” or “strong.” More and stronger scientific evidence is needed and would be helpful. To not provide this assessment would be, we believe, the “half of the story” problem, and may not lead to sustained policy changes. We believe our policy-makers should have accurate information available as they deliberate, no matter how nuanced, so they can appreciate the gaps in knowledge and can recognize the work that remains to be done. We reject the notion that access to the best science that is available would dissuade partners from engaging in dialogue. Providing biased, inaccurate, or oversimplified conclusions risks ill-conceived or even very harmful policy decisions, particularly when addressing complex interdependent systems. Doing so may also erode trust in and respect for the scientific community.5 Refraining from a biased presentation, however important we know adolescent sleep to be from a clinical and parental perspective, seemed the best scholarly and scientific approach.
The authors have indicated no financial conflicts of interest.
Morgenthaler TI, Dort L, Mullington J. Transparency and partnership. J Clin Sleep Med. 2017;13(5):763.
Owens J, Troxel W, Wahlstrom K. Commentary on healthy school start times. J Clin Sleep Med. 2017;13(5):761
Morgenthaler TI, Hashmi S, Croft JB, Dort L, Heald JL, Mullington J. High school start times and the impact on high school students: what we know, and what we hope to learn. J Clin Sleep Med. 2016;12(12):1681–1689. [PubMed]
Kirby M, Maggi S, D'Angiulli A. School start times and the sleep-wake cycle of adolescents: a review and critical evaluation of available evidence. Educational Researcher. 2011;4(2):56–61
Rychetnik L, Frommer M, Hawe P, Shiell A. Criteria for evaluating evidence on public health interventions. J Epidemiol Community Health. 2002;56(2):119–127. [PubMed Central][PubMed]
Tanner C, Elvers HD, Jandrig B. The ethics of uncertainty. In the light of possible dangers, research becomes a moral duty. EMBO Rep. 2007;8(10):892–896. [PubMed Central][PubMed]