ADVERTISEMENT

Issue Navigator

Volume 14 No. 07
Earn CME
Accepted Papers
Classifieds





Special Articles
Free

Evaluation of Clinical Tools to Screen and Assess for Obstructive Sleep Apnea

Charlene Gamaldo, MD1; Luis Buenaver, PhD1; Oleg Chernyshev, MD, PhD2; Stephen Derose, MD, MSHS3; Reena Mehra, MD, MS4; Kimberly Vana, DNP5; Harneet K. Walia, MD4; Vanessa Gonzalez, MPH6; Indira Gurubhagavatula, MD, MPH7,8
1Johns Hopkins University, Baltimore, Maryland; 2Louisiana State University, Shreveport, Louisiana; 3Kaiser Permanente, Pasadena, California; 4Cleveland Clinic, Cleveland, Ohio; 5Arizona State University, Phoenix, Arizona; 6American Academy of Sleep Medicine, Darien, Illinois; 7University of Pennsylvania, Philadelphia, Pennsylvania; 8Corporal Michael Crescenz VA Medical Center, Philadelphia, Pennsylvania

ABSTRACT

Obstructive sleep apnea (OSA) is a globally recognized medical condition, associated with development of long-term adverse health consequences, including cardiovascular disease, cerebrovascular disease, neurocognitive deficiencies, and vehicular and occupational accidents. OSA can be screened effectively, because it can be identified well before the manifestation of the aforementioned poor health and public safety consequences. Additionally, appropriate management of OSA includes an assessment of outcomes before and after therapeutic intervention initiation. OSA clinical screening and outcome assessment tools exist; however, a key existing knowledge gap is identifying which tools are most clinically relevant and efficient to use in clinical practice models. The American Academy of Sleep Medicine (AASM) commissioned a task force (TF) of sleep medicine experts to identify and evaluate current OSA screening and assessment tools for adult patients and determine if they are reliable, effective, and feasible for use in clinical settings. No single tool met all the TF's objective criteria and subjective evaluation for clinical validity and feasibility to be recommended by the AASM. The TF provides several suggestions for the development of new tools or modifications to existing tools that would enhance their functionality in adults.

Citation:

Gamaldo C, Buenaver L, Chernyshev O, Derose S, Mehra R, Vana K, Walia HK, Gonzalez V, Gurubhagavatula I; OSA Assessment Tools Task Force of the American Academy of Sleep Medicine. Evaluation of clinical tools to screen and assess for obstructive sleep apnea. J Clin Sleep Med. 2018;14(7):1239–1244.


INTRODUCTION

Obstructive sleep apnea (OSA) is common among middle-aged adults in the United States.1 Because of the increased prevalence in obesity, a major risk factor for OSA, the estimates of prevalence of OSA have risen by 14% to 55% in recent years.2 Despite increased recognition, OSA continues to remain largely undiagnosed, with 80% of cases unrecognized.3 An indepth cost analysis found that unrecognized OSA produces a large economic burden, costing the United States $149.6 billion annually.3 Therefore, identifying valid, convenient, effective and inexpensive tools to identify those individuals at high risk for OSA is of utmost importance. An ideal screening tool is one that incorporates easily-obtained information to predict the presence or absence of disease in a patient. OSA screening tools that are currently available rely on classic observations of OSA: obesity, snoring, and excessive daytime sleepiness. The United States Preventive Services Task Force (USPSTF) recently reviewed available screening tools for sleep apnea and concluded there was insufficient evidence available to inform the accuracy, benefits, and harms of screening tools for OSA to recommend their use in asymptomatic adults.4

An effective outcome assessment tool is one that accurately quantifies disease severity and response to treatment. The currently available OSA assessment tools utilize a diverse spectrum of health and behavioral markers to serve as outcome measures. Daytime sleepiness, occupational functioning, and social functioning are three of the most common measures. However, a comprehensive review of assessment tools has yet to be performed to assist in informing implementation in clinical practice.

In 2016, the American Academy of Sleep Medicine (AASM) commissioned a task force (TF) of sleep medicine experts to identify and evaluate current clinical screening and outcome assessment tools for OSA and determine if they are reliable, effective, and feasible for use in clinical settings. All members of the TF were independent reviewers and were not involved in the development of any of the tools evaluated, thereby, reducing risk of inherent bias. The following report summarizes these findings and recommendations of the TF regarding: (1) whether particular tools should be recommended by the AASM, (2) ways to improve current tools, and (3) key features or elements to include in the future development of new tools.

METHODS

The TF was charged with (1) developing a comprehensive list of metrics that would be clinically relevant to sleep medicine providers and that could be used to objectively evaluate OSA clinical screening and outcome assessment tools for adult patients, (2) identifying existing tools through a comprehensive literature search, and (3) determining which tools, if any, met the diverse needs of a clinical sleep practice.

Development of Checklists to Evaluate Tools for Clinical Screening and Outcome Assessment of OSA

The TF developed separate checklists to objectively evaluate OSA clinical screening and outcome assessment tools to determine if any of the existing tools met the various clinically relevant outcome metrics and were validated sufficiently to warrant recommendation by the AASM. Both checklists consisted of (1) standard validation metrics to determine if the tools were validated appropriately in OSA patients seen in clinical practices, (2) target populations and demographics to determine if screening tools were studied in patients at-risk for OSA or if assessment tools were studied in patients before or after intervention, and (3) functional metrics to determine how the tools can be used in a clinical setting (Table 1 and Table 2). Target populations were selected based on the AASM's paper: Quality Measure for Screening for Adult Obstructive Sleep Apnea by Primary Care Physicians.5 For the evaluation of outcome assessment tools, the checklist also included clinically important outcomes by TF consensus (Table 2).

Metrics evaluated for clinical screening of OSA tools.

jcsm.14.7.1239.t01.jpg

table icon
Table 1

Metrics evaluated for clinical screening of OSA tools.

(more ...)

Metrics evaluated for clinical outcome assessment of OSA tools.

jcsm.14.7.1239.t02.jpg

table icon
Table 2

Metrics evaluated for clinical outcome assessment of OSA tools.

(more ...)

Identifying and Evaluating Tools for Clinical Screening of OSA

The TF identified screening tools based on the work already published in (1) the AASM's Clinical Practice Guideline for Diagnostic Testing for Adult Obstructive Sleep Apnea6 by evaluating whether the tools that were initially reviewed for the diagnosis of OSA in the publication could be used for OSA screening and (2) the USPSTF's review, Screening for Obstructive Sleep Apnea in Adults.4 For a list of identified tools see Table S1 in the supplemental material. The TF evaluated whether the screening tools were used in the appropriate target populations (Table S3 in the supplemental material), validated among the target OSA patient populations (Table S2 in the supplemental material), and functional in a clinical setting (Table S5 in the supplemental material).

Identifying and Evaluating Tools for Clinical Outcome Assessment of OSA

To identify OSA outcome assessment tools, the TF conducted a PubMed literature search for studies published between January 2010 and December 2016 with outcomes relevant for assessing OSA before and after intervention. The TF, by consensus, selected this data range, based on the following rationale: this range would incorporate the most recent literature, and the tools that are most commonly used today, while literature published before 2010 would add little incremental value. For a list of identified tools see Table S7 in the supplemental material.

The TF reviewed whether each tool adequately assessed outcome domains of interest (Table S8 in the supplemental material). The TF also reviewed whether each tool assessed specific health or safety outcome measures that have well-established associations to OSA (Table S9 in the supplemental material). Finally, the TF evaluated in which patient populations the tools have been studied (Table S11 in the supplemental material), functionality of the tools in a clinical setting (Table S13 in the supplemental material), and validation of the tool among specific OSA patient populations (Table S10 in the supplemental material).

TOOLS FOR CLINICAL SCREENING OF OSA: RESULTS OF EVALUATION

A total of 10 tools used for screening OSA were evaluated to determine their usefulness in clinical settings (Table 3).

List of evaluated clinical screening and assessment of OSA tools.

jcsm.14.7.1239.t03.jpg

table icon
Table 3

List of evaluated clinical screening and assessment of OSA tools.

(more ...)

Validation of Screening Tools in Clinical Settings and Various Populations

Of the 10 tools reviewed, the TF identified the Berlin Questionnaire, Multivariable Apnea Prediction (MVAP), Symptomless Multivariable Apnea Prediction (sMVAP) and STOP-BANG (Snoring, Tiredness, Observed apnea, high blood Pressure, Body mass index (BMI), Age, Neck circumference, male Gender) as individual tools meeting a majority of the predetermined validation metrics (Table S2).710 Across the 10 tools evaluated, limited validation data exist pertaining to the utilization of these individual tools in high-risk OSA populations. STOP-BANG appears to have the most diverse portfolio of target population validation data, and sMVAP had a validation study conducted in a perioperative patient population exclusively (Table S3).

Although the Epworth Sleepiness Scale (ESS) is used commonly as an OSA screening tool in various demographic groups (Table S4 in the supplemental material), the TF review demonstrated that validation data in targeted OSA patient populations was lacking (Table S3).11 Overall, the TF found that there was a lack of validation data for the identified clinical screening tools in commercial drivers and stroke patient populations. However, the TF noted that performance characteristics of the STOP-BAG (Snoring, Tiredness, Observed apnea, high blood Pressure, BMI, Age, male Gender), which was modified specifically from the STOP-BANG questionnaire, was assessed in the stroke population and demonstrated superior performance in predicting OSA in this population.12 Although the TF found sufficient validation data for the STOP-BAG within a stroke population, additional data were not available to support its use outside of the stroke patient population.

The Berlin Questionnaire, sMVAP, OSA 50, Wisconsin Sleep Questionnaire, and STOP (Snoring, Tiredness, Observed apnea, high blood Pressure) had limited use among minority populations (Table S4).13,14 Additionally, the evaluation uncovered gaps in the validity, utility, and feasibility of all the reviewed screening tools in younger adults (18–44 years of age).

Functionality of Screening Tools

OSA screening tools varied in terms of costs and permission for use, often requiring that the developer be contacted for details. No tool was freely available for clinical use without permission. The available platforms for administering a tool were predominantly face-to-face and paper (Table S5). The tools required a combination of objective measures reported by clinicians (eg, BMI) and subjective measures reported by patients (eg, daytime sleepiness and snoring). The required completion time generally ranged from 1–5 minutes and some tools, such as the STOP, STOP-BANG, Berlin Questionnaire, and ESS, were available in many languages (Table S6 in the supplemental material). The Flesch-Kincaid score, performed by the TF, revealed reading grade levels ranging from third to seventh grade.

In a stroke population at high risk for OSA, the STOP-BAG variables, which exclude the neck circumference, were identified to have comparable performance characteristics to the traditional STOP-BANG instrument when considering mild to moderate levels of OSA.12 The TF recognized that the potential increase in reliability offered by objectively-measured physical findings (eg, BMI) must be balanced against the increased burden on the health care team to collect these clinical variables.

TOOLS FOR CLINICAL OUTCOME ASSESSMENT OF OSA: RESULTS OF EVALUATION

A total of 20 OSA assessment tools were evaluated to determine their usefulness in clinical settings (Table 3).

Outcome Domains of Assessment Tools

The TF reviewed whether each tool adequately assessed outcome domains of interest (Table S8). The TF identified the Functional Outcomes of Sleep Questionnaire (FOSQ) as the assessment tool that includes the most relevant outcome measures.15 However the FOSQ may be subject to having questions skipped by patients (eg, questions on sexual function), thus, skewing the scoring and leading to misinterpretation of results. Additionally, the TF noted that some of these assessment tools report on general outcomes (eg, 36-Item Short Form Health Survey [SF-36] for quality of life) and may lack more immediate outcomes, such as adherence, that are useful to the clinician in assessing a patient's OSA post-intervention response (Table S9).16

Validation of Outcome Assessment Tools in Clinical Settings and Various Populations

While none of the assessment tools were validated sufficiently to warrant a recommendation, the FOSQ best met most of the validation metrics for assessing OSA (Table S10). Most of the remaining tools were not designed specifically to assess patients with OSA, and thus, did not meet the validation metrics deemed important in assessing the OSA population. Only the Calgary Sleep Apnea Quality of Life Index (SAQLI), Sleep Apnea scale of the Sleep Disorders Questionnaire (SA-SDQ), and Symptoms of Nocturnal Obstruction and Related Events (SNORE) tools were developed for assessment of patients with OSA.1719 However, even these OSA-specific tools lacked many of the domains of interest and supporting evidence in validating their use in specific target populations (Table S11).

Most of the assessment tools generally have been used in both men and women, multi-national populations, and have involved middle-aged and elderly individuals (Table S12 in the supplemental material). The ESS, FOSQ, and Pittsburgh Sleep Quality Index (PSQI) have been studied extensively across a wide variety of demographic groups. The ESS has been widely studied in a variety of populations including general medical, cardiac, obese, stroke, type 2 diabetes mellitus, and preoperative bariatric surgery candidates (Table S11). The PSQI also has been examined in various clinical populations including cardiac, general medical, and obese populations.20 The Beck Anxiety Inventory, Brief Fatigue Inventory, ESS, FOSQ, and PSQI have been studied in minorities.

Functionality of outcome assessment tools

Most of the assessment tools were not associated with a known cost (Table S13). The tools were available predominantly through face-to-face, paper, and electronic platforms. All the tools have the capability for manual scoring, and most tools have electronic scoring options. Most of the tools did not require completion by providers, caregivers, and staff, which allowed for full patient-report administration. Flesch-Kincaid readability for the tools ranged from second to eleventh grade, and reported duration of completion ranged from 1 minute to 10 minutes.

CONCLUSION AND RECOMMENDATIONS

The TF performed a comprehensive literature review to identify both OSA screening and outcome assessment tools currently utilized in the health care setting. Of the 10 screening and 20 assessment tools identified and evaluated, no single tool met all the TF's objective criteria and subjective assessment for clinical validity and feasibility to be recommended by the AASM. To fulfill this unmet need, researchers in the field of sleep medicine should pursue the development and validation of OSA screening and OSA outcome assessment tools, ideally with the following features:

  1. 10 or less questions (subjective symptoms or objective physical findings, alone or in combination) written at or below the 5th grade-school reading level21

  2. Completion in less than 5 minutes by any member of the health care team

  3. Tiered system for completion, scoring, and interpretability: (1) patient-reported, (2) bed partner-reported, and (3) provider-reported measures

  4. Compatibility with electronic health record platform for future monitoring of clinical outcomes and analysis

  5. Capability of patient self-tracking or capability to monitor progress

  6. Availability as an application platform with electronic scoring or paper format with easy manual scoring

  7. Availability in the various languages that represent communities with high OSA presence

  8. Adaptability to general and sleep patient populations

  9. Adaptability for specific, at-risk OSA populations (eg, stroke patients, atrial fibrillation) or those individuals with unique occupational or public health risk (eg, commercial drivers, pilots)

  10. Available to the public

When left unidentified and untreated, OSA remains a health care condition linked with numerous personal health and public safety consequences. As such, the identification and implementation of feasible, valid, and effective OSA screening and outcome assessment tools are paramount to our overall global and domestic public health interests. Broad implementation of clinical screening and outcome assessment tools that incorporate the aforementioned TF recommendations would improve the ability of the sleep field to achieve the overarching goal of enhancing methods of OSA detection and improving patient outcomes.

DISCLOSURE STATEMENT

The development of this position paper was funded by the American Academy of Sleep Medicine. Ms. Gonzalez is employed by the American Academy of Sleep Medicine. The other authors report no conflicts of interest.

ACKNOWLEDGMENTS

The AASM OSA Assessment Tools Task Force acknowledges the following individuals for their participation and contributions: Charlene Gamaldo, MD (Chair); Indira Gurubhagavatula, MD (Vice Chair); Jonathan L. Heald, MA (AASM Staff); and Dennis Hwang, MD.

REFERENCES

1 

Young T, Palta M, Dempsey J, Skatrud J, Weber S, Badr S. The occurrence of sleep-disordered breathing among middle-aged adults. N Engl J Med. 1993;328(17):1230–1235. [PubMed]

2 

Peppard PE, Young T, Barnet JH, Palta M, Hagen EW, Hla KM. Increased prevalence of sleep-disordered breathing in adults. Am J Epidemiol. 2013;177(9):1006–1014. [PubMed Central][PubMed]

3 

Frost & Sullivan; American Academy of Sleep Medicine. In an age of constant activity, the solution to improving the nation's health may lie in helping it sleep better: what benefits do patients experience in treating their obstructive sleep apnea. https://aasm.org/advocacy/initiatives/economic-impact-obstructive-sleep-apnea/. Published August 8, 2016. Accessed November 7, 2017.

4 

US Preventive Services Task Force. Screening for obstructive sleep apnea in adults: US Preventive Services Task Force recommendation statement. JAMA. 2017;317(4):407–414. [PubMed]

5 

Aurora RN, Quan SF. Quality measure for screening for adult obstructive sleep apnea by primary care physicians. J Clin Sleep Med. 2016;12(8):1185–1193. [PubMed Central][PubMed]

6 

Kapur VK, Auckley DH, Chowdhuri S, et al. Clinical practice guideline for diagnostic testing for adult obstructive sleep apnea: an American Academy of Sleep Medicine clinical practice guideline. J Clin Sleep Med. 2017;13(3):479–504. [PubMed Central][PubMed]

7 

Netzer NC, Stoohs RA, Netzer CM, Clark K, Strohl KP. Using the Berlin Questionnaire to identify patients at risk for the sleep apnea syndrome. Ann Intern Med. 1999;131(7):485–491. [PubMed]

8 

Maislin G, Pack AI, Kribbs NB, et al. A survey screen for prediction of apnea. Sleep. 1995;18(3):158–166. [PubMed]

9 

Gurubhagavatula I, Nkwuo JE, Maislin G, Pack AI. Estimated cost of crashes in commercial drivers supports screening and treatment of obstructive sleep apnea. Accid Anal Prev. 2008;40(1):104–115. [PubMed]

10 

Chung F, Subramanyam R, Liao P, Sasaki E, Shapiro C, Sun Y. High STOP-Bang score indicates a high probability of obstructive sleep apnoea. Br J Anaesth. 2012;108(5):768–775. [PubMed Central][PubMed]

11 

Johns MW. A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–545. [PubMed]

12 

Katzan IL, Thompson NR, Uchino K, Foldvary-Schaefer N. A screening tool for obstructive sleep apnea in cerebrovascular patients. Sleep Med. 2016;21:70–76. [PubMed]

13 

Chai-Coetzer CL, Antic NA, Rowland LS, et al. A simplified model of screening questionnaire and home monitoring for obstructive sleep apnoea in primary care. Thorax. 2011;66(3):213–219. [PubMed]

14 

Chung F, Yegneswaran B, Liao P, et al. STOP questionnaire: a tool to screen patients for obstructive sleep apnea. Anesthesiology. 2008;108(5):812–821. [PubMed]

15 

Weaver TE, Laizner AM, Evans LK, et al. An instrument to measure functional status outcomes for disorders of excessive sleepiness. Sleep. 1997;20(10):835–843. [PubMed]

16 

Ware JE, Sherbourne CD. The MOS 36-Item Short-Form Health Survey (SF-36): I. Conceptual Framework and Item Selection. Med Care. 1992;30(6):473–483. [PubMed]

17 

Flemons WW, Reimer MA. Measurement properties of the Calgary sleep apnea quality of life index. Am J Respir Crit Care Med. 2002;165(2):159–164. [PubMed]

18 

Douglass AB, Bornstein R, Nino-Murcia G, et al. The Sleep Disorders Questionnaire. I: creation and multivariate structure of SDQ. Sleep. 1994;17(2):160–167. [PubMed]

19 

Piccirillo JF. Outcomes research and obstructive sleep apnea. Laryngoscope. 2000;110 Suppl 94:16–20. [PubMed]

20 

Buysse DJ, Reynolds CF, Monk TH, Berman SR, Kupfer DJ. The Pittsburgh Sleep Quality Index: a new instrument for psychiatric practice and research. Psychiatry Res. 1989;28(2):193–213. [PubMed]

21 

Stossel LM, Segar N, Gliatto P, Fallar R, Karani R. Readability of Patient Education Materials Available at the Point of Care. J Gen Intern Med. 2012;27(9):1165–1170. [PubMed Central][PubMed]

22 

Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: Psychometric properties. J Consult Clin Psychol. 1988;56(6):893–897. [PubMed]

23 

Beck AT, Ward CH, Mendelson M, Mock J, Erbaugh J. An inventory for measuring depression. Arch Gen Psychiatry. 1961;4:561–571. [PubMed]

24 

Mendoza T, Wang XS, Cleeland CS, Morrissey M, Johnson BA, Wendt JK, Huber SL. The rapid assessment of fatigue severity in cancer patients: use of the Brief Fatigue Inventory. Cancer. 1999;85(5):1186–1196. [PubMed]

25 

Guy W. ECDEU Assessment Manual for Psychopharmacology. Rockville, MD: U.S. Department of Health, Education, and Welfare; 1976.

26 

The EuroQol Group. EuroQol-a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208. [PubMed]

27 

Krupp LB, LaRocca NG, Muir-Nash J, Steinberg AD. The fatigue severity scale. Application to patients with multiple sclerosis and systemic lupus erythematosus. Arch Neurol. 1989;46(10):1121–1123. [PubMed]

28 

Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand. 1983;67(6):361–370. [PubMed]

29 

Subramanian S, Hesselbacher SE, Aguilar R, Surani SR. The NAMES assessment: a novel combined-modality screening tool for obstructive sleep apnea. Sleep Breath. 2011;15(4):819–826. [PubMed]

30 

Hunt SM, McKenna SP, McEwen J, Backett EM, Williams J, Papp E. A quantitative approach to perceived health status: a validation study. J Epidemiol Community Health. 1980;34(4):281–286. [PubMed Central][PubMed]

31 

Cella D, Riley W, Stone A, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005-2008. J Clin Epidemiol. 2010;63(11):1179–1194. [PubMed Central][PubMed]

32 

Brazier J, Roberts J, Deverill M. The estimation of a preference-based measure of health from the SF-36. J Health Econ. 2002;21(2):271–292. [PubMed]

33 

Hoddes E, Zarcone V, Smythe H, Phillips R, Dement WC. Quantification of sleepiness: a new approach. Psychophysiology. 1973;10(4):431–436. [PubMed]

34 

Spielberger CD, Gorsuch RL, Lushene R, Vagg PR, Jacobs GA. Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press; 1983.

35 

Young T, Palta M, Dempsey J, Peppard PE, Nieto FJ, Hla KM. Burden of sleep apnea: rationale, design, and major findings of the Wisconsin Sleep Cohort study. WMJ. 2009;108(5):246–249. [PubMed Central][PubMed]


Supplemental Material

Supplemental Material

(.pdf | 204 KB)