Measurement of Quality to Improve Care in Sleep Medicine

INTRODUCTION

High quality health care is safe, timely, effective, efficient, equitable, and designed to maximize the patient's interests (patient centered).¹ Although there has always been an imperative to deliver high quality care,^2–4 recent assessment has shown that overuse, misuse, and underuse of healthcare services are common. These shortcomings in healthcare lead to excessive costs and in many cases lost opportunity for improving the health of our fellow citizens.^1,5 The pursuit of improving healthcare is best aligned with three simultaneous achievements, known as “The Triple Aim”: improving the quality of care, improving the health of populations, and reducing per capita costs of healthcare.

High quality sleep medicine care has always been a focus of the American Academy of Sleep Medicine (AASM). The mission of the AASM is to improve “sleep health and promote high quality patient-centered care through advocacy, education, strategic research, and practice standards.”⁶ Pursuant to this mission, the AASM was among the first professional societies to produce evidence-based practice standards (Practice Parameters, Best Practice Guidelines, and Clinical Practice Guidelines) that explicitly describe best practices for sleep medicine.⁷ In light of growing demands to deliver and demonstrate high value care and the goals of the Patient Protection and Affordable Care Act, the AASM has developed measures for five common sleep disorders to evaluate the quality of care in sleep medicine.^8–12 This document explains the process used to develop these quality measures and factors related to their appropriate application and implementation. It will also describe the values and tradeoffs considered in developing these measures including anticipated challenges to their utilization in the healthcare setting.

The accompanying Appendix should guide clinicians on how to implement these sleep quality measures in their practices. The Appendix outlines what the clinician needs to do at the initial evaluation, when follow-ups should be done, and what the clinician should complete at these follow-up visits.

WHY MEASURE QUALITY OF CARE?

The two primary purposes of measuring quality are (1) to serve as essential management tools for improvement, and (2) to evaluate the value of healthcare (see endnote).^13,14 Healthcare professionals are well-acquainted with using measurements to assist in patient management. The success of treatment for hypertension, hyperlipidemia, diabetes, and obesity are judged in large part on the basis of measures such as blood pressure, lipid level, hemoglobin glycosylation (hemoglobin A1c), and body mass index. Experienced healthcare providers treat patients for these conditions with serial measurements to track the response to therapy. Similarly, the development of measures is an essential first step towards managing quality improvement efforts. Measures of healthcare quality help providers identify opportunities for improvement, evaluate the effectiveness of interventions to improve quality, and prioritize areas for improvement.

Other stakeholders, including patients, third-party payers, employers, and society at large, are increasingly concerned with the value of healthcare, which can be represented as a ratio of quality to patient costs (see equation 1).^15,16 Ultimately, quality is represented as various outcomes of interest, including meeting certain goals, avoiding harm, and experiencing healthcare with dignity. Costs are often distributed between patient and third parties initially, but ultimately become reflected in costs directly born by the patient.

One way to improve value is to reduce stakeholder cost. Physicians and other healthcare professionals have experienced increasing pressure to reduce cost. In sleep medicine, this has been particularly evident as systems and incentives promote expanded use of home sleep apnea testing (HSAT), reduced reimbursement for technical and cognitive services, and restricted patient access to sleep specialists. However, exclusive focus on cost reduction to the exclusion of quality of care in sleep medicine is myopic and may decrease quality of care. There is growing evidence of the influence of healthy sleep in adequate quantity on other medical conditions and overall improvement in quality of life. The best applications of the rapid advances in sleep medicine may alleviate suffering, improve productivity, and save costs over time. Beyond these considerations, growing evidence suggests that improving quality substantially reduces the cost of healthcare delivery.¹⁷ Therefore, it is necessary to continue to focus on the numerator of the value equation with at least as much intensity as the denominator. We must measure quality to move sleep healthcare forward.

HOW SHOULD WE MEASURE THE QUALITY OF CARE FOR PATIENTS WITH SLEEP DISORDERS?

Quality of care is typically measured across three dimensions: structure, process, and outcome. Each has distinct advantages and disadvantages.^18,19

Outcome Measures

Outcome measures refer to “a health state of a patient resulting from health care,” and are therefore patient-centric.²⁰ For example, completion of the Epworth Sleepiness Scale (ESS) at a follow up visit could be a process measure (a dichotomous nominal measure), while the ESS numeric score (0–24, a continuous ordinal measure) is an outcome measure. Outcome measures span a broad range of potential measures, such as mortality, stroke rate, sleepiness measures, quality of life measures, and patient-solicited information such as satisfaction and global scale measures of improvement.^16,21–23 The advantage of outcome measures is that they often represent the overall goals of healthcare, and the systematic measurement of outcomes often leads to awareness of gaps in processes to achieve those goals.

While, ideally, outcome measures measure what actually matters most—the health of the patient or population—many challenges arise from efforts to obtain and use outcome measures for improvement and benchmarking. First, many health-care outcomes are the product of not only processes of care within the control of a healthcare system, but also of socioeconomic, hereditary, cultural, and other influences beyond the control of a healthcare facility. As a result, although outcome measures are readily adopted for internal benchmarking, such as tracking a given outcome in a specific system or context over time, controversy often ensues when utilized for external benchmarking comparisons or accountability. Comparisons between contexts require risk adjustment to control for such factors. Ideally, the risk adjustment will be based upon the patient's situation prior to receiving a particular health care service. Usually such risk adjustment is complex, inadequate, or at least controversial, leading to excessive expenditure of resources to optimize risk adjustment or to argue against external benchmarking rather than focusing on actual quality improvement. To minimize complexities, risk-adjustment has tended to focus on measurement of outcomes of very specific diseases or procedures that have well defined risk factors.

In addition, many meaningful healthcare outcomes lag the process of care by years or decades, making it necessary for the temporality of the outcome measure to be precise and often protracted. In many circumstances, the period necessary to relate outcome to process increases complexity due to the difficulties and expense of locating patients, who may migrate during the relevant period. For these reasons, the best outcome measures for quality improvement are those most closely tied in causality and temporality to processes of care within control of the healthcare system. The relationship between structure, process, and outcome measures is depicted in Figure 1.

Realizing some of the inherent limitations of the structural, process, and disorder-specific outcome measures, a next step in maturation of quality measurement is to focus on patient-reported outcomes (PRO), with assessment of patients' preferences, symptoms, and functional and emotional status across episodes of care, along with other risk-adjusted health outcomes. Development of appropriate PROs, particularly those applicable to sleep medicine, is in early stages.

QUALITY MEASURES FOR THE TREATMENT OF SLEEP DISORDERS

The AASM chartered five Workgroups in June of 2013 to develop quality measures for assessment and management of the following sleep disorders:

• Restless Legs Syndrome

• Insomnia

• Narcolepsy

• Adult Patients with Obstructive Sleep Apnea

• Pediatric Patients with Obstructive Sleep Apnea

Formation and Charge of the Workgroups

Workgroup members were sleep specialists solicited via AASM member sections and completed detailed conflict-of-interest statements. Conflicts of interest, if present, were managed according to the conflict of interest policies of the AASM, and the AASM Board of Direcotors approved Task Force membership. The Workgroups were charged to develop both process and outcome measures that, if implemented, would lead to improvements in quality of care and allow for comparisons between care contexts. Their charge was to develop 2–10 process measures, and 1–3 outcome measures following the process outlined below and shown in Table 1.²⁴

Table 1 Process for development of quality measures for sleep disorders.

The process was convened with a face-to-face meeting at the 2013 Association of Professional Sleep Society Sleep meeting in Baltimore, MD. Subsequently, a webinar was conducted to acquaint Task Force members with basic issues related to quality measures and to clarify the charge of the Quality Measures Task Force.

Development of Candidate Measures

For each Workgroup, a literature review was performed to identify published evidence regarding the measurement of quality, care processes, or validated outcome tools. Focus was also placed on literature that would support or measure the strength of linkage between certain care processes and desired outcomes.

A comprehensive search was conducted to identify any publications that addressed any of the four sleep disorders in conjunction with quality care. The following disorder search terms, and associated MeSH terms, were used: restless legs syndrome, Willis-Ekbom, sleep apnea, narcolepsy, and sleep initiation and maintenance disorders. The disorder search terms were used in conjunction with quality terms such as quality indicators, quality measure, quality assurance, outcome measurement/assessment, process measurement/assessment, validation, performance assessment and best practices.

For each disorder, an additional search was conducted to identify clinical practice guidelines, measures, systematic reviews, meta-analyses, and consensus recommendations published by the AASM or other specialty societies in the National Guidelines Clearinghouse, the National Quality Measures Clearing-house, PubMed, EMBASE, PsycInfo, and the Cochrane Library. Workgroup members also performed “pearling,” where references from full articles found through the literature search were examined to identify any additional relevant evidence.

All searches were limited to articles published between 2002 and 2013 and in the English language. Publication types such as news, letters, editorials, and case reports were excluded.

Two Workgroup members reviewed the titles and abstracts of all articles. A third Workgroup member resolved any disagreements. Full articles of publications thought to be relevant were obtained and reviewed in full to identify and provide support for the drafted quality measures.

In general, with the exception of adult OSA, little or no antecedent literature directly addressed measurement of quality in the management of sleep patients. However, published evidence did exist on best practices for care processes relevant to all of the disorders. The Workgroups worked to identify candidate processes linked to outcomes of importance and then schematized the relationship between candidate processes and outcomes of importance in driver diagrams. An example is shown in Figure 2.

Sleep disorder specific examples are provided in the accompanying articles.^8–12 Workgroups next graded the available evidence for the strength of association between the proposed process and the desired outcome (S in Figure 2). The grading scheme utilized is shown in Table 2. Additionally, Work-groups used literature, when available, to develop an estimate of how the measure might address any current gaps in quality of care. Adult OSA Workgroup members evaluated current literature to assure that gaps in preexisting OSA measures were addressed.

Table 2 Strength of association between process measure and desired outcome.

Development of the Selected Measures

Workgroups next drafted the technical definitions of numerators and denominators (if any) for measures utilizing a measure specification template (Table 3). Important outputs of this activity included clear statements of numerators and denominators of the proposed measures, including any exclusions to the numerators or denominators. Workgroups also provided rationales for the selection of the particular measure, along with estimates of the strength of evidence supporting their choices. For each measure, known gaps in the present state of care that might be assessed by measurement were illuminated. The choice of measure was made keeping in mind that the ability to perform measurement should not be limited by the setting or environment in which care was delivered. Administrative or billing data make automation possible across multiple platforms. Where possible, Workgroups strove to identify patients in the numerator and denominator using descriptors characterized by Current Procedural Terminology (CPT) codes that would typically be recorded on claims or billing forms.

Table 3 Quality measure template.

Workgroups next engaged in a Plan, Do, Study, Act (PDSA) process to refine the measures for use.²⁵ Developed measures were first piloted in Task Force members' individual sleep medicine practice locations. As this was part of a quality improvement activity rather than research, patient-specific data were not retained, and no data from the reviews were to be published, patient consent and institutional review board approval were not required. Through this iterative process of assessment and improvement, the Workgroup members developed a working draft of their measures.

Expression of Performance on Quality Measures

The AASM has chosen to display performance as the proportion of patients meeting the quality criteria (equation 2).

Generally, in order to track performance over time, it is useful to display performance measures graphed as a function of time, a display called a run chart (Figure 3).

NEXT STEPS AND LIMITATIONS

The AASM recommends the use of these measures as part of a quality improvement program that will enhance the ability to improve the quality of care for patients with sleep disorders. With additional refinement and validation, these measures may be helpful for external benchmarking. This first exercise in development of measures for quality care in sleep medicine is a beginning, but almost certainly not an end in the effort to measure and improve the quality of sleep medicine. With the exception of management of adult OSA, little prior experience existed upon which to draw in sleep medicine quality measures development. Absent are mature literature and long experience in quality improvement in sleep medicine, this effort is a first and early step, with many opportunities for improvement in the future.

First, most of the measures developed are process measures. Nearly a century ago Codman urged us to focus on the “end results” of care, so in some ways many of our new proposed measures may seem far afield from both our roots and more recent trends that are beginning to focus on patient reported outcomes.²⁶ However, process measures are often the first focus for healthcare quality improvement efforts for important reasons.¹⁹ Measurement and improvement in care processes that are tightly linked to relevant outcomes obviates the need for risk adjustment and focuses on improvement of “what we do” in healthcare. Improvement activities associated with measurement of care processes tends to reduce variation in care delivered and improve the reliability with which key processes of care are rendered. Nonetheless, potential problems with a primary focus on process measures include (a) the measurement of the many processes required to influence outcomes may utilize and compete with resources that would otherwise be directed to activities that have more meaning to the healthcare system, (b) selected processes measured may not be tightly linked or sufficiently influential to important outcomes, and (c) outcomes linked to the chosen processes of care may not be maximally important to patients.

Like limitations of process measures, potential limitations of outcome measures also exist: (a) the measurement of outcomes may utilize and compete with resources that would otherwise be directed to activities that have more meaning to the health-care system, (b) selected outcomes may only measure a limited facet of desirable outcomes of importance to the patient, and (c) comparisons of outcomes between healthcare contexts are usually not adequately controlled for risk factors and confounders.¹⁸ Furthermore, the outcome measures chosen may require additional measurement tools not currently in routine use in many practices. The Workgroups have chosen the final outcomes primarily for two reasons: (a) they are important objectives in the care of patients with these disorders, and (b) these outcomes of care should be a focus in routine practice and, in the context of clinical quality improvement activities, would lead to important changes in the effectiveness of care rendered.

The reasons various processes and outcomes were selected for the care of patients with specific sleep disorders are described in each of the accompanying papers, along with estimates regarding the strength of evidence underlying the recommended measures. In many cases, evidence is not plentiful. This underscores opportunities for clinical research to reveal best practices in sleep medicine.

Regarding the proliferation of quality measures, the last decade clearly has seen a significant increase in the quantity and comprehensiveness of process and outcome measures used for quality improvement. For example, in 2005 there was one CMS inpatient quality reporting system which required reporting of 10 measures (mostly process measures); in 2014 there are ten CMS quality systems with over 350 measures. Additional to these are many other CMS pay-for-reporting and meaningful use incentive programs, and yet other measure programs linked either to various accreditations (such as the Joint Commission ORYX reporting system), to certifications (e.g., National Database of Nursing Quality Indicators for Magnet Hospital status, Get with the Guidelines measures from the American Heart Association for heart failure), or various state-mandated quality measurement systems.¹³ Many healthcare systems are already straining under quality reporting requirements. Although this proliferation may provoke the question of why the AASM would want to offer yet other measures, recent history and legislation shows that the future of healthcare will demand improved and quantified value. The AASM believes that measures developed for the care of patients with sleep disorders are most aptly conceived by those most knowledgeable in sleep medicine.

One important issue with quality measures relates to evaluation of measure validity. The AASM's stakeholder review appeared to support at least face validity. Although minimal construct validity was tested by panel-member field pilots, a need remains for more extensive field testing and formal evaluation.

It is additionally acknowledged that, like most of the measures listed above, chart abstraction of clinical data seems necessary, given the state of most electronic medical records (EMRs). This requirement stems from the fact that many of the processes of care are either not documented by discrete data fields in our EMRs or are not represented by CPT codes.²⁷

To move toward improvement, the AASM recommends the following:

1. Professionals or healthcare systems that provide care to patients with OSA, insomnia, narcolepsy, or restless legs syndrome should commit to a systematic program to evaluate and improve the quality of care rendered to patients with one or more of these sleep disorders. A broad range of healthcare providers may utilize these measures, not only those who practice in sleep centers or sleep specialists. Patients who receive care for OSA, for example, should not experience one standard of care in one setting and a different one in another. Care providers should adopt one or more measure sets to begin assessment of the quality of care they render. Which set(s) to select might be influenced by factors such as perceived quality gaps, practice-specific prevalence of the disorder, perceived impact on payment or referral advantage, resources available to commit to improvement activities, or healthcare burden of disease. These and other factors relevant to measure selection might vary between contexts and over time. Use of a formal prioritization matrix might prove helpful in the decision process.²⁸ Sleep centers accredited by the AASM might consider such measures and related improvement activities as appropriate targets for quality improvement programs.

2. Data should be kept in a confidential database (compliant with the Health Insurance Portability and Accountability Act of 1996-HIPAA) and be used as part of meaningful quality improvement activity.²⁹ Data could be stored in HIPAA compliant registries shared across care contexts for further manipulation, risk adjustment, and benchmarking. Such data often reveal potential areas for enhanced improvement focus and a commitment to best practices.³⁰ The AASM would anticipate that such data registries might be a part of integrated sleep care management systems.³¹

3. The AASM anticipates that accreditation requirements will continue to develop increased emphasis on quality improvement and hope that these measures will serve well for this purpose.

Measurement of quality is a first and important step towards fulfilling one of the most important missions of the AASM, to improve sleep health and promote high quality patient-centered care. These quality measures apply to the ambulatory care model of today, but will almost certainly need evolution to apply to new models of care that will replace our current ones. The AASM regards this first step with pride and humility, recognizing there is much work yet to do to harmonize clinical work, quality measurement, and improve the value of rendered care.

ENDNOTE

The National Quality Forum suggests that quality measurement may (1) inform consumers of healthcare, (2) influence payment and payment programs, (3) drive improvement via external comparison or accreditation/regulatory standards, or (4) help with internal quality improvement. The authors feel these all are subsumed into the two listed purposes of quality measurement.

DISCLOSURE STATEMENT

This was not an industry supported study. Dr. Aronsky is employed by CareCentrix, Inc., a benefit management company and is a past member of the American Academy of Sleep Medicine Board of Directors. Dr. Carden has received royalties from UpToDate. Dr. Chervin has consulted for MC3 and Zansors and has received royalties from UpToDate and Cambridge University Press. Dr. Thomas is an employee of the American Academy of Sleep Medicine. Drs. Morgenthaler, Chervin, Watson, and Carden are all current members of the American Academy of Sleep Medicine Board of Directors.

REFERENCES

APPENDIX

Clinician's Guide for Implementing Sleep Quality Measures

The clinician is responsible for documenting all steps of care in the patient's chart.