To evaluate the effect of continuing medical education (CME) activities on patient reported outcomes with regard to (1) screening for excessive sleepiness (ES) and obstructive sleep apnea (OSA) and (2) appropriate referral and treatment.
A total of 725 patients were recruited from 75 providers who either participated or did not participate in Transtheoretical Model (TTM)-based OSA CME activities. Patient reported outcomes from participating (n = 36) and non-participating providers (n = 39) were compared using generalized estimating equations examining random effects of provider as unit of assignment.
Patients' reports demonstrate that participating physicians were 1.7 times more likely to initiate discussion of sleep problems than non-participating physicians (t1,411 = 3.71, p = 0.05) and 2.25–2.86 times more likely to administer validated measures for OSA (Epworth Sleepiness Scale and STOP-BANG). Patient reports also indicated that participating clinicians (79.9%) were significantly more likely to recommend seeing a sleep specialist compared to non-participating clinicians (60.7%; t1,348 = 9.1, p < 0.01, OR = 2.6). Furthermore, while 89.4% of participating clinicians recommended a sleep study, only 73.2% of the non-participating physicians recommended one (t1,363 = 11.46, p < 0.001, OR = 3.1).
Participation in TTM-based OSA CME activities was associated with improved patient reported outcomes compared to the non-participating clinicians.
Johnson SS, Castle PH, Van Marter D, Roc A, Neubauer D, Auerbach S, DeAguiar E. The effect of physician continuing medical education on patient-reported outcomes for identifying and optimally managing obstructive sleep apnea. J Clin Sleep Med 2015;11(3):197–204.
The American Academy of Sleep Medicine (AASM) reports that obstructive sleep apnea (OSA), the most common form of sleep-related breathing disorder (SRBD), “is characterized by repetitive episodes of complete or partial upper airway obstruction occurring during sleep” that are typically associated with “reduction in blood oxygen saturation and are usually terminated by brief arousals from sleep.”1 OSA is associated with significant social, functional, and clinical consequences including shortened life expectancy, increased risk of hypertension, heart attack, stroke, congestive heart failure, irregular heartbeat, coronary artery disease, and depression2–5 as well as increased risk of peri-operative complications.6
The prevalence of SRBD is increasing steadily. Peppard et al.3 report double-digit increases in prevalence since 1988– 1994. Based on their cohort study, the population prevalence of moderate to severe SDB (defined as apnea-hypopnea index ≥ 15 or apnea-hypopnea index ≥ 5 and symptoms of daytime sleepiness) in adults aged 30–70 is nearly 20%. The prevalence of OSA is even higher (≥ 50%) in patients with cardiac or metabolic disorders.7
Current Knowledge/Study Rationale: Despite the increasing prevalence and harmful sequelae of obstructive sleep apnea (OSA), 80% to 90% of adults with OSA remain undiagnosed. There is therefore an urgent need to promote performance change related to its identification and management.
Study Impact: CME activities that included behavior change messages increased the likelihood that physicians would initiate a discussion about sleep problems, administer measures of excessive sleepiness and OSA, recommend sleep studies, or refer to a sleep specialist. The integration of CME and behavior change messages designed for clinicians in all stages of readiness to implement recommended practices for the identification and optimal management of OSA is a promising approach to increasing the extent to which those guidelines are adopted.
Comprehensive clinical guidelines outlining diagnostic, assessment, treatment (e.g., initiation of positive airway pressure (PAP) therapies, use of behavioral therapies, oral appliances, surgical options, or adjunctive therapies), and management algorithms have been issued for patients with potential or diagnosed OSA.8,9 Guidelines recommend, among other things, that questions about OSA should be included in routine health evaluations and that a comprehensive sleep examination should be performed for patients with suspected OSA.8
Primary care physicians (PCPs) should therefore consider OSA in patients with predisposing risk factors such as obesity, large neck circumferences, snoring, daytime sleepiness, and hypertension.4 Remarkably, however, 80% to 90% of adults with OSA remain undiagnosed.6,10 Early reports indicate that PCPs did not recognize or diagnose OSA in their patients.11,12 Furthermore, many clinicians do not routinely screen for sleep conditions. Despite the availability of easy-to-administer self-report measures, such as the Epworth Sleepiness Scale13 and the STOP-BANG questionnaire,14 screening tools are under-used, resulting in a failure to identify hallmark symptoms of OSA, such as excessive sleepiness and snoring.15 Nearly 40% of responding clinicians reported that they do not screen their patients for excessive sleepiness, the hallmark symptom of OSA.
One promising approach for increasing the diagnosis and optimal management of OSA is clinician education. Continuing medical education (CME), which has been identified as a promising strategy for improving clinical outcomes,16 has a long and well-established history of improving clinician knowledge.17 Increasingly, the focus of CME is shifting from the traditional provision of knowledge to improvements in performance and clinical practice.18 Strategies for maximizing the effectiveness of CME (e.g., tailoring content to the educational needs and goals of the learner19) are being explored. Controlled trials demonstrated that internet-based CME significantly increases the probability of making evidence-based clinical decisions.20,21 Tailoring educational content to readiness to adopt a performance change is also likely to play a key role.22–25
The Transtheoretical Model (TTM) is an integrative model that describes behavior change as a progression through a series of five stages of readiness: (1) Precontemplation: not intending to adopt the best practices, (2) Contemplation: intending to adopt the best practices in the next six months, (3) Preparation: intending to adopt the best practices in the next 30 days, (4) Action: adopted the best practices less than six months ago, and (5) Maintenance: adopted the best practices more than six months ago. The TTM could provide a useful framework for tailoring CME content in that leveraging the principles of the TTM permits the inclusion of behavior change statements to ensure activities meet the needs of all physicians, even those who are not prepared to adopt the desired performance. Messages on the benefits of adopting the best practices are important for clinicians in the Precontemplation stage, for example, whereas suggestions for how to overcome the barriers to adopting the best practices and messages about being inspired by other clinicians who have already adopted the best practices are more important for clinicians in the Contemplation stage. Clinicians in the Action stage benefit from recommendations about how to structure their practice to facilitate implementation of the best practices (e.g., prompts in EMR) and how to overcome situations in which it might be difficult to implement the best practices. Previous research demonstrates that applying the TTM can improve performance significantly more than standard education. Specifically, physicians randomized to educational content matched to their readiness to diagnose, treat, and make referrals for depression demonstrated significantly higher performance improvements on those clinical practices two months post-intervention as compared to those randomized to standard education.23 Johnson et al.26 demonstrated that incorporating TTM messages into CME activities for individualizing care for rheumatoid arthritis increases the likelihood that physicians will move to more advanced stages of change for adopting a performance change, but no research to date has examined the impact of TTM-based CME activities on patient-reported outcomes.
The goal of this study was to evaluate the effect of accredited CME activities with a foundation in the TTM on patient-reported outcomes regarding screening and treatment for OSA.
As part of a larger study, the TTM was applied to the design and evaluation of an internet-based CME curriculum including three activities designed to promote the identification and optimal management of obstructive sleep apnea (OSA) and one of its primary symptoms, excessive sleepiness (ES). For clinician participants, that performance improvement was defined by the adoption of the following best practices:
Incorporating sleep-related questions to screen for the presence of OSA during:
Comprehensive physical examinations
Routine office visits when symptoms (e.g., tiredness, fatigue, ES, insomnia, bed partner report of snoring or snorting, abrupt awakening from sleep with gasping or choking), physical findings (e.g., big neck, crowded oral pharynx, elevated body mass index), or comorbidities (e.g., hypertension, obesity, diabetes, congestive heart failure) suggest an increased likelihood of OSA
Assessing degree and severity of ES using a validated self-assessment tool (e.g., Epworth Sleepiness Scale)
Assessing likelihood of OSA using a brief validated screening tool (e.g., modified Berlin, STOP-BANG)
Referring patients to a sleep medicine center when OSA is suspected or the differential diagnosis is not clear
Developing an appropriate treatment plan for OSA that includes positive airway pressure devices (e.g., continuous positive airway pressure), patient education, behavioral strategies (e.g., weight loss), and symptomatic treatment with antihypertensive medications and wake-promoting medications for residual ES as indicated
Monitoring patients for adherence to OSA treatment regimen as well as effectiveness and effects of OSA treatment on ES and associated morbidities
The three activities, described in more detail in Table 1, were delivered free of charge in varied formats online by Medscape Education.27 The first was a Clinical Anthology (a 3-article collection activity focused on various aspects of OSA) entitled Evidence-Based Best Practices in Obstructive Sleep Apnea Management28 that included an expert interview and 2 case reports. The second was an interactive case-based activity that employed a “test, then teach” methodology intended to challenge clinician participants to demonstrate their skills in the care of patients with OSA, from screening through intervention, and provide relevant feedback immediately to improve their clinical skills.29 The final activity was an Ask the Expert Collection, in which the 6 most common questions collected from the initial 2 activities were addressed by experts.30 Each of the activities included a brief pretest to assess the participating clinician's readiness to identify and optimally manage obstructive sleep apnea (OSA) and one of its primary symptoms, excessive sleepiness (ES) as defined above. Clinicians could report that they were not intending to do so in the next 6 months (Precontemplation stage); intending to do so in the next 6 months (Contemplation stage); intending to do so in the next month (Preparation stage); already doing so for < 6 months (Action stage); or already doing so for > 6 months (Maintenance stage).
Educational content: learning objectives, content focus, and learner experience.
Educational content: learning objectives, content focus, and learner experience.
The educational content for each activity included behavior change messages relevant to clinicians in all stages of change for adopting the performance improvement. The messages were based on the well-established theoretical principles of the TTM, as well as on information gathered from thought leader interviews conducted during the development of the activities which addressed topics such as key benefits and barriers and difficult situations. The Ask the Expert collection, for example, included the following messages to overcome common barriers to screening for OSA:
“It is crucial to remind ourselves that time screening for OSA is well spent given the potential consequences prevented and improved quality of life and emotional well-being patients experience when it is treated. Other clinicians have said it is extremely rewarding to hear from patients whose lives have been dramatically improved by treating their OSA.”
“It is possible to reduce the time needed for screening (e.g., patients completing questionnaires at home prior to visit or in waiting room). It will be helpful to use cues to prompt screening (e.g., flag in EMR, particularly for patients with any of the likely co-morbid conditions).”
In addition, at the conclusion of each activity, clinician participants received behavior change suggestions matched to their individual readiness to engage in the performance improvement (see examples from one activity in the Appendix). The behavior change recommendations varied from one activity to the next.
To mimic real-world conditions as closely as possible, clinicians registered on Medscape had access to all 3 activities regardless of previous participation in CME for OSA. The activities were promoted via regularly scheduled outreach messages. Clinicians could choose to participate in 1, 2, or 3 activities when they were released.
For the current study, English-speaking US clinicians (physicians, nurse practitioners and physician assistants) were eligible to participate if they were licensed to practice medicine and reported a specialty of primary care, pulmonology, or psychiatry. While it is possible that some of these clinicians had full or part-time involvement in sleep medicine, sleep medicine specialists were not targeted, and no clinician reported that as his or her primary specialty. English-speaking patients aged ≥ 21 years were eligible if they were experiencing excessive sleepiness, were suspected of having OSA because they report other symptoms suggestive of OSA, or had recently been diagnosed with OSA.
Clinicians who met the eligibility criteria and had participated in one or more of the accredited CME activities categorized from the TTM-based OSA curriculum (n = 4,500) were identified by Medscape, the CME provider. A group of clinicians from the same specialties who had not participated in any of CME activities (n = 4,500) were also identified. Each group received 2 email invitations to join the study. The invitations, which were sent by Medscape, explained the requirements of the study and the incentive ($500) for participation. Interested clinicians could click on a link in the email to review the informed consent documentation, complete a brief survey, and provide their contact information to the research team. The brief clinician survey included the same assessment of each clinician's readiness to identify and optimally manage OSA and ES. Clinicians were then mailed survey packets for distribution to 40–50 patients treated in their practice that met the patient eligibility criteria (outlined above). All patient survey packets included an informed consent form for the patient and the 45-item patient survey, as well as a postage-paid return envelope. The second email invitation was similar to the first but also offered the option of paper or online versions of survey packets for patients. Patients opting for the online survey option (n = 15 eligible patients from participating clinicians and n = 49 eligible patients from non-participating clinicians) provided informed consent prior to responding to the survey. Clinicians were eligible to receive the incentive (in the form of a check) when ≥ 11 eligible patients returned assessments. The study was approved by the Institutional Review Board at Pro-Change Behavior Systems, Inc.
The following measures were administered to the patients.
Epworth Sleepiness Scale (ESS)
The ESS is a widely used, validated 8-item scale measuring subjective sleepiness. Response options are 0 = unlikely to fall asleep to 3 = high chance of falling asleep.13,31
The STOP-BANG is an 8-item screening tool for OSA focusing on snoring, tiredness during daytime, observed apnea, high blood pressure, body mass index, age, neck circumference, and gender.14
Frequency of Scale Administration
A single question for each scale asked the patient to report whether the clinician asked the patient ESS and STOP-BANG questions in the past. Response options include (1) No, (2) Yes, once before, and (3) Yes, more than once before.
Discussion of Sleep Problems
Two questions assessed whether patients had spoken with their physicians about a sleep problem and who had initiated the discussion.
Recommendations for Tests and Treatments
A series of 11 questions assessed recommendations clinicians provided to patients for various methods of diagnosing (seeing a sleep specialist, having a sleep study) and treating a sleep disorder (taking a prescription drug, taking an OTC sleep aid, taking an herbal supplement, using a PAP device, changing sleep habits, using a non-medical treatment, or trying alternative medicine).
The primary outcome measures were patient reports of whether clinicians initiated a discussion about sleep; administered the ESS and/or STOP-BANG; and recommended follow-up tests and treatment when appropriate. The generalized estimating equation method of analysis was used to control for intra-class correlations within each provider. Datasets were analyzed using complete data methodology—in this case, generalized estimating equations (GEE) examining random effects of provider as unit of assignment—and the results pooled by using SAS v9.1 PROC MIXED for the continuous outcomes and PROC GLIMMIX for the categorical outcomes using either the binary or multinomial distributions. Statistical significance of the results was evaluated using a t-test. GEE allows analysis of data for continuous and categorical outcomes.32 All GEEs, which were run with an unstructured variance matrix, used the control group (i.e., non-participating clinicians) as reference. Odds ratios were calculated for each analysis.
A total of 176 participating clinicians accessed the brief survey from the link in the email invitation. 78.1% of the participating clinicians took part in a single CME activity. Another 19.4% participated in 2 activities.
Sixteen participants were removed due to fraudulent practices (e.g., unlicensed providers) or no data. Fraudulent practices included 2 unlicensed physicians returning 11 patient surveys each that they had completed themselves in the same ink, all of which requested payment be mailed to the same address (n = 2). As the “patient” surveys all arrived within a 2–3 day period, they were flagged and excluded by astute research assistants. In other instances, multiple requests from parties with the same last name were flagged as suspicious. If attempts to verify participation of a “suspicious” patient were unanswered by the patient, a participant was deemed fraudulent and excluded. Ninety-two (57.5%) of the remaining clinicians provided contact information. Of these, 38 had patients who returned surveys. However, 2 of these providers did not have any eligible patients, resulting in final sample of 36 providers who had participated in the TTM-based OSA educational activities and recruited eligible patients. The number of eligible patient surveys returned per treatment group provider ranged from 1 to 27, with a mean of 7.1.
Of the 168 non-participating clinicians who accessed the survey, 11 did not provide any data and 5 were excluded for fraudulent practices. Roughly 59% (n = 90) of the remaining clinicians provided contact information. Of these, 42 had patients who returned surveys. However, 3 of these providers did not have any eligible patients, resulting in a final sample of 39 providers who had not participated in any of the educational activities. The number of eligible patient surveys returned per non-participating provider ranged from 1 to 30, with a mean of 11.8. Table 2 describes the specialty of clinician participants included in the patient outcome study. Over half (68.3%) of the providers reported that their specialty was primary care. The remainder of the sample included psychiatrists (24%), pulmonary medicine physicians (6.4%), and geriatricians (1.3%). Nearly half of the primary care participants were physicians (48.4%), and an almost equivalent number were nurses/advanced practice nurses (45.8%).
Patient outcomes study: specialties.
Patient outcomes study: specialties.
A total of 725 patients were recruited by eligible providers to participate in the current study. The majority of patient participants were White, non-Hispanic (70.2%); just over half (51.8%) were female. Ages ranged from 21 to 89, with a mean age of 52.8 (Table 3).
Discussing Sleep Problems
Patients of participating providers were more likely to report that their physician initiated a discussion about sleep compared to patients of non-participating clinicians (43.4% versus 31.7%), t1,411 = 3.71, p = 0.05, OR = 1.67 (see Table 4).
Proportion of clinicians initiating discussion about sleep, administering objective measures, and making referrals by group.
Proportion of clinicians initiating discussion about sleep, administering objective measures, and making referrals by group.
Administering Validated Measures
Epworth Sleepiness Scale (ESS)
As Table 4 indicates, patients of participating clinicians (57.1%) reported that their clinicians were more likely to have asked the ESS at least once in the past compared to non-participating clinician patients (43.3%), t1,533 = 5.41, p < 0.01. Follow-up tests revealed that participating clinicians were more likely than non-participating clinicians to administer the ESS more than once t1,533 = 3.04, p < 0.01, OR = 2.25. No significant differences were found between the patients of participating versus non-participating clinicians on ESS scores.
Significant differences were found for the STOP-BANG, with the patients of participating clinicians endorsing more symptoms of OSA (mean = 4.86) than non-participating clinician patients (mean = 4.09), t1,513 = 7.6, p < 0.01. As shown in Table 4, patients of participating clinicians (47.7%) reported that their clinicians were more likely to have asked the STOP-BANG questions at least once in the past compared to non-participating clinician patients (35.3%), t1,548 = 4.75, p < 0.01. Follow-up tests revealed that the participating clinicians were more likely than non-participating clinicians to administer the STOP-BANG more than once, t1,548 = 3.04, p < 0.01, OR = 2.86.
Referral to Sleep Specialist of Sleep Study
As reported by the patients, participating clinicians (79.9%) were significantly more likely to recommend seeing a sleep specialist compared to non-participating clinicians (60.7%; t1,348 = 9.1, p < 0.01, OR = 2.6; see Table 4). Similarly, while 89.4% of participating clinicians recommended a sleep study, only 73.2% of the non-participating physicians recommended one, t1,363 = 11.46, p < 0.001, OR = 3.1 (see Table 4).
Patients of participating clinicians (73.7%) reported that their clinicians were significantly more likely to make recommendations for further tests/treatment compared to patients of non-participating clinicians (53.1%; t1,591 = 9.81, p < 0.01, OR = 2.72; see Table 4).
The aim of the current analyses was to evaluate the effect of accredited CME activities with a foundation in the TTM on patient-reported outcomes regarding the extent to which clinicians adopted best practices for the identification and optimal management of OSA. The effect of the curriculum was evidenced by changes in how likely clinicians were to screen for OSA and ES and make recommendations for follow-up tests and treatments.
Given the under-recognition of OSA despite its increasing prevalence and the harmful sequelae,3 there is an urgent need to promote performance change related to its identification and management. Patient-reported outcomes indicate that clinicians who participated in one or more of the CME activities were more likely to (1) initiate discussions about sleep problems, (2) administer standardized assessments to diagnose and monitor OSA and excessive sleepiness, (3) recommend a sleep study, (4) refer patients to sleep specialist, and (5) recommend further tests and treatment. Compared to non-CME participation, the odds ratios for implementing well-established diagnosis, assessment, and treatment guidelines for OSA were 1.7 to 3 times higher for clinicians participating in CME activities.
The results described here have important implications for informing interventions to promote performance change related to SRBDs, particularly OSA, a major challenge in today's health care system. The majority of participating clinicians took part in a single CME activity. Almost 20% participated in only two activities. Despite the limited “intervention dose,” the curriculum had a significant impact on clinician behavior, perhaps due in part to the embedded behavior change messages.26 These results add to the growing evidence to support clinician education based on readiness to change.22–24 The findings are similar to those of Shirazi and colleagues,24 who reported that education tailored to different stages of readiness to address depression in primary care enabled clinicians to move to more advanced stages of readiness, and the findings of Johnson et al.26 who reported that TTM-based CME for specialists increased readiness to employ best practices in individualizing early and aggressive treatment of rheumatoid arthritis. The appeal of this type of approach to improve clinical practice is enhanced when one considers how cost-effectively online CME can be provided and the convenience if offers clinicians.
This study had a number of strengths, including reliance on patient-reported outcomes rather than clinician self-report— which incidentally was consistent with the patient-reported data. Other strengths of the study include the inclusion of a comparison group, the nationwide sample of patients and clinicians, and the rigorous definition of the desired performance change. The strength of the analytic approach and use of the physician as the unit of analysis are also noteworthy.
While the sample size was sufficient to detect the effects present, the response rate was lower than expected. Another concern is that self-selection bias was operating among participating clinicians in that only those who had succeeded in adopting the guidelines agreed to participate in this patient study. It is also possible that clinicians invited patients they presumed most likely to return the surveys in an effort to meet the requirements for the incentive. A final concern is that the study did not include a non-TTM education control group for comparison. Thus, the results need to be replicated in a larger study. Future studies may benefit from another means of distributing the assessment to the patient rather than relying on the clinician to deliver them. Alternatively, patient-clinician interactions could be audio-recorded34 or charts audited to obtain objective data. Although the quasi-experimental design may be considered a limitation, randomization was not possible. Inviting non-participating clinicians from the same specialties as participating clinicians reduces some of the concern about lack of a true experimental design.
In conclusion, this study demonstrates that the integration of CME with behavior change messages designed for clinicians in all stages of readiness to implement recommended practices for the identification and optimal management of OSA is a promising approach to increasing the extent to which those guidelines are adopted. While likely not a sufficient solution in and of itself, the fact that the prevalence of sleep-disordered breathing has increased as much as 55% in some subgroups in the past two decades3 indicates that all available tools should be brought to bear on this challenge.
The educational activities were developed through independent educational grants from ResMed, Cephalon, and Philips Respironics. The work was performed at Pro-Change Behavior Systems, Inc. The authors have indicated no financial conflicts of interest.