To revise and enhance the current Behavioral Risk Factor Surveillance System (BRFSS) sleep questions for detection of sleep/wake disorders that contribute to health burden.
A descriptive qualitative design was used to guide the investigation. The three methods were (1) a review of the current evidence on sleep related screening questions (including the results from the parent study validating the current BRFSS questions), (2) interviews with sleep experts about the questions they use in their clinical practice to screen for sleep problems, and (3) interviews with lay people to discuss contextual meanings, feelings, and beliefs about sufficient and restful sleep and not feeling rested.
Recommendations for revisions of the current BRFSS questions.
The current BRFSS questions should be refined to better screen for sleep disorders.
Jungquist CR, Klingman KJ, Dickerson SS. Revisions to the Behavioral Risk Factor Surveillance System sleep questions. J Clin Sleep Med 2016;12(12):1585–1592.
The Centers for Disease Control and Prevention (CDC) conducts ongoing telephone health surveys called the Behavioral Risk Factor Surveillance System (BRFSS) to track health risk behaviors, preventive health practices, and health care access primarily related to chronic disease and injury. These surveys were initiated in the early 1980s and are conducted in the 50 United States including District of Columbia, Puerto Rico, the U.S. Virgin Islands, and Guam. Several countries have also used the BRFSS questions and procedures to develop their own national surveys. In 2015, there were over 400,000 surveys administered across the nation. The survey results are used by many organization as well as state agencies, the CDC, and National Institutes of Health to develop policies, health promotion strategies, and establish funding to address health problems that cause burden to individuals, healthcare systems, and society at large.1
In 1995 one question related to sleep was added to the BRFSS. In 2000 the National Sleep Foundation initiated a review of the sleep related screening questions as the science of sleep had developed. Four additional questions were developed by a panel of sleep experts and the National Sleep Foundation, then added to the survey. The current BRFSS uses five questions to screen for sleep related symptoms that may indicate the presence of a sleep disorder or self-induced sleep restriction, both known to be associated with burden to self, families, work, and society. The current questions are: (BRFSS#1) During the past 30 days for about how many days have you felt you did not get enough rest or sleep? (BRFSS#2) On average, how many hours of sleep do you get in a 24-hour period? Think about the time you actually spend sleeping or napping, not just the amount of sleep you think you should get. (BRFSS#3) Do you snore? (BRFSS#4) During the past 30 days, for about how many days did you find yourself unintentionally falling asleep during the day? and (BRFSS#5) During the past 30 days, have you ever nodded off or fallen asleep, even just for a brief moment, while driving?
Current Knowledge/Study Rationale: The BRFSS is an important and influential tool for establishing health problems in our society. Health burden secondary to sleep disorders and insufficient sleep is significant and warrants sensitive screening questions.
Study Impact: Using qualitative methods, this study resulted in recommendations for changing the current BRFSS sleep questions to include the concepts of (1) insufficient opportunity to sleep, (2) sleep disorders, (3) daytime consequences of insufficient sleep or a sleep disorder, and (4) drowsy driving and/or performing other hazardous activities while excessively sleepy that cause harm to self or others in society. Using the most sensitive and specific questions will result in more accurate estimates of this societal problem and will lead to public health strategies that include awareness programs and funding for future research studies.
In 2012 the CDC funded the authors of this study to perform a study to validate the BRFSS sleep related questions. The two-phased study was performed in Upstate New York between 2012 and 2014. In phase I of the study, the sleep related questions were assessed for validity and reliability as well as for sensitivity and specificity in detecting sleep/wake problems that were associated with health burden. The most serious health burdens associated with sleep problems are automobile accidents from drowsy driving, cardiovascular disease, impaired cognitive and work performance, and mood dysregulation.2–7 The questions were assessed using two main concepts: sleep/wake disorders and insufficient sleep opportunity. Three hundred subjects from the community at large in Upstate New York were recruited using Research Match, posters, and Craigslist. All subjects underwent a one-night screening sleep study to assess for sleep disordered breathing, responded to questionnaires, and wore an PRO-Diary Acti-watch for two weeks.8 During phase II, subjects were interviewed to determine commonly used words as well as subjects' meanings of adequate and inadequate sleep as well as their opinions and understanding of the wording of the current sleep related questions. This report will include the results of phase II. For full results of Phase I, see Jungquist.8
This is a report of a study (Phase II) that was performed to refine the five BFRSS questions. The underlying assumptions used to develop this phase of the study are that screening for sleep problems in society should encompass questions that tap into the presence of: (1) insufficient opportunity to sleep, (2) sleep disorders, (3) daytime consequences of insufficient sleep or a sleep disorder, (4) drowsy driving and/or performing other hazardous activities while excessively sleepy that cause harm to self or others in society.
This study was reviewed and approved by the University at Buffalo and the University of Rochester Institutional Review Boards. A descriptive qualitative design was used to guide the investigation. The three methods were (1) a review of the current evidence on sleep related screening questions (including the results from the parent study validating the current BRFSS question), (2) interviews with sleep experts about the questions they use in their clinical practice to screen for sleep problems, and (3) interviews with lay people to discuss contextual meanings, feelings, and beliefs about sufficient and restful sleep and not feeling rested.
Review of Literature on Validated Questions that Screen for Sleep/Wake Disorders
The objective of the BRFSS sleep screening questions is to efficiently detect sleep problems that result in health burden. Therefore, a review of the literature was completed to discover evidence of validation of questions that screen for sleep disordered breathing, lack of sleep opportunity, and drowsy driving. Search strategy included search of Medline on sleep (title), and screening or questionnaire (keyword). Articles retrieved were 7,382. Limits were placed “humans,” “all adults,” and “past 10 years,” thus reducing articles to 3,594. Ten years was chosen to control for the number of articles needing review. Articles were screened for appropriateness. Three hundred fifty articles were reviewed for study specific criteria. Seventy-five articles were found relevant for studies that validated screening questionnaires, but no validation of single questions was found. A systematic review of screening questionnaires has been recently published by our group.9
The researchers met with the group of 15 sleep experts (5 physicians boarded in sleep medicine with 2 of their fellows in training, 1 dentist certified in sleep medicine, 1 psychologist who works in a sleep disorders center, 2 nurse practitioners working in sleep centers, and 4 nurse sleep researchers) during a one-hour session that started with the group members responding in writing to the BRFSS questions. The results of the parent Phase I study were then presented. The sleep experts were then asked to write down a list of questions that they use during patient interviews in their clinical practice. Participants were then asked to give their professional opinions of the current BRFSS questions and how they would refine the questions. The group meeting was scribed by a research assistant. The research investigators, for verification of accuracy of content, reviewed the transcription of the group discussion.
A structured interview method was used. A random sample of 30 subjects from the parent study was obtained to reflect a cross section of the population studied. To select interview participants, a random number generator was used to yield 30 numbers between 1 and 300. Parent study participants with ID numbers corresponding to those random numbers were then contacted per IRB-approved protocol to request participation in the interviews. If the contacted person was not interested in the interview, another set of random numbers was generated until the sample reached 30 participants. The interviewer was blinded to the participants' demographics and results as well as their individual findings from the parent study.
The structured interviews were performed via telephone and were recorded. Recorded interviews were transcribed by an independent company and then assessed for accuracy of transcription by the research team. During the first part of the interviews, subjects were asked to describe (1) typical good and bad sleep, (2) what contributes to their sleep experience, (3) how they know they had a good night's sleep, and (4) the importance of sleep in their lives.
During the second part of the interviews, the subjects responded to the BRFSS questions and then commented on their interpretations of each of the 5 BRFSS sleep questions. Subjects were encouraged to discuss their thoughts about the wording of the questions and whether they thought the questions required any improvement in wording or response options.
Qualitative methods were used to analyze and synthesize the data from the validation study, review of the literature of current validated screening questions, participant's interviews, and sleep expert interviews.
Transcripts of the participant's interviews were downloaded in NVIVO for analysis by two separate study investigators. The investigators independently read each transcript and identified: themes of words used by the subjects to describe good and bad sleep, common responses to the BRFSS questions, and suggestions for improvement. During research team meetings, all investigators' analyses of the interview interpretations were compared and discussed to identify any discrepancies. Investigators referred back to original transcripts to resolve any discrepancies by consensus. The researchers developed matrixes to compare contextual narrative descriptions of good and bad sleep. Once the themes were identified, the subjects' individual comments as well as responses to the 5 BRFSS questions, their demographics, and results from the Phase I study procedures were revealed. An additional matrix was created with the subject characteristics to identify and verify sleep related issues that emerged from the qualitative findings. The research team used the themes as well as individual responses to evaluate (1) the wording of the original BRFSS questions for appropriateness to access the subjects' sleep related symptoms and (2) structure of the questions and how a person could respond to them.
The research team then met to combine and synthesize the results of the review of the literature (current validated questionnaires [Table 1] and results Phase I BRFSS validation study), interviews with sleep experts, and interviews with participants to achieve consensus on recommended changes to the current BRFSS sleep questions. Discussions and results were developed keeping in mind that the Center for Disease Control and Prevention and administrators of the BRFSS try to keep the number of questions to less than 6 for each disease content and the timeframe for their questions is standardized “to the previous 30 days.”
Summary of questions from instruments that screen for sleep problems.
Summary of questions from instruments that screen for sleep problems.
Summary of Phase I Validation Study Results
BRFSS #1 was found reliable over time but not sensitive to detect lack of sleep opportunity or sleep duration. BRFSS#1 was more closely associated with measures of insomnia, excessive daytime sleepiness, function, pain, and mood disorders. BRFSS #2 was found to be a valid question for obtaining data on the number of hours of sleep. BRFSS#3 was not found to be effective to screen for obstructive sleep apnea (positive likelihood ratio is 3.25 and negative likelihood ratio is 0.67; positive predictive value was 76.62%, and negative predictive value was 59.71%). In the sample of subjects who reported, on average 10% of the past 30 days they unintentionally fall asleep, BRFSS#4 was a reliable question over 14 days only. BRFSS#4 correlated with total score on the Epworth Sleepiness Scale, and was found to be associated with higher fatigue, decreased physical function, and higher AHI. BRFSS#5 is used to screen for drowsy driving. As it was impossible to assess the predictive value of falling asleep while driving, the question was tested against the Epworth Sleepiness Scale to detect a total score > 10 on the ESS (> 10 is considered clinically relevant excessive sleepiness). BRFSS#5 was found 86.76% (95%CI 76.35 to 93.75) sensitive and 65.46% (95% CI 59.82 to 70.80) specific with a PPV of 35.98% (95% CI 28.94 to 43.83) and NPV of 95.67 (95% CI 91.94 to 98.00).
Review of the Literature
Screening Questionnaires for Sleep Disorders and Disturbance in General
Seven validated questionnaires that screen for sleep disorders were found. Two of the seven contained questions that screen for lack of sleep opportunity. Of the seven questionnaires, three contain succinct questions screening for sleep disorders. The Global Sleep Assessment Questionnaire (GSAQ), Holland Sleep Disorders Questionnaire (HSDQ), and the Auckland Sleep Questionnaire (ASQ) are efficient and validated to screen for and differentiate certain types of sleep disorders.10–12 See Table 1 for comparison of validated questionnaires and questions.
Screening Questionnaires that Only Screen for Obstructive Sleep Apnea
There are three instruments commonly used to screen for obstructive sleep apnea that have undergone extensive testing in several populations: (1) Berlin, (2) STOP, and (3) the Epworth Sleepiness Scale. The Berlin questionnaire consists of 10 questions that ask about (1) snoring, (2) grade of loudness of snoring, (3) witnessed pauses in breathing during sleep, (4) feeling tired, fatigued, or not up to par during the day, (5) nodding off while driving, and (6) diagnosis of hypertension. The STOP questionnaire consists of 4 questions that ask (1) do you snore loudly, (2) do you often feel tired, fatigued or sleepy during the day, (3) has anyone observed you stop breathing during sleep, (4) have you ever been told you have high blood pressure. The Epworth Sleepiness Scale was developed to assess daytime sleepiness but has been used to screen for OSA also. The questionnaire asks how likely you are to fall asleep in 8 different settings during the day. Unfortunately all the questionnaires are only fair predictors of OSA with ROC values ranging 0.63 to 0.75 in studies.13–16 There is also evidence that the STOP-BANG questionnaire, which adds physical measurement of neck circumference, as well as age, gender, and BMI to the STOP questions, may result in false negative findings in 18% of patients screened in the preoperative setting.17
This sample was predominantly Caucasian females with a mean age of 39 years. See Table 2 for sample characteristics of Phase II cohort. Fifty percent of the Phase II cohort reported clinically relevant mild insomnia (ISI total score > 7) but scored normal on the measures of mood and function.
Phase II sample characteristics (n = 30).
Phase II sample characteristics (n = 30).
Expert Opinion Interviews
There was agreement during group discussion that the current: (1) BRFSS #1 question “during the past 30 days for about how many days have you felt you did not get enough rest or sleep” does not specifically differentiate between having lack of sleep opportunity or having a sleep disorder, (2) BRFSS #3 was not sufficient to screen for OSA as a stand-alone question and that there are many people who sleep alone and have no idea if they snore, (3) BRFSS #5 asks specifically about drowsy driving, yet there are plenty of occupational injuries that occur from falling asleep while working. The group agreed that BRFSS questions #2 and #4 were efficient, well-worded, and necessary.
A summary of conclusions made from interviews with subjects and recommendations for refining questions is listed per BRFSS question below. When considering timeframe to base responses, in general, the interviewer noticed that subjects with poorer sleep had some hesitation when answering questions that asked them to remember back 30 days. They often answered in ranges or said it was hard to remember.
BRFSS#1: During the past 30 days for about how many days have you felt you did not get enough rest or sleep?
Interviews revealed BRFSS#1 to be an easily understood question. See Table 3 for summary of comments. In general, some subjects commented that their sense of feeling rested was different on weekdays than weekends, and varied according to their work schedules and seasons of the year. There were also comments about having difficulty with recalling how they felt 30 days ago.
Participant Interviews (n = 30): comments regarding each BRFSS question.
Participant Interviews (n = 30): comments regarding each BRFSS question.
BRFSS#2: On average, how many hours of sleep do you get in a 24-hour period? Think about the time you actually spend sleeping or napping, not just the amount of sleep you think you should get.
Interviews revealed the general opinion that BRFSS#2 was an easily understood question. Comments included minor difficulty with answering if a regular work and sleep schedule was not kept.
Subjects had great difficulty in answering this question with yes or no. There was usually a caveat about sleeping alone and never been told they snore, snoring only when they have allergies or a cold, or only snoring when in one sleep position such as on their back or after drinking alcohol. Interviews with subjects revealed their difficulty answering this question. Most recommended that the question be changed to reflect more certainty, such as asking if anyone had ever told them that they snored or being able to answer using an open ended response.
BRFSS#4: During the past 30 days, for about how many days did you find yourself unintentionally falling asleep during the day?
Thematic analysis revealed that many subjects don't allow themselves to actually fall asleep, but do report that they feel like they were struggling to stay awake. Individual comments revealed some subjects thought the question was asking if they had narcolepsy. In discussion about how the subjects would recognize that they were excessively sleepy during the day, they reported the urge to sleep and having difficulty concentrating with lack of focus and attention as well as experiencing heavy eyes. Subjects associated the sensation of sleepiness with being drowsy, tired, and/or having a low blood sugar. Discussions revealed that the question could be modified to include wakefulness producing behaviors. Subjects expressed that modifying the question would better capture excessive sleepiness.
BRFSS#5: During the past 30 days, have you ever nodded off or fallen asleep even just for a brief moment while driving?
Automatic responses to this question revealed that some subjects felt morally judged by this question and that most people would not allow themselves to nod off or fall to sleep while driving. Therefore. it is likely that the question in this format would suffer from social desirability bias. Other concerns were that if the person did not drive, the screen would miss excessive sleepiness occurring in other unsafe situations, such as operating heavy equipment or working on a manufacturing line. All subjects readily described behaviors of what they do to prevent themselves from nodding off at inappropriate times.
Summary of Results: Review of Literature, Expert Opinion, and Participant Interviews
Once all interviews with subjects and sleep experts were completed and transcribed, the research team met to interpret the findings. Cumulative findings are presented per content category.
Screening for Insufficient Opportunity to Sleep
The BRFSS sleep questions currently do not contain a question that specifically asks if the person is not getting enough sleep due to circumstances in their life. There are currently two questions that are being used as part of screening questionnaires: (1) Did work or other activities prevent you from getting enough sleep and (2) Despite having plenty of opportunity to sleep in, I don't get enough sleep. The problem with both of these questions is the word “enough.” Using the word enough allows for subjective determination on how many hours is enough sleep for them. There is now evidence that adults require on average at least 7 hours of sleep for optimum health.18 Therefore, the recommended wording for the question screening for lack of sleep opportunity is “Do work or other activities prevent you from getting at least 7 hours sleep?”
Screening for Sleep Disorders
The current BRFSS sleep questions do not include a question that specifically screens for insomnia. The most common sleep disorder is insomnia, and patients with other sleep disorders will also endorse diagnostic criteria for insomnia. See Table 1 for questions commonly used in questionnaires. As it is common for patients to think that they sleep “fine” yet wake in the morning feeling unrestored, there was consensus that the question should include non-restorative sleep. The question that appears succinct and inclusive of diagnostic criteria of insomnia and non-restorative sleep is “Do you have difficulty falling asleep, staying asleep, or do you feel poorly rested in the morning?”
As obstructive sleep apnea is known to be associated with negative health consequences and motor vehicle accidents, it is likely that the Center for Disease Control and Prevention would prefer to have a question that is sensitive and specific for obstructive sleep apnea. As there is not one question that has been successfully validated to screen for OSA, and the BRFSS prefers brief questions, we are recommending a combination of signs and symptoms into one question, “Have you been told that you hold your breath, have breathing pauses, snore and/or stop breathing in your sleep?”
Screening for Daytime Consequences of Insufficient Sleep or a Sleep Disorder
Fatigue and daytime sleepiness are the most frequently reported symptoms related to insufficient sleep or sleep disorders. But daytime consequences of poor or insufficient sleep also include inability to concentrate, mood changes, irritability, memory problems, and lack of energy as per the participant interviews. BRFSS # 4 is an effective screening question for excessive daytime sleepiness when comparing to the Epworth Sleepiness Scale, but could be refined to capture symptoms that occur just before actually unintentionally nodding off. An example could be “During the past month, how many days have you felt so tired that your eyes were heavy, your head was nodding, and you felt the urge to go to sleep at a time when you should be awake?”
Although excessive daytime sleepiness is the most severe and dangerous daytime consequence of poor or insufficient sleep, the concept does not capture more common daytime consequences of poor or insufficient sleep. See Table 1 for examples of questions used to assess daytime consequences. We are recommending an additional question such as “Do you suffer from fatigue, sleepiness, bad mood, poor concentration, memory problems, or lack of energy related to insufficient or non-restorative sleep more than 3 days a week?”
Screening for Drowsy Driving and/or Performing Other Hazardous Activities while Excessively Sleepy that Cause Harm to Self or Others in Society
There are two questions found in the literature that are used to screen for drowsy driving: the BRFSS question and “have you fallen asleep at the wheel and not crashed.”19–21 As the CDC is interested in capturing drowsy driving specifically, we recommend that the wording of the question actually capture drowsy driving instead of falling asleep at the wheel. For a question that focuses on just driving, the recommendation is “During the past month have you felt the uncontrollable urge to sleep or fallen asleep while driving?”
In cases where the question should encompass excessive daytime sleepiness in the general public, a more inclusive question would be “During the past month have you felt the urge to sleep or fallen asleep while driving or doing a hazardous task?”
Screening for sleep disturbance and sleep disorders can be effectively performed using a small set of questions, but the questions may differ according to the mission of the survey. When screening for sleep problems in society, there are two important and often independent concepts that must be covered: insufficient sleep opportunity and the presence of a sleep disorder. Using qualitative methods, this study resulted in recommendations for changing the current BRFSS sleep questions to include the concepts of (1) insufficient opportunity to sleep, (2) sleep disorders, (3) daytime consequences of insufficient sleep or a sleep disorder, and (4) drowsy driving and/or performing other hazardous activities while excessively sleepy that cause harm to self or others in society. These recommendations are based on the uses of the BRFSS surveys that includes prevalence of sleep disorders and insufficient sleep in society and the consequences of the before-mentioned such as impaired daytime function and motor vehicle or work accidents. Using the most sensitive and specific questions will result in more accurate estimates of this societal problem and will lead to public health strategies that include awareness programs and funding for future research studies.
When developing questions for a telephone survey, it is important to encompass brevity, clarity, and wording that is commonly understood by the population at large. It is also important to word the questions to achieve answers that can be accurately and easily analyzed and understood. The BRFSS commonly anchors their questions “in the past 30 days.” Therefore, the questions recommended in this study are anchored in this time frame. The questions were worded for either a yes/no response or anchored in number days out of the past 30 days.
Scoring of responses will at times require adding together responses from two or more questions. For example, when inquiring about insufficient sleep opportunity, the question “Do work or other activities prevent you from getting at least 7 hours sleep?” could be combined with “On average, how many hours of sleep do you get in a 24-hour period? Think about the time you actually spend sleeping or napping, not just the amount of sleep you think you should get.” to quantify how much sleep the person reports getting. These questions could also be combined with “Do you suffer from fatigue, sleepiness, bad mood, poor concentration, memory problems, or lack of energy more than 3 days a week?” to affirm that insufficient sleep also results in impaired daytime function. The comparison of these questions will also allow the investigator to assess if receiving less than 7 hours (recommended by Academy of Sleep Medicine) is because of life events resulting in lack of sleep opportunity or if the person is a “short sleeper” or perhaps has a sleep disorder.
When screening for sleep disorders, it is important to establish if difficulty initiating and/or maintaining sleep is resulting in daytime consequences. This assessment will require the investigator to use the results from more than one question. An example would be combining the results of “Do you have difficulty falling asleep, staying asleep, or do you feel poorly rested in the morning?” with “Do you suffer from fatigue, sleepiness, bad mood, poor concentration, memory problems, or lack of energy more than 3 days a week?” The investigator could then add the results of “Have you been told that you hold your breath, have breathing pauses, snore and/or stop breathing in your sleep?” to differentiate if the sleep disorder may be obstructive sleep apnea as opposed to just insomnia.
Qualitative approaches promote understanding of the meaning of everyday experiences in the context of the individuals' experiences and help with developing questions that are understood by the target audience. Qualitative studies do not provide objective validation and sensitively testing.
Attention to assessing the appropriateness of the timeframe of recall (30 days) was mostly ignored in this study. According to the CDC, the 30-day timeframe is the standard for most of the BRFSS questions. Therefore, the investigators of this study did not pay special attention to testing if this time frame was best for accessing accurate sleep data.
Before the refined questions should be adopted, a validation study is needed to document sensitivity and specificity of the newly developed questions. Traditionally, the number of BRFSS screening questions for each disease is held to a minimum to allow for brevity of the survey. This constricts the development of an increased number of questions that may be necessary to capture health burden that is specifically related to sleep.
The BRFSS is an important and influential tool for establishing health problems in our society. Health burden secondary to sleep disorders and insufficient sleep is significant and warrants sensitive screening questions. Although the current questions are helpful, improving their sensitivity is necessary to accurately detect sleep related problems that will in turn allow for increased state and federal funding for treatment and prevention research and programs.
This was not an industry supported study. This manuscript was supported by Cooperative Agreement Numbers U48DP001910 and U48DP001910-05S1 from The Centers for Disease Control and Prevention (CDC). Its contents are solely the responsibility of the authors and do not necessarily represent the official views of the Centers for Disease Control and Prevention. The authors have indicated no financial conflicts of interest. The study was conducted at the University at Buffalo under contract with University of Rochester.