The Bordeaux Sleepiness Scale (BOSS): a new questionnaire to measure sleep-related driving risk
ABSTRACT
Study Objectives:
Sleepiness is a well-known risk factor for traffic accidents. Our study presents a new questionnaire, the Bordeaux Sleepiness Scale (BOSS), specifically designed to evaluate sleep-related driving risk in patients with sleep disorders.
Methods:
The BOSS was designed by gathering data on sociodemographics, sleepiness, driving items, and traffic accident exposure (kilometers driven) in the past year of 293 patients followed for sleep disorders at a French sleep clinic. It was then validated on data from a large population-based cohort of 7,296 highway drivers. Its performance was compared to the Epworth sleepiness scale and to self-reported episodes of severe sleepiness at the wheel. Receiver operating characteristic curves were computed.
Results:
The sensitivity and specificity of the BOSS (cutoff = 3) to predict sleep-related near-misses or accidents was, respectively, 82% and 74%, with an area under the receiver operating characteristic curve of 0.83. In a cohort of patients and a large population-based cohort, the area under the curve of the BOSS was significantly larger than that of the Epworth sleepiness scale (P < .001). Although the areas under the curve were equivalent between the BOSS and sleepiness at the wheel, the specificity of the BOSS was higher.
Conclusions:
The BOSS scale combining exposure (kilometers driven) and self-perception of situational sleepiness provides a simple and reliable evaluation of sleep-related driving risk. This short, specific questionnaire should be promoted as a first-line tool to evaluate the risk of traffic accidents in sleepy patients.
Citation:
Philip P, Micoulaud-Franchi J-A, Taillard J, et al. The Bordeaux Sleepiness Scale (BOSS): a new questionnaire to measure sleep-related driving risk. J Clin Sleep Med. 2023;19(5):957–965.
BRIEF SUMMARY
Current Knowledge/Study Rationale: Sleepiness at the wheel is a well-known risk factor for traffic accidents. While the ability of untreated and treated patients to drive is attracting growing interest, there is still no specific questionnaire to predict accident risk in sleep-disorder patients defined by the occurrence of near-misses or sleep-related traffic accidents.
Study Impact: This study proposes a novel questionnaire to measure the risk of sleepiness-related traffic accidents compared to reference questionnaires used in clinical practice. The Bordeaux Sleepiness Scale is a short, specific tool that should be promoted as a first-line tool for evaluating the risk of traffic accidents.
INTRODUCTION
Excessive daytime sleepiness (EDS) is a common symptom among the general population.1–3 It is also a very common risk factor for car accidents, causing many deaths and injuries and carrying a high socioeconomic burden.4 A previous study on a large sample of highway drivers showed that male sex and exposure to driving were associated with the occurrence of sleep-related accidents.5 Self-reported sleepiness-related near-misses and accidents in the past years are also very strong predictors of future accident risk.6–9 Importantly, 10–20% of car accidents are caused by drivers falling asleep at the wheel.10–12 EDS is also strongly associated with sleep disorders. Obstructive sleep apnea syndrome (OSAS) is the most frequent sleep disorder; more than 8% of the general population report nocturnal breathing disorders,13 and the vast majority of these patients report nocturnal sleep complaints and EDS. Studies confirmed that patients with OSAS present a 2-fold greater risk of being involved in sleep-related accidents than drivers free of sleep disorders.5,14 Since then, OSAS has been recognized by the European Respiratory Society as a risk factor for road accidents,15 and the European Respiratory Society suggests that physicians should systematically explore EDS in these patients for their potential accident risk.16
Because of its frequency, EDS deserves to be explored by simple questionnaires before objective measurements like the Multiple Sleep Latency Test or the Maintenance of Wakefulness Test are administered. Standardized scales have been developed such as the Stanford sleepiness scale,17 the Karolinska sleepiness scale,18 and the Epworth sleepiness scale (ESS),19 which is the most widely used. The latter measures the propensity of falling asleep in very heterogeneous situations from reading a book to talking to someone. Two specific questions from the scale specifically address the occurrence of sleepiness in automobiles. Several studies have investigated its ability to predict traffic accidents.8,9,20–25 Results, however, are heterogeneous, possibly because sleepiness in the context of transport accounts for only 25% of the global score (2 questions out of 8 relate to in-car situations). Since the launch of the ESS in 1991,19 several studies have highlighted the specific accident risk related to EDS.8,10,21–25 With further investigation, it turned out that specific questions on sleepiness while driving were strongly related to the risk of near-misses and/or sleep-related accidents.5,20 Sleepiness at the wheel (SAW) is a single Likert-scale question on the frequency of occurrence of episode of severe sleepiness at the wheel (Never, < 1/month, ≥ 1/month, ≥ 1/week).5,20,26 A recent study on a sample of 71,017 patients recorded for suspicion of nocturnal breathing disorders showed that self-reported SAW was a very good predictor of sleep-related traffic accidents.23 In the same study, a 3-point increase on the ESS accounted for a 1.23-fold increased risk of accidents. A meta-analysis showed that reporting sleepiness while driving carried a 2.5-fold increased risk of having a traffic accident.20 In a recent study on patients routinely investigated in a French sleep clinic,9 more than 90% of patients who reported a sleepiness-related traffic near-miss or accident in the past year (30% of the sample) reported at least 1 severe sleepiness episode at the wheel in the past year and 77% had EDS (ESS ≥ 11). On the other hand, 35% of patients who did not report a sleepiness-related traffic near-miss or accident in the past year reported at least 1 severe sleepiness episode at the wheel in the past year and 40% had EDS (ESS ≥ 11). Therefore, both scales (ie, ESS and SAW) identify at-risk patients but tend to overestimate the risk of patients not involved in sleep-related accidents. Some studies found that drivers with chronic sleepiness on the ESS have an increased risk of accidents,8,21,23,24 whereas others did not.22,25 A lack of consensus in clinicians’ judgment of fitness to drive is common for both untreated and treated patients with OSAS.27,28 A limitation of patient self-reporting is related to the fact that they do not rely on external data from Department of Motor Vehicles reports, accounting possibly for underestimation of the actual rate of accidents. However, an agreement is emerging regarding self-reported sleepiness while driving and past experience of sleepiness at the wheel, which are associated with an increased risk of traffic accidents.8,20,26,29,30
In summary, the issue regarding the best predictive scale to evaluate the sleep-related risk of accidents remains unanswered.31 Therefore, we decided to design a new tool to explore sleep-related driving risk and compare its sensitivity and specificity to the reference for measuring EDS (ie, the ESS). The Bordeaux Sleepiness Scale (BOSS) was aimed at assessing sleep-related driving risk in patients with sleep disorders followed in sleep centers. Validity of the new questionnaire was explored using 2 independent databases, one of patients with sleep disorders followed in a sleep center (ie, sleep center cohort) and one of regular drivers on highways from the general population (ie, internet-based cohort).
METHODS
From October 2017 to March 2020, patients followed for sleep disorders at the University Hospital of Bordeaux sleep clinic (France) completed questionnaires during their clinical follow-up on sleepiness and traffic accident exposure in the past year. A minimal selection was made in order to obtain a representative sample of patients consulting at a sleep clinic (sleep clinic cohort).
The study was sponsored by a grant from the French Sleep Society (Société Française de Recherche et Médecine du Sommeil) and received the agreement of its ethical committee. In accordance with the French ethics legislation, observational studies from data obtained without any additional therapy or monitoring procedure do not require written consent. Participants were informed of the objectives of the study and were not opposed to the collection of their data. Data were collected and stored anonymously, according to the recommendations of the French Data Protection Authority, which ensures the ethical use of medical and scientific data collected for research purposes.
Participants
Patients with a sleep disorder were included if they (1) had no treatment or a stable treatment in the past year or just had a minimal change (not affecting the average level of alertness over the day) of treatment; (2) had their driving license and were currently driving; and (3) were willing to take part in the study. Patients affected by sleep disorders (sleep-related breathing disorders, central disorders of hypersomnolence, sleep-related movement disorders, insufficient sleep syndrome) were diagnosed at the Bordeaux University Hospital Sleep Clinic based on the Third edition of the International Classification of Sleep Disorders.32 The diagnosis was established by a board-certified sleep specialist who carried out the patients’ clinical evaluations.
Questionnaires
Patients completed a questionnaire exploring whether they had had traffic near-misses or actual accidents related to sleepiness in the past year, as well as the ESS.19 Their sociodemographic characteristics, medical history, professional status, working time, driving behaviors, and frequency of severe sleepiness episodes at the wheel were also collected.
BOSS
The BOSS was generated a posteriori by using selected items from questionnaires (ie, sociodemographic characteristics, clinical characteristics including ESS and driving behaviors/events) completed by patients during their consultation. SAW is the best-known predictor of sleep-related near-misses and accidents reported in the past year.5,20,26 This question refers to the occurrence of episodes of sleepiness at the wheel in terms of frequency (Never, < 1/month, ≥ 1/month, ≥ 1/week). Improving its psychometric properties and its ability to discriminate at-risk patients8,9,20,23 could be done adding an intensity component of SAW and known risk factors in accidentology in the clinical population consulting in sleep centers. The intensity of SAW (never, slight, moderate, or high chance of dozing) is evaluated by 2 items in the ESS relating to in-car situations (item 4: As a passenger in a car for an hour without a break and item 8: In a car while stopped for a few minutes in traffic).
Within the scope of the internal consistency of the questionnaire, the sex of the patients, the distance driven per year, the item closest to a monotonous and undemanding situation from the ESS for its intensity component, and the history of severe sleepiness episodes while driving for its frequency component were retained. The sum of the scores for these items constitutes the global score on the BOSS. Scores range from 0 to 8 (see supplemental material).
No weight ponderation of items was performed given that the best predictors of near-misses and accidents are the items related to situational sleepiness at the wheel and both these items were already rated on a 4-point scale vs a binary response for the other 2 items (ie, sex and distance driven per year).
Statistical analyses
Descriptive statistics for quantitative variables (mean ± standard deviation) and for qualitative variables (frequency and percentage) were expressed for sociodemographic, clinical, and driving characteristics in patients who did or did not self-report a sleep-related traffic near-miss or accident in the past year.
Between-group comparisons were performed with the Mann-Whitney U test for continuous variables or the chi-square test (χ2) for categorical variables to compare groups.
Predictive validity of occurrence of sleep-related traffic near-miss or accident and external validity (vs ESS and SAW) were tested. Receiver operating characteristic (ROC) curves33 were computed to evaluate the performance of a binary classifier system (ie, the performance of the BOSS in detecting patients who self-reported sleep-related traffic near-misses or accidents). ROC curves determine the area under the curve (AUC) with 95% confidence intervals (CI), sensitivity/specificity, and positive/negative predictive values. These parameters are a measure of how well a parameter can distinguish between 2 groups (case and control). Sensitivity refers to the probability that a test result is positive when the event is present (true positive). Specificity refers to the probability that a test result is negative when the event is not present (true negative). Positive predictive value refers to the probability that the event is present when the test is positive. Negative predictive value refers to the probability that the event is not present when the test is negative.
External validity on an independent cohort (internet-based cohort) was explored by reiterating these analyses.
Data analyses were carried out using the SPSS statistical software package (Version 18, PASW Statistics, IBM, Armonk, New York) and MedCalc software (Version 14.8 for Windows, MedCalc Software Ltd, Ostend, Belgium). The alpha criterion was set at P ≤ .05.
RESULTS
Demographic, clinical, and driving characteristics (sleep clinic cohort)
The flow chart of the sleep clinic cohort in Figure 1 shows that 560 patients were screened, of whom 293 were included (101 cases who self-reported sleep-related traffic near-misses or accidents in the past year and 192 controls who did not self-report sleep-related traffic near-misses or accidents in the past year). Sociodemographic, clinical, and driving characteristics are reported in Table 1 in patients who did or did not self-report a sleep-related traffic near-miss or accident in the past year.
| All Patients | Self-Reported Sleep-Related Traffic Near-Miss or Accident | P | ||
|---|---|---|---|---|
| No | Yes | |||
| n | 293 | 192 (65.5%) | 101 (34.5%) | |
| Sociodemographic characteristics | ||||
| Age (years) (mean ± SD) | 47.7 ± 14.7 | 48.5 ± 14.6 | 46.3 ± 14.7 | ns |
| Male, n (%) | 184 (65.9%) | 119 (66.1%) | 65 (65.7%) | ns |
| Marital status, n (%) | ||||
| Married | 160 (54.6%) | 105 (54.7%) | 55 (54.5%) | ns |
| Single | 81 (27.6%) | 58 (30.2%) | 23 (22.8%) | |
| Divorced | 46 (15.7%) | 27 (14.1%) | 19 (18.8%) | |
| Workers, n (%) | 187 (63.8%) | 116 (60.4%) | 71 (70.3%) | ns |
| Work, n (%) | ||||
| Diurnal work | 143 (76.5%) | 90 (77.6%) | 53 (74.6%) | ns |
| Nocturnal work | 6 (3.2%) | 5 (4.3%) | 1 (1.4%) | |
| Shift work | 33 (17.7%) | 19 (16.3%) | 14 (19.8%) | |
| Clinical characteristics | ||||
| BMI (kg/m2) (mean ± SD) | 28.6 ± 6.8 | 28.9 ± 7.1 | 28.1 ± 6.1 | ns |
| BMI > 30, n (%) | 106 (38.7%) | 73.0 (40.6%) | 33 (35.1%) | ns |
| ESS | ||||
| ESS score (mean ± SD) | 11.4 ± 5.4 | 9.8 ± 5.1 | 14.2 ± 4.7 | <.001 |
| ESS score ≥ 11, n (%) | 161 (57.3%) | 84 (45.7%) | 77 (79.4%) | <.001 |
| ESS score ≥ 16, n (%) | 70 (24.9%) | 28 (15.2%) | 42 (43.3%) | <.001 |
| BOSS score (mean ± SD) | 2.6 ± 1.9 | 1.8 ± 1.5 | 4.0 ± 1.7 | <.001 |
| Alcohol addiction, n (%) | 6 (2.1%) | 6 (3.1%) | 0 (0.0%) | ns |
| Use of anxiolytic/hypnotic, n (%) | 19 (6.6%) | 11 (5.9%) | 8 (8.2%) | ns |
| OSAS, n (%) | ||||
| Not treated | 59 (33.1%) | 38 (32.5%) | 21 (34.4%) | ns |
| Treated with CPAP or DO | 119 (66.9%) | 79 (67.5%) | 40 (65.6%) | |
| Narcolepsy or idiopathic hypersomnia, n (%) | ||||
| Not treated | 15 (22.4%) | 9 (19.6%) | 6 (28.6%) | ns |
| Treated drug | 52 (77.6%) | 37 (80.4%) | 15 (71.4%) | |
| Insufficient sleep syndrome, n (%) | 13 (4.6%) | 5 (2.7%) | 8 (8.2%) | <.05 |
| Restless legs syndrome, n (%) | 18 (6.3%) | 12 (6.4%) | 6 (6.2%) | ns |
| Driving behaviors | ||||
| Kilometers driven per year, n (%) | ||||
| ≥ 20,000 km | 66 (47.8%) | 33 (38.8%) | 33 (62.3%) | <.01 |
| Professional driver, n (%) | 137 (47.4%) | 80 (42.6%) | 57 (56.4%) | <.05 |
| Severe sleepiness episode at the wheel in past year, n (%) | ||||
| No, never | 128 (43.7%) | 122 (63.5%) | 6 (5.9%) | <.001 |
| < 1/month | 71 (24.2%) | 42 (21.9%) | 29 (28.7%) | |
| ≥ 1/month | 46 (15.7%) | 16 (8.3%) | 30 (29.7%) | |
| ≥ 1/week | 48 (16.4%) | 12 (6.3%) | 36 (35.6%) | |
Patients self-reporting or not self-reporting sleep-related traffic near-misses or accidents
Patients were predominantly middle-aged, overweight males. More than half of them had EDS (ESS score ≥ 11) and a quarter of these had severe EDS (ESS score ≥ 16). They were evaluated for OSAS (62.5%), idiopathic hypersomnia/narcolepsy (23.5%), restless legs syndrome (6.3%), and insufficient sleep syndrome (4.6%). Almost two-thirds of the sample were mainly day workers. About half of them were professional drivers, explaining why half of the sample drove more than 20,000 km/y. More than half of them had experienced at least 1 episode of severe sleepiness at the wheel in the past year.
More than a third (101/293 patients, 34.5%) of the sleep clinic cohort reported a sleep-related traffic near-miss or accident in the past year. A higher proportion of patients with a near-miss or accident had EDS (79.4% had an ESS score ≥ 11 and 43.3% had an ESS score ≥ 16) compared to patients without a near-miss or accident (45.7% had an ESS score ≥ 11 and 15.2% had an ESS score ≥ 16) (χ2 = 29.533 [degrees of freedom (df) = 1]), P < .001 and χ2 = 26.777 [df = 1], P < .001, respectively). Patients who reported a sleep-related traffic near-miss or accident in the past year were more exposed to driving (56.4% were professional drivers and 62.3% drove more than 20,000 km/y) than those without a traffic near-miss or accident (42.6% were professional drivers and 38.8% drove more than 20,000 km/y) (χ2 = 5.079 [df = 1], P < .05 and χ2 = 7.188 [df = 1], P < .01, respectively). Patients with a sleep-related traffic near-miss or accident were more likely to experience sleepiness at the wheel in the past year (χ2 = 105.699 [df = 3], P < .001) (Table 1).
Patients reporting or not reporting severe sleepiness episode at the wheel in the past year
More than half of the patients (165/293 patients, 56.3%) of the sleep clinic cohort reported at least 1 episode of severe sleepiness at the wheel that made driving difficult or forced them to stop the car in the past year. A higher proportion of patients with a severe sleepiness episode at the wheel were workers (70.3% vs 55.5% of those who reported no severe sleepiness episode at the wheel, χ2 = 16.750 [df = 7], P < .05). They experienced more EDS (75.6% had an ESS score ≥ 11 and 36.9% had an ESS score ≥ 16) compared to patients without a severe sleepiness episode at the wheel (33.1% had an ESS score ≥ 11 and 9.1% had an ESS score ≥ 16), (χ2 = 51.022 [df = 1], P < .001 and χ2 = 28.433 [df = 1], P < .001, respectively). Patients who reported a severe sleepiness episode at the wheel in the past year were more exposed to driving (56.1% drove more than 20,000 km/y) than those who reported no severe sleepiness episode at the wheel (35.7% drove more than 20,000 km/y) (χ2 = 5.541 [df = 1], P < .05).
Conditions of occurrence of self-reported sleep-related traffic near-misses or accidents
Sleep-related traffic near-misses or accidents occurred preferentially during the day in the morning and early afternoon (59.5%) rather than at night (3.0%). More than half of the patients considered that they had slept enough (53.5%) and that their sleep was of good quality the night before the accident (62.4%). There was no difference in the characteristics of sleep-related traffic near-misses or accidents in patients who did or did not experience at least 1 severe sleepiness episode at the wheel in the past year or in the consumption of drugs or alcohol.
Performances of the BOSS (sleep clinic cohort)
The statistical ROC analyses of ESS, SAW, and the BOSS in the sleep clinic cohort are shown in Table 2. ROC curves of BOSS and ESS are presented in Figure 2.
| Psychometric Properties for Identifying Sleep-Related Traffic Near-Miss or Accident | AUC | P | Criterion | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value |
|---|---|---|---|---|---|---|---|
| Sleep clinic cohort (n = 293) | |||||||
| ESS | 0.73 [0.68–0.78] | <.001 | >13 | 0.63 [0.52–0.72] | 0.75 [0.68–0.81] | 0.57 [0.47–0.66] | 0.79 [0.72–0.85] |
| ESS ≥ 11 | 0.67 [0.61–0.72] | <.001 | ≥11 | 0.79 [0.70–0.87] | 0.54 [0.47–0.62] | 0.48 [0.40–0.56] | 0.83 [0.75–0.89] |
| ESS ≥ 16 | 0.64 [0.58–0.70] | <.001 | ≥16 | 0.43 [0.33–0.54] | 0.85 [0.79–0.90] | 0.60 [0.48–0.71] | 0.74 [0.67–0.80] |
| SAW | 0.84 [0.80–0.88] | <.001 | >0 | 0.94 [0.87–0.98] | 0.63 [0.56–0.70] | 0.58 [0.50–0.65] | 0.95 [0.90–0.98] |
| BOSS | 0.83 [0.79–0.87] | <.001 | >2 | 0.82 [0.73–0.89] | 0.74 [0.68–0.80] | 0.63 [0.54–0.71] | 0.88 [0.83–0.93] |
| Internet-based cohort (n = 7,296) | |||||||
| ESS | 0.67 [0.66–0.68] | <.001 | >9 | 0.60 [0.57–0.62] | 0.65 [0.64–0.66] | 0.29 [0.27–0.30] | 0.87 [0.86–0.88] |
| ESS ≥ 11 | 0.62 [0.61–0.64] | <.001 | ≥11 | 0.50 [0.48–0.53] | 0.74 [0.73–0.75] | 0.31 [0.29–0.33] | 0.87 [0.86–0.87] |
| ESS ≥ 16 | 0.54 [0.53–0.55] | <.001 | ≥16 | 0.11 [0.09–0.12] | 0.97 [0.96–0.97] | 0.45 [0.40–0.51] | 0.82 [0.81–0.83] |
| SAW | 0.78 [0.77–0.79] | <.001 | >0 | 0.96 [0.94–0.97] | 0.45 [0.44–0.47] | 0.29 [0.28–0.31] | 0.98 [0.97–0.98] |
| BOSS | 0.76 [0.75–0.77] | <.001 | >2 | 0.71 [0.69–0.73] | 0.69 [0.68–0.70] | 0.35 [0.33–0.37] | 0.91 [0.90–0.92] |
BOSS = Bordeaux Sleepiness Scale, ESS = Epworth Sleepiness Scale.
Discrimination performances
ROC analysis discrimination of patients who reported or did not report a sleep-related traffic near-miss or accident in the past year showed an AUC of 0.67 (95% CI, 0.61–0.72), P < .001 for ESS ≥ 11 and an AUC of 0.64 (95% CI 0.58–0.70), P < .001 for ESS ≥ 16. ESS ≥ 11 had a higher level of sensitivity (79%) than of specificity (54%); ESS ≥ 16 had a lower level of sensitivity (43%) than of specificity (85%).
ROC analysis discrimination of patients who reported or did not report a sleep-related traffic near-miss or accident in the past year showed an AUC of 0.84 (95% CI 0.80–0.88), P < .001 for the SAW. For the BOSS, the AUC was 0.83 (95% CI 0.79–0.87, P < .001). Regarding prediction of the risk of having a sleep-related traffic near-miss or accident, the BOSS had a sensitivity of 82% and a specificity of 74% at a cutoff ≥ 3 out of 8.
Differences between AUCs
The AUC of the BOSS was significantly higher than that of the ESS (difference between areas = 0.100, P < .001), the ESS ≥ 11 (difference between areas = 0.164, P < .001), and the ESS ≥ 16 (difference between areas = 0.192, P < .001). There was no difference between the AUC of ESS ≥ 11 and ESS ≥ 16 (difference between areas = 0.028, not significant) or between the AUC of the BOSS and the AUC of the SAW (difference between areas = 0.009, not significant).
Performances of the BOSS (internet-based cohort)
Discrimination performances
In addition to internal and predictive validity, the BOSS score was also validated (ie, external validity) in an independent large cohort of regular registered highway drivers. Out of 37,648 internet-based questionnaires completed by frequent highway users registered in an electronic payment system,5 7,296 drivers who self-reported at least 1 sleep disorder (OSAS, restless legs syndrome, insomnia, or narcolepsy/hypersomnia) with complete data on the BOSS items were selected (74.1% male, mean age 49.5 ± 12.5 years, body mass index 26.4 ± 4.6). Of these, 31.4% believed they had OSAS, 16.4% restless legs syndrome, 59.4% insomnia, and 2.2% narcolepsy/hypersomnia. Drivers self-reported 1,396 sleep-related traffic near-misses or accidents in the previous year. As for the sleep clinic cohort (n = 293), the performance of the BOSS on AUC, sensitivity, specificity, positive predictive value, and negative predictive value was better than ESS scores, including clinical cut-off values (ESS ≥ 11 and ESS ≥ 16), in the internet-based cohort (Table 2).
Regarding the internet-based cohort, ROC analysis of patients who reported or did not report sleep-related traffic near-misses or accidents in the past year showed an AUC of 0.62 (95% CI 0.61–0.64), P < .001 for ESS ≥ 11 and an AUC of 0.54 (95% CI 0.53–0.55), P < .001 for ESS ≥ 16. Sensitivity of ESS ≥ 11 and ESS ≥ 16 was low: 50% and 11%, respectively.
ROC analysis of patients who reported or did not report sleep-related traffic near-misses or accidents in the past year showed an AUC of 0.78 (95% CI 0.77–0.79), P < .001 for SAW.
ROC analysis of patients who reported or did not report sleep-related traffic near-misses or accidents in the past year showed an AUC of 0.76 (95% CI 0.75–0.77), P < .001 for the BOSS. The BOSS had a reasonable level of sensitivity (71%) and specificity (69%) with a cutoff value ≥ 3.
Differences between AUCs
The AUC of the BOSS was significantly higher than that of the ESS (difference between areas = 0.085, P < .001), ESS ≥ 11 (difference between areas = 0.135, P < .001), and ESS ≥ 16 (difference between areas = 0.220, P < .001). The AUC of ESS ≥ 11 was significantly higher than that of ESS ≥ 16 (difference between areas = 0.085, P < .001). The AUC of the BOSS was significantly different compared to the SAW (difference between areas = 0.017, P < .001). The latter was significantly higher than that of ESS ≥ 11 (difference between areas = 0.154, P < .001) and ESS ≥ 16 (difference between areas = 0.239, P < .001).
DISCUSSION
The BOSS, which has been developed and tested on both a sleep clinic and general population sample of drivers, has significantly better positive predictive value for sleep-related traffic accidents than the ESS and SAW for the cohort of patients. This means that a driver is more likely to have an accident when the BOSS is positive compared to a positive ESS or SAW. It also has the best balance between sensitivity and specificity regarding the accident risk in both groups. By including sleep clinic patients and the general population interviewed via internet we were able to demonstrate its validity. Although the BOSS is intended to identify patients at risk in the clinic, the validation on the large internet cohort of the general population is interesting since the situational risk factor SAW is derived from the latter.5
This study also confirms the limitations of the ESS regarding the measurement of driving risk in nonsleepy and very sleepy drivers. The BOSS is based on driving behaviors and, unlike the ESS, does not cover nonrisky situations (eg, reading a book or watching TV). Furthermore, by incorporating a question about sleepiness at the wheel and a question from the ESS (Do you fall asleep in a car stopped for a few minutes in traffic?) its sensitivity was increased. The SAW is based on the frequency of a severe episode of sleepiness at the wheel, and adding the ESS item on intensity of in-car drowsiness results in an increase in the specificity of the questionnaire (ie, ability of the questionnaire to exclude risk-free drivers). We also included sex and distance driven per year, 2 well-known factors responsible for sleep-related accidents.5 Indeed, serious accidents are associated with males.34 The BOSS is also shorter (5 items) and simpler to administer than the ESS (8 items).
We compared measures between the BOSS and the overall score of ESS and several levels of sleepiness measured on the ESS (ESS ≥ 11 and ESS ≥ 16) both in sleep clinic patients and the general population. The 2 samples showed similar differences in trend between the BOSS scale and the ESS. Regarding the BOSS, a score ≥ 3 out of 8 best predicts the risk of having a sleep-related near-miss or accident.
Interestingly, using high levels of ESS scores (ie, ESS ≥ 16) increased the specificity to a higher value than that of the BOSS but reduced the sensitivity to a considerably lower level to predict accident risk. While it is useful to identify patients with no risk of accidents, the main purpose of such predictive scales is to quantify disability, ie, risk of accidents. Using ESS ≥ 11 improves its sensitivity but significantly decreases its specificity. Compared to the ESS at both levels (11 and 16), the BOSS provides an optimal balance between sensitivity (82%) and specificity (74%), thus allowing at-risk patients to be identified but also providing robust evaluation of safe drivers. The BOSS also has a better AUC (0.83) than the ESS (0.73) and gives the best positive predictive value and negative predictive value. Compared with self-reported SAW, the BOSS has lower sensitivity (0.82) but better specificity (0.74); in addition, the SAW item presents the overall best performances to identify patients at risk of having sleep-related traffic near-misses or accidents (sensitivity 0.94) but poorly identifies safe drivers (specificity 0.63). Indeed, in the SAW a patient is at risk as soon as he/she declares having been confronted with an episode of severe drowsiness, regardless of the frequency of occurrence. By identifying risk on a single item, the danger is that the number of patients free of accident risk will be underestimated. On the other hand, a scale with multiple items optimizes the balance between sensitivity and specificity.
The study has some limitations. First, data were self-reported and not obtained from Department of Motor Vehicles records. Further studies on patients’ victims of accidents confirmed by the police forces would reinforce the predictive value of the BOSS. However, self-reported sleepiness-related near-misses and accidents in the past years are very strong predictors of future accident risk.6–9 Sleepiness-related near-miss accidents are dangerous precursors to actual driving accidents. The study by Powell et al6 was the first to show the predictable risk of near-misses related to sleepiness in the occurrence of future sleep-related accidents. Second, we did not compare BOSS values before and after treatment in sleepy patients,35 a dimension that requires future exploration. Third, other scales evaluating sleep complaints and excessive sleepiness exist but were not considered in this study due to a lack of evidence of association between these questionnaires and driving risk. For example, the Functional Outcomes of Sleep Questionnaire assesses the impact of excessive sleepiness on activities of daily life36 and includes items related to driving difficulties related to drowsiness/fatigue. Drivers with a higher impact of sleepiness on the Functional Outcomes of Sleep Questionnaire seem to have an increased risk of accidents.21 In this questionnaire, the items related to driving take into account the effect of drowsiness on the duration of driving (short or long duration > 1 hour), items partly redundant with the exposure to driving evaluated in our study by the distance traveled per year. The Barcelona Sleepiness Index is a simple, brief instrument for measuring self-reported EDS in sleep-disordered breathing37 but has no question on driving risk, and the Hypersomnia Severity Index was designed to assess the severity and impairment of hypersomnolence in psychiatric disorders.38 To our knowledge, no study exists concerning the relationship between Barcelona Sleepiness Index or Hypersomnia Severity Index scores and sleep-related near-misses or accidents. Other scales including a few items related to the impact of drowsiness induced by central hypersomnia on driving exist,39,40 but again no association between these scales including a few items on the intensity of the effect of drowsiness on driving and driving risk was demonstrated.
The BOSS was constructed from data collected on patients seen in a sleep clinic for suspicion of sleep disorders. A large majority of patients recruited were male, which could justify a further study to compare the BOSS performances between men and women exposed to driving risk. Our patients were also diagnosed with EDS-associated disorders such as OSAS, narcolepsy/hypersomnia, restless legs syndrome, and insufficient sleep syndrome. It would now be interesting to test it on a large sample of the general population to obtain additional data. In line with identified risk factors for sleep-related traffic near-misses or accidents,8,21,23,24,29 the respective weights of the items related to situational chronic or instantaneous sleepiness at the wheel are higher than those relating to sex or exposure factors such as kilometers driven per year.
To conclude, the BOSS assesses the risk and nonrisk of sleepiness-related traffic accidents more accurately than the ESS. If SAW presents a higher sensitivity than the BOSS, the lower specificity makes us believe that in the overall evaluation the BOSS is a more balanced scale than SAW to evaluate global fitness to drive. It is short, specific tool that should be promoted as a first-line tool to evaluate the risk of traffic accidents.
ABBREVIATIONS
| AUC | area under the curve |
| BOSS | Bordeaux Sleepiness Scale |
| CI | confidence interval |
| df | degrees of freedom |
| EDS | excessive daytime sleepiness |
| ESS | Epworth sleepiness scale |
| OSAS | obstructive sleep apnea syndrome |
| ROC | receiver operating characteristic |
| SAW | sleepiness at the wheel |
DISCLOSURE STATEMENT
All authors approved the manuscript. The authors report no conflicts of interest.
REFERENCES
1. . From wakefulness to excessive sleepiness: what we know and still need to know. Sleep Med Rev. 2008;12(2):129–141.
2. . Consequences of shiftworking on sleep duration, sleepiness, and sleep attacks. Chronobiol Int. 2010;27(3):575–589.
3. . Prevalence of obstructive sleep apnea in the general population: a systematic review. Sleep Med Rev. 2017;34:70–81.
4. . The economic and societal burden of excessive daytime sleepiness in patients with obstructive sleep apnea. Sleep Med Rev. 2020;51:101275.
5. . Sleep disorders and accidental risk in a large group of regular registered highway drivers. Sleep Med. 2010;11(10):973–979.
6. . Sleepy driver near-misses may predict accident risks. Sleep. 2007;30(3):331–342.
7. . Sleep disorders, sleepiness, and near-miss accidents among long-distance highway drivers in the summertime. Sleep Med. 2014;15(1):23–26.
8. . Risk factors for sleepiness at the wheel and sleep-related car accidents among patients with obstructive sleep apnea: data from the French Pays de la Loire Sleep Cohort. Nat Sci Sleep. 2021;13:1737–1746.
9. . Self-perceived sleep during the Maintenance of Wakefulness Test: how does it predict accidental risk in patients with sleep disorders? Sleep. 2021;44(11):zsab159.
10. . Driver sleepiness and risk of serious injury to car occupants: population based case control study. BMJ. 2002; 324(7346):1125.
11. . Fatigue, alcohol, and serious road crashes in France: factorial study of national data. BMJ. 2001; 322(7290):829–830.
12. National Representatives as Study Collaborators. Sleepiness at the wheel across Europe: a survey of 19 countries. J Sleep Res. 2015;24(3):242–253.
13. . Prevention and care of respiratory failure in obese patients. Lancet Respir Med. 2016;4(5):407–418.
14. . Systematic review of motor vehicle crash risk in persons with sleep apnea. J Clin Sleep Med. 2006;2(2):193–200.
15. ; ERS Task Force. European Respiratory Society. Public health and medicolegal implications of sleep apnoea. Eur Respir J. 2002;20(6):1594–1609.
16. . Sleepiness behind the wheel and the implementation of European driving regulations. Sleep Med Clin. 2021;16(3):533–543.
17. . Quantification of sleepiness: a new approach. Psychophysiology. 1973;10(4):431–436.
18. . Subjective and objective sleepiness in the active individual. Int J Neurosci. 1990;52(1-2):29–37.
19. . A new method for measuring daytime sleepiness: the Epworth sleepiness scale. Sleep. 1991;14(6):540–545.
20. . Risk of motor vehicle accidents related to sleepiness at the wheel: a systematic review and meta-analysis. Sleep. 2017;40(10):zsx134.
21. . Sleepiness, sleep-disordered breathing, and accident risk factors in commercial vehicle drivers. Am J Respir Crit Care Med. 2004;170(9):1014–1021.
22. . Habitually sleepy drivers have a high frequency of automobile crashes associated with respiratory disorders during sleep. Am J Respir Crit Care Med. 2000;162(4 Pt 1):1407–1412.
23. . Self-reported sleepiness and not the apnoea hypopnoea index is the best predictor of sleepiness-related accidents in obstructive sleep apnoea. Sci Rep. 2020;10(1):16267.
24. . Sleepy driving: accidents and injury. Otolaryngol Head Neck Surg. 2002;126(3):217–227.
25. . Sleep-disordered breathing and motor vehicle accidents in a population-based sample of employed adults. Sleep. 1997;20(8):608–613.
26. . Sleep restriction, sleep hygiene, and driving safety: the importance of situational sleepiness. Sleep Med Clin. 2019; 14(4):407–412.
27. . Assessment of sleepiness in drivers: current methodology and future possibilities. Sleep Med Clin. 2019;14(4):441–451.
28. . Comparison of self-report and objective measures of driving behavior and road safety: a systematic review. J Safety Res. 2018;65:141–151.
29. . Self-reported sleepiness while driving as a risk factor for traffic accidents in patients with obstructive sleep apnoea syndrome and in non-apnoeic snorers. Respir Med. 2000;94(10):971–976.
30. . I think I’m sleepy, therefore I am - awareness of sleepiness while driving: a systematic review. Sleep Med Rev. 2021;60:101533.
31. . Epidemiology, physiology and clinical approach to sleepiness at the wheel in OSA patients: a narrative review. J Clin Med. 2022;11(13):3691.
32. American Academy of Sleep Medicine. International Classification of Sleep Disorders. 3rd ed. Darien, IL: American Academy of Sleep Medicine; 2014.
33. . Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. Clin Chem. 1993;39(4):561–577.
34. . Factors associated with serious vehicular accidents: a cross-sectional study in hospital emergency rooms. Br J Clin Pharmacol. 2021;87(2):612–621.
35. . Car crashes and central disorders of hypersomnolence: a French study. PLoS One. 2015;10(6):e0129386.
36. . An instrument to measure functional status outcomes for disorders of excessive sleepiness. Sleep. 1997;20(10): 835–843.
37. . The Barcelona Sleepiness Index: a new instrument to assess excessive daytime sleepiness in sleep disordered breathing. J Clin Sleep Med. 2015;11(11):1289–1298.
38. . Development and validation of the Hypersomnia Severity Index (HSI): a measure to assess hypersomnia severity and impairment in psychiatric disorders. Psychiatry Res. 2019;281:112547.
39. . Measurement of symptoms in idiopathic hypersomnia: the Idiopathic Hypersomnia Severity Scale. Neurology. 2019;92(15):e1754–e1762.
40. . Narcolepsy Severity Scale: a reliable tool assessing symptom severity and consequences. Sleep. 2020;43(6):zsaa009.
