In this month's issue of the Journal of Clinical Sleep Medicine, Schade et al1 present findings from a validation study of the S+ from ResMed, a non-wearable, non-contact motion detector that tracks body movements, including respiration, from radio-frequency biomotion sensors and uses proprietary algorithms to estimate sleep, wake and perform surrogate sleep staging. The outputs are sent to a mobile phone app and uploaded to a web server. The approach to validation used in the study was to compare sleep/wake time from the S+ to simultaneous polysomnography (PSG) and actigraphy. PSG provides the reference for sleep staging and actigraphy a comparison with the widely used indirect sleep/wake assessment. Epoch-by-epoch agreement was used in addition to summary metrics commonly used in clinical and research applications. Duration of the epochs was 30 seconds. Twenty-seven participants without sleep complaints were enrolled, one of whom was excluded due to presence of obstructive sleep apnea. Because of technical issues, 22 completed the study on the S+. The authors are to be commended for the excellent methodology followed for this “validation” study. However, as they acknowledge, clinical patients with sleep disorders were not enrolled.
The results of this study reaffirm that non-electroencephalography (EEG) techniques are not very good (∼65% agreement) when it comes to specific sleep staging. However, like actigraphy, the S+ seems to detect motion well and may have some advantages in that it underestimates wake before sleep onset less than actigraphy but performs similarly to actigraphy for wake after sleep onset (it overestimates sleep by missing arousals). The problems are exaggerated in poor sleepers in the study, and patients with sleep disorders were not tested. It is clear from the figures that agreement in all metrics obtained from non-PSG is best when sleep is “good” and is most inaccurate when it is disturbed. The implications for clinical testing are obvious.
While the authors used an excellent validation approach to the data collected, the results do not go very far to change one's opinion of motion sensing devices and current analysis algorithms as surrogates for sleep testing with EEG. The devices work well for those with normal sleep and less so for those with abnormal sleep, failing particularly to detect some of the sleep disruption that would presumably be the goal of a study in a patient with sleep complaints. This is not to impugn the utility of actigraphy (and by extension the radiofrequency approach) to see circadian trends and overall approximate sleep times with essentially no patient burden. There is little doubt such devices appeal to the home consumer interested in self monitoring, but this study and others suggest a limited role in clinical situations and research, at least with the present performance for motion-based sleep scoring, except to define broad sleep habits. The breakdown of sleep stages in the S+ algorithm is of limited value as it is 20% less accurate than interscorer technician agreement on a PSG, and thus likely not to capture staging with sufficient accuracy to be useful clinically, particularly in those with more disrupted sleep.
An interesting side aspect of the present study was that the algorithm for the S+ analysis changed during the study, allowing the authors to compare the accuracy of the algorithms before and after the change. As might be expected, the small improvement in wake specificity was offset by a decrease in sleep sensitivity. The authors correctly comment on the fact that this “improved” algorithm may impact someone using it without awareness of the change.
Overall, this study adds to the budding literature evaluating the plethora of consumer devices purporting to monitor sleep. It is possible (even likely) that one day we will have minimal or no-contact sensors that can capture something similar to what we score on PSG, but it is clear we have a long way to go. Since the consumer use of such devices is unregulated, the present study serves to provide a reminder to clinicians to use data brought to them by patients from motion based sensors with caution if there are clinical repercussions. Furthermore, the likely unappreciated changes that can occur in the algorithms embedded pose an additional problem for longitudinal clinical and research monitoring.
Dr. Rapoport reports no conflicts of interest.
Rapoport DM. Non-contact sleep monitoring: are we there yet? J Clin Sleep Med. 2019;15(7):935–936.
Schade MM, Bauer CE, Murray BR, et al. Sleep validity of a non-contact bedside movement and respiration-sensing device. J Clin Sleep Med. 2019;15(7):1051–1061