ADVERTISEMENT

Issue Navigator

Volume 13 No. 11
Earn CME
Accepted Papers
Classifieds





Emerging Technologies

Rate My Sleep: Examining the Information, Function, and Basis in Empirical Evidence Within Sleep Applications for Mobile Devices

Peta A. Lee-Tobin, MProfPsych1,2; Rowan P. Ogeil, PhD1,2; Michael Savic, PhD1,2; Dan I. Lubman, PhD, FRANZCP, FAChAM1,2
1Eastern Health Clinical School, Monash University, Box Hill, Victoria, Australia; 2Turning Point, Eastern Health, Fitzroy, Victoria, Australia

ABSTRACT

Study Objectives:

Sleep applications (apps) have proliferated in online spaces, but few studies have examined the validity of the information contained within the apps. This study aimed to examine the information and functions found within sleep apps, determine if the information is based on empirical evidence, and whether or not user ratings were affected by these factors.

Methods:

Sleep apps found in the Google Play store (n = 76) were coded using content analysis to examine the types of information, functions, and evidence base of each app.

Results:

Only 32.9% of sleep apps contained empirical evidence supporting their claims, 15.8% contained clinical input, and 13.2% contained links to sleep literature. Apps also contained information on how sleep is affected by alcohol or drugs (23.7%), food (13.2%), daily activities (13.2), and stress (13.2%). A mean difference in average user rating was found between apps that contained at least one source of information compared those that did not. App user ratings were not associated with an app having multiple functions, or from an app drawing on multiple sources of evidence (except for sleep literature only). Last, there was a higher average user rating among apps that contained a sleep tip function.

Conclusions:

Sleep apps are increasingly popular, demonstrated by the large number of downloads in the Google Play store. Users favored apps that contained sleep tips; however, these tips and other information in the apps were generally not based on empirical evidence. Future research in the area of sleep apps should consider constructing sleep apps derived from empirical evidence and examining their effectiveness.

Citation:

Lee-Tobin PA, Ogeil RP, Savic M, Lubman DI. Rate my sleep: examining the information, function, and basis in empirical evidence within sleep applications for mobile devices. J Clin Sleep Med. 2017;13(11):1349–1354.


INTRODUCTION

Disturbed sleep and excessive sleepiness are reported by 20% to 35% of the adult population on a regular basis.1 Disturbed sleep negatively affects quality of life,2 productivity,3 and is associated with an increased risk of a motor vehicle accident.4 Sleep disorders also increase the risk of psychiatric conditions including depression5 and substance use disorders6 and collectively place a large burden on the economy.7

Insomnia (6.9%) and obstructive sleep apnea (OSA) (4.9%) are the most commonly reported sleep conditions.1 Few people with these sleep disorders seek professional advice, with many undergoing trials of self-help approaches to combat symptoms.8 Traditionally, self-help approaches have included behavioral strategies (eg, using relaxation techniques), and pharmacological strategies (eg, alcohol or sedative medications).912 Newer technological strategies such as smartphone applications (apps) are becoming popular alternatives for self-assessment, monitoring, and/or treatment of sleep issues.13 Although sleep apps are easily accessible, there is scant evidence of the validity of the information provided by these apps to support their utility.13,14

Information provided via sleep apps can affect how people understand and interpret their sleep.15 Therefore, if a sleep app diagnoses a sleep disorder, the individual may come to understand themselves or their bodies as “abnormal” and may be distressed about this.16,17 However, diagnosis can provide a sense of relief and validation, and facilitate access to resources and professional help.17 Because a diagnosis inevitably carries consequences, it is important to subject the claims of sleep apps to analysis. Many sleep apps propose to accurately monitor and diagnose sleep disorders,14 but these claims have not been thoroughly examined. Although the validity of apps and the information they provide is a matter of concern for sleep and other health professionals, it is unclear whether app users harbor such concerns about their validity.

To date, there has been limited research in this area. Behar and colleagues reviewed approximately 40 sleep apps accessible on the Google Play store and Apple App store, and found that none of the apps were based on strong scientific evidence.13 Similarly, other studies examining apps related to a single symptom of sleep (apps only monitoring snoring) found that the apps tested were not accurate enough to replace common diagnostic measures.18 Finally, while research examining an app to assess sleep by detecting sleep and wake states, daily sleep quality, and global sleep quality demonstrated high accuracy percentages among the sleep states, the data were not compared to objective sleep measures.19

This study explored the different kinds of information, functions, and evidence features sleep apps contained and what percentage of sleep apps contained these features. In addition, this study aimed to investigate what sleep app content was associated with higher app user rating by comparing the average user rating among features presented in each app. If sleep app users are concerned with the validity of information provided, one would hypothesize that apps that claim to draw on empirical evidence would have higher average ratings than those that do not.

METHODS

Identifying Sleep Apps

The authors identified sleep apps in the Google Play store, one of the largest stores for downloading apps for devices that run the Android operating system (Google, Mountain View, California, United States). The Android operating system is the most widely used smartphone operating system,20 with 87.7 million smartphone users in 2014.20 The Google Play store was selected as the forum to search for sleep apps given that it has a high volume of sleep apps, and also provides user ratings and number of downloads per app.

Search terms used to identify sleep apps were: “sleep,” “sleep problems,” “insomnia,” and “sleep apnea.” The search generated 439 apps though after evaluation of the apps, 322 of the apps were excluded due to the following inclusion/exclusion criteria. Apps were included if they aimed at identifying, monitoring, and providing support for sleep problems. Similarly, apps that exclusively played relaxing music were not included as well as alarms and ebooks. Apps that were not in English were also excluded.

Data Collection

Seventy-six sleep apps met inclusion criteria, and data were collected using the app descriptions and screenshots provided by the developers. Content on the type of information the app contained (eg, information on how alcohol and other drugs, food, exercise, daily activities, stress, people, space and technology can affect sleep), the functions of the apps (eg, sleep tips, action lists, progress of sleep, virtual rewards, referral number, service locator, online community, and screening for sleep and mood functions), and sources of evidence (eg, what portions of the app draw upon sleep literature, national guidelines, expert input, general evidence, and clinical input and background) was coded. The authors developed a preliminary coding framework by refining the structure employed by Savic et al.21 to ensure it was relevant to sleep apps and the sleep literature. The authors then piloted the preliminary coding framework on 5 apps each, and revised the framework accordingly to arrive at the final coding framework.

Data on each app were coded by 2 authors, and cross-checked by a third to ensure data integrity. Where there was a discrepancy in the coded data, the 3 researchers discussed the discrepancies in the coding and agreed upon the final data set.

Data were analyzed in SPSS version 22 software (IBM, Armonk, New York, United States). Descriptive data demonstrating inclusion of specific content are presented as frequencies, whereas Kruskal-Wallis tests were used to explore rank differences between app categories and the average user ranking of the app. To explore differences in average user rating among apps that had at least one category present in the content compared to those that did not and to see if average user rating differed for each of the categories that made up lifestyle information, evidence and function categories, t tests were used.

RESULTS

The primary sleep complaint that apps purported to address included: OSA (n = 23, 30.3%), insomnia (n = 9, 11.8%), sleep cycles or circadian rhythms (n = 32, 42.1%), or general sleep (n = 8, 10.5%). Most were categorized in the Google Play store as either “health & fitness” (n = 46, 60.5%) or “medical” (n = 20, 26.3%) apps. Most were free to download (n = 61, 80.3%) at a maximum cost of $6.82. Of the 76 sleep apps, 6 (7.9%) had been downloaded up to 5 million times and of all the sleep apps, the average minimum installation of the apps was 124,674.68 (standard deviation [SD] = 286,165.5) and a maximum average installation was 503,213.36 (SD = 1,348,263.4). The mean duration between the date the app was last updated to when the app was analyzed for this study was 595 days (SD = 469 days) with a range between 6 and 2,200 days.

Common characteristics of the 10 most downloaded sleep apps (approximately 1 to 5 million downloads) included that 9 of the 10 had a rating of 4 or more stars (representing high user satisfaction). These apps had a central focus on sleep cycles or circadian rhythms, and all were free to download. However, only one of these apps was identified as containing evidence. Of the 10 least downloaded apps, 8 of the 10 were given ratings under 4 stars with most of them focusing on OSA. Most were associated with costs between $0.99 and $6.82. Three of the apps mentioned sources of evidence in their descriptions, though few apps contained informational content (eg, information on how alcohol and other drugs can affect sleep) in their product.

There were 76 developers identified, with no developer creating more than one of the sleep apps included in this sample. Most of the app developers were from the United States (n = 19, 25%), with European countries (n = 14, Russia 2.6%, Sweden 2.6%, France 3.9%), Canada (n = 4, 5.3%) and Japan (n = 3, 3.9%) developing a smaller number. Most of the identified developers created other apps, although only 25% (n = 19) had developed other sleep apps (not used in this sample due to exclusion criteria). All of the developers had provided an email address for support, with 78.9% providing a link to an external website, and 48.7% providing another form of contact information, such as a physical address.

The number and percentages of apps categorized into “lifestyle information,” “functions” and “evidence” are presented in Table 1. Some sleep apps contained information on how alcohol and other drugs (23.7%), food (13.2%), daily activities (13.2), and stress (13.2%) affect sleep. In relation to functions, more than half of the sleep apps contained at least one function, with information on sleep tips and progress of sleep being the most popular. A total of 32.9% of the sleep apps reportedly were derived from empirical evidence.

Frequencies and percentages of features in sleep apps (n = 76).

jcsm.13.11.1349.t01.jpg

table icon
Table 1

Frequencies and percentages of features in sleep apps (n = 76).

(more ...)

Table 2 presents the frequencies and percentages of how many apps contained multiple content features of information, functions, and evidence and highlights the number of apps that did not contain content within the investigated categories (eg, 45 of the apps contained no content on information).

App distribution categories based on amount of lifestyle information, evidence, and function features (n = 76).

jcsm.13.11.1349.t02.jpg

table icon
Table 2

App distribution categories based on amount of lifestyle information, evidence, and function features (n = 76).

(more ...)

Ranking Between Content and Average User Ratings

Because of large discrepancies in sample sizes, 3 Kruskal-Wallis H tests were used to explore the rank differences between amount of information, evidence, and functions content available in the sleep apps and average user rating (eg, an app may have contained information features such as how alcohol and other drugs, daily activities, and stress can affect sleep and therefore that app would have 3 information features as it covered content in those 3 areas). There was no significant rank difference between the amount of information (χ2(4) = 6.7, P = .15), function (χ2(4) = 3.95, P = .41), and evidence (χ2(4) = 4.16, P = .38) an app contained. Although no statistically significant differences were found in the average user rating, apps that contained 2 types of information content had a higher average user ranking than other apps. Additionally, apps that contained the most amount of functions (4+ functions) and apps that contained 3 types of evidence had a higher average user rating.

Relationship Between Average User Ratings for Apps With at Least One Function, Evidence, and Information Factor

The average user rating of identified sleep apps (measured on a scale of 1 to 5) was 3.7 (SD = 0.79, n = 72 with 4 apps not rated). To explore whether apps that contained at least one content feature of information, functions, or evidence had a higher average user rating than those apps that did not, 3 separate independent t tests were performed (see Table 3).

Independent samples t tests to compare differences in average user rating of sleep apps that contained at least one feature to those that contained none.

jcsm.13.11.1349.t03.jpg

table icon
Table 3

Independent samples t tests to compare differences in average user rating of sleep apps that contained at least one feature to those that contained none.

(more ...)

Table 3 shows that apps with lifestyle information factors were rated higher than apps without. The results also highlighted that app users' ratings did not significantly differ between the apps that contained at least one function factor and those that did not and app users' ratings did not differ in relation to apps that drew on at least one form of evidence and those that did not.

Relationship Between Average User Ratings for Subgroups of Information, Functions, and Evidence

Table 4 shows that there was a mean difference in average user ratings between apps that contained alcohol and other drugs and food and daily activities information and those that did not. In relation to apps with content supported by evidence, there was a significant difference, suggesting that apps that contained sleep literature were rated more highly than those that did not. There was no difference in the average user rating between apps that were sourced from other sources of evidence. Last, in relation to apps that had function content, sleep apps that contained sleep tips, service locators and referral numbers were rated significantly higher than those that did not.

Independent samples t tests to compare differences in average user rating of sleep apps between apps that contained information, function, and evidence features.

jcsm.13.11.1349.t04.jpg

table icon
Table 4

Independent samples t tests to compare differences in average user rating of sleep apps between apps that contained information, function, and evidence features.

(more ...)

DISCUSSION

This study illustrates that sleep apps are increasingly sought after, with the most popular sleep app being downloaded by more than 5 million android users. This study also highlighted that few apps drew from empirical evidence, supporting past research that many sleep apps are based on a limited evidence base.13,18 Furthermore, from our evaluation of the sleep app descriptions and screenshots provided by the developers, all of the apps except one (the most downloaded and highest rated app) failed to mention the 2-process model of sleep. This model is important as it describes the interaction between circadian processes, which run on a 24-hour rhythm controlled by the central pacemaker in the suprachiasmatic nucleus, and homeo-static processes, which relate to the accumulation of hypnogenic substances within the central nervous system that form during wake hours and dissipate during sleep period,22 and is vital to understand sleep-wake cycles.

Sleep apps might influence sleep behavior in several ways. For example, sleep apps may provide users with a helpful tool to manage their own sleep, and self-recorded data provided by apps may prompt those experiencing a sleep problem to seek professional assessment as findings from this study suggest that people may “self-diagnose” and worry about having a “sleep disorder” on the basis of information which may not be evidence based.23 However, app users may also be exposed to their smartphones more frequently before bedtime, a habit known to inhibit the onset of sleep due to artificial light exposure.24

Our study found no difference in average user rating and whether an app drew on one or more sources of evidence. There was a difference in average user rating for those apps that contained sleep literature only. This suggests that app users are not always concerned about whether apps are evidence based when rating apps or may not be sufficiently aware of whether an app draws on evidence to factor this into their judgement about the quality of an app. One way to draw attention to, and encourage the use of, evidence-based apps may be to introduce a badge system agreed on by a reputable professional society to verify sleep apps that are approved for public health. In a similar way as with the Heart Foundation tick of approval for food in Australia,25 sleep apps that meet strict credibility and validity requirements could be eligible for accreditation.

This study also highlighted that users preferred sleep apps that contained information about how alcohol and other drugs, stress, and daily activities can affect one's sleep. They also preferred sleep apps that contained sleep tips, service locators and referral numbers. This implies that such information about these factors was sought after by those downloading the apps, and may have positively contributed to their sleep habits therefore resulting in higher user ratings. However, when each subgroup of evidence was tested, there were significantly higher average user ratings for apps that mentioned sleep literature specifically. Using the results of this study, creators of future sleep apps may benefit from constructing apps with features that app users in this study rated highly and use empirical-based research to support these features. For example, app creators could develop sleep tips and information content on how alcohol and other drugs, stress, and daily activities can affect sleep that are based on empirical sources.

This study also found an average of 595 days between updates of the sleep apps, which may suggest that app developers may not be updating their respective sleep apps, which is important to ensure that users are provided with the most up-to-date information. Noninclusion of such information and/ or functions could negatively affect the app user's experience. Empirical studies are needed to ascertain the effects on and the effectiveness and experiences of individuals using sleep apps. This would enable recommendations about effective apps to be made, and thus facilitate access to evidence-based apps to those who may be suffering from disturbed sleep. Furthermore, empirically derived sleep apps could provide established self-assessment tests of sleep-wake cycles to help identify patients with psychiatric conditions (such as major depression and bipolar) who have contributing disturbed circadian profiles.5

A limitation was that only apps found in the Google Play store were explored and apps were analyzed based on the summary descriptions and screenshots provided by developers. Although this study provides the first systematic coding of features related to the evidence base of commonly used sleep apps, future research that downloads and explores the functionality and usability of apps is needed. Additionally, this study did not explore the specific written reviews of users; only the star ratings were provided. Without taking the written user comments into account it is unclear what factored into users' ratings. For instance, a user may have enjoyed the content of an app but experienced a technical fault and therefore decided to give the app a lower rating. Last, we did not directly explore the aesthetics or marketing of sleep apps in the Google Play store, and future research should explore how these factors influence use of particular apps over others.

CONCLUSIONS

This study illustrates the popularity of sleep apps despite many not drawing on empirical evidence to substantiate their claims. Users should be cautious of relying on individual apps in the assessment or diagnosis of a sleep disorder. Given their widespread appeal, future evidence-based apps have the potential to reach large populations and play a role in promoting good sleep.

DISCLOSURE STATEMENT

All authors have seen and approved the manuscript for submission. Rowan Ogeil is the recipient of an NHMRC Peter Doherty Fellowship from the NHMRC. Dan Lubman has received speaking honoraria from AstraZeneca, Janssen and Lundbeck, and has provided consultancy advice to Lundbeck. All other authors report no conflicts of interest.

EDITOR'S NOTE

The Emerging Technologies section focuses on new tools and techniques of potential utility in the diagnosis and management of any and all sleep disorders. The technologies may not yet be marketed, and indeed may only exist in prototype form. Some preliminary evidence of efficacy must be available, which can consist of small pilot studies or even data from animal studies, but definitive evidence of efficacy will not be required, and the submissions will be reviewed according to this standard. The intent is to alert readers of Journal of Clinical Sleep Medicine of promising technology that is in early stages of development. With this information, the reader may wish to (1) contact the author(s) in order to offer assistance in more definitive studies of the technology; (2) use the ideas underlying the technology to develop novel approaches of their own (with due respect for any patent issues); and (3) focus on subsequent publications involving the technology in order to determine when and if it is suitable for application to their own clinical practice. The Journal of Clinical Sleep Medicine and the American Academy of Sleep Medicine expressly do not endorse or represent that any of the technology described in the Emerging Technologies section has proven efficacy or effectiveness in the treatment of human disease, nor that any required regulatory approval has been obtained.

ABBREVIATIONS

app

application

OSA

obstructive sleep apnea

SD

standard deviation

REFERENCES

1 

Hillman DR, Lack LC. Public health implications of sleep loss: the community burden. Med J Aust. 2013;199(8):S7–S10. [PubMed]

2 

Strine TW, Chapman DP. Associations of frequent sleep insufficiency with health-related quality of life and health behaviors. Sleep Med. 2005;6(1):23–27. [PubMed]

3 

Lockley SW, Barger LK, Avas NT, Rothschild JM, Czeisler CA, Landrigan CP. Effects on health care provider work hours and sleep deprivation on safety and performance. Jt Comm J Qual Patient Saf. 2007;11 Suppl:7–18.

4 

Barger LK, Cade BE, Ayas NT, et al. Extended work shifts and the risk of motor vehicle crashes among interns, the Harvard Work Hours, Health, and Safety Group. N Engl J Med. 2005;352:125–134. [PubMed]

5 

Hickie IB, Naismith SL, Robillard R, Scott EM, Hermens DF. Manipulating the sleep-wake cycle and circadian rhythms to improve clinical management of major depression. BMC Med. 2013;11:79. [PubMed Central][PubMed]

6 

van Straten A, Cuijpers P. Self-help therapy for insomnia: a meta-analysis. Sleep Med Rev. 2009;13(1):61–71. [PubMed]

7 

Mansfield DR, Hillman DR, Antic NA, McEvoy RD, Rajaratnam SM. Sleep loss and sleep disorders. Med J Aust. 2013;199(8):S5–S6. [PubMed]

8 

Morin CM, LeBlanc M, Daley M, Gregoire JP, Mérette C. Epidemiology of insomnia: prevalence, self-help treatments, consultations and determinants of help-seeking behavior. Sleep Med. 2006;7:123–130. [PubMed]

9 

Johnson EO, Roehrs T, Roth T, Breslau N. Epidemiology of alcohol and medication as aids to sleep in early adulthood. J Sleep Res. 1998;21(2):178–186. [PubMed]

10 

Ancoli-Israel S, Roth T. Characteristics of insomnia in the United States: results of the 1991 National Sleep Foundation Survey. Sleep. 1999;22 Suppl 2:347–353. [PubMed]

11 

Kaneita Y, Uchiyama M, Takemura S, et al. Use of alcohol and hypnotic medication as aids to sleep among the Japanese general population. Sleep Med. 2007;8(7-8):723–732. [PubMed]

12 

Doghramji K, Jangro WC. Adverse effects of psychotropic medications on sleep. Sleep Med Clin. 2016;11(4):503–514. [PubMed]

13 

Behar J, Roebuck A, Domingos S, Gederi E, Clifford GD. A review of current sleep screening applications for smartphones. Physiol Meas. 2013;34(7):29–46. [PubMed]

14 

Bhat S, Ferraris A, Gupta D, et al. Is there a clinical role for smartphone sleep apps? Comparison of sleep cycle detection by a smartphone application to polysomnography. J Clin Sleep Med. 2015;11(7):709–715. [PubMed Central][PubMed]

15 

Lupton D. Critical perspectives on digital health technologies. Sociology Compass. 2014;8(12):1344–1359.

16 

Fosket J. Constructing ‘high-risk women’: the development and standardization of a breast cancer risk assessment tool. Sci Technol Human Values. 2014;29(3):291–313.

17 

Savic M, Barker SF, Hunter B, Lubman DI. ‘Holy shit, didn't realise my drinking was high risk’: an analysis of the way risk is enacted through an online alcohol and drug screening intervention. Health Risk Soc. 2016;17(7-8):565–591.

18 

Stippig A, Hübers U, Emerich M. Apps in sleep medicine. Sleep Breath. 2014;3(1):411–417. [PubMed]

19 

Min JK, Doryab A, Wiese J, Amini S, Zimmerman J, Hong JI. Toss ‘n’ turn: smartphone as sleep and sleep quality detector. Proc SIGCHI Conf Hum Factor Comput Syst. 2014;2014:477–486.

20 

Number of Android smartphone users in the United States from 2014 to 2016 (in millions). statista website. https://www.statista.com/statistics/232786/forecast-of-andrioid-users-in-the-us. Accessed November 21, 2016.

21 

Savic M, Best D, Rodda S, Lubman DI. Exploring the focus and experiences of smartphone applications for addiction recovery. J Addict Dis. 2013;32(3):310–319. [PubMed]

22 

Borbely AA, Daan S, Wirz-Justice A, Deboer T. The two-process model of sleep regulation: a reappraisal. J Sleep Res. 2016;25(2):131–143. [PubMed]

23 

Van den Bulck J. Sleep apps and the quantified self: blessing or curse? J Sleep Res. 2015;24:121–123. [PubMed]

24 

Technology & Sleep. Sleep Health Foundation website. https://www.sleephealthfoundation.org.au/public-information/fact-sheets-a-z/802-technology-sleep.html. Published May 12, 2016. Accessed September 19, 2017.

25 

Heart Foundation Tick. Heart Foundation website. https://www.heartfoundation.org.au/healthy-eating/heart-foundation-tick. Accessed November 25, 2016.