Reliability and Validity of the Turkish Version of the Observation and Interview Based Diurnal Sleepiness Inventory
Article information
Abstract
Objective
The Observation and Interview Based Diurnal Sleepiness Inventory (ODSI) is a valid 3-item tool used for assessment of excessive daytime sleepiness (EDS). The aim of this study was to investigate the reliability and validity of the ODSI in the Turkish language.
Methods
Linguistic validation of the ODSI was performed by forward-backward translation. The Turkish version of the ODSI and the Epworth Sleepiness Scale (ESS) were administered in EDS and control groups.
Results
The ODSI was tested in 106 older patients. The median age of the patients was 73 (65–89) years and 55.7% were female. The EDS group was older and more dependent on instrumental activities of daily living than the control group. The inter-rater reliability of the ODSI was high (interclass correlation coefficient [ICC]: 0.851, 95% confidence interval [CI]: 0.540–0.958, p<0.001). Test-retest reliability was also high for the total sample (ICC: 0.871, 95% CI: 0.632–0.959, p<0.001). Positive strong correlations were found with the ESS (Speraman’s rho=0.876, 95% bootstrap CI [0.813–0.918], p<0.001). ROC curve analysis showed an area under the curve of 0.968 (95% CI: 0.937–0.998), a cutoff score of ≥6, a sensitivity of 94.1%, a specificity of 87.6%, a positive predictive value of 76.55%, and a negative predictive value of 97.2%.
Conclusion
Our data validate the ODSI for application in Turkish-speaking populations. The simplicity, reliability, and the apparent lack of relevant influences of cultural background on performance of the 3-item ODSI make it a valuable tool for clinical management and research.
INTRODUCTION
Excessive daytime sleepiness (EDS), which is described by a high sleep propensity during wakefulness in situations of lessened attention and concentration, is an underdiagnosed and untreated problem especially for older adults [1]. There are some troubles to assess prevalence of EDS due to the lack of a standard definition criteria. Previous studies showed that EDS prevalence was approximately 10%–30% in older people [2]. It is associated with falls, cognitive impairment, frailty, increased dependence of activities of daily living, and mortality [3-5].
EDS can be assessed by several methods including objective sleep laboratory tests and subjective sleepiness questionnaires. However, these laboratory tests and questionnaires are not interchangeable [6] and explore different aspects of sleepiness in different environmental conditions. To be precise, for the same patient more than one subjective questionnaire may be useful to assess multiple dimensions of its drowsiness [7] with or without objective tests.
The Epworth Sleepiness Scale (ESS) is the most commonly used and Turkish validated subjective 8-item sleepiness questionnaire [8,9]. The ESS has adequate psychometric properties and has been extensively used in epidemiological studies including diverse age groups to evaluate sleep propensity. It was successfully administered for screening daytime sleepiness in the educated, active, and healthy older adults [10] and in independent community dwelling older sleep apnea patients [11].
However, the scale does not include information from a caregiver, partner, or other outside source; and thus, the information provided may be limited, possibly underestimating risk in some groups of older adults (e.g., sedentarity, lower socioeconomic status, and cognitive impairment) [12].
In order to complete the offer of sleepiness assessment questionnaires that can be administered to older adults, Onen et al. [11] developed the Observation and Interview Based Diurnal Sleepiness Inventory (ODSI). The ODSI is a simple and standardized 3-item tool being used in English and French speaking countries. This tool can be administered as an interview to patients and/or their relatives and that can evaluate the daytime sleepiness of the older individuals without important cultural, scholarly, or professional references. The ODSI validation study showed that this tool has high validity, internal consistency, and intra-rater and inter-rater reliability.
The purpose of this study was to translate the ODSI into Turkish and then assess its relevance and validity in this new version.
METHODS
Linguistic validation
The linguistic validation was carried out following the international guidelines [13]. First, the original ODSI developers (FO and SHO) were contacted for permission to use the instrument and invited to become involved in the translation and validation process.
Second, two native target language translators (PÜ, BBD) independently performed a forward translation of the ODSI from English into Turkish including original items, instructions, and response choices. Afterwards, the two translators discussed their translations and produced a combined version.
Third, after production of the combined version, both of two translators and the translation committee consisting of three geriatricians (GŞA, ME, CB) discussed the translation and agreed on a single reconciled version. In order to improve this version, a third translation from French to Turkish (forward translation SHO) was performed blindly. As a result, the first version of the ODSI Turkish for Türkiye was reached after some minor changes.
Fourth, a backward translation was performed by a translator who is a native speaker of English and bilingual in the target language.
Fifth, the translation committee reviewed and compared the backward translation and the original ODSI. They changed only two words in Turkish version to ensure the conceptual equivalence of the translation. Thus, after harmonization, the second version was created.
Sixth, the previous improved translation was applied face-toface to ten non-demented native Turkish speaker patients. The participants had no difficulty understanding and answering all three questions of the test. In other words, the cognitive debriefing was successful.
Seventh, the finalized translation is proofread to check for minor errors (spelling, grammatical, and/or other errors) which have been missed during the translation process.
Finally, this improved translation was adopted to conduct the reliability and validity study of the ODSI in Turkish language (Turkish version provided in Supplementary Materials in the online-only Data Supplement).
Patients and procedure
A total of 106 consecutive patients who were attending to the university hospital geriatric outpatient clinic were included in this study. Patients with dementia, delirium, known psychiatric diseases, using psychotropic drugs, and who were not able to speak Turkish were not included in the study.
All participants underwent comprehensive geriatric assessment. Age, gender, education, and living status were recorded. Geriatric syndromes including malnutrition, incontinence, osteoporosis, depression, polypharmacy, falls, insomnia, and other sleep related disorders were evaluated. Katz Activities of Daily Living (ADL) scale [14], Lawton Brody Instrumental Activities of Daily Living scale (Lawton-Brody) [15,16], Mini-Nutritional Assessment– short form (MNA-SF) [17], Mini-Mental State Examination (MMSE) [18], and Yesevage Geriatric Depression Scale short form [19] were administered to all patients.
Patients were asked a single question “Are you excessively sleepy during the day?” to state their daytime sleepiness as “never, rarely, regularly, or frequently.’’ They were enrolled in the group “EDS” when the answer was “regularly” or “frequently” and in the group “control” when it was “never” or “rarely” [20]. In addition, the ESS and the ODSI were filled in.
The study protocol was approved by the Hacettepe University Ethics Committee (GO 18/744-47). Informed consent was obtained from all individual participants included in the study.
ESS
The ESS is an 8-question scale developed by Johns [9] that assesses sleepiness in daily activities. For every question in the scale, ‘’0’’ points out no chance of sleepiness, “1,” “2,” and “3” indicate mild, moderate, and significant sleepiness. Total score range is 0–24 points and a cutoff >10 points is accepted to indicate EDS. The Turkish version of the ESS has been validated in 150 patients with sleep disorders [8].
ODSI
The ODSI, composed of 3 standardized questions, is a short and easy-to-administer test that assesses daytime sleepiness. The ODSI is administered by a healthcare professional as an interview with the patients themselves and/or their relatives who observe the patient in activities of daily living. While administering the ODSI, the relatives/or proxies of the patients who were with them were asked whether “they agreed with the patient’s response” and whether “there was anything he/she wanted to add.’’ Responses to the 3 items are summed to yield a total score from 0 (non-sleepy) to 24 (extremely sleepy).
Question-1 helps to assess sleepiness or falling asleep in active situations or situations where a high level of stimulation is required such as washing, getting dressed, eating, talking, and driving. Because sleepiness during active situations is likely more unsafe, experts suggested adding 6 to the first item while keeping a “0” unchanged. Thus, scoring for the first item would be 0, 7 (1+6), 8 (2+6), etc., and first item scores would range from 0 to12.
Question-2 helps to assess sleepiness or falling asleep in passive situations or situations where a low level of stimulation is required such as reading, watching TV, listening to a conversation, and listening to music. The score range is 0–6.
Question-3 helps to estimate the average total duration of sleep during the day, including feeling sleepy, dozing off, and taking naps. The score range is 0–6.
In the study of 133 patients, 73 of whom were obstructive sleep apnea syndrome (OSAS), the ODSI sensitivity was found 0.842 (95% confidence interval [CI]: 0.624–0.945) and specificity was found 0.851 (95% CI: 0.761–0.911). The ODSI has demonstrated internal consistency and a reliability coefficient (Pearson and Spearman coefficients) of 0.70. The intra-class correlation (r=0.856, 95% CI: 0.796–0.902) and test-retest sessions (r=0.970, 95% CI: 0.898–0.991) were high. Receiver operating characteristic (ROC) curve analysis showed that a cutoff point of 6 is valuable for classifying EDS for older people [11].
Reliability
In order to assess its test-retest reliability, the inventory was used twice in 13 patients seven days apart. For interrater reliability, the ODSI was performed on 11 patients by two geriatricians (PÜ, GŞA) who did not know each other’s scores in a different examination room on the same day.
Statistical analysis
The minimum sample size required to have an internal consistency above 70% and type 1 error level below 5% was found to be 90 for 3-item scale. The sample size was determined by PASS Power Analysis and Sample Size software program 11 (NCSS, LLC, Kaysville, UT, USA). Continuous variables were presented as mean±standard deviation for normally distributed variables and median (minimum–maximum) for skew distributed variables. Categorical variables were presented as number and frequency. The following procedures were used for the item analysis. Item scores were summarized using median and minimum–maximum values. In order to validate the consistency of the rating scale, inter-item correlations (Pearson and Spearman coefficients) and correlations between item scores and summated scale scores were evaluated. Interclass correlation coefficient (ICC) was used to examine test-retest reliability and interrater reliability. The construct validity was assessed by Spearman correlation analyses between the ODSI and the ESS. The accuracy of the test was measured by the area under the curve (AUC) from ROC curve analysis. The optimal cutoff score for detecting EDS (scored greater than 10 from the ESS) of the ODSI was assessed by the ROC curve analysis. Cohen’s kappa statistics (using a cutoff of the ESS >10) was calculated. Statistical analyses were performed by SPSS Statistics 23.0 program (IBM Corp., Armonk, NY, USA). A 5% type I error level was used to infer statistical significance.
RESULTS
Totally 106 patients (53 EDS and 53 control) were included in the study. The median age of the patients was 73 (65–89) years and 55.7% (n=59) were female. The EDS group was older than the control group (74 [65–89] vs. 72 [65–89], p=0.033). Gender, education, and living status were similar in both groups. The EDS group was more dependent on instrumental activities of daily living than the control group (7.3 [0–8] vs. 7.9 [6–8], p=0.004). There was no difference between the groups regarding Katz ADL, MNA-SF, MMSE, and Yesavage Depression Scale scores. Insomnia and other sleep disorders were higher in the EDS group (27 [50.9%] vs. 17 [32.1%], p=0.049), but other geriatric syndromes such as malnutrition, incontinence, polypharmacy, and falls were similar in both groups. The demographic and clinical characteristics of the patients are demonstrated in Table 1.
Comparison of demographic and clinical characteristics of patients with and without EDS
Internal consistency
Item scale correlation
There was a strong correlation between the total score and the second (r=0.946, p<0.001) and third questions (r=0.723, p<0.001); there was a weak correlation with first question (r=0.290, p=0.003). A moderate correlation was observed between the second and third questions (r=0.493, p<0.001) (Table 2).
Correlation of total score and each question
Test-retest and inter-rater reliability
The ODSI was administered to 13 patients twice in a one-week period by the same geriatrician. Test-retest reliability was high for the total sample (ICC: 0.871, 95% CI: 0.632–0.959, p<0.001). The inter-rater reliability of the ODSI was also high for the total sample (ICC: 0.851, 95% CI: 0.540–0.958, p<0.001).
Construct validity
Construct validity was assessed by correlating the ODSI with the ESS. Positive strong correlations were found with the ODSI total score and the ESS (Pearson r=0.856, 95% bootstrap CI [0.815–0.895], p<0.001; Speraman’s rho=0.876, 95% bootstrap CI [0.813– 0.918], p<0.001).
Cutoff scores
ROC curve analysis, assuming the gold standard ESS >10, showed an AUC of 0.968 (95% CI: 0.937–0.998), a cutoff score of ≥6, a sensitivity of 94.1%, a specificity of 87.6%, a positive predictive value of 76.55% (95% CI: 64.95–85.18), and a negative predictive value of 97.2% (95% CI: 83.83–99.57) (Figure 1).
Receiver operating characteristic (ROC) analysis curve. AUC, area under the curve.
The kappa agreement coefficients between the ESS >10 and ODSI ≥6 cutoff points, for the frequency of EDS was good (κ=0.660, p<0.001).
DISCUSSION
Our study demonstrated that the Turkish version of the ODSI is a valid and reliable tool for screening EDS in older people. It has high test-retest and interrater reliability. In addition, the strong correlation was ascertained between the ESS and the ODSI total scores.
The construct validity was assessed to evaluate the validity of the ODSI Turkish version questionnaire. Test-retest and inter-rater reliability, and internal consistency reliability tests were performed to evaluate its reliability. Kappa agreement coefficients were also obtained.
In the original validation study, the ODSI has an internal consistency and a reliability coefficient of 0.70 (Pearson and Spearman tests) for its three items [11]. These results suggest a strong internal consistency and reliability.
Our results also showed that there was a strong correlation between the total score and the second and third questions, but only the first question had weak correlations. This weak correlation may be due to the fact that most of the participants had zero points from the first question and falling asleep in active situation is not acceptable.
The inter-rater and intra-rater reliability of the ODSI was high (ICC: 0.851, ICC: 0.871, respectively). The ESS’s test-retest reliability was moderate evidence between fair and good, while the test-retest reliability of the ODSI was high [21]. In a study evaluating 181 patients with EDS, a significantly positive correlation was found between the ODSI and the ESS (r=0.547, 95% CI [0.436–0.642], p<0.001) [7]. Similar to previous studies, there was a high correlation between the ODSI and the ESS in our study (r=0.876). This results testify that the ODSI is reliable tool for detecting EDS.
The recommended cut-off score is ≥6 or above for detecting daytime sleepiness in patients with OSAS [11]. In this study, we found a cut-off score of ≥6, a sensitivity of 94.1%, and a specificity of 87.6%. The ESS, commonly used among subjective methods and accepted as the gold standard, has good sensitivity but poor specificity to ascertain objectively evaluated EDS [22].
In our study, we found that the EDS group was more dependent on instrumental activities of daily living like the previous literature. In the data obtained during the sleep visits of the Fractures in Men Study, worse ESS scores were correlated with greater instrumental activities of daily living impairment [23]. In a study evaluating 2,968 community dwelling older women, daytime sleepiness on the ESS was significantly related with functional impairment [24]. This is an important factor demonstrating that daytime sleepiness has a negative impact on functionality of older people, therefore it is crucial to detect.
The strength of this study lies in its linguistic validation process, which meets all the criteria for questionnaire validation set forth by the ISPOR task force [13]. The original ODSI developers played an active role in this validation study. In addition, it is a prospective real-life study including consecutive patients attending a geriatric outpatient clinic.
Some limitations must nonetheless be emphasized. First, we included the independent community dwelling older patients who attended an outpatient clinic. Patients with dementia and/or psychiatric diseases were excluded. Nevertheless, there are no clear design rules for sampling a target population to validate a health measurement instrument. Previous studies recommended simply an adequate description of the target population to be able to judge the completeness and applicability of the questionnaire in other populations [25,26].
Second, we assessed daytime sleepiness only with subjective questionnaires without comparing them to so-called “objective methods” including the Maintenance of Wakefulness Test and the Multiple Sleep Latency Test. However, these objective measurements have been criticized for their non-ecological nature, far removed from real-life conditions [27]. Their correlation with subjective tests is low [28,29] and their measurements in older age groups are inconsistent [30-32]. Finally, these highly standardized “objective methods” with a lot of constraints are not suitable for use in geriatrics.
Geriatric sleep medicine is a young field that is organizing itself and creating its own assessment tools [33]. The availability of the ODSI questionnaire in Turkish, in addition to its English and French versions, will allow the assessment of a larger number of adults in the routine clinical practice as well as in research studies.
In conclusion, the ODSI Turkish version is a valid and reliable tool that can be used in Turkish-speaking patients. The simplicity, reliability, and the apparent lack of relevant influences of cultural background on performance of the 3-item ODSI make it a valuable tool for clinical management and research.
Supplementary Materials
The online-only Data Supplement is available with this article at https://doi.org/10.33069/cim.2023.0001
Gözleme ve Mülakata Dayalı Gündüz Uykululuk Ölçeği (Observation and Interview Based Diurnal Sleepiness Inventory) (ODSI)
Notes
Funding Statement
None
The authors have no potential conflicts of interest to disclose.
Availability of Data and Material
The datasets generated or analyzed during the study are available from the corresponding author on reasonable request.
Author Contributions
Conceptualization: Pelin Unsal, Gozde Sengul Aycicek, Burcu Balam Dogu. Formal analysis: Cafer Balci, Meltem Koca, Yelda Ozturk, Ilker Boga, Suna Burkuk, Erdem Karabulut. Investigation: Cafer Balci, Meltem Koca, Yelda Ozturk, Ilker Boga, Suna Burkuk, Erdem Karabulut. Methodology: Pelin Unsal, Olgun Deniz, Mert Esme, Ayse Dikmeer. Supervision: Burcu Balam Dogu, Meltem Halil, Mustafa Cankurtaran, Fannie Onen, SHakki Onen. Writing—original draft: Pelin Unsal, Burcu Balam Dogu, Fannie Onen, S-Hakki Onen. Writing—review & editing: Pelin Unsal, Burcu Balam Dogu, Fannie Onen, S-Hakki Onen, Meltem Halil, Mustafa Cankurtaran.