East Asian Arch Psychiatry 2017;27:3-10

ORIGINAL ARTICLE

Validation of the Chinese Version of the Revised Clinical Interview Schedule: Findings from Hong Kong Mental Morbidity Survey

Dr Wai-Chi Chan, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Ms Corine Sau-Man Wong, MSocSc, Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Prof. Eric Yu-Hai Chen, MA, MD, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Dr Roger Man-Kin Ng, FRCPsych, FHKAM (Psychiatry), Department of Psychiatry, Kowloon Hospital, Hong Kong SAR, China.
Dr Se-Fong Hung, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, Kwai Chung Hospital, Hong Kong SAR, China.
Prof. Linda Chiu-Wa Lam, MD, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The Chinese University of Hong Kong, Hong Kong SAR, China.
Dr Eric Fuk-Chi Cheung, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, Castle Peak Hospital, Hong Kong SAR, China.
Prof. Pak-Chung Sham, BA, MSc, PhD, MRPsych, Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Prof. Helen Fung-Kum Chiu, FRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The Chinese University of Hong Kong, Hong Kong SAR, China.
Dr Ming Lam, MRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, Castle Peak Hospital, Hong Kong SAR, China.
Dr Wing-Chung Chang, MRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Dr Edwin Ho-Ming Lee, MRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, The University of Hong Kong, Hong Kong SAR, China.
Dr Tin-Po Chiang, MRCPsych, FHKCPsych, FHKAM (Psychiatry), Department of Psychiatry, Castle Peak Hospital, Hong Kong SAR, China.
Prof. Joseph Tak-Fai Lau, BSSc, MA, PhD, FFPH, Jockey Club School of Public Health and Primary Care, The Chinese University of Hong Kong, Hong Kong SAR, China.
Prof. Jim van Os, MD, PhD, Department of Psychiatry and Psychology, Maastricht University Medical Centre, Maastricht, The Netherlands.
Prof. Glyn Lewis, PhD, FRCPsych, Division of Psychiatry, University College London, London, United Kingdom.
Prof. Paul Bebbington, PhD, FRCPsych, Division of Psychiatry, University College London, London, United Kingdom.

Address for correspondence: Dr Wai-Chi Chan, Department of Psychiatry, 2/F New Clinical Building, Queen Mary Hospital, Pokfulam, Hong Kong SAR, China.
Email: waicchan@hku.hk

Submitted: 19 May 2016; Accepted: 8 November 2016

pdf Full Paper in PDF

Introduction

Common mental disorders (CMDs) such as depressive and anxiety disorders constitute a significant disease burden. They are associated with psychosocial disability,¹ poorer physical health,² and loss of productivity.³ Apart from increased morbidity, CMDs also lead to significant mortality. In a meta-analysis of 10 prospective cohort studies involving 68,222 adults, those with anxiety or depressive disorder were found to have a reduced life expectancy.⁴

Changes in mortality demonstrated a strong dose-response effect that remained significant after adjusting for socio- demographic characteristics, alcohol and smoking habits, and social class.⁴

Given their immense burden,⁵ there is a pressing need for updated prevalence data on CMDs. Large- scale community studies, which produce generalisable information on prevalence, are preferred to traditional sources of data such as psychiatric hospital admission and attendance records.⁶ Nonetheless conducting community surveys of psychiatric disorders was not possible until the 1980s when fully structured diagnostic interviews started to emerge. The development of diagnostic interviews like the Diagnostic Interview Schedule,⁷ Composite International Diagnostic Interview,⁸ and Revised Clinical Interview Schedule (CIS-R)⁹ have made ascertainment of psychiatric diagnoses by trained lay interviewers in large-scale community studies possible.⁶ These structured interviews have contributed to the international growth in the number of large-scale psychiatric epidemiological studies over the past 2 decades.

The CIS-R is one of the most widely used structured diagnostic assessments for CMDs. Developed from the Clinical Interview Schedule,¹⁰ CIS-R has demonstrated concordance with the Schedules for Clinical Assessment in Neuropsychiatry¹¹ and the Structured Clinical Interview for the DSM-IV (SCID).¹² Compared with other fully structured diagnostic interviews, CIS-R requires less administration time, is user-friendly for lay interviewers, and is highly acceptable to the respondents. It retains adequate validity in assessing depressive and anxiety disorders in different age and ethnic minority groups.^13,14

To further its use in large-scale psychiatric epidemiological studies, the CIS-R has been translated into various languages. Its original and translated versions have been used in European countries (United Kingdom, Greece, Italy, Romania, Spain, and France)^11,12,15-17 and beyond (Australia, Sri Lanka, Malaysia, Brazil, Tanzania, and Zimbabwe).^18-23 Validation studies of various versions of CIS-R are listed in Table 1.

To the best of our knowledge, there are no published data on the psychometric properties of the Chinese version of CIS-R (C-CIS-R). The present study aimed to examine the validity and reliability of C-CIS-R and identify its optimal cut-off points in the local population.

Methods

Participants

This study was conducted as preparation for the Hong Kong Mental Morbidity Survey, the first territory-wide psychiatric epidemiological study in Hong Kong.^24,25 A total of 301 participants (140 psychiatric patients and 161 healthy controls) took part in the study. Patients were recruited from psychiatric outpatient clinics at Alice Ho Miu Ling Nethersole Hospital, Castle Peak Hospital, and Tuen Mun Hospital in Hong Kong from September 2010 to April 2011. Inclusion criteria comprised: (1) Chinese ethnicity, (2) age 16 to 75 years, and (3) CMD diagnosis established by psychiatrists according to the ICD-10 criteria. Persons with severe mental illness such as schizophrenia and bipolar disorder, significant cognitive impairment or poor physical health were excluded. Age- and gender-matched control participants, who had no current or past psychiatric diagnosis as verified by SCID, were recruited from the community sources. The study protocol was approved by the Clinical Research Ethics Committees of the Chinese University of Hong Kong, the University of Hong Kong, and the participating hospitals. The study was carried out in accordance with the latest version of the Declaration of Helsinki.

Instruments

The CIS-R⁹ determines the presence of 14 non-psychotic symptoms in the past week, namely somatic symptoms, fatigue, concentration and forgetfulness, sleep problems, irritability, worry about physical health, depression, depressive ideas, worry, anxiety, phobias, panic, compulsions, and obsessions. Score for each symptom section ranges from 0 to 4 (except for 0-5 for depressive ideas). The total score of CIS-R indicates the overall severity of psychological symptoms. The ICD-10 diagnoses of CMDs, including depressive episode, generalised anxiety disorder, mixed anxiety and depressive disorder, phobias, panic disorder, and obsessive-compulsive disorder, can be derived from a standard algorithm based on the CIS-R responses.

Chinese-bilingual SCID is a semi-structured clinical interview designed to generate a psychiatric diagnosis. It is well validated and extensively used in the local Chinese population with an overall kappa for inter-rater reliability of 0.84 and rater-clinician reliability of 0.77.^26,27 In the present study, modules A and D (mood disorders) and F (anxiety disorders) were administered to participants.

The 12-item General Health Questionnaire (GHQ- 12)²⁸ is a self-administered questionnaire that measures general mental health in both clinical and community settings. Each item is rated on a 4-point scale with a higher score representing greater psychological distress. The Chinese version gives satisfactory sensitivity and specificity.²⁹

The Hospital Anxiety and Depression Scale (HADS)³⁰ includes 14 questions probing depression and anxiety. Each question is scored on a 4-point scale from 0 to 3, with a higher score representing a higher frequency of occurrence of a symptom. The scale includes 7 questions related to depression and 7 to anxiety. Score for each domain ranges from 0 to 21, with a maximum total score of 42. The Chinese version demonstrates good validity and reliability for screening psychiatric morbidity in the local population.³¹

The Social and Occupational Functioning Assessment Scale (SOFAS)³² measures overall social and occupational functioning of an individual due to their physical limitations, general medical condition, and mental impairment. The global rating ranges from 0 to 100, where higher score indicates better functioning.

Procedures

Forward and Backward Translation

The CIS-R questionnaire was first translated into Chinese then back-translated into English by 2 independent teams of mental health professionals. To ensure conceptual equivalence, the teams comprised experienced bilingual psychologists and psychiatrists who had no prior knowledge of the scale. A consensus Chinese version was developed by comparing the original and back-translated versions. Disagreement and reconciliation were discussed in a focus group involving the principal investigator, translators, and psychiatrists in the research team. The consensus version was pilot tested in a sample of 5 patients (3 depression and 2 anxiety), and the questionnaire further refined as the final version of C-CIS-R.

Validity and Reliability

The Chinese versions of CIS-R, GHQ-12, and HADS were evaluated by research assistants who underwent video training and supervised administration of the instruments. Psychiatrists who were blinded to previous assessment results then conducted SCID interviews to establish psychiatric diagnoses. All psychiatrists had more than 5 years of clinical experience and received training on SCID. The inter-rater reliability of C-CIS-R was evaluated by observer co-ratings of a random subsample of 35 participants. For test-retest reliability of C-CIS-R, another random subsample of 24 participants was reassessed by the same rater 2 weeks later.

Statistical Analyses

Statistical analyses were performed using STATA version 12.0 (StataCorp). Descriptive statistics were expressed as means, with standard deviations, percentages, or ranges. Criterion validity was evaluated by comparing diagnoses generated by C-CIS-R and SCID using kappa coefficients (κ) with standard errors. Spearman’s correlations between total scores of GHQ-12, HADS, and C-CIS-R were calculated in the patient group (n = 140) to determine the convergent validity. Internal consistency of questions in each symptom section was measured by Cronbach’s alpha. Intraclass correlation coefficients (ICC) of the C-CIS-R total score were computed for inter-rater and test-retest reliability. Two receiver operating characteristic (ROC) curves (sensitivity vs. 1-specificity) were generated. The first ROC curve was constructed to identify the optimal cut-off score for significant non-psychotic symptoms (i.e. diagnosable CMDs). By including patients and healthy controls in the analysis, the score that best differentiated those with CMDs from those without was decided using the patient-control criterion. The second ROC curve determined the cut-off score for symptoms that require clinical attention. As the vast majority of CMD patients in need of treatment suffer impaired function and had an SOFAS score of ≤ 70,³³ we defined the group needing clinical treatment as participants with SOFAS score of ≤ 70 when determining the corresponding threshold. Area under the curve (AUC)³⁴ was computed for each ROC curve. Sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and percentage of correct classification were compared across different cut-off points. The level of significance was set at two-sided p < 0.05.

Results

Validity and Reliability

Table 2 presents the demographic characteristics of the 140 patients and 161 healthy controls. Their mean (± standard deviation) age was 47.4 ± 13.6 years. Of them, 104 (34.6%) were men and 197 (65.4%) women. Among the patients, 60.7% had a current psychiatric disorder as confirmed by SCID (29.3% with depressive disorders, 40.7% with anxiety disorders). The criterion validity of C-CIS-R, as evident from the association between C-CIS-R– and SCID-generated diagnoses, was moderate (Table 3). Kappa values linking C-CIS-R– and SCID-generated diagnoses of depressive disorder, anxiety disorder, and any CMD were 0.68 (standard error, 0.06), 0.41 (0.06) and 0.67 (0.05), respectively.

The convergent validity of C-CIS-R was determined by its correlation with GHQ-12 and HADS scores in the patient group (n = 140). There was a significant and sizeable correlation between the C-CIS-R and GHQ-12 (r = 0.71, p < 0.001). The correlation between the C-CIS-R and HADS was also significant (r = 0.76, p < 0.001).

The internal consistency was high with Cronbach’s alpha ranging from 0.685 to 0.931 in all sections. The inter- rater and test-retest reliability of the C-CIS-R were both satisfactory, in which the ICC were 0.997 (p < 0.001) and 0.894 (p < 0.001) respectively.

Optimal Cut-off Points

Receiver operating characteristic curve analyses were performed to determine the threshold points for the C- CIS-R. To detect significant non-psychotic symptoms (i.e. diagnosable CMDs) in the patient group, an ROC curve with AUC of 0.810 was constructed (Fig 1). A cut-off score of ≥ 12 gave an optimal sensitivity of 69.3% (95% confidence interval [CI], 60.9-76.8%) and specificity of 92.7% (95% CI, 87.6-96.2%).

To identify those participants who needed clinical attention (i.e. SOFAS score ≤ 70), an ROC curve with an AUC of 0.827 was constructed (Fig 2). A cut-off score ≥ 18 gave a sensitivity of 70.2% (95% CI, 59.9-79.2%) and specificity of 95.2% (95% CI, 91.4-97.7%). The diagnostic accuracy at different cut-off points is listed in Table 4.

Discussion

In view of the significant morbidity and mortality associated with CMDs,^1-4 it is important for mental health providers to offer early identification and treatment for people with depressive and anxiety disorders. Developing validated structured diagnostic instruments for CMDs is therefore essential. It allows conduction of psychiatric epidemiological studies that provide accurate prevalence data of mental health problems in the community, and facilitates appropriate planning of service and formulation of mental health policies.

This paper reports the first study of the psychometric properties of the C-CIS-R. Compared with previous validity studies (Table 1), the overall agreement between C-CIS-R and SCID in our study was either fair or moderate (κ = 0.68 for depression, 0.41 for anxiety disorders and 0.67 for any disorder). The capacity of C-CIS-R to predict SCID- generated diagnoses was moderate to good. It performed well in identifying depression and any disorder (PPV = 66% and 69.4%, respectively) and moderately so for anxiety disorders (PPV = 44.9%). The high NPV of above 90% for any disorder also indicates the accurate detection of most non-CMD cases in the community. In addition, the C-CIS-R showed good convergent validity with GHQ-12 and HADS, and demonstrated excellent inter-rater and test- retest reliability.

The cut-off points of 11/12 indicated the presence of diagnosable CMDs and 17/18 a need for treatment and clinical attention. At both points, high specificity (11/12: 92.7% and 17/18: 95.2%) and moderate sensitivity (around 70%) were achieved. They concurred with the threshold points reported in the Adult Psychiatric Morbidity Survey in the United Kingdom.³⁵

The strength of our study was the use of SCID, a gold standard in establishing psychiatric diagnosis, when validating the psychometric properties of C-CIS-R. Another strength of the study is the examination of a wide array of CMDs of different severities among psychiatric patients. Nonetheless there are a number of limitations worth noting. Firstly, the use of the self-reported questionnaires might render the results subject to recall bias. Second, the optimal cut-off points might differ with respect to age and gender, and were not investigated in the current study. The scale was also not tested on patients with severe psychiatric problems such as schizophrenia, major neurocognitive disorders, or substance use disorders. Finally, the study did not collect information on co-morbidity with physical illnesses and mental disorders other than CMDs. Further investigation is required to study the generalisability of our findings to other cohorts. Despite these limitations, the current study provides the first-ever evidence that the psychometric properties of C-CIS-R are satisfactory, and similar to CIS-R in other languages. This confirms the applicability of C-CIS-R as a diagnostic instrument of CMDs in Chinese communities.

Declaration

The authors declared no potential conflicts of interest with respect to the research, authorship, and / or publication of this article.

Acknowledgement

This study was supported by the Health and Health Services Research Fund, Food and Health Bureau of the Government of the Hong Kong Special Administrative Region (Grant No. 09101601). The authors would like to thank the psychiatrists from Alice Ho Miu Ling Nethersole Hospital, Castle Peak Hospital, and Tuen Mun Hospital for their generous support in the field work.

References

1. Druss BG, Hwang I, Petukhova M, Sampson NA, Wang PS, Kessler RC. Impairment in role functioning in mental and chronic medical disorders in the United States: results from the National Comorbidity Survey Replication. Mol Psychiatry 2009;14:728-37.
2. Cooke D, Newman S, Sacker A, DeVellis B, Bebbington P, Meltzer H. The impact of physical illnesses on non-psychotic psychiatric morbidity: data from the household survey of psychiatric morbidity in Great Britain. Br J Health Psychol 2007;12(Pt 3):463-71.
3. Mykletun A, Overland S, Dahl AA, Krokstad S, Bjerkeset O, Glozier N, et al. A population-based cohort study of the effect of common mental disorders on disability pension awards. Am J Psychiatry 2006;163:1412-8.
4. Russ TC, Stamatakis E, Hamer M, Starr JM, Kivimäki M, Batty GD. Association between psychological distress and mortality: individual participant pooled analysis of 10 prospective cohort studies. BMJ 2012;345:e4933.
5. Kessler RC, Aguilar-Gaxiola S, Alonso J, Chatterji S, Lee S, Ormel J, et al. The global burden of mental disorders: an update from the WHO World Mental Health (WMH) surveys. Epidemiol Psichiatr Soc 2009;18:23-33.
6. Jenkins R, Bebbington P, Brugha T, Farrell M, Gill B, Lewis G, et al. The National Psychiatric Morbidity surveys of Great Britain — strategy and methods. Psychol Med 1997;27:765-74.
7. Robins LN, Helzer JE, Croughan J, Ratcliff KS. National Institute of Mental Health Diagnostic Interview Schedule. Its history, characteristics, and validity. Arch Gen Psychiatry 1981;38:381-9.
8. Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, et al. The Composite International Diagnostic Interview. An epidemiologic instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Arch Gen Psychiatry 1988;45:1069- 77.
9. Lewis G, Pelosi AJ, Araya R, Dunn G. Measuring psychiatric disorder in the community: a standardized assessment for use by lay interviewers. Psychol Med 1992;22:465-86.
10. Goldberg DP, Cooper B, Eastwood MR, Kedward HB, Shepherd M. A standardized psychiatric interview for use in community surveys. Br J Prev Soc Med 1970;24:18-23.
11. Jordanova V, Wickramesinghe C, Gerada C, Prince M. Validation of two survey diagnostic interviews among primary care attendees: a comparison of CIS-R and CIDI with SCAN ICD-10 diagnostic categories. Psychol Med 2004;34:1013-24.
12. Pez O, Gilbert F, Bitfoi A, Carta MG, Jordanova V, Garcia-Mahia C, et al. Validity across translations of short survey psychiatric diagnostic instruments: CIDI-SF and CIS-R versus SCID-I/NP in four European countries. Soc Psychiatry Psychiatr Epidemiol 2010;45:1149-59.
13. McBride O, Bebbington P, Cooper C. Could the lower prevalence of affective disorder in older people be due to measurement error? Reliability of the Revised Clinical Interview Schedule in younger and older adults. J Affect Disord 2013;148:310-5.
14. Das-Munshi J, Castro-Costa E, Dewey ME, Nazroo J, Prince M. Cross-cultural factorial validation of the Clinical Interview Schedule– Revised (CIS-R); findings from a nationally representative survey (EMPIRIC). Int J Methods Psychiatr Res 2014;23:229-44.
15. Taub NA, Morgan Z, Brugha TS, Lambert PC, Bebbington PE, Jenkins R, et al. Recalibration methods to enhance information on prevalence rates from large mental health surveys. Int J Methods Psychiatr Res 2005;14:3-13.
16. Brugha TS, Bebbington PE, Jenkins R, Meltzer H, Taub NA, Janas M, et al. Cross validation of a general population survey diagnostic interview: a comparison of CIS-R with SCAN ICD-10 diagnostic categories. Psychol Med 1999;29:1029-42.
17. Skapinakis P, Bellos S, Koupidis S, Grammatikopoulos I, Theodorakis PN, Mavreas V. Prevalence and sociodemographic associations of common mental disorders in a nationally representative sample of the general population of Greece. BMC Psychiatry 2013;13:163.
18. Patton GC, Coffey C, Posterino M, Carlin JB, Wolfe R, Bowes G. A computerised screening instrument for adolescent depression: population-based validation and application to a two-phase case- control study. Soc Psychiatry Psychiatr Epidemiol 1999;34:166-72.
19. Wickramasinghe SC, Rajapakse L, Abeysinghe R, Prince M. The Clinical Interview Schedule–Sinhala version: validation in a community setting in Sri Lanka. Int J Methods Psychiatr Res 2002;11:169-77.
20. Subramaniam K, Krishnaswamy S, Jemain AA, Hamid A, Patel V. The Clinical Interview Schedule–Revised (CIS-R)–Malay Version, Clinical Validation. Malays J Med Sci 2006;13:58-62.
21. Botega NJ, Pereira WA, Bio MR, Garcia Júnior C, Zomignani MA. Psychiatric morbidity among medical in-patients: a standardized assessment (GHQ-12 and CIS-R) using ‘lay’ interviewers in a Brazilian hospital. Soc Psychiatry Psychiatr Epidemiol 1995;30:127-31.
22. Ngoma MC, Prince M, Mann A. Common mental disorders among those attending primary health clinics and traditional healers in urban Tanzania. Br J Psychiatry 2003;183:349-55.
23. Patel V, Mann A. Etic and emic criteria for non-psychotic mental disorder: a study of the CISR and care provider assessment in Harare. Soc Psychiatry Psychiatr Epidemiol 1997;32:84-9.
24. Lam LC, Chan WC, Wong CS, Chen EY, Ng RM, Lee EH, et al. The Hong Kong Mental Morbidity Survey: background and study design. East Asian Arch Psychiatry 2014;24:30-6.
25. Lam LC, Wong CS, Wang MJ, Chan WC, Chen EY, Ng RM, et al. Prevalence, psychosocial correlates and service utilization of depressive and anxiety disorders in Hong Kong: the Hong Kong Mental Morbidity Survey (HKMMS). Soc Psychiatry Psychiatr Epidemiol 2015;50:1379-88.
26. So E, Kam I, Leung CM, Chung D, Liu Z, Fong S. The Chinese- bilingual SCID-I/P project: stage 1 — reliability for mood disorders and schizophrenia. Hong Kong J Psychiatry 2003;13:7-18.
27. So E, Kam I, Leung CM, Pang A, Lam L. The Chinese-bilingual SCID- I/P project: stage 2 — reliability for anxiety disorders, adjustment disorders and ‘no diagnosis’. Hong Kong J Psychiatry 2003;13:19-25.
28. Goldberg D, Williams P. A user’s guide to the General Health Questionnaire. Windsor, United Kingdom: NFER-Nelson; 1988.
29. Pan PC, Goldberg DP. A comparison of the validity of GHQ-12 and CHQ-12 in Chinese primary care patients in Manchester. Psychol Med 1990;20:931-40.
30. Zigmond AS, Snaith RP. The hospital anxiety and depression scale. Acta Psychiatr Scand 1983;67:361-70.
31. Leung CM, Ho S, Kan CS, Hung CH, Chen CN. Evaluation of the Chinese version of the Hospital Anxiety and Depression Scale. A cross-cultural perspective. Int J Psychosom 1993;40:29-34.
32. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders. 4th ed. Washington, DC: American Psychiatric Association; 1994.
33. Williams JB. Mental health status, functioning, and disabilities measures: Global Assessment of Functioning Scale; Social and Occupational Functioning Assessment Scale. In: Rush AJ, editor. Handbook of psychiatric measures. Washington DC: American Psychiatric Association; 2000: 93-116.
34. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982;143:29-36.
35. McManus S, Meltzer H, Brugha T, Bebbington P, Jenkins R, editors. Adult psychiatric morbidity in England 2007: results of a household survey. National Health Service Information Centre for Health and Social Care; 2009.