East Asian Arch Psychiatry 2023;33:65-70 | https://doi.org/10.12809/eaap2258

ORIGINAL ARTICLE

pdf Full Paper in PDF

Ho Nguyen Yen Phi, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Bui Xuan Manh, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Tran Anh Ngoc, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Pham Thi Minh Chau, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Truong Quoc Tho, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Nguyen Trung Nghia, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam; Mental Health Unit, Hoan My Sai Gon Hospital, Ho Chi Minh City, Vietnam
Tran Trung Nghia, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Huynh Ho Ngoc Quynh, Department of Health Education and Psychology in Medicine, Faculty of Public Health, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Nguyen Tien Huy, School of Tropical Medicine and Global Health, Nagasaki University, Nagasaki, Japan
Ngo Tich Linh, Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam
Pham Le An, The Center of Training Family Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam; Department of Psychiatry, Faculty of Medicine, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam; Mental Health Unit, Hoan My Sai Gon Hospital, Ho Chi Minh City, Vietnam

Address for correspondence: Dr Ho Nguyen Yen Phi, Department of Psychiatry, University of Medicine and Pharmacy at Ho Chi Minh City, Vietnam. Email: honguyenyenphi@ump.edu.vn

Submitted: 2 December 2022; Accepted: 5 May 2023

Introduction

Vietnam is a lower-middle-income country with a chronic shortage in mental healthcare resources. Although major depressive disorder (MDD) is the second-most common mental health condition in Vietnam, with a prevalence of 2.8%,¹ the estimated number of psychiatrists per 100 000 Vietnamese people is 0.35.² Additionally, as a result of internalised stigma,³ associated somatic symptoms,^4,5 and co-occurring chronic physical conditions,^6,7 patients with mental health problems are more likely to seek care at primary healthcare centres (PHCs) rather than specialist clinics. However, PHCs lack human resources to recognise patients with MDD,^8,9 who therefore have been underdiagnosed and not received appropriate intervention.¹⁰ Given the high prevalence (15.8%) of depressive disorders among PHCs in Vietnam¹¹ and the lower capacity for PHCs than psychiatric teams to perform mental evaluations (11.1% vs 93.7%),12 it is critical to use the Vietnamese versions of depression rating scales to screen MDD in the PHC settings.

The self-reported and clinician-rated versions of the Quick Inventory of Depressive Symptomatology (QIDS-SR and QIDS-C, respectively) have been widely used, especially in the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial.¹³ The QIDS-SR and QIDS-C comprise 16 items across nine domains to measure symptom severity according to the DSM-IV criteria for MDD.¹⁴ Both scales have contributed to the assessment of depression and treatment response.¹⁵ Clinician-rated scales are marginally more accurate and specific than the self-reported scales for screening older adults¹⁶ and adolescents¹⁷ in PHCs. In a meta-analysis, the QIDS-SR and QIDS-C have high internal consistency, with a Cronbach’s alpha of 0.65 to 0.89.¹⁸ For the QIDS-SR, a cut-off score of 6 is recommended when screening for MDD, with 79% sensitivity and 81% specificity.¹⁴ Nonetheless, to date, no study has assessed the sensitivity and specificity of the QIDS-C as a screening tool in PHCs.

The Patient Health Questionnaire (PHQ-9) has been widely used to screen for depression and measure symptom severity.¹⁹ In PHCs, the PHQ-9 is easy to administer with outstanding discriminatory performance and accurate psychometric properties.²⁰ The Vietnamese PHQ-9 has been validated in sexual minority women,²¹ patients with epilepsy,²² patients undergoing methadone therapy,²³ patients with cancer,²⁴ and medical students.²⁵ It has good internal consistency, with a Cronbach’s alpha of 0.81 for medical students and 0.86 for sexual minority women.^21,25 At the cut-off point of ≥10, the specificity and sensitivity of PHQ-9 for detecting MDD in the general population were both 0.88.¹⁹

We aimed to evaluate the internal consistency of the Vietnamese versions of the QIDS-SR, QIDS-C, and PHQ-9, to determine the optimal cut-off scores for screening for MDD, and to examine the correlations between the QIDS- SR, QIDS-C, and PHQ-9.

Methods

This study was carried out in accordance with the Declaration of Helsinki. The study protocol was approved by the Institutional Review Board of the University of Medicine and Pharmacy (reference: IRB-VN01002). Informed consent from each participant was obtained. This study was conducted in the outpatient ward of the Family Medicine Facility at the University Medical Center, Ho Chi Minh City. All patients who visited the ward between January 2022 and February 2022 were invited to participate. The inclusion criteria were age ≥18 years and the ability to communicate in Vietnamese. Patients were excluded if they were diagnosed by psychiatrists with psychosis, bipolar disorder, dementia, or intellectual disability (based on the DSM-5 diagnosis criteria) or on psychotropic medications for 2 weeks prior to recruitment.

Participants completed the QIDS-SR and the PHQ-9 under the supervision of interviewers. For participants with visual disability or low reading ability, the interviewers read out each item of the QIDS-SR and PHQ-9 and recorded their responses. In addition, participants were assessed independently by two psychiatrists using the QIDS-C and the Structured Clinical Interview for DSM-5. Participants were assigned to the healthy group if they did not meet the DSM-5 diagnostic criteria for MDD. The psychiatrists would consult with the physicians to ensure that patients with MDD and/or suicidal thoughts would be referred for intervention.

For QIDS-SR and QIDS-C, participants were asked to rate the severity of their depressive symptoms over the prior 7 days on a 4-point Likert scale that ranges from 0 to 3 Cut-off scores of 5, 10, 15, and 20 indicate the severity of depressive symptoms as normal, mild, moderate, severe, and very severe, respectively. The QIDS-SR and QIDS-C consist of 16 items across nine domains including sleep disturbance (items 1 to 4), sad mood (item 5), appetite/ weight change (items 6 to 9), concentration (item 10), self-criticism (item 11), suicidal ideation (item 12), interest (item 13), energy/fatigue (item 14), and agitation/retardation (items 15 and 16).

Permission was obtained from Mapi Research Trust to translate the QIDS-SR and QIDS-C into Vietnamese according to its guidance.²⁶ Some modifications were made as follows. Pounds were converted to kilograms. The first possible response to item 4 was modified to include a midday nap, as nearly half of Vietnamese people, particularly older adults, have the habit of napping at midday. The phrase “read books, newspapers, or watch screens (television, phones, or computers)” was added to the final response to item 10, as the younger generation often prefers watching television, YouTube, or reading the news from websites or Facebook rather than reading books or newspapers. The initial responses to items 6 to 9 were changed from “no change” to “no decrease/increase” in usual weight or appetite.

The Vietnamese version of the PHQ-9 comprises nine items across nine domains: interest, sad mood, sleep disturbance, energy, appetite, self-criticism, concentration, agitation, and suicidal ideation.²⁵ Symptom frequency during the prior 2 weeks was rated on a 4-point Likert scale. The severity of depression symptom (mild, moderate, moderately severe, and severe) was determined using cut-off scores of 5, 10, 15, and 20, respectively.¹⁹

SPSS (Windows version 28.0; IBM Corp, Armonk [NY], US) was used to analyse the data. Receiver operating characteristic curves were used to assess criterion validity. The optimal cut-off score was determined by the best sensitivity, specificity, and Youden index. Sensitivity was set higher than specificity to detect all potential cases.²⁷ Cronbach’s alpha was used to determine internal consistency; a value of ≥0.6 was considered satisfactory.²⁸ Pearson correlation coefficients between the QIDS-SR, QIDS-C, and PHQ-9 were calculated.

Results

Of 520 patients recruited, eight were excluded because of antidepressant use (n = 2) or benzodiazepine use (n = 6). Among the 512 eligible participants, 506, 509, and 507 were assessed using the QIDS-SR, QIDS-C, and PHQ-9, respectively. Data from 506 participants who were assessed by all three scales were analysed (Table 1).

The QIDS-SR, QIDS-C, and PHQ-9 demonstrated good validity, with robust areas under the curve of 0.901 (95% confidence interval [CI] = 0.863-0.939), 0.967 (95% CI = 0.950-0.983), and 0.864 (95% CI = 0.084-0.924), respectively (Figure). Table 2 summarises the psychometric property of the three scales at various cut-off scores. The Cronbach’s alphas for the QIDS-SR, QIDS-C, and PHQ-9 were 0.709, 0.813, and 0.745, respectively, indicating acceptable internal consistency between domains/items of the three scales (Table 3).

The PHQ-9 highly correlated with the QIDS-SR (r = 0.77, 95% CI = 0.73-0.80, p < 0.001) and the QIDS-C (r = 0.75, 95% CI = 0.71-0.78, p < 0.001), indicating good concurrent validity of the QIDS-SR and QIDS-C.

Discussion

To the best of our knowledge, this is the first study to validate the Vietnamese versions of the QIDS-SR, QIDS-C, and PHQ-9 in a PHC setting; the sample size was large (>500 participants) and conformed to the respondent-to-item ratio of 5:1 or greater, as specified by the validation guidelines for questionnaires.²⁹ The three scales had acceptable internal consistency, acceptable concurrent validity, and adequate discriminative validity for screening for MDD.

When screening for MDD in a PHC setting, a cut-off score of 6 could achieve the optimal balance of sensitivity and specificity for both the QIDS-SR (87.8% and 77.8%, respectively) and QIDS-C (97.6% and 86.2%, respectively). This finding is consistent with that in a US study of 596 outpatients with chronic, nonpsychotic MDD, which suggested a cut-off score of 6 for the QIDS-SR to achieve 79% sensitivity and 81% specificity.¹⁴ Additionally, another study reported that a cut-off score of 6 was used to determine whether a patient was in remission.³⁰

Regarding the PHQ-9, a cut-off score of 4 was established for screening for MDD in the PHC setting, with 82.9% sensitivity and 70.1% specificity. This finding is consistent with the finding of a study of Vietnamese cancer patients that reported 87% sensitivity and 58% specificity.²⁴ In contrast, other studies have suggested higher cut-off scores for screening for MDD, namely a score of 5 in methadone maintenance patients (95% sensitivity and 91% specificity),²³ a score of 7 in medical students (82% sensitivity and 71% specificity),²⁵ and a score of 8 in patients with epilepsy (87% sensitivity and 82% specificity).²² Some international studies have suggested higher cut-off scores when screening for MDD in primary care, including a score of 6 (91% sensitivity and 72% specificity),³¹ a score of 10 (88% sensitivity and 88% specificity),¹⁹ and a score of 11 (83% sensitivity and 83% specificity).³²

The Vietnamese QIDS-SR, QIDS-C, and PHQ-9 demonstrated acceptable internal consistency with Cronbach’s alphas of 0.709, 0.813, and 0.745, respectively. The lowest item-total correlation coefficients were found in the domains of ‘self-outlook’ in the QIDS-SR (0.302) and ‘thought of death’ in the QIDS-C (0.303) and PHQ-9 (0.194). Whereas the lowest item-total correlation was found in the ‘self-view’ domain in a study examining the QIDS-SR,³³ and in the ‘sleep problem’ domain in other QIDS-C studies.^33-35 These distinctions may be explained by the fact that suicide and self-perception are central themes in traditional Vietnamese culture. Although many patients had a high risk of suicide and low self-esteem, they tended to be resistant to or dismissive of these issues. This tendency is considered a cultural phenomenon and is common in Asian populations.³⁶

This study had several limitations. First, although the sample size is large, it was in a single general hospital, and the results may not be generalised to all Vietnamese population. Second, although the sensitivity and specificity of the PHQ-9 at a cut-off score of 4 were acceptable for screening, this appears to be lower than those reported in previous studies, with cut-off scores ranging from 5 to 15.³⁷ Third, the cross-sectional nature of the study cannot determine the test reliability and causal relationships. Fourth, the traditional Lunar New Year holiday, during which some participants returned to their hometown, interrupted the recruitment process. Two weeks before the holiday, some participants were anxious and depressed regarding their economic situation caused by the COVID-19 pandemic. Finally, the Medical University Center is overcrowded and overburdened. Participants were assessed in a hurry after an extended period of waiting. The latter two may have negative psychological effects. Nonetheless, the present study had a large sample size and was conducted in the context of a PHC setting, where patients with MDD typically first seek medical assistance.

Conclusions

The Vietnamese versions of the QIDS-SR, QIDS-C, and PHQ-9 demonstrated good psychometric properties, including acceptable internal reliability, reasonable concurrent validity, and adequate discriminative validity for screening for MDD. For primary care MDD screening, a cut-off score of 6 was recommended for both the QIDS-SR and the QIDS-C, whereas a cut-off score of 4 was recommended for the PHQ-9.

Contributors

HNYP contributed to the conceptual design, data collection, statistical analysis, provision of the original draft. BXM, TAN and PTMC contributed to the conceptual design, data collection, statistical analysis, and provision of the original draft. NTN and TQT contributed to data collection, statistical analysis, and editing of the manuscript. TTN, HHNQ, and NTH contributed to the conceptual design, statistical analysis, and editing of the manuscript. NTL and PLA supervised the conceptual design, statistical analysis, and final manuscript preparation. All authors read and approved the final manuscript.

Conflicts of Interest

No authors have disclosed any conflicts of interest.

Funding/Support

This study received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

Data Availability

All data generated or analysed during the present study are available from the corresponding author on reasonable request.

Ethics Approval

This study was approved by the University of Medicine and Pharmacy at Ho Chi Minh City Board of Ethics in Biomedical Research (reference: IRB-VN01002). The patients were treated in accordance with the tenets of the Declaration of Helsinki. The patients provided written informed consent for all treatments and procedures.

Acknowledgements

We would like to thank Mr Tran Cong Khanh, a teacher at the Vietnam-Australia International School, and Ms Ai Ngoc Phan, a psychologist at the Department of Psychiatry, University of Medicine and Pharmacy at Ho Chi Minh City, for their support during the translation process. We also specially thank the patients and the staff of the Outpatient Ward of Family Medicine at the University Medical Center in Ho Chi Minh City who kindly participated in and supported this study.

References

1. Vuong DA, Van Ginneken E, Morris J, Ha ST, Busse R. Mental health in Vietnam: burden of disease and availability of services. Asian J Psychiatr 2011;4:65-70. Crossref
2. World Health Organization. WHO-AIMS Report on Mental Health System in Viet Nam. World Health Organization; 2006.
3. Alemayehu Y, Demilew D, Asfaw G, Asfaw H, Alemnew N, Tadesse A. Internalized stigma and associated factors among patients with major depressive disorder at the outpatient department of Amanuel Mental Specialized Hospital, Addis Ababa, Ethiopia, 2019: a cross-sectional study. Psychiatry J 2020;2020:7369542. Crossref
4. Simon GE, VonKorff M, Piccinelli M, Fullerton C, Ormel J. An international study of the relation between somatic symptoms and depression. N Engl J Med 1999;341:1329-35. Crossref
5. Yen Phi HN, Minh Chau PT, Xuan Manh B, et al. Somatic symptoms in major depressive disorder: a cross-sectional study in a mental health setting, Vietnam. Medpharmres 2022;7:42-8. Crossref
6. Singh GP. Medical illnesses in psychiatric patients--a review. East Asian Arch Psychiatry 2005;15:127-31.
7. Read JR, Sharpe L, Modini M, Dear BF. Multimorbidity and depression: a systematic review and meta-analysis. J Affect Disord 2017;221:36-46. Crossref
8. Nguyen T, Tran T, Tran H, Tran T, Fisher J. Challenges in integrating mental health into primary care in Vietnam. In: Innovations in Global Mental Health. Springer; 2019:1-21. Crossref
9. Murphy J, Corbett KK, Linh DT, Oanh PT, Nguyen VC. Barriers and facilitators to the integration of depression services in primary care in Vietnam: a mixed methods study. BMC Health Serv Res 2018;18:641. Crossref
10. Thornicroft G, Chatterji S, Evans-Lacko S, et al. Undertreatment of people with major depressive disorder in 21 countries. Br J Psychiatry 2017;210:119-24. Crossref
11. Yen Phi HN, Quoc Tho T, Xuan Manh B, et al. Prevalence of depressive disorders in a primary care setting in Ho Chi Minh City, Vietnam: a cross-sectional epidemiological study. Int J Psychiatry Med 2023;58:86-101. Crossref
12. Chan CY, Yong SW, Mhaisalkar AS, Sin GL, Poon SH, Tan SM. Audit of mental capacity assessment by primary care physicians versus consultation-liaison psychiatrists. East Asian Arch Psychiatry 2018;28:95-100.
13. Alpert JE, Biggs MM, Davis L, et al. Enrolling research subjects from clinical practice: ethical and procedural issues in the Sequenced Treatment Alternatives to Relieve Depression (STAR*D) trial. Psychiatry Res 2006;141:193-200. Crossref
14. Rush AJ, Trivedi MH, Ibrahim HM, et al. The 16-Item Quick Inventory of Depressive Symptomatology (QIDS), clinician rating (QIDS-C), and self-report (QIDS-SR): a psychometric evaluation in patients with chronic major depression. Biol Psychiatry 2003;54:573-83. Crossref
15. Zimmerman M, Walsh E, Friedman M, Boerescu DA, Attiullah N. Are self-report scales as effective as clinician rating scales in measuring treatment response in routine clinical practice? J Affect Disord 2018;225:449-52. Crossref
16. Doraiswamy PM, Bernstein IH, Rush AJ, et al. Diagnostic utility of the Quick Inventory of Depressive Symptomatology (QIDS-C16 and QIDS-SR16) in the elderly. Acta Psychiatr Scand 2010;122:226-34. Crossref
17. Russell PS, Basker M, Russell S, Moses PD, Nair MK, Minju KA. Comparison of a self-rated and a clinician-rated measure for identifying depression among adolescents in a primary-care setting. Indian J Pediatr 2012;79(Suppl 1):S45-51. Crossref
18. Reilly TJ, MacGillivray SA, Reid IC, Cameron IM. Psychometric properties of the 16-item Quick Inventory of Depressive Symptomatology: a systematic review and meta-analysis. J Psychiatr Res 2015;60:132-40. Crossref
19. Kroenke K, Spitzer RL, Williams JB. The PHQ-9: validity of a brief depression severity measure. J Gen Intern Med 2001;16:606-13. Crossref
20. Miller P, Newby DA, Walkom E, Schneider J, Li SC, Evans TJ. The performance and accuracy of depression screening tools capable of self-administration in primary care: a systematic review and meta- analysis. Eur J Psychiatry 2021;35:1-18. Crossref
21. Nguyen TQ, Bandeen-Roche K, Bass JK, German D, Nguyen NT, Knowlton AR. A tool for sexual minority mental health research: the Patient Health Questionnaire (PHQ-9) as a depressive symptom severity measure for sexual minority women in Viet Nam. J Gay Lesbian Ment Health 2016;20:173-91. Crossref
22. Le Hoang Ngoc T, Le MT, Nguyen HT, et al. Patient Health Questionnaire (PHQ-9): a depression screening tool for people with epilepsy in Vietnam. Epilepsy Behav 2021;125:108446. Crossref
23. Mughal AY, Stockton MA, Bui Q, et al. Validation of screening tools for common mental health disorders in the methadone maintenance population in Hanoi, Vietnam. BMC Psychiatry 2021;21:488. Crossref
24. Nguyen TQ, Do TM, Pham TA. Screening for psychological distress in Vietnamese cancer patients: an evaluation of the distress thermometer. Cancer Med 2021;10:7793-803. Crossref
25. Nguyen NT, An PL, Dong Phuong Tien N. Reliability and validity of Vietnamese version of Patient Health Questionnaire 9 items (PHQ-9) among UMP medical freshmen. In: 8th International Conference on the Development of Biomedical Engineering in Vietnam. Springer; 2020:901-23. Crossref
26. Mapi Research Trust. Linguistic Validation Guidance of a Clinical Outcome Assessment. 2016.
27. Warner J. Clinicians’ guide to evaluating diagnostic and screening tests in psychiatry. Adv Psychiatr Treat 2004;10:446-54. Crossref
28. Ursachi G, Horodnic IA, Zait A. How reliable are measurement scales? External factors with indirect influence on reliability estimators. Procedia Econ Finance 2015;20:679-86. Crossref
29. Tsang S, Royse CF, Terkawi AS. Guidelines for developing, translating, and validating a questionnaire in perioperative and pain medicine. Saudi J Anaesth 2017;11(Suppl 1):S80-9. Crossref
30. Liu J, Xiang YT, Lei H, et al. Guidance on the conversion of the Chinese versions of the Quick Inventory of Depressive Symptomatology-Self- Report (C-QIDS-SR) and the Montgomery-Asberg Scale (C-MADRS) in Chinese patients with major depression. J Affect Disord 2014;152- 154:530-3. Crossref
31. Sung SC, Low CC, Fung DS, Chan YH. Screening for major and minor depression in a multiethnic sample of Asian primary care patients: a comparison of the nine-item Patient Health Questionnaire (PHQ-9) and the 16-item Quick Inventory of Depressive Symptomatology - Self- Report (QIDS-SR16). Asia Pac Psychiatry 2013;5:249-58. Crossref
32. Thapar A, Hammerton G, Collishaw S, et al. Detecting recurrent major depressive disorder within primary care rapidly and reliably using short questionnaire measures. Br J Gen Pract 2014;64:e31-7. Crossref
33. Bernstein IH, Rush AJ, Stegman D, Macleod L, Witte B, Trivedi MH. A comparison of the QIDS-C16, QIDS-SR16, and the MADRS in an adult outpatient clinical sample. CNS Spectr 2010;15:458-68.Crossref
34. Bernstein IH, Rush AJ, Suppes T, et al. A psychometric evaluation of the clinician-rated Quick Inventory of Depressive Symptomatology (QIDS-C16) in patients with bipolar disorder. Int J Methods Psychiatr Res 2009;18:138-46. Crossref
35. Liu J, Xiang YT, Wang G, et al. Psychometric properties of the Chinese versions of the Quick Inventory of Depressive Symptomatology - Clinician Rating (C-QIDS-C) and Self-Report (C-QIDS-SR). J Affect Disord 2013;147:421-4. Crossref
36. Chu J, Lin M, Akutsu PD, Joshi SV, Yan LH. Hidden suicidal ideation or intent among Asian American Pacific Islanders: a cultural phenomenon associated with greater suicide severity. Asian Am J Psychol 2018;9:262-9. Crossref
37. Levis B, Benedetti A, Thombs BD; DEPRESsion Screening Data (DEPRESSD) Collaboration. Accuracy of Patient Health Questionnaire-9 (PHQ-9) for screening to detect major depression: individual participant data meta-analysis. BMJ 2019;365:l1476. Crossref