East Asian Arch Psychiatry 2020;30:12-9 | https://doi.org/10.12809/eaap1848

ORIGINAL ARTICLE 

Chi-chiu Lee, MRCP, FRCPsych, Kwai Chung Hospital, Hong Kong
William WH Chui, MRCPsych(UK), FHKAM(Psych), Castle Peak Hospital, Hong Kong
Sui-lung Wong, BSc Nursing, MSc Nursing, Kwai Chung Hospital, Hong Kong
Terry CB Wong, BSW, MSW, New Life Psychiatric Rehabilitation Association, Hong Kong
Sharon PF Lau, BSc Nursing, Kwai Chung Hospital, Hong Kong
Po-keung Kwong, FRCPsych(UK), FHKAM(Psych), Kwai Chung Hospital, Hong Kong
Se-fong Hung, FRCPsych(UK), FHKAM(Psych), Department of Psychiatry, Chinese University of Hong Kong, Hong Kong
Sania SW Yau, BSW, MSW, New Life Psychiatric Rehabilitation Association, Hong Kong

Address for correspondence: Dr Chi-chiu Lee, Kwai Chung Hospital, Hong
Kong.
Email: lccz04@ha.org.hk

Submitted: 15 May 2018 Accepted: 16 November 2018 

pdf Full Paper in PDF

Introduction

The community, rather than in an institution, is the preferred setting for recovery from mental illness, deinstitutionalisation can lead to undesirable consequences such as homelessness, criminal offences, increased hospital readmission, and non- compliance with treatment.^1,2 To address these issues, a case management model has been developed to improve the quality of life of patients in community.^3-5

In Hong Kong, after a tragedy in 1982 in which a person with mental illness killed several people, including children in a kindergarten, a registry was established for psychiatric patients with a history of violence requiring priority follow-up.⁶ This initiative was the first step in risk assessment and stratification in the public mental health service in Hong Kong.

There has been a substantial increase in resources and reforms in the community mental health services provided by both the government (ie, the Hospital Authority) and non-government organisations (NGOs). A recovery- oriented multi-disciplinary case management model was implemented, with a primary focus on patients with severe mental illness. Preliminary results were positive in the psychiatric^7-10 and social service settings.^11,12

In 2010, the Hospital Authority community psychiatric service (CPS) launched the personalised care programmes (PCPs) for patients with severe mental illness. Subsequently, the CPS evolved into three tiers of service: the intensive care team, PCPs, and standard CPS care. For each patient receiving CPS, a profile of needs and risks is identified, and their level of risk is stratified. In addition, the priority follow-up system was revised into the special care system to specify the levels of multi-disciplinary care required for each category (including tracing of defaulters) and to more clearly describe the upgrading or downgrading procedure (eg, endorsed by a multi-disciplinary meeting or by a senior doctors) for psychiatric patients.¹³ The special care system includes intensive care for those at risk of committing serious violence, special care for those at risk of committing violence, and conventional care. Tier 1, the intensive care team, has a case manager–to-patient ratio of 1:25; it adopts an assertive community treatment model for highest-risk patients. The intensive care team has more senior case managers than other tiers, and its case managers have additional training in crisis intervention, specific risk, and need assessment for patients of very high risk or with specific diagnosis, and liaison with police or other disciplinary forces. Management of each patient is reviewed in multi-disciplinary case conference on a 3- monthly basis, whereas PCP and standard CPS care are on a 4-monthly and 6-monthly basis, respectively. Tier 2, the PCPs, has a case manager–to-patient ratio of 1:50; it covers most of the special care patients and those with some risks in community. The PCPs have the largest proportion of patients. Tier 3, standard CPS care, has a case manager–to- patient ratio of 1:80; it supports patients with a lower level of risk. The staff-to-patient ratio is based on available resources and service demand. Unless conditionally discharged under the mental health ordinance, patients can opt out from the CPS and can be stepped up or down the tiers according to the ongoing risk assessment by case managers.

In Hong Kong, 11 NGOs operate 24 integrated community centres for mental wellness (ICCMWs). These district-based ICCMWs target early identification of those in need of mental health service, supporting patients in the community, and promoting mental health literacy and mental well-being to the general public. Case management service is provided for patients whose needs and risks are complex. Potential patients can approach the ICCMWs directly or be referred from medical social service units, integrated family service centres, or the CPS as step-down cases. Patients with predominantly medical needs are taken up by the CPS, whereas those with predominantly social needs are supported by ICCMWs. ICCMWs have taken up some mental health–related interventions that were previously provided by integrated family service centres, whose major role is to arrange welfare and manage family issues in the community.

To facilitate recovery of patients in community, risk stratification in PCPs and ICCMWs must be valid in terms of the level of needs and identified risks of patients and intensity of care. As the case management service may transition from PCPs to ICCMWs, agreement on risk stratification between them is an important concern.

In this study, we examined the validity of risk stratification in PCPs by correlating clinical condition of patients. Because violence and suicide incidents are rare, it is not feasible to use future occurrence of these incidents to assess the validity of a risk assessment. If a psychiatric patient was identified as being at risk, this patient would also be rated to have more severe psychopathology and poorer functioning and support. We investigated whether the identified risk matched the intensity of care. We used service utilisation data as proxy of intensity of care and hypothesised that those at risk should receive higher intensity of care in community. Finally, we examined the concordance in risk stratification between PCPs and ICCMWs. We hypothesised that there is no difference in epidemiological and clinical characteristics of patients between PCPs and ICCMWs.

Methods

The North Lantau and Mongkok districts were selected for the present study because both are served by the PCPs of Kwai Chung Hospital and ICCMWs of the New Life Psychiatric Rehabilitation Association. All patients who received case management services in the two institutions between 1 April 2014 and 30 June 2015 were identified. Data from the PCPs were obtained from the Hospital Authority Clinical Management System; data from the ICCMWs were provided by case managers.

The PCPs use the Standardised Risk Assessment (SRA) for risk stratification. It is a 14-item scale adapted from the clinical assessment of risk decision support used in the United Kingdom.¹⁴ The scale was completed by case managers on patient’s risks in violence, suicide, and neglect upon admission into the PCP and then 6-monthly and before discharge. Endorsement of any one item on the SRA results in the patient being considered as at risk.

The ICCMWs use another structured assessment tool based on the Modified Scale for Suicidal Ideation¹⁵ and the Two-minute Risk Assessment of violence¹⁶ to determine the integrated risk level (IRL). The IRL is considered low if there is no endorsement from both scales, moderate if there is endorsement from either scale, and high if there is endorsement from both scales.

Epidemiological and service utilisation data were retrieved, including the number of patients who received case management service in PCPs and/or ICCMWs, the IRL and SRA assessments, home visits and telephone contacts, and for those in PCP, Brief Psychiatric Rating Scale scores (lower scores indicate less severe symptoms)¹⁷ and Health of the Nation Outcome Scales scores (lower scores indicate better mental condition, functioning, and support).¹⁸

The Chi-squared test was used to examine the distribution of epidemiological characteristics among different groups of patients. Receiver operating characteristics (ROC) analysis was used to examine the sensitivity of SRA in predicting IRL. IRL was presumed to be the gold standard owing to its satisfactory psychometric validation results in suicidal risk assessment¹⁵ and its case-by-case design in violent risk assessment,^1,16 whereas SRA was originally developed as a brief screening tool. Correlations among service utilisation, psychopathological score, and risk stratification of patients were determined.

Results

In total, 644 patients (356 from North Lantau and 288 from Mongkok) attended ICCMWs from 1 April 2014 to 30 June 2015. These patients did not differ significantly in terms of sex, age, education level, and psychiatric diagnosis (Table 1). In the same study period, 4313 patients from the two districts were cared for in psychiatric specialist outpatient clinics (SOPDs). Of them, 953 received case management services from Hospital Authority: 41 received intensive care team service and 912 received PCP or standard CPS care. Among these 953 patients, six required intensive care and received CPS (all 6 by intensive care team) and 114 required special care, 112 (98.3%) of whom received CPS (5 by intensive care team, 107 by PCP). Among these 112 patients, 13 also received case management services from ICCMWs (Figure 1).

Among the 644 patients receiving case management services from ICCMWs, 492 attended psychiatric SOPDs. The 644 patients were divided into three subgroups: ICCMW only (n = 152), ICCMW + SOPD (n = 335), and ICCMW + PCP (n = 157). These three subgroups were comparable in terms of sex, age, and education level. The ICCMW + SOPD subgroup had a higher proportion of patients with mental disorders or neurotic conditions than did the ICCMW + PCP subgroup (52.3% vs 32.3%). In contrast, the ICCMW + PCP subgroup had a higher proportion of psychotic patients than did the ICCMW + SOPD subgroup (54.9% vs 33.3%). Increasing intensity of services from psychiatric SOPDs or PCPs was associated with decreasing involvement from integrated family service centres, from 32.89% for ICCMW only to 21.79% for ICCMW + SOPD to 18.47% for ICCMW + PCP (Table 2).

For the 157 patients receiving services from both ICCMWs and PCPs, we performed cross-validation among the IRL status, the SRA status, and special care system status. However, between IRL and SRA, there was a time difference in risk assessment between ICCMWs and PCPs. We thus calculated the mean of time difference (TD) as the mean of the square root of the square of each time-gap in each case, mean(TD) = ∑{√[(TD)²]}/N, which is 3.25 (range, -6.41 to 22.57) months for all 157 patients.

Cross-validation between IRL and SRA

We selected 136 patients with TD <6 months for cross- validation between IRL and SRA. Of them, 3 were categorised as high risk, 12 as moderate risk, and 121 as low risk according to the IRL. Because of the small sample size of the high-risk group, those with high or moderate risk were grouped as at risk and those with low risk as not at risk. The SRA identified 101, 39, 10, and 4 patients as at risk, with a caseness threshold of 1 item (lowest threshold), 2 items, 3 items, and 4 items (highest threshold), respectively. Of the at-risk patients identified by SRA, only 13, 7, 1, and 0 patients, respectively, were also categorised as at risk according to the IRL.

Using the IRL as the gold standard, ROC analysis found that the convergence between the two tests was modest only (area under ROC curve = 0.483 to 0.601), correlation was best when a 2-item threshold in SRA was used (Figure 2). However, in clinical practice, a patient was identified as at risk by the PCP case manager if there was any one item endorsed in the SRA. Subsequent analyses used the 1-item threshold in SRA.

Cross-validation between IRL and SRA in terms of special care system

The IRL identified 16 at-risk patients from 157 patients receiving services from both ICCMWs and PCPs, including only three of the 13 special care cases. This generated a sensitivity of 23.08%, specificity of 90.97%, positive predictive value of 18.75%, and negative predictive value of 92.91%. The SRA using 1-item threshold identified 111 at-risk patients from the 157 patients, including all the special care cases. This generated a sensitivity of 100%, specificity of 31.94%, positive predictive value of 11.71%, and negative predictive value of 100%.

Cross-validation between IRL and SRA in terms of service utilisation

PCPs delivered more home visits and telephone contacts in 6 months to the 15 at-risk patients than the 121 not-at- risk patients identified by IRL (7.47 vs 5.68), but ICCMW delivered fewer home visits to the at-risk patients than the not-at-risk patients (4.33 vs 4.44); there were no data on the telephone contacts from ICCMW. However, ICCMWs and PCPs together provided more home visits and telephone contacts to the at-risk patients than the not-at-risk patients (11.80 vs 10.12). At-risk patients (103 identified by IRL or SRA, 101 identified by SRA only, and 13 identified by both IRL and SRA) received more community services from ICCMW and PCP altogether than not-at-risk patients did (10.63 vs 9.27, 10.65 vs 9.29, and 12.15 vs 10.11, respectively) [Table 3].

Cross-validation between IRL and SRA in terms of Health of the Nation Outcome Scales and Brief Psychiatric Rating Scale

No significant associations were found between at-risk and not-at-risk patients identified by IRL or SRA in terms of the Health of the Nation Outcome Scales score or the Brief Psychiatric Rating Scale score (Table 3).

Discussion

The two districts studied in this study are served by the same NGO and hospital; therefore, their ICCMWs and PCPs should share policies and service guidelines with their corresponding organisations. Epidemiological and clinical data of patients from the two districts were not significantly different.

All intensive care and 98.3% of special care cases were cared for by the intensive care teams or PCPs with minimal overlap with ICCMWs. More patients with psychosis received services from both ICCMWs and PCPs than from ICMMWs only. Furthermore, the involvement from integrated family service centres decreased with increasing intensity of services from the Hospital Authority. There is specialisation of services between ICCMWs, integrated family service centres, and the Hospital Authority in Hong Kong.

Using special care system status as a standard, IRL had a sensitivity of only 23.08%, leaving 76.92% of the special care cases not identified as at risk. Similarly, SRA with 1-item threshold had 100% negative predictive value and 31.94% specificity; this tool was sensitive for risk stratification, but a large proportion of the screened-positive patients were wrongly placed, with a potential wastage of case manager resources. If we validate the tool with a much lower real incidence of violence or suicide, the positive predictive value will further decrease.

Using 1- or 2-item threshold, SRA identified more at-risk cases than IRL did (101 and 39 vs 13 and 7, respectively). However, if 3- or 4-item threshold was used, the SRA identified at-risk cases (10 and 4, respectively) become insensitive to IRL. Such results concurred with the fair agreement between the two assessment tools, as suggested by the low correlation coefficient (area under ROC curve = 0.483-0.646). Reasons for non-concordance between the two assessment tools are multifactorial. First, the focus between IRL and SRA is marked difference. IRL is adapted from the Modified Scale for Suicidal Ideation and the Two-minute Risk Assessment, both of which focus on the immediate thoughts and behaviours of the respondents.¹⁵

In contrast, SRA focuses on a history of risky behaviours. Second, case managers from ICCMWs and PCPs have unequal access of information to complete their IRL and SRA. The ICCMW case managers primarily use clinical interviews, with supplementary information from carers and significant others or clinical summaries from case psychiatrists and PCP case managers. In contrast, PCP case managers have access to all medical records in the Hospital Authority, including reports of previous violence/suicide or other risky behaviours. Third, even widely used and comprehensive assessment tools such as the Psychopathy Checklist-revised¹⁹ or the Historical, Clinical and Risk Management-20²⁰ have a low predictive value of risky behaviours, because the rare occurrences of violence lead to a high false-positive rate, and they have non-standardised durations of prediction.²¹ A meta-analysis found that three commonly used suicide risk assessment scales (the Beck Hopelessness Scale, the Suicide Intent Scale, and the Scale for Suicide Ideation) have insufficient evidence to support their use to predict suicide following self-harm.²¹ The NICE guidelines do not recommend these risk-assessment tools for predictive purposes²² and regarded them as complementary to facilitate defensible decision.¹⁰

Nevertheless, it is reassuring from the service utilisation data that at-risk patients received more interventions from ICCMWs and PCPs. Case managers from either organisation were not required to inform their counterpart about the findings of their assessments. Therefore, the increased community interventions to at-risk patients likely resulted from the clinical judgement of each case manager, independent of their knowledge on the risk assessment tools used by their counterpart. Informal communication platforms between case managers of either organisation on the patients’ risk profile may facilitate matching between level of risk and intensity of care.

We found no consistent correlation between both scores of the Brief Psychiatric Rating Scale and the Health of the Nation Outcome Scales and identification of at-risk patients in either ICCMW or PCP. It is possible that most patients were residing in the community and were free from most symptoms. Moreover, violence and suicide behaviours were associated with social factors such as major life events and interpersonal conflicts, and these factors might not be adequately captured by these rating scales. Although past history of violence and suicide is a predictor of future suicide and violence, the interaction of three domains (current clinical condition, current psychosocial environment, and the past history of these behaviours) is more determining on the risk of occurrence. Therefore, assessment tools that focus only one or two domains is unlikely to be informative in risk prediction.

The observed discrepancy in assessment of risk level between ICCMWs and CPS has highlighted the importance of using a standardised assessment framework by all service providers along the entire recovery journey of patients, and sharing of all clinical and social information among

all service providers. This finding supports the direction stated in the Service Framework of Personalised Care for Adults with Severe Mental Illness in Hong Kong.²³ The service framework listed five essential components of effective delivery of personalised care; one of these is using a standardised assessment framework of needs, strengths, and risk. To put this service framework into action, in 2017, the Need-Strength-Risk Assessment for Adults with Severe Mental Illness was developed; it is a tool to facilitate a structured clinical judgement by considering a wide array of factors unique to the person. This assessment should be completed when a patient is cross-referred between ICCMWs and CPS. Our findings support the use of this assessment and enhancement of information flow between ICCMWs and CPS, as limited access to patient information is a major barrier to accurate risk stratification. The next step is to evaluate the agreement between ICCMWs and CPS on assessment of needs, strengths, and risks, rather than merely risk stratification. Apart from assessing patients in terms of a scoring profile of numerous rating scales, we put more emphasis on listening and responding to the patients’ unique experience. Our findings suggest that it is unlikely to be fruitful to use any cut-off score to determine which patient is in pain or has recovered.

The present study has several limitations. Both ICCMWs and PCPs were new projects to introduce a multi- disciplinary recovery-oriented case management model. Although we tried to capture all data collected from these projects, these projects were not designed to answer any specific research hypothesis. To our knowledge, this is the first study in Hong Kong that NGOs and the Hospital Authority have contributed their in-house administrative data for in-depth analysis of the quantity and quality of services jointly delivered to psychiatric patients. Therefore, there were no precedent studies to benchmark the standard of our services or validity of different risk assessment tools used. Our samples were recruited from two districts only; caution is required to generalise our findings to other regions. Nevertheless, our findings are a pioneering attempt for future investigations on collaboration in community mental health services.

Conclusion

In Hong Kong, community mental health services follow a case management model that involves mainly the government-organised PCP and NGO-run ICCMWs. Identified at-risk patients are provided with higher intensity care by these two sectors together. There is room for further improvement in the concordance of risk stratification between the two sectors. Assessment rating scales do not appear to be able to provide any optimal cut-off score for risk stratification. Therefore, instead of relying on these scores, a structured clinical judgement based on comprehensive and accurate information of past clinical and psychosocial background of protective and risk factors should be used for risk stratification in community mental health care.

Therefore, we recommend better free flow of information and closer collaboration between ICCMWs and PCPs.

Acknowledgements

We thank Prof Fanny Cheung, chairperson of New Life Psychiatric Rehabilitation Association, for advice. We also thank the case managers of the personalised care programmes and integrated community centres of mental wellness in North Lantau and Mongkok, Hong Kong, and all members of the Community Service Subcommittee of the New Life Psychiatric Rehabilitation Association, Hong Kong.

Declaration

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

1. Lamb HR, Weinberger LE. The shift of psychiatric inpatient care from hospitals to jails and prisons. J Am Acad Psychiatry Law 2005;33:529-34.
2. Montgomery P, Kirkpatrick H. Understanding those who seek frequent psychiatric hospitalizations. Arch Psychiatr Nurs 2002;16:16- 24.
3. Marshall M, Gray A, Lockwood A, Green R. Case management for people with severe mental disorders. Cochrane Database Syst Rev 2000;2:CD000050.
4. Marshall M, Lockwood A, Gath D. Social services case-management for long-term mental disorders: a randomised controlled trial. Lancet 1995;345:409-12.
5. Rosen A, Teesson M. Does case management work? The evidence and the abuse of evidence-based medicine. Aust N Z J Psychiatry 2001;35:731-46.
6. Pang AH, Yip KC, Cheung HK, Yeung OC. Community psychiatry in Hong Kong. Int J Soc Psychiatry 1997;43:213-6.
7. Chan S, MacKenzie A, Ng DT, Leung JK. An evaluation of the implementation of case management in the community psychiatric nursing service. J Adv Nurs 2000;31:144-56.
8. Lee CC, Liem SK, Leung J, Young V, Wu K, Wong KKK, et al. From deinstitutionalization to recovery-oriented assertive community treatment in Hong Kong: What we have achieved. Psychiatry Res 2015;228:243-50.
9. Lee CC, Liem SK, Leung JS, Wong K, Yuen SK, Lee WL, et al. Assertive community treatment for psychiatric patients with frequent hospitalization. Hong Kong Med J 2015;21(Suppl 2):37-40.
10. Lee CC, Ip G, Chu M, Lo TL, Ip YC. From psychiatric rehabilitation to recovery-focused practice in Kwai Chung Hospital, a mental hospital in Hong Kong. Asia Pac J Soc Work Dev 2014;24:17-28.
11. Tsoi WS, Tse S, Fukui S, Jones S. Study protocol for a controlled trial of Strengths Model Case Management in mental health services in Hong Kong. BMJ Open 2015;5:e008303.
12. Wong DFK, Yeung MPM, Ching CK. Evaluating a case management model for people with severe mental illness in Hong Kong: a preliminary study. Hong Kong J Psychiatry 2009;19:11-7.
13. Hospital Authority. Operation Manual on the Special Care System for Psychiatric Patients. Hong Kong: Hospital Authority Head Office PFU Revamp Task Group; 2010: 7-23.
14. Watts D, Bindman J, Slade M, Holloway F, Rosen A, Thornicroft G. Clinical assessment of risk decision support (CARDS): the development and evaluation of a feasible violence risk assessment for routine psychiatric practice. J Ment Health 2004;13:569-81.
15. Miller IW, Norman WH, Bishop SB, Dow MG. The Modified Scale for Suicidal Ideation: reliability and validity. J Consult Clin Psychol 1986;54:724-5.
16. Brennan W. Safer lone working: assessing the risk to health professionals. Br J Nurs 2010;19:1428-30.
17. Overall JE, Gorham DR. The Brief Psychiatric Rating Scale. Psychol Rep 1962;10:799-812.
18. Wing J, Curtis RH, Beevor A. Health of the Nation Outcome Scales (HoNOS). Glossary for HoNOS score sheet. Br J Psychiatry 1999;174:432-4.
19. Venables NC, Hall JR, Patrick CJ. Differentiating psychopathy from antisocial personality disorder: a triarchic model perspective. Psychol Med 2014;44:1005-13.
20. Pyott J. Use of HCR–20 in routine psychiatric practice. Psychiatr Bull 2005;29:352.
21. Chan MK, Bhatti H, Meader N, Stockton S, Evans J, O’Connor RC, et al. Predicting suicide following self-harm: systematic review of risk factors and risk scales. Br J Psychiatry 2016;209:277-83.
22. National Institute for Health and Clinical Excellence. Longer-Term Care and Treatment of Self-Harm. London: NICE; 2012.
23. Hospital Authority and Social Welfare Department. Service Framework of Personalised Care for Adults with Severe Mental Illness in Hong Kong. Hong Kong: 2016: 8-41.