Training frontline workforce on psychosis management: a prospective study of training effects
© Sørlie et al. 2015
Received: 11 March 2015
Accepted: 31 August 2015
Published: 19 November 2015
The care situation for persons experiencing severe mental illness is often complex and demands good coordination, communication, and interpersonal relationships among those involved from the primary and specialized mental health care systems. For 15 years, professional care providers from different service levels within the same geographical areas in Norway have been trained together in a 2-year local onsite training program with the aim of increasing skills, joint understanding, and collaboration in their work with individuals experiencing severe mental illness.
The key aspects of competence addressed by the training program were measured at baseline, after 1 year, and at the end of the training period. Professional education and experience were also rated at baseline. Data were collected between 1999 and 2005 and were analyzed by estimating a linear mixed model.
Results showed a significant increase in participants’ experienced competence in all training goals, especially for the understanding of psychosis and relationship building. There was no significant variance at the program level, indicating consistent implementation of local programs.
This prospective study indicates that the training program was successful in increasing perceived competence in the areas addressed, and training staff from different service levels together probably contributed to more collaboration. This training model still operates in Norway.
KeywordsEvaluation Multidisciplinary cooperation Psychosis management Staff training
In the current era of deinstitutionalization, people experiencing long-term severe mental illness are increasingly relegated to living at home where they often have to cope with various socioeconomic challenges. Complex care situations demand good coordination, communication, and interpersonal relationships among providers from primary and specialized mental care systems  and represent one of the greatest challenges to solving the dilemma of obtaining a sufficient mental health workforce . The World Health Organization is urging countries to direct resources toward the development of robust primary health care services to improve access to mental health treatment . There has also been an emphasis on the importance of functional treatment networks that include both primary and specialized care. This implies a need for collaborative training programs for mental health professionals who provide interdisciplinary services across agencies that offer adequate treatment, rehabilitation, and support for citizens living with severe mental health challenges [1, 4]. One of the ideas underlying the training program described here is that collaboration among participants may be developed, learned, and improved through common training contexts. Another central aspect of the learning philosophy is that service users and family members are involved in program development and in training sessions. This ensures that learning is more easily transferred to real life situations. Thus, this training model focuses on the needs of service users, health care providers, and the health services.
Studies of the salient components of effective mental health skills training programs highlight the importance of the following factors: the use of small groups of trainees led by expert facilitators or supervisors who meet regularly over extended periods of time to discuss cases, the use of interactive learning, and the inclusion of performance feedback . Other emphasizes the importance of a facilitative learning environment  and the sharing of emotions arising from clinical experiences . The latter may have relevance for both the learning process and the wellbeing of professionals . Training programs that systematically focus on students’ own working experiences may increase motivation because they feature relevant personal experiences and provide an opportunity for students to master their needs. The training program described here included long-term small group clinical supervision with an emphasis on experiential material.
The training program
The Center for Psychotherapy and Psychosocial Rehabilitation of Psychoses (SEPREP) was founded in 1990 to improve treatment and care for people who are experiencing severe mental health problems. SEPREP’s main activity is training professionals.
The goals of the training program, which still operates, are as follows: (1) better understanding of psychosis, psychosocial rehabilitation, and treatment; (2) improved skills in building and maintaining relationships with persons in need of services; (3) improved awareness of interactions with those persons to better understand their experiences and needs; (4) improved participation in multidisciplinary collaboration; (5) improved teamwork and collaboration across service levels; (6) increased support of carers; and (7) improved ability to take care of oneself while performing clinical work.
This 2-year training program consists of lectures 1 day a month (80 h), supervision in small multidisciplinary groups every 2 weeks (80 h), discussion and reflective work on theory in small groups once a month (40 h), and writing two clinical case stories integrating theory and literature (group task). The lecturers are clinicians, researchers, and users are involved in about 20 % of the lectures. The supervisors are clinicians with extensive experience in the treatment of people with severe mental illness and in the provision of clinical supervision. Participation in the training is part-time while the students continue in their clinical job, facilitating interactions between training and practice. Each supervision group consists of health workers from both community mental health centers (CMHCs) and primary care providers. By training the workforce together, the program offers a combination of individual training and system-level intervention. The programs are planned and implemented locally in cooperation between representatives of a CMHC, collaborating municipalities, users and SEPREP.
This article will report on the effects of the training program in accordance with its learning goals.
Is there an increase in experienced individual competence in relation to the training goals of the program?
Are patterns of change in competence different across training goals? How do service level and previous clinical experience influence the patterns of change in experienced individual competence during the training program?
Are changes in experienced individual competence different across local training programs, and if so, what factors may explain the differences?
Design and procedure
Prospective longitudinal questionnaire based cohort study. Changes in participants’ experienced competences related to the individual program goals were assessed at the start of the program, after 1 year, and at the end of the 2-year program. Each participant was asked to include on their anonymous questionnaire a personal id-number consisting of the last two digits of their mother’s birth year and the first two letters of their mother’s first name. This enabled questionnaires from the same persons to be kept anonymous and still be linked across the three data collection times, as this information was expected to be reliably recreated each time. Data collection took place between 1999 and 2005. Since the evaluation form was changed in 2005, later evaluation data could not be included in the present study.
The study was not based on individual sensitive material. Thus, approval from the Regional Ethics Committee was not necessary.
Data consisted of 1846 questionnaires from 56 local programs at the start of the training, 1492 from 55 programs after 1 year, and 1258 from 53 programs at the end of the 2-year training. Based on the data from 55 local programs, the average response rate compared with the number who started the training was 93 % at the start of the program, 83 % after 1 year, and 72 % after 2 years. The completion of questionnaires during program sessions contributed to the high response rates. It is likely that the response rate decreased over time mainly because some participants left the programs.
46 % of the participants were from primary care and 54 % from CMHCs. 20 % had university educations (6 years), 67 % had college educations (3 years), and 4 % had 1-year professional educations. 20 % had little clinical experience with severely mentally ill people, 33 % had some, 25 % had substantial experience, and 14 % had a moderate amount of clinical experience. 10 % of the sample was aged 20–29 years old, 35 % was aged 30–39, 22 % was aged 40–49, and 33 % was 50 or older.
Questionnaire and variables
Content of subscales based on factor analysis of the questionnaire
Subscale (with Cronbach’s alpha/year) and itemsa in subscale
N = 842
N = 470
N = 207
(1) Understanding psychoses and treatment (alpha: 0.90; 0.80; 0.86)
3. Understand much of the experience of having a psychosis
4. Know a good deal about how different causes may contribute to psychosis
5. Know a good deal about how different severe mental disorders appear
6. Understand psychosis from a psychological development perspective
7. Know how medical and psychosocial treatments should be used for psychosis
8. Know what tasks I have in the treatment of patients with severe mental illness
9. Have adequate knowledge and training for treatments I may contribute to
10. Have a clear picture of psychotherapy for patients with a psychosis
(2) Building and maintaining relationships with patients (alpha: 0.83; 0.78; 0.75)
16. Uncertain how to establish a confident relationship with a psychotic patient
17. Feel confident in how to keep and develop my contact with a psychotic patient
18. Fully understand how to relate to a psychotic patient
(3) Using own feelings and reactions to understand patients (alpha: 0.66; 0.49; 0.67)
12. Often aware of my own feelings while observing and interacting with patients
13. My reactions to patients often help me to understand how to relate to them
15. Emotional reactions of different staff can help my understanding of the patient
(4) Participating in multidisciplinary collaboration (alpha: 0.66; 0.66; 0.65)
21. Experience myself as able to contribute greatly in multidisciplinary collaboration
22. Have many thoughts that I do not express in multidisciplinary collaboration
23. Unclear picture of my role or tasks in multidisciplinary collaboration
24. Have good experiences of multidisciplinary collaboration for persons with psychosis
(5) Teamwork and collaboration (alpha: 0.64; 0.45; 0.65)
25. Fully aware of who my local coworkers are in relation to patients with psychosis
26. In this area, we usually do not collaborate across services regarding psychoses
27. Psychotic patients have several needs for which there is no local competence
28. Easy to ask other collaborators for advice regarding patients with psychosis
29. Many collaborators do not know how my team works with psychotic patients
(6) Supporting family/relatives of patients (alpha: 0.58; 0.77; 0.76)
30. Where I work, it is not typical to collaborate with relatives of psychotic patients
32. Where I work, relatives of patients in the acute phase get little information
33. Where I work, we try to give relatives of psychotic patients support and care
34. We usually inform relatives about local organizations for relatives
(7) Taking care of myself (alpha: 0.57; 0.63; 0.59)
35. Feel content in my work
37. Do not have opportunities to share my feelings regarding patients with my colleagues
38. Make sure to take breaks when I am tired
39. Make sure to get supervision regarding patients I have problems relating to
Multilevel data analysis was performed by estimating a linear mixed effects model  for each subscale (SAS PROC MIXED procedure). The normality assumption was assessed by examining the histograms. First, an unconditional growth model was estimated to assess a time trend in the dependent variable. A second-order time component was included in the model where significant. Next, bivariate analyses with person- and program-level covariates as fixed effects were performed. The interactions between person-level covariates (sex, age, working place, education, clinical experience with the severely mentally ill) and time were tested and left in the model if significant. Finally, the multivariate linear mixed model with time components and person- and program-level covariates (size of program, proportion from primary care, dropout rate) was estimated. The models containing both fixed and random effects were estimated using unstructured covariance. If significant, random effects for both intercepts and time on person- and program-level were included. Subscale 7 (taking care of oneself) was excluded from the multilevel analysis because it was not considered a treatment competence and because there were no significant differences across the three points in time. To avoid hypothesis fishing, data splitting was employed . The data were randomly split into two parts by local programs, with all data from one local program used in the same part. The first data set with 30 % of cases was used to identify a model. The second data set was used to test the model. Only those relationships that were significant in both parts of the data were considered significant when fitting the model to the entire data set. The results are presented as regression coefficients estimated on the entire data set, each with specified level of significance. Model assumptions were tested by standard methods. Effect sizes are reported as regression coefficients (R2). The analyses were performed using SAS version 9.2 (SAS Institute Inc., Cary, NC, USA).
Results of linear mixed models for changes in experienced competence and predictors (regression coefficients: R2)
Using own reactions
Teamwork and collaboration
Years in training
Years in training × years in training
Service level (primary/specialized)
Length of professional education
Clinical experience with SMI
Years in training × service level
Years in training × education
Years in training × clinical experience
Local program variables
Number of students
Portion from primary care
Proportion of total variance explained by individual factors
No random slopes
No random slopes
A significant increase in experienced individual competence was observed for all training goals; this increase was greatest for understanding of psychosis and relationship building and smaller for awareness of own reactions, multidisciplinary work, teamwork, and family support. As expected, the change was greatest in those with less experienced competence at baseline, particularly among those from primary care, leading to less variation in experienced competence after the training.
One of the ideas underlying this training program is that collaboration among participants may be developed, learned, and improved through common training contexts. The greater improvement in experienced competence for the two first subscales (understanding psychoses and relationship building) during the first year of training among may partly relate to networking effects both within the training program and in the participants’ daily working environment. The greater increase in these two subscales among those with less experienced competence at baseline may also indicate that that it was most new to learn in the first year. We have not been able to find other studies reporting similar findings. Competence in this field is usually considered to be strongly influenced by the quality of professional networks, both in terms of the accessibility of competent coworkers and their support and sharing when caring for individuals experiencing long-term severe mental illness. In line with these considerations, one can also assume that the supervision in small multidisciplinary groups every 2 weeks may have had particular significance for the observed skills development. The significance of clinical supervision on patient and educational outcomes is well documented .
The multilevel regression analysis did not show any significant differences in experienced competence across the local programs, and the program-level variables (size of program, proportion from primary care, dropout rate) explained only an insignificant portion of the total variance. The lack of significant variation across local training programs may indicate that they were implemented in a consistent way. The main reasons for this are probably the well-developed manual for organizing and running the local programs and the close collaboration with, and monitoring of, all local programs by regional coordinators from the SEPREP national training program. There is also a joint pool of lecturers and supervisors who are engaged across local programs.
The significant increase in experienced competence revealed in our study is in agreement with external evaluations commissioned by the national health authorities [12, 13]. They found that participants and leaders of services were satisfied with the program and they reported increased clinical competence, improved understanding, and mutual respect between professional groups and employees of different service levels.
This national training program was supported by the Norwegian Action Plan for Mental Health 1999–2008, which shared some of the same goals as a proposed national action plan for workforce development in the United States . The program has a strong emphasis on user and caretaker perspectives, clinical supervision, building of relationships, and improvements in collaboration across services. Reviews of these types of training programs have resulted in several recommended training approaches and best practices , including the involvement of users and caretakers in program design  and as teachers . In view of these recommendations, the present program could probably put more emphasis on evidence-based practices with regard to user and family member involvement and collaborative work.
This was a prospective study. It was unlikely that the participants were able to remember their initial questionnaire responses across the 1-year interval between measurements. The response rate was high and the results may be considered representative for the whole sample of participants. The last assessment took place at the very end of the 2-year training program. Although participants continued in their usual work during the training, follow-up assessments might have captured participants’ more stable daily work evaluations of their own competencies with no influences from the group processes and mutual sharing experiences in the training program.
This study focused only on changes in participants’ experienced competence. We would have liked to measure how the local training programs influenced clinical practice and collaboration, especially regarding the extent to which patients experienced improvement in services. The issues of clinical practice and collaboration are partly explored in the two mentioned evaluation reports [12, 13]. Resources were not available to measure the patients’ experiences. Another possible limitation of the study is the fact that the data used in the study was collected between 1999 and 2005. However, since the group of participants and the structure, content and pedagogic approach of the program still is very much alike, the representativeness of our results for the current courses is considered large.
Quality of care for the seriously mentally ill is also related to continuity of care. The study did not allow us to evaluate whether participation in the training influenced employment stability. Most likely there is such a correlation. WHO’s policy of improving retention of rural health-care workers recommends that governments ‘design continuing education and professional development programs that meet the needs of rural health workers and that are accessible from where they live and work, so as to support their retention .
The training program has pioneered a model that we have not found elsewhere: training mental health frontline workers from collaborative services together and with systematic involvement of service users and family members to achieve changes on the individual, professional, and systemic level. Our prospective measurement of experienced competence indicates that several of the goals of the training program were reached. Future evaluations should aim to include measurements of fidelity to the training model and possibly measure the impact of training on health workers’ practices, their employment stability, and service users’ experiences.
TS and TR conceived the paper and developed a first draft. TS, TR, MB, KBF, and GBH contributed significantly to the development and implementation of the training program and the data collection. JSB conducted the statistical analyses. All authors read and approved the final manuscript.
This national training program was developed by the SEPREP Foundation and economically supported by the Norwegian Action Plan for Mental Health 1999–2008.
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Committee on Crossing the Quality Chasm. Improving the quality of health care for mental and substance-use conditions. Washington DC: Institute of Medicine, National Academies Press; 2006.Google Scholar
- Faulkner LR, Scully JH Jr, Shore JH. A strategic approach to the psychiatric workforce dilemma. Psychiatr Serv. 1998;49:493–7.View ArticlePubMedGoogle Scholar
- WHO/Europe. What are the advantages and disadvantages of restructuring a health care system to be more focused on primary care services? 2004. http://www.euro.who.int/HEN/NewsArchive.
- The Norwegian Directorate on Health and Social Affairs. People with severe mental illness and needs to facilitate specific deals (in Norwegian). Rapport. IS 1554; 2007.Google Scholar
- Murrihy RC, Byrne MK, Gonsalvez CJ. Testing an empirically derived mental health training model featuring small groups, distributed practice and patient discussion. Med Educ. 2009;43:140–5.View ArticlePubMedGoogle Scholar
- McNeil K, Mitchell R, Parker V. The paradoxical effects of workforce shortages on rural interprofessional practice. Scand J Caring Sci. 2015;29:73–82.View ArticlePubMedGoogle Scholar
- McKensey A, Sullivan L. Balint groups—helping trainee psychiatrist make even better use of themselves. Aus Psych. 2015;. doi:https://doi.org/10.1177/1039856215598870.Google Scholar
- Høifødt TS, Talseth AG, Olstad R. A qualitative study of the learning process in young physicians treating suicidal patients: from insecurity to personal pattern knowledge and self-confidence. BMC Med Educ. 2007;6:7–21.Google Scholar
- Fitzmaurice GM, Laird NM, Ware JH. Applied longitudinal analysis. Hoboken: Wiley; 2004.Google Scholar
- Dahl FA, Grothe M, Šaltytė Benth J, Natvig B. Data splitting as countermeasure against hypothesis fishing: with a case study of predictors for low back pain. Eur J Epidem. 2008;23:237–42.View ArticleGoogle Scholar
- Farnan JN, Petty LA, Georgitis E, Matin S, Chiu E, Prochaska M, Arora VM. A systematic review: the effect of clinical supervision on patient and residency education outcomes. Acad Med. 2012;87:428–42.View ArticlePubMedGoogle Scholar
- Management Rambøll. Evaluation of SEPREP multidisciplinary training program in relation to The Escalation Plan for Mental Health (1999–2008) (In Norwegian). Oslo: The Norwegian Directorate on Health and Social Affairs; 2005.Google Scholar
- Ludvigsen K, Helgesen MK. Increase of competence in mental health work. Educations in university colleges and SEPREP multidisciplinary training program (In Norwegian). Oslo: Norwegian Institute for Urban and Regional Research (NIBR); 2004.Google Scholar
- Hoge MA, Morris JA, Stuart GW, Huey LY, Bergeson S, Flaherty MT, et al. A national action plan for workforce development in behavioral health. Psychiatr Serv. 2009;60:883–7.View ArticlePubMedGoogle Scholar
- Hoge MA, Huey LY, O’Connell MJ. Best practices in behavioral health workforce education and training. Adm Policy Ment Health. 2004;32:91–106.View ArticlePubMedGoogle Scholar
- Moran MM, Coyle J, Pope R, Boxall D, Nancarrow SA, Yong J. Supervision, support and mentoring interventions for health practitioners in rural and remote contexts: an integrative and thematic synthesis of the literature to identify mechanisms for successful outcomes. Hum Resource Health. 2014. http://www.human-resources-health.com/content/12/1/10.
- Repper J, Breeze J. User and carer involvement in the training and education of health professionals: review of the literature. Int J Nurs Stud. 2007;44:511–9.View ArticlePubMedGoogle Scholar
- World Health Organization. Increasing access to health workers in remote and rural areas through increased retention. Geneva. 2010.Google Scholar