- Open Access
Satisfaction with primary care and mental health care among individuals with severe mental illness in a rural area: a seven-year follow-up study of a clinical cohort
International Journal of Mental Health Systemsvolume 10, Article number: 33 (2016)
The Erratum to this article has been published in International Journal of Mental Health Systems 2016 10:40
Most studies of services for people with severe mental illness have been performed in cities. Our 7-year follow-up study aimed to investigate clinical course and satisfaction with services among individuals with severe mental illness who received community mental health services in a rural area. The services were provided by primary care and a community mental health center (CMHC), which worked in close collaboration and emphasized individually tailored case management, relationship-building and continuity of care.
All 57 patients with severe mental illness who were seen by the CMHC in 1992–1993 and were still alive in 1999 were asked to participate. Retrospective ratings were performed for the first month of contact in 1992–1993 based on patient records and detailed notes. A semi-structured interview was conducted in 1999–2000 with the 40 patients (70.2 %) who gave written consent to participate in the study. DSM-IV diagnoses were made using OPCRIT. The retrospective baseline ratings and the follow-up interview included assessments of symptoms and functioning using the following instruments: the Brief Psychiatric Rating Scale Expanded version 4 (BPRS-E), the Health of the Nation Outcome Scales (HoNOS), the Global Assessment of Functioning Scale (split version), and the Practical and Social Functioning Scale (PSF).
The ratings revealed improvements in psychiatric problems and functioning. Patients with schizophrenia spectrum disorders improved primarily in psychotic symptoms, while patients with severe affective disorders improved primarily in affective symptoms. Large variations in the use of primary care and mental health services were observed, with more intensive specialized mental health services for individuals with schizophrenia spectrum disorders than severe affective disorders. Overall, the patients were satisfied with the provided services. They were most satisfied with GPs and more satisfied with local outpatient and inpatient services than with hospital inpatient services and medication.
Patients with severe mental illness in a rural area value local services that emphasize relationships and close collaborations among the CMHC, GPs and primary health and social care. Even in an area with a fairly well-staffed CMHC, the highest patient satisfaction was reported for GPs, indicating the potential key role of GPs for this patient group.
This paper reports patient satisfaction and clinical course from a 7-year follow-up study of a cohort of service users with severe mental illness (SMI) who received collaborating services from primary care (including general practitioners, GPs) and a community mental health center (CMHC) in a rural area on the west coast of Norway. As large-scale development of community mental health care is still in an early phase in many countries [1, 2], experiences related to the close collaboration of primary care services and a pioneering CMHC in Norway 1992–2000 may be of interest for others who are currently developing local mental health services in rural areas.
Most studies of community mental health services for individuals with SMI have been performed in cities. Studies of shared care by collaboration of primary care including GPs and specialized mental health care have been performed primarily for patients with depression, and few studies are available concerning the experience of combined treatment from primary care and CMHC for individuals with SMI [3, 4]. This type of combined treatment may be the most common type of services for this target group, especially in rural areas.
According to patients and health professionals, GPs are important for patients with SMI . The engagement of GPs in the treatment of individuals with SMI depends on their interest in mental health and other factors like the support available from collaborating mental health services . Good communication between services is important for the quality of the services . In addition, mental health nurses, primary care mental health teams and social services are important collaborators for the GP and the CMHC.
In Norway, the development of CMHCs with outpatient clinics and inpatient wards was one of the main strategies in a national plan for mental health 1999–2008 . Much of the country consists of rural areas, and during the ten-year plan, resources were increased for CMHCs and for primary care in the municipalities. However, a report on all of the CMHCs in the country in 2013 revealed that there is still great variation in the available resources and the degree of implementation of the different types of community mental health services . In rural areas the number of psychiatrists is one of the critical factors for capacity and quality of outpatient and local inpatient treatment at CMHCs for persons with SMI. In two recent national surveys of GPs’ assessment of the local CMHC showed that many CMHCs still have great potential for the improvement of their services [10, 11].
Countries and areas that are developing community mental health services based on collaboration by primary care and CMHCs may take into account the experiences from similar processes during the last three decades in Norway. Studies describing the content of shared care and patient satisfaction with this care over time may be especially valuable. However, recent Norwegian studies have been primarily cross-sectional studies describing the types of services provided, but not following the patients over time and reporting course and outcome. One 6-year study followed a group of patients who were receiving long-term inpatient care at the time of the baseline evaluation [12, 13], but this study reported limited information concerning the content of the treatment provided in the communities after discharge from institutions, and it had no measures of patient satisfaction.
The local area and the collaborating services
Nordfjord is a rural area surrounding a fjord on the west coast of Norway. There are six small municipalities in the area, with a total population of 30,000. An epidemiological study found a lower prevalence of affective disorders but a similar prevalence of non-affective psychoses in this rural county in comparison to the Norwegian capital Oslo . The CMHC is located in the middle of the area, with a 1 hour drive to most of the municipality centers. At the time of the follow-up study, 20 GPs were in the area and each municipality provided primary care by mental health workers, social services and supported housing for persons with SMI. In the small municipalities, the GPs and other primary care workers had a good knowledge of the population and the local community. High stability and low turnover among GPs and health workers contributed to high continuity of care and long-term personal contact with patients. Most GPs were highly engaged in serving patients with SMI and other mental disorders, and the GPs expressed that the close collaboration with the CMHC encouraged them to take more active responsibility for these patients.
Nordfjord CMHC was one of the first CMHCs in Norway to provide the full range of outpatient, day patient, mobile and inpatient services, as intended in national plans. The CMHC became fully operational in 1992 and has subsequently provided treatment in close collaboration with GPs and other municipal primary health and social services in the catchment area. At the time of the follow-up study, the CMHC had an outpatient clinic with seven clinicians, a mobile rehabilitation team with a staff of four individuals, a day unit with a staff of three individuals, and two nine-bed inpatient wards with a staff-patient ratio 1.5:1.0. One of these wards was for patients with SMI, and the other unit was for all other patient groups.
Four psychiatrists were available at the CMHC. The staffing was considered to be fairly good for a small CMHC, and the CMHC has been rated highest in quality in two national surveys of GPs’ ratings of the local CMHC [10, 11]. This indicates that the quality of local mental health care and collaboration in our area was considered to be fairly good.
Most of the needs for psychiatric inpatient services and all of the needs for psychiatric outpatient services in the local area were covered by the CMHC. The CMHC also collaborated closely with the mental health clinic at the county central hospital 2–3 h from Nordfjord, which had acute and closed inpatient psychiatric wards. On average, four inpatients from Nordfjord were present in these wards. Involuntary admissions could only be processed at the mental health clinic at the county central hospital, but involuntary treatment could be transferred to the CMHC and continued there.
A major component of the care available from the CMHC for people with SMI was clinical case management by the staff of the mobile rehabilitation team, as well as of outpatient and inpatients units. Building and maintaining relationships and alliances with the patients and continuity of care were emphasized. Primary care or CMHC staff met SMI patients weekly or more often in their homes, in the community or in localities of the services. Important treatment components included help and support in everyday living, meaningful activities, training in practical and social skills, medication, supportive psychotherapy and meeting the family . During the years of the follow-up study, clinical guidelines and evidence-based treatments were emerging and began to influence clinical practice, but specific models and fidelity measures were still scarce and not implemented in the services.
The shared care provided by the primary care and mental health care services was implemented partly as joint service delivery and partly by close coordination of the services provided by each agency. The psychiatrists and other clinicians from the CMHC spent 1 day every week working in the municipalities with the GPs and primary care. This service included joint consultations, home visits, family sessions and supervision. The close collaboration led to good working relationships based on mutual knowledge and respect and increased the overall competence of the shared care. The coordination of the total services for each patient was accomplished in meetings every 6–8 weeks for all professionals involved with the patient, with one case manager from the primary care or the CMHC as coordinator. The CMHC also had close contact with the psychiatric inpatient department at the central hospital, facilitating early discharge to the community or transfer to the CMHC inpatient ward.
In 1996–1998, more than thirty health professionals from the CMHC and municipality primary care services participated in a comprehensive 2-year local training program in community mental health care for people with SMI. This program was arranged in collaboration with the Center for Psychotherapy and Psychosocial Rehabilitation of Psychoses (SEPREP), which is a national network of clinicians, service users and caregivers. The participants received supervision every second week in joint supervision groups for health workers from both the CMHC and primary care, attended 1 day of lectures every month, and met in groups to discuss clinical literature every month . The training was part-time and completed in parallel to clinical work and aimed to increase the competency of clinical practice during the training. From 1999, SEPREP was commissioned by the government to build a national program with local training programs based on this model. This national program has been widely disseminated and is still running with strong impacts throughout Norway.
The aims of this paper are to: (1) Describe the clinical course of patients with SMI during the 7-year follow-up period and (2) Report patient satisfaction with service components and with the collaboration of services.
This study is a 7-year follow-up study of a 2-year clinical cohort of patients with severe mental illness. The baseline assessment was performed retrospectively but before the follow-up interview.
The 2-year clinical cohort was defined as the individuals with SMI (schizophrenia spectrum disorders and severe affective disorders) who had been outpatients and/or inpatients at the CMHC in 1992–1993. These years were the first 2 years that the CMHC was in full operation, and we identified 64 patients who fulfilled the criteria. All of the 57 patients who were still alive in 1999 were contacted by letter and invited to take part in the follow-up study, and 40 patients (70.2 %) gave written consent. No significant differences in gender, age or retrospective baseline ratings of the severity of psychiatric problems were observed between the 40 patients who gave consent and the 17 patients who did not give consent.
Instruments and variables
DSM-IV diagnoses were made using Operational Checklist for Psychotic Symptoms, version 3.4 (OPCRIT) with algorithms based on 90 criteria from the WHO Schedule for Clinical Assessment in Neuropsychiatry (WHO-SCAN) . We used all available information to answer the 90 diagnostic criteria for psychotic and affective symptoms, and used the OPCRIT software to get a lifetime diagnosis. The type and severity of psychiatric symptoms were rated using the Brief Psychiatric Rating Scale, Expanded Version 4.0 (BPRS-E) . The type and severity of recent problems in major problem areas in relation to SMI was rated using the Health of the Nation Outcome Scales (HoNOS) developed to cover these areas listed in Table 1 . Functioning was measured using the Global Assessment of Functioning Scale (GAF) (split version, with separate scoring of symptoms and function)  and the Rating Scale for Practical and Social Functioning (PSF) .
A questionnaire measuring patient satisfaction with each of the local service components was designed for the study as such specific components were not covered in established instruments for measuring patient satisfaction with services. We also included questions concerning the amount of each service component received by each patient.
The three psychiatrists doing the follow-up interviews discussed the rating scales thoroughly when preparing for the study. Each psychiatrist did their part of the 40 interviews, and for 18 of the interviews it was possible for one of the other psychiatrist to be present as a listener and do independent ratings on BPRS-E, GAF and PSF so that we could calculate inter-rater reliability (ICC) for the ratings on these instruments . The ICC values were 0.85–0.94 for the four subscales of the BPRS-E, 0.92 and 0.91 for the GAF scales on symptoms and functioning, respectively, and 0.73–0.94 for the ten PSF subscales.
Data collection and data analysis
The patients were retrospectively rated for psychopathology and level of functioning for the first month (the index month) of contact in 1992–1993, using all available information in written records and detailed notes that were made by the staff who worked with the patient. These ratings were made before the patients were contacted for a follow-up interview. The three mental health nurses in the project group conducted the retrospective baseline ratings based on patient records and detailed notes from any inpatient stays at that time, with support from the three psychiatrist in rating BPRS-E and HoNOS. The three psychiatrists conducted some months later the ratings in the follow-up interview without having access to the retrospective baseline ratings done earlier. None of the raters were blind to the aims of the study to learn about the clinical course and experiences of the patients.
In the follow-up interview in 1999–2000, which was 7 years after the index month, the patients were assessed using the same rating scales for psychopathology and the level of functioning that were used in the retrospective baseline ratings. They also answered the questions on patient satisfaction with service components. Going through the list of the service components listed in Table 2, the patient was first asked if he had received this type of service during the 7 years. If so, three additional questions were asked, reading the answers to choose among for each question: How much did you have of this during the years (from 0 = nothing to 4 = very much or all the time/most of the time)? How important has this been to you (from 1 = not important to 4 = very important)? How satisfied have you been with this type of service (1 = very dissatisfied, 2 = quite dissatisfied, 3 = mixed, 4 = quite satisfied, 5 = very satisfied)? One patient with extreme withdrawal and one patient with severe cognitive dysfunction were not able to answer all of the questions on patient satisfaction.
Data analyses were performed using SPSS for Windows version 18 and included descriptive statistics and analyses of the significance of differences that employed the Chi square test and the Student’s t test.
The clinical course of patients
Table 1 shows the clinical characteristics of the sample at baseline and follow-up. The group with schizophrenia spectrum disorders included 17 patients with schizophrenia and 6 patients with schizoaffective disorder. The group with severe affective disorders included 11 patients with bipolar disorder and 6 patients with depression with psychotic features. All the patients had a well-established long term severe mental illness with one or more illness episodes at the index month in 1992–1993, and none had a recent onset first episode. The mean time since the start of the severe illness was 14 years (SD 14 years).
There were 10 women and 13 men with schizophrenia and 8 women and 9 men with severe affective disorders. Distribution on age groups was fairly equal in the two patient groups. Two patients with schizophrenia and seven with severe affective disorder were living with a spouse or partner at follow-up. There was an increase in the number with disability pension from 14 to 20 of those with schizophrenia and from 6 to 10 of those with affective disorder. Six with affective disorder were in paid work at follow-up compared to four at the index month, while none with schizophrenia were in paid work at follow-up compared to one at the index month.
Significant improvements of psychiatric problems and functioning were observed for several subscales, as shown in Table 1. The severity of psychiatric symptoms was reduced during the follow-up period for both groups, but these changes had different patterns. Patients with schizophrenia spectrum disorders improved significantly with respect to positive and negative symptoms on the BPRS and in the BPRS mean total score, with no significant changes in the subscales on affective symptoms. The patients with severe affective disorders showed the opposite pattern, with a significant reduction on the depression subscale but no significant changes in positive and negative symptoms, which were fairly low at baseline. The ratings on the GAF symptom subscale improved significantly for both groups. Both groups also experienced significant improvements in practical and social functioning, as measured using the PSF.
Care and satisfaction with care
Figure 1 shows the distribution of intensity of care for the total sample and for the two subsamples with schizophrenia spectrum disorders and severe affective disorders. A clear difference was observed between the two groups, with a higher intensity of care for the patients with schizophrenia spectrum disorders than for those with affective disorders. The six individuals who required inpatient care during most or all of the 7 years all had schizophrenia spectrum disorders, and the five patients who needed limited care all had affective disorders.
Table 2 shows the proportion of patients who reported contact with various service components within primary care and mental healthcare during the 7 years of follow-up. The pattern was largely the same for both groups of patients. Nearly all of the patients used medication, and most of the patients had been inpatients at the CMHC. Most patients reported contact with GPs and primary care nurses, but more variation was observed in the amount of contact with psychiatrists, social services and other types of services. Most patients also reported support from family, and half of the patients reported support from friends.
Table 2 also shows patient satisfaction with the services with which the patients had been in contact. Within each service level, the service components are ranked according to the level of satisfaction. The patients were generally satisfied with the services and were most satisfied with GPs. The patients were also satisfied with outreach from the CMHC, sheltered work, support from family, inpatient stays at the CMHC and contact with the case manager and psychiatrist at the CMHC. All of these services were on average rated 4 (satisfied) or above. The patients were somewhat less satisfied with inpatient stays at the county central hospital, family sessions, their own influence on the treatment, medication, information and group therapy.
Our 7-year follow-up study aimed to investigate clinical course and satisfaction with services among persons with severe mental illness who received community mental health services in a rural area. The services were provided by GPS and other primary care services and a CMHC. These services worked in close collaboration and emphasized individually tailored case management, relationship-building and continuity of care. The follow-up study showed improvements in psychiatric problems and functioning. Patients with schizophrenia spectrum disorders improved primarily in psychotic symptoms, while patients with severe affective disorders improved primarily in affective symptoms. Large variations in the use of primary care and mental health services were observed, with more intensive specialized mental health services for individuals with schizophrenia spectrum disorders than individuals with severe affective disorders. Overall, the patients were satisfied with the provided services. The patients were most satisfied with GPs and more satisfied with local outpatient and inpatient services than with hospital inpatient services and medication.
The 23 patients with schizophrenia spectrum disorders and 17 with severe affective disorders were 0.09 and 0.07 % of the adult population in the CMHC catchment area, respectively. An epidemiological study in our county found a life time prevalence of 0.4 % for non-affective psychoses and 0.2 % for bipolar disorder . Taking into account some uncertainty of such estimates due to low prevalence of the disorders, our cohort seems to have a fairly representative balance of the two patient groups.
The clinical course
There were improvements in both psychiatric problems and functioning during the 7 years from the index month to follow-up. In this observational study with retrospective rating for baseline we do not claim that the improvement is due to treatment effect. But part of the improvement may be due to the services given, as the results are in line with randomized controlled trials and other studies on integrated care for patients with schizophrenia . A 6-year follow-up study of case management for patients with SMI in Sweden at the same time found an improvement in self-reported psychiatric symptoms, psychosocial functioning, social networks and quality of life . But the Swedish study did not measure patient satisfaction with the services.
In our study there were large variations in the use of primary care and mental health services. Patients with schizophrenia spectrum disorders needed more intensive psychiatric outpatient services and more psychiatric inpatient services than patients with severe affective disorders, which may be expected from the severity of symptoms and functional impairment.
In our area with small municipalities and good stability and continuity of staffing for most services, the primary care and the CMHC were usually able to continue treatment and contact for as long as the user wanted and needed. The amount of contact and types of services used were thus largely based on the patient’s choice. In our treatment approach, we emphasized treatment that was individually adjusted according to changing needs through different phases of the clinical course. This makes it difficult to know whether more services are always better than less. We have found no established methods for measuring the level of individual adjustment of treatment, which could be useful in health services research.
Patient satisfaction with care
Overall, the patients were satisfied with the provided services. The patients were most satisfied with GPs and were more satisfied with local outpatient and inpatient services than with hospital inpatient services and medication. The highest satisfaction was reported for services that require personal contact with the same person over time.
We are not aware of any similar follow-up study of SMI patients measuring satisfaction with primary care including GPs and specialized mental health care in rural areas. The Swedish 6 years follow-up study on case management reported a reduction in the use of mental health services after inclusion in case management, but it did not measure patient satisfaction with the services .
The patients were more satisfied with GPs than with any other service. This finding demonstrates the importance of the GP for individuals with SMI, even in an area in which the CMHC was rated highest in quality by GPs in national surveys. The continuity of a personal relationships between the patients and the GP may be one aspect contributing to this, as this has been shown to be one of the key factors in satisfaction of SMI patients with the GP . GPs are in the local community, they know the local context and local services, and they may refer to local and specialized health service. GPs are providing both somatic and mental health care, as well as coordinating various services that may be needed.
The patients were also more pleased with inpatient stays at the CMHC than with inpatient stays at the county central hospital. This difference may be partially due to the different roles of these inpatient wards, as involuntary admissions were done at the central hospital. There are almost no studies done on inpatient treatment in CMHC, but a study of CMHC inpatient units in the UK also showed higher patient satisfaction at local inpatient units .
The patients were less satisfied with their own influence on the care and with the coordination of the services. This was an unexpected finding, as we aimed to put emphasis on patient involvement and coordination.
Strengths and limitations of the study
Our sample consisted of most of the patients from the total 2-year cohort and may be considered to be a representative sample of the patients with severe mental illness during the inclusion period. However, the sample was fairly small and gave limited possibilities for sophisticated statistical analyses. In a small rural area, it is not possible to get the large samples that are more common in studies in urban areas. Our study also has other limitations. The rating of the clinical state at baseline was done retrospectively after 7 years. Thus, our knowledge of the later clinical course and the present state of some of the patients may have influenced our ratings despite our efforts to avoid such bias. The ratings at follow-up were done by psychiatrists who had been in charge of some of the treatment during the 7 years, and not by independent researchers. This may have given some unknown bias due to wishes to see positive changes in the patients. The retrospective ratings were based on the first period of contact with the patient in 1992–1993, and their condition were probably more severe immediately following referral compared with later. Regression to the mean during the follow-up period obviously contributes to the differences between the baseline and follow-up ratings, and we cannot prove what part of the improvement may be treatment effect of the services. The patients’ ratings of their satisfaction with the services were done by the patients, but in interviews done by the psychiatrists. This may have influenced the patients’ ratings of the CMHC services to be more positive, but still the satisfaction was highest with the GPs.
Conclusions and implications
Patients with SMI in rural areas value local services that emphasize relationships and close collaborations among the CMHC, GPs and other primary care. Even in an area with a fairly comprehensive CMHC, the highest patient satisfaction was reported for GPs, indicating the potential key role of GPs for this patient group.
Clinical implications from our experiences and follow-up study are that GPs are important for patients with SMI, both as primary physicians and as collaborators with other primary care services and specialized mental health services. GPs are accessible close to where patient live and in less stigmatizing settings than specialized mental health services, they know the context in the local community, they know other local services as well as specialized mental health service and general hospital services, and they are the gateway to such services. GPs are also the main coordinators of the combination of somatic and mental health care. GPs often represent continuity of care, which is essential for building and maintaining a good personal relationship with the patient. The CMHC may give important comprehensive mental health services that are accessible to the patient, and the CMHC may support the GPs in their care for patients with SMI. Close collaboration between GPs, other primary care services and specialized mental health services like CMHCs is crucial for giving the comprehensive shared care that is needed.
Research implications from our experiences and study are that there is a need for more studies on how comprehensive care can be delivered in rural communities by collaboration among GPs, other primary care services and specialized mental health services. Patient experiences and recommendations should be studied in greater detail and with persons with service user experiences, caregivers and GPs involved in defining the research questions. Multi-center studies involving many rural areas should be done to have large enough samples and to learn from variations across sites.
Implications for service providers and policy makers are that they should support the aims for clinical practice and research described above. Further development of health services must both take into account the situation and needs in rural areas, as well as learn from experiences in rural areas by identifying and studying models and practices that seems function well.
Ethics approval and consent to participate
The study was assessed by the Regional Committee on Medical Research Ethics in Health Region III as being a follow-up and quality assurance study of the patients of the CMHC and did not require approval (reg. no. 87/99-16.99). All patients included in the study gave written informed consent to participate on a form approved by the committee.
Brief Psychiatric Rating Scale Extended version
community mental health center
Diagnostic and Statistical Manual of Mental Disorders, 4th Edition
Global Assessment of Functioning Scale
Health of the Nation Outcome Scales
Operational Checklist for Psychotic Symptoms
Rating Scale for Practical and Social Functioning
Center for Psychotherapy and Psychosocial Rehabilitation of Psychoses
severe mental illness
Statistical Package for the Social Sciences
Amaddeo F, Becker T, Fioritti A, et al. Reforms in community care: the balance between hospital and community-based mental health care. In: Knapp M, editor. Mental health policy and practice across Europe. Berkshire: Open University Press; 2007.
Semrau M, Barley EA, Law A, et al. Lessons learned in developing community mental health care in Europe. World Psychiatry. 2011;10:217–25.
Reilly S, Planner C, Gask L, et al. Collaborative care approaches for people with severe mental illness. Cochrane Database Syst Rev. 2013. doi:10.1002/14651858.CD009531.pub2.
Kelly BJ, Perkins DA, Fuller JD, et al. Shared care in mental illness: a rapid review to inform implementation. Int J Ment Health Syst. 2011;5:31.
Lester H, Tritter JQ, Sorohan H. Patients’ and health professionals’ views on primary care for people with serious mental illness: focus group study. BMJ. 2005;330:1122.
Oud MJ, Schuling J, Slooff CJ, et al. How do General Practitioners experience providing care for their psychotic patients? BMC Fam Pract. 2007;8:37.
Bindman J, Johnson S, Wright S, et al. Integration between primary and secondary services in the care of the severely mentally ill: patients’ and general practitioners’ views. Br J Psychiatry. 1997;171:169–74.
Stortingsmelding nr 25 (1996–97). Åpenhet og helhet. Om psykiske lidelser og tjenestetilbudet [White paper no 25 to the Storting (1996–97). Openness and wholeness. On psychiatric disorders and the services]. Oslo: Departement of Social and Health Affairs; 1997.
Pedersen PB and Lilleeng SE, Community mental health services 2013 Perfomance indicators for community mental health centers [Distriktspsykiatriske tjenester 2013 Driftsindikatorer for distriktspsykiatriske sentre]. Oslo: Norwegian Directorate of Health [Helsedirektoratet]; 2014.
Bjertnæs ØA, Oltedal S, Garratt A, et al. Fastlegenes vurdering av kvaliteten ved distriktspsykiatriske sentre. Tidsskr Nor Laegeforen. 2006;126:2787–91.
Bjertnaes OA, Garratt A, Ruud T. Family physicians’ experiences with community mental health centers: a multilevel analysis. Psychiatr Serv. 2008;59:864–70.
Ruud T, Martinsen E, Friis S. Chronic patients in psychiatric institutions: psychopathology. level of functioning and need for care. Acta Psychiatr Scand. 1998;97:55–61.
Martinsen E, Ruud T, Borge L, et al. The fate of chronic in-patients after closure of psychiatric nursing homes in Norway. A personal follow-up 6 years later. Acta Psychiatr Scand. 1999;98:360–5.
Kringlen E, Torgersen S, Cramer V. Mental illness in a rural area: a Norwegian psychiatric epidemiological study. Soc Psychiatry Psychiatr Epidemiol. 2006;41:713–9.
Kanter J. Clinical case management: definitions, principles, components. Hosp Community Psychiatry. 1989;40:361–8.
Ruud T, Flage KB, Kolbjornsrud OB, et al. A Two-Year Multidisciplinary Training Program for the Frontline Workforce in Community Treatment of Severe Mental Illness. Psychiatr Serv. 2015. doi:10.1176/appi.ps.201500199.
McGuffin P, Farmer A, Harvey I. A polydiagnostic application of operational criteria in studies of psychotic illness. Development and reliability of the OPCRIT system [news]. Arch Gen Psychiatry. 1991;48:764–70.
Ventura J, Lukoff D, Nuechterlein KH, et al. Appendix 1: Brief Psychiatric Rating Scale BPRS. Expanded version 4.0. scales, anchor points and administration manual. Int J Methods Psychiatr Res. 1993;3:227–43.
Wing JK, Beevor AS, Curtis RH, et al. Health of the Nation Outcome Scales (HoNOS). Research and development. Br J Psychiatry. 1998;172:11–8.
Karterud S, Pedersen G, Loevdahl H et al., Global Assessment of Functioning-Split Version (GAF-S): Background and Scoring Manual. Oslo; 1998.
Rishovd Rund B and Ruud T, Måling av funksjonsnivå og funksjonsendring i psykiatrisk behandling. Oslo: The Norwegian Board of Health Supervision; 1994.
Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull. 1979;86:420–8.
Schottle D, Karow A, Schimmelmann BG, et al. Integrated care in patients with schizophrenia: results of trials published between 2011 and 2013 focusing on effectiveness and efficiency. Curr Opin Psychiatry. 2013;26:384–408.
Bjorkman T, Hansson L. Case management for individuals with a severe mental illness: a 6-year follow-up study. Int J Soc Psychiatry. 2007;53:12–22.
Lester H, Tritter JQ, England E. Satisfaction with primary care: the perspectives of people with schizophrenia. Fam Pract. 2003;20:508–13.
Boardman AP, Hodgson RE, Lewis M, et al. North Staffordshire Community Beds Study: longitudinal evaluation of psychiatric in-patient units attached to community mental health centres. 1: methods, outcome and patient satisfaction. Br J Psychiatry. 1999;175:70–8.
All authors contributed to the planning of the study, the collection of the data and the discussion of the results. The psychiatrists TR, TA and BB performed the ratings for the baseline visit and the follow-up interviews. SAK, JS, RK and PSH collected data from the detailed nursing notes in the patient records and data concerning the services provided by primary care. TR was the principal investigator, analyzed the data and wrote the paper. All authors contributed with comments on the drafts of the manuscript, and all authors read and approved the final manuscript.
This study was funded in part by the Norwegian Council for Mental Health. Most of this work was performed as part of the work at the Nordfjord Psychiatric Center.
The authors declare that they have no competing interests.