- Study protocol
- Open Access
Understanding the mental health and recovery needs of Canadian youth with mental health disorders: a Strategy for Patient-Oriented Research (SPOR) collaboration protocol
International Journal of Mental Health Systemsvolume 13, Article number: 6 (2019)
While considerable progress is being made to understand the health and self-management needs of youth with mental health disorders, little attention has focused on the mental health and recovery needs that the youth themselves identify—this despite a national priority to incorporate patient-oriented research into the development and assessment of mental health services. To address this gap, estimates of the extent to which existing patient-reported outcome measures (PROMs)—originally developed for use amongst adult populations—are clinically meaningful and psychometrically fit for use among youth are needed. In tandem, a recovery profile for youth can be constructed incorporating the youth perspective of the services provided within a community mental health setting.
This study will utilize a mixed methods design incorporating qualitative focus group interviews and cross-sectional survey. Our process will begin with the hiring of a youth peer research partner who will provide lived experience expertise through all phases of the study. We will advertise, recruit, and conduct four focus groups with youth who receive services from the Foundry Vancouver Granville located in British Columbia, Canada. In the first two focus groups, we will recruit youth aged 15–18 years (n = 10). In the second two focus groups, we will recruit young adults aged 19–24 years (n = 10). In parallel, we will conduct a cross-sectional survey of the recovery and mental health needs of youth, informed by ten widely used and validated PROM. Thematic analysis techniques will guide the identification of predominant thematic trends in the qualitative focus group data. We will use Classical and Rasch measurement methods to test and analyze the reliability and validity of selected PROM measures for youth populations.
The proposed study has the potential to produce a preliminary conceptual and measurement model for understanding the mental health and recovery needs of youth with mental health disorders. This evidence will inform how youth mental health services can grow, support, and sustain the capacity for a collaborative, interdisciplinary and innovative patient-oriented research environment. Findings will also contribute much needed evidence to improve the standard of care for youth who experience mental health disorders in Canada and beyond.
Mental health disorders affect approximately one in four Canadian youth [1, 2], with those aged 12–24 experiencing the highest incidence of mental disorders among any age group [3, 4]. Adolescence and early adulthood are considered the peak periods for the onset of mental disorders, with 75% of all diagnoses having an onset before age 25 [5, 6]. With data estimating the economic burden of mental health disorders in Canada at $51 billion annually [7, 8], there is an urgent need to re-examine services for this high priority group [8,9,10,11,12]. Paradoxically, it is during adolescence and young adulthood that access to mental health services is the poorest  and current mental health services are “largely inadequate and unsuited to [youth] needs” [12, 14]. To reduce the impact of mental health disorders on Canadian youth, transformative change and service redesign are necessary.
In September 2014, we submitted a proposal entitled Transforming Access to Health and Social Services for Transition-Aged Youth (12–25) to the Select Standing Committee on Children and Youth in British Columbia (BC), Canada. The proposal called for the creation of a network of health and social service centres across BC that would provide youth- and family-centred services to youth with mental health disorders. This proposal resulted in the establishment of Foundry and the first six Foundry centres, which provide integrated mental health and substance use services, primary care, social services and youth and family peer supports to youth aged 12–24 (foundrybc.ca). With the goal of implementing timely, evidence-based, and youth-centred services across the province, our team recognized early that there is a significant gap in evidence on the recovery and mental health needs of youth, identified by youth themselves [15, 16]. We also identified a lack of measurement tools fit to measure recovery and mental health outcomes for youth.
In Canada, the term “recovery” in mental health care extends beyond symptom reduction to emphasize outcomes associated with “a way of living a satisfying, hopeful, and contributing life, even with the limitations caused by illness” . This dates back to the late 1980’s, when the patient-led recovery movement was established [17, 18], with the purpose of providing a voice for patients with mental health disorders, and emphasised health rather than illness. Recovery and “health” have become overarching goals for mental health service provision globally, and in the last decade they have become the primary objectives of mental health system reforms in Canada. Service organizations now face increasing pressure to articulate how their interventions and outcomes fit within a person-centred, recovery-oriented framework [19,20,21]. Although the principles of person-centred care are gaining acceptance for adults with mental health disorders in Canada, less attention is centred on the unique needs and priorities of youth. For instance, despite the emphasis on providing mental health services that align with person-centred principles, a recent systematic review identified that youth are “rarely actively involved” in their mental health treatment . Moreover, consensus on the definitions of terms such as “health”, “mental health”, and “recovery” remains illusive . To advance the science of patient-centred youth mental health care, stakeholders in the field must clearly identify, conceptualize, and prioritize targeted outcomes of care. Clear targets will be important to help stakeholders learn to systematically measure outcomes and to drive services according to the needs of youth.
One challenge with measuring youth outcomes in mental health care is that they are often not directly observable—they have to do with how youth feel, function, and perceive wellness and health. To infer the extent to which treatments and health services are effective, data need to be gathered from patients . Often this data is gathered utilizing patient-reported outcome measures (PROMs). PROMs have the potential to capture outcomes such as sustained symptom reduction, return to functioning, and optimization of patient mental health and recovery [21, 25]. Unfortunately, as the demand increases for accountability of a broad range of mental health services to be patient-centered, there is limited consensus on rating scales fit to measure these outcomes for youth [26, 27]. Successful youth engagement in the development of conceptually-driven measurement models are needed to ensure that research and clinical innovation in youth mental health services in Canada are relevant for this patient population.
The overall goal of this project is to address a national priority to incorporate patient-oriented research into the development and assessment of community mental health services for youth. The specific research aims are to:
Understand the mental health and recovery needs of Canadian youth, as identified by youth;
Use existing outcome assessments designed to measure mental health and recovery in adults to determine what evidence can be gleaned about the recovery profile of youth;
Estimate the extent to which available PROMs are clinically meaningful and psychometrically fit for purpose to measure the mental health and recovery needs of youth.
To meet the objectives of this project, we will use Canada’s Strategy for Patient-Oriented Research (SPOR) Patient Engagement Framework  and the recent Emerging Guidelines for Patient Engagement in Research [29, 30] throughout the four phases of this mixed methods study. The total duration of the study is 1 year. We will also use the core values of youth- centred care at Foundry to guide study design and procedures. We will obtain ethical approval at relevant institutions for all parts of this study.
Phase 1. Hiring and Training a Youth Patient Research Partner (PRP) and Youth Research Assistant (YRA) (months 1–2)
Patient and family engagement strategies have been formally embedded in Foundry’s organizational structure since its initial development stages, including at its primary care site, Foundry Vancouver Granville centre. During this first phase, our team will develop a job description in consultation with knowledge users to recruit a youth Patient Research Partner (PRP) and Youth Research Assistant (YRA). As recommended by Kirwan et al. , we will seek to identify candidates with strong communication skills, motivation, and assertiveness in a team setting. The research lead will ensure that YRA will be recognized in the project, including publications and knowledge translation activities.
Upon hiring of the PRP and YRA, the knowledge users (researchers, Foundry leadership) will create a safe environment that promotes honest interactions, cultural competence, training, and education. Support will also involve financial compensation for the PRP’s involvement, with the PRP hired officially as a staff member on the study team. We will document shared learnings about the strengths and limitations of the hiring process and report any concerns to leadership. Once the entire team is established, we will hold a team meeting to outline project details, develop and sign a team contract outlining members’ roles, and co-create a vision of team values, including the key principle of mutual respect for all team members and a commitment to collaboration and excellence.
Phase 2. Identification of recovery needs of youth, from the perspective of youth and young adults (months 4–6)
We will advertise and recruit four focus groups, which we will conduct with youth participants who receive services from Foundry Vancouver Granville. In focus groups 1 and 2, youth aged 15–18 years will be recruited (n = 10), while groups 3 and 4 will have young adults aged 19–24 years (n = 10). If youth are under the age of 19 years and assent is required, the PRP will meet with the youth and their parent or designated caregiver. If no parent or caregiver is available to consent for the youth, we will consider ethical issues of assent guided by the Behavioral Research Ethics Board at the University of British Columbia. This will include having youth review an assent form written in lay terms, that clearly explains what the study is about and why it is being done, and spells out and explains any acronyms in simple terms. As well, the assent form will also explain summarily and accurately what the participant will have to go through, including the total time that the participants will spend participating in the study and who is doing the study, who to contact if they have questions, and information about confidentiality.
Focus group questions will include: (1) What does health and recovery mean to you?; (2) What does it look like to go from low to high recovery?; (3) What supports are needed to help you achieve your health and recovery goals in the short-term?; and (4) What supports are needed to help you achieve your health and recovery goals long-term?. The PRP and YRA for the study will moderate the focus groups with the assistance of the research lead. These group interviews will be audio-recorded, transcribed without identifiers and uploaded to NVivo 10 qualitative management software to facilitate analysis. Inductive thematic analysis  will identify themes within participants’ experiences with mental health services and recovery, along with their perceived needs to achieve short- and long-term recovery. Two members of the research team with expertise in qualitative analysis will independently review and generate a series of broad codes. The study team will then collectively organize codes into themes to be compared and contrasted within and across focus groups. Initial themes derived from the data will be verified by the multidisciplinary research team and study participants, using an iterative process to refine participant perspectives on what is required to support their recovery needs. Comprehensive reporting of all analysis stages will be framed using the Consolidated Criteria for Reporting Qualitative Research (COREQ) .
Phase 3. PROMs: assessment of the recovery and mental health needs of youth (months 6–10)
Recruitment will occur at Foundry Vancouver Granville centre, which receives 1200 visits per year from youth aged 12–24. Staff at the centre will support recruitment of participants in two ways: (1) they will post flyers about the project in high traffic areas in the centre; and (2) they will schedule times for a research team member to be present in the clinic to meet potential participants and provide study information.
Interested participants, aged 15–24 years, who speak English, will meet with the PRP who will answer questions about the study. We will obtain informed consent prior to the study. After consent is obtained, we will give the participant a study assessment package. As noted above, if youth are under the age of 19 years and assent is required, the PRP will meet with the youth and their parent or designated caregiver or follow guidelines for assent. The study assessment package will include two paper copies of the consent form, a study participant number, and a tablet with the demographic questionnaire and PROMs programmed in (See Table 1 for list of patient-reported outcomes included). Pilot testing of the included survey has been completed by a pilot group of ten youth who receive mental health services from Foundry. Overall, youth in the pilot testing enjoyed this form of administration over the paper version and felt comfortable with the data collection platform . Youth also enjoyed with engaging with a YRA. As a result, the YRA will be responsible for all procedures related to recruitment, consenting, and data collection. Moving forward, data will be collected via a secure web-based application, Research Electronic Data Capture (REDCap), hosted by the Canadian HIV Trials Network. Participants will be compensated $15 (Canadian) for their time, which is estimated at 30 min.
Phase 4. Psychometric testing of PROMs (months 10–12)
The aim of this phase is to pilot test and estimate the extent to which the existing PROMs are fit for purpose to measure recovery and mental health of the youth with mental health disorders receiving Foundry services. A set of ten PROMs have been identified with global and regional expert consultation. The PROMs to be tested are listed in Table 1. The results from this section will inform the selection of measures for a larger scale multi-centre psychometric study in 2019. We will use Classical and Rasch measurement theory approaches to assess the psychometric properties each PROM for score reliability, construct validity, and ability to detect change.
Classical psychometric analysis
Data from the PROMs collected in Phase 3 will be examined for completeness, equality of variances, normality, scale to sample targeting (score means; standard deviations [SD]; floor and ceiling effects), internal consistency (Cronbach’s alpha), and magnitude of item-total correlations . Sociodemographic characteristics of participants will be compared by employing independent-sample t-tests and Chi square goodness-of-fit tests. Convergence and discriminant construct validity will be tested by examining correlations between the PROMs tested and demographic variables (gender, age, ethnicity and diagnosis).
Rasch measurement testing
Responses will also be analyzed using modern psychometric methods. A confirmatory analysis will be carried out, followed by Rasch analysis [59, 60] which tests the extent to which the items from each PROM form a linear continuum. The ordering of the response options will be verified, and the unique location of each item estimated to identify redundant items. We will use RUMM 2030 as the software for the analysis . A minimum sample size of 250 will provide stable person and item estimates of location on the continuum . This sample size has been shown to be consistent across health fields [63,64,65], including recent research conducted by the research lead [24, 66, 67]. To be conservative we have set our target sample to 350.
Ours is the first study that we know of to explore and measure the health and recovery needs of youth with mental health disorders. Evidence generated from this study will build capacity for patient-oriented research in the area of youth mental health. Specifically, it will inform how youth mental health services can grow, support, and sustain a collaborative and interdisciplinary patient-oriented research environment. We will ensure the PRP is well integrated into the research process in the following capacities: longitudinal data collection, PROM development and testing, interpreting datasets, and encouraging networking and relationship building opportunities in the mental health research community. By including PRPs in all phases of this research, we will gain valuable knowledge about the impact, benefit, and feasibility of our approach for enhancing the quality of clinical care and service delivery for youth. This project will provide a preliminary model for Foundry and other Canadian youth services, guiding the implementation of collaborative research projects to shape clinical initiatives and future research based on youth-identified needs.
Researchers investigating health services in Canada are increasingly required to involve patients and their families at the core of their work. Our study team has included youth and family collaborators in the preparation of the research project (agenda formulation, grant writing, funding procurement), and there is commitment to maintaining this standard throughout the execution of this research (study conduct, data analysis, interpretation of results) and the translation of results into action. To do so, ongoing training will be provided to the PRP, YRA, and knowledge users, to ensure they are well equipped with the skills for meaningful engagement in patient-oriented research. Through weekly meetings, the aim will be to achieve excellence in training for all team members, thereby ensuring that the methodology and outcomes of this project produces high quality, rigorous research that brings real benefits for patients in their daily lives.
With the inception of Foundry, we see an unprecedented opportunity to work closely with youth research partners to design and evaluate innovative services that are tailored to meet the specific needs of youth living with mental health and substance use disorders in BC and beyond. By closely documenting our process and sharing our results in a timely manner, this preliminary work will catalyse new patient-oriented initiatives within Foundry. This collaborative approach will enhance the results of the proposed project and will lead to better study results that are realistic and patient\-centered. Most importantly, the results of this collaboration between youth, knowledge users, and researchers will contribute to improved quality of care for youth with mental health disorders living in Canada and beyond.
The anticipated outcomes of this project include
Developing a profile of the mental health and recovery needs of youth from the perspectives of patients.
This profile of 350 Foundry youth (survey) and 40 youth (focus groups) will provide a comprehensive overview of the areas that youth consider to be important and priorities for mental health and recovery. Altogether, these results will guide Foundry and other Canadian youth mental health services to better understand and assist patients with their mental health and recovery goals. Also, this study will contribute evidence towards the feasibility of using PROMs in a clinical setting to identify and measure outcomes important to youth.
Create a culture for patient-oriented research for youth receiving mental health services in BC and Canada.
As noted in Canada’s SPOR document (2015), “health services research studies document a long history of advocating for more involvement of patients in decision research priorities and agendas, and there is increasing evidence on how valuable this can be” (p. 16). Little documented evidence has been identified to support this statement among youth with mental health disorders. By creating a culture of youth engagement in this project, we will be able to guide how services are delivered and meet the needs of key stakeholders.
Develop preliminary evidence for a set of items that are fit for purpose to measure the recovery and mental health needs of youth.
This study will provide preliminary evidence for understanding how current measures of mental health and recovery can collectively be used as reliable and valid instruments for youth. The psychometric summary of the proposed measures will be a starting point to understand the conceptual underpinnings and measurement of mental health and personal recovery in youth. This information will be used to design future research studies that will develop and test outcome assessments that are fit for purpose for youth. Alternatively, should the outcome assessments be shown to be reliable and valid, this evidence can then be used by key stakeholders and researchers within Foundry to understand the recovery trajectories of youth who receive Foundry services. In this study, we will also gain valuable information from our Information Technology collaborators to implement relevant measures into the current and future clinical data collection systems at Foundry to enhance clinical practice and patient-centred care.
Bridge the research-to-practice valley through the establishment of a Research Steering Committee and establishment of a culture for patient-oriented research.
In October 2016, Foundry started a Research Steering Committee to guide research priorities and ensure the methodological rigor of research stemming from the initiative. The team includes youth, families, health researchers, health funders, and knowledge users from Canada, the United States, and Australia. This proposal is the first research study put forward and endorsed by this Committee. Our study team commits that we will consult and report back to the Research Steering Committee every 3 months on timelines, deliverables, and evaluation metrics, as well as project budget. These frequent reports and consultations will allow for early dissemination of preliminary results. This will also lay groundwork for a process to establish youth-oriented needs and priorities and address them in a timely and feasible manner.
Contribution to health care systems.
This study will provide evidence of Foundry’s impact on community health outcomes for youth with mental health and substance use disorders. In recognition of lack of a widely accepted definitions and measurement of mental health and recovery amongst the youth population, this study will provide an overview of needs from the perspectives of youth. It will highlight strategic, cultural, technical, and structural benefits and challenges associated with patient engagement in health-improvement activities and research. We will conclude this study by offering recommendations for policy and practice to Foundry and other youth health services to support the inclusion of youth across all sites.
Census of Canada http://www12.statcan.gc.ca/census-recensement/index-eng.cfm. Accessed 1 Nov 2018.
Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walters EE. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005;62(6):593–602.
Employment and unemployment. http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/health71a-eng.htm. Accessed 1 Nov 2018.
Gore FM, Bloem PJ, Patton GC, Ferguson J, Joseph V, Coffey C, Sawyer SM, Mathers CD. Global burden of disease in young people aged 10–24 years: a systematic analysis. Lancet. 2011;377(9783):2093–102.
Jones PB. Adult mental health disorders and their age at onset. Br J Psychiatry Suppl. 2013;54:s5–10.
Viner RM, Ozer EM, Denny S, Marmot M, Resnick M, Fatusi A, Currie C. Adolescence and the social determinants of health. Lancet. 2012;379(9826):1641–52.
Mental Health Commission of Canada. Making the case for investing in Mental Health in Canada. Ottawa: Mental Health Commission of Canada; 2013.
Mokdad AH, Forouzanfar MH, Daoud F, Mokdad AA, El Bcheraoui C, Moradi-Lakeh M, Kyu HH, Barber RM, Wagner J, Cercy K, et al. Global burden of diseases, injuries, and risk factors for young people’s health during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. Lancet. 2016;387(10036):2383–401.
Forchuk C, Dickins K, Corring DJ. Social determinants of health: housing and income. Healthc Q. 2016;18(Suppl):27–31.
Purcell R, Jorm AF, Hickie IB, Yung AR, Pantelis C, Amminger GP, Glozier N, Killackey E, Phillips LJ, Wood SJ, et al. Demographic and clinical characteristics of young people seeking help at youth mental health services: baseline findings of the Transitions Study. Early Interv Psychiatry. 2015;9(6):487–97. https://doi.org/10.1111/eip.12133.
McGorry P, Bates T, Birchwood M. Designing youth mental health services for the 21st century: examples from Australia, Ireland and the UK. Br J Psychiatry Suppl. 2013;54:s30–5.
Coughlan H, Cannon M, Shiers D, Power P, Barry C, Bates T, Birchwood M, Buckley S, Chambers D, Davidson S, et al. Towards a new paradigm of care: the International Declaration on Youth Mental Health. Early Interv Psychiatry. 2013;7(2):103–8.
The Canadian Community Health Survey: Mental Health 2012. http://www.statcan.gc.ca/daily-quotidien/130918/dq130918a-eng.htm. Accessed 1 Nov 2018.
Mental Health Commission of Canada: Making transitions a priority. In: Consensus Conference on the mental health of emerging adults: 2015; Ottawa; 2015.
Barbic S, Mathias S, Gillingham S, Irving S: What are the recovery needs of Canadian youth?: a scoping review. Manuscript in preparation.
Barbic S, Kidd SA, Durisko ZT, McKenzie K: What are the Personal Recovery needs of community-dwelling individuals with mental illness? Preliminary findings from the Canadian Personal Recovery Outcome Measurement (C-PROM) study. Can J Community Ment Health. In press?.
Anthony WA. Recovery from mental illness: the guiding vision of the mental health service system in the 1990’s. Psychosoc Rehabil J. 1993;16(4):11–23.
Davidson LJ, O’Connell M, Tondora J, Stryton T, Kangas K. The top ten concerns about recovery encountered in mental health system transformation. Psychiatr Serv. 2006;57(5):640–5.
Kidd SA, McKenzie KJ, Virdee G. Mental health reform at a systems level: widening the lens on recovery-orientated care. Can J Psychiatry. 2014;59(5):243–9.
Kidd SA, George L, O’Connell M, Sylvestre J, Kirkpatrick H, Browne G, Thabane L. Fidelity and recovery orientation in Assertive Community Treatment. Community Ment Health J. 2010;46:342–50.
Slade M, Amering M, Farkas M, Hamilton B, O’Hagan M, Panther G, Perkins R, Shepherd G, Tse S, Whitley R. Uses and abuses of recovery: implementing recovery-oriented practices in mental health systems. World Psychiatry. 2014;13(1):1–12.
Gondek D, Edbrooke-Childs J, Velikonja T, Chapman L, Saunders F, Hayes D, Wolpert M. Facilitators and barriers to person-centred care in child and young people mental health services: a systematic review. Clin Psychol Psychother. 2017;24(4):870–86.
Manwell LA, Barbic SP, Roberts K, Durisko Z, Lee C, Ware E, McKenzie K. What is mental health? Evidence towards a new definition from a mixed methods multidisciplinary international survey. BMJ Open. 2015;5(6):e007079.
Barbic S, Cano S. The application of Rasch Measurement Theory (RMT) to psychiatric clinical outcomes research. Br J Psychiatry. in press.
Thornicroft G, Slade M. New trends in assessing the outcomes of mental health interventions. World Psychiatry. 2014;13:118–24.
Kozloff N, Stergiopoulos V, Adair CE, Cheung AH, Misir V, Townley G, Bourque J, Krausz M, Goering P. The Unique needs of homeless youths with mental illness: baseline findings from a housing first trial. Psychiatric Serv. 2016;67:1083–90.
What do we mean by youth?. http://www.unesco.org/new/en/social-and-humansciences/. Accessed 1 Nov 2018.
Canadian Institutes for Health Research. Canada’s strategy for patient-oriented research—patient engagement framework 2015.
Kirwan JR, Bartlett SJ, Beaton DE, Boers M, Bosworth A, Brooks PM, Choy E, de Wit M, Guillemin F, Hewlett S, et al. Updating the OMERACT filter: implications for patient-reported outcomes. J Rheumatol. 2014;41(5):1011–5.
Bartlett SJ, Barnes T, McIvor RA. Integrating patients into meaningful real-world research. Ann Am Thorac Soc. 2014;11(Suppl 2):S112–7.
Kirwan J, de Wit M, Frank L, Haywood KL, Salek S, Brace-McDonnell S, Lyddiatt A, Barbic S, Alonso J, Guillemin F et al. Emerging guidelines for patient engagement in research. Value Health. 1–6 (in press).
Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Q Health Care. 2007;19(6):349–57.
Richardson C, Barbic S, Mathias S. Implementation evaluation of a tablet-based waiting room survey of patient reported outcomes in an integrated youth health clinic. In: Quality of Life Research 2015; Vancouver: Springer; 2015.
Kessler RC, Andrews G, Colpe LJ, Hiripi E, Mroczek DK, Normand SL, Walters EE, Zaslavsky AM. Short screening scales to monitor population prevalences and trends in non-specific psychological distress. Psychol Med. 2002;32(6):959–76.
Andrews G, Slade T. Interpreting scores on the Kessler Psychological Distress Scale (K10). Aust N Z J Public Health. 2001;25(6):494–7.
Brooks RT, Beard J, Steel Z. Factor structure and interpretation of the K10. Psychol Assess. 2006;18(1):62–70.
Lowe B, Unutzer J, Callahan CM, Perkins AJ, Kroenke K. Monitoring depression treatment outcomes with the patient health questionnaire-9. Med Care. 2004;42(12):1194–201.
Elhai JD, Contractor AA, Tamburrino M, Fine TH, Prescott MR, Shirley E, Chan PK, Slembarski R, Liberzon I, Galea S, et al. The factor structure of major depression symptoms: a test of four competing models using the Patient Health Questionnaire-9. Psychiatry Res. 2012;199(3):169–73.
Manea L, Gilbody S, McMillan D. A diagnostic meta-analysis of the Patient Health Questionnaire-9 (PHQ-9) algorithm scoring method as a screen for depression. Gen Hosp Psychiatry. 2015;37(1):67–75.
Titov N, Dear BF, McMillan D, Anderson T, Zou J, Sunderland M. Psychometric comparison of the PHQ-9 and BDI-II for measuring response during treatment of depression. Cogn Behav Ther. 2011;40(2):126–36.
Lowe B, Decker O, Muller S, Brahler E, Schellberg D, Herzog W, Herzberg PY. Validation and standardization of the Generalized Anxiety Disorder Screener (GAD-7) in the general population. Med Care. 2008;46(3):266–74.
Spitzer RL, Kroenke K, Williams JB, Lowe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006;166(10):1092–7.
Swinson RP. The GAD-7 scale was accurate for diagnosing generalised anxiety disorder. Evid Based Med. 2006;11(6):184.
Kertz S, Bigda-Peyton J, Bjorgvinsson T. Validity of the generalized anxiety disorder-7 scale in an acute psychiatric sample. Clin Psychol Psychother. 2013;20(5):456–64.
Beck AT, Epstein N, Brown G, Steer RA. An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988;56(6):893–7.
Steer RA, Kumar G, Ranieri WF, Beck AT. Use of the Beck Anxiety Inventory with adolescent psychiatric outpatients. Psychol Rep. 1995;76(2):459–65.
Steer RA, Rissmiller DJ, Ranieri WF, Beck AT. Structure of the computer-assisted Beck Anxiety Inventory with psychiatric inpatients. J Pers Assess. 1993;60(3):532–42.
McDonell MG, Comtois KA, Voss WD, Morgan AH, Ries RK. Global appraisal of individual needs short screener (GSS): psychometric properties and performance as a screening measure in adolescents. Am J Drug Alcohol Abus. 2009;35(3):157–60.
Stucky BD, Edelen MO, Ramchand R. A psychometric assessment of the GAIN individual severity scale (GAIN-GISS) and short screeners (GAIN-SS) among adolescents in outpatient treatment programs. J Subst Abus Treat. 2014;46(2):165–73.
Barbic S, Kidd SA, McKenzie K. Development and testing of the personal recovery outcome measure. In Preparation.
Barbic S, Kidd SA, McKenzie K. Development and testing of the personal recovery outcome measure. Under Review.
Salyers MP, Godfrey JL, Mueser KT, Labriolo S. Measuring illness management outcomes: a psychometric study of clinician and consumer rating scales for illness self-management and recovery. Community Ment Health J. 2007;43(5):459–80.
EuroQoL group. EuroQol—a new facility for the measurement of health-related quality of life. Health Policy. 1990;16(3):199–208.
Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, Egmar AC, Greiner W, Gusi N, Herdman M, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Q Life Res. 2010;19(6):875–86.
Ravens-Sieberer U, Wille N, Badia X, Bonsel G, Burstrom K, Cavrini G, Devlin N, Egmar AC, Gusi N, Herdman M, et al. Feasibility, reliability, and validity of the EQ-5D-Y: results from a multinational study. Q Life Res. 2010;19(6):887–97.
Keetharuth A, Brazier J, Connell J, Carlton J, Taylor Buck E, Ricketts T, Barkham M. Development and validation of the Recovering Quality of Life (ReQoL) outcome measures. In: EEPRU Research Report 050. Edited by Interventions PRUiEEoHaC. Universities of Sheffield and York; 2017.
Keetharuth AD, Brazier J, Connell J, Bjorner JB, Carlton J, Taylor Buck E, Ricketts T, McKendrick K, Browne J, Croudace T, et al. Recovering Quality of Life (ReQoL): a new generic self-reported outcome measure for use with people experiencing mental health difficulties. Br J Psychiatry. 2018;212(1):42–9.
Cronbach LJ. Coefficient alpha and the internal structure of tests. Psychometrika. 1951;16:297–334.
Andrich D. Rating scales and Rasch measurement. Expert Rev Pharmacoecon Outcomes Res. 2011;11(5):571–85.
Andrich D. A rating formulation for ordered response categories. Psychometrika. 1978;43:357–74.
Andrich D, Sheridan B, Luo G. RUMM2020. In: Perth, editor. Rasch unidimensional measurement models software. Western Australia: RUMM laboratory; 2007.
Blanchin M, Hardouin JB, Guillemin F, Falissard B, Sebille V. Power and sample size determination for the group comparison of patient-reported outcomes with Rasch family models. PLoS ONE. 2013;18(2):e57279.
Mundy LR, Miller HC, Klassen AF, Cano SJ, Pusic AL. Patient-reported outcome instruments for surgical and traumatic scars: a systematic review of their development, content, and psychometric validation. Aesthetic Plast Surg. 2016;40:792–800.
Klassen AF, Cano SJ, Alderman A, Soldin M, Thoma A, Robson S, Kaur M, Papas A, Van Laeken N, Taylor VH, et al. The BODY-Q: a patient-reported outcome instrument for weight loss and body contouring treatments. Plast Reconstr Surg Glob Open. 2016;4(4):e679.
Cleanthous S, Isenberg DA, Newman SP, Cano SJ. Patient Uncertainty Questionnaire-Rheumatology (PUQ-R): development and validation of a new patient-reported outcome instrument for systemic lupus erythematosus (SLE) and rheumatoid arthritis (RA) in a mixed methods study. Health Qual Life Outcomes. 2016;14:33.
Barbic SP, Kidd SA, Davidson L, McKenzie K, O’Connell MJ. Validation of the brief version of the recovery self-assessment (RSA-B) using Rasch measurement theory. Psychiatric Rehabil J. 2015;38:349.
Barbic SP, Durisko Z, Andrews PW. Measuring the bright side of being blue: a new tool for assessing analytical rumination in depression. PLoS ONE. 2014;9(11):e112077.
SB and SM conceptualized the study, contributed to study design and writing of the protocol. AL, AS, PA, CH, SI, KT, SG, RZ, SB-D, EJ, TB, and RP contributed to the protocol design and writing. All authors read and approved the final manuscript.
The authors wish to thank Dr. Letitia Henville for her editorial support reviewing this manuscript.
The authors declare that they have no competing interests.
Availability of data and materials
Data is available upon request.
Consent for publication
Ethics approval and consent to participate
Ethics approval has been obtained from the University of British Columbia Behavioral Research Ethics Board (H17-0027).
Funding for this study has been received by the Canadian Institutes for Health Research Strategy for Patient Oriented Research Collaborating Grant and the St. Paul’s Hospital Foundation.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.