The Refugee-Mental Health Assessment Package (R-MHAP); rationale, development and first-stage testing amongst West Papuan refugees
© Tay et al. 2015
Received: 3 May 2015
Accepted: 26 June 2015
Published: 10 July 2015
Researchers and practitioners are concerned with a range of inter-related factors that impact on the mental health of refugees, including the traumatic events (TEs) of conflict, conditions of post-migration adversity, and the broader psychosocial disruptions that these societies have experienced. In addition, there is growing recognition that refugees are at risk of a wide range of common mental disorders that include but extend beyond the commonly studied category of post-traumatic stress disorder (PTSD). There is a need therefore to establish an integrated and comprehensive package of measures to assess all elements of relevance to the mental health of refugees. This report describes the rationale, design, development and first stage testing of the four modules (TEs, ongoing adversity, psychosocial impacts, and mental disorder categories/reactions) of the Refugee Mental Health Assessment Package (R-MHAP), drawing on data from a survey of West Papuan refugees (n = 230) residing in Port Moresby, Papua New Guinea.
We detail the sequence of qualitative (based on informant interviews, focus groups) and psychometric tests undertaken in the development of the four modules.
Preliminary modules were adapted according to our qualitative findings. Psychometric testing of the relevant modules yielded coherent factorial structures and demonstrated sound convergent and discriminant validity. For TEs, the factors comprised conflict-related experiences, traumatic losses, witnessing murder, childhood adversities, and health stress; for ongoing adversity, factors included deprivation of basic needs, difficulties accessing health care/health problems, displacement, and conditions of safety in the community; and for psychosocial impacts, factors included threats to safety; ruptured relationships; injustice related to past human rights violations; challenges to identity; and existential meaning (related to expression of political aspirations). There was strong agreement in assignment of cases of mental disorder (as a composite grouping) between the R-MHAP administered by field workers and structured clinical interview conducted blind by a psychologist. Symptom means for the majority of diagnostic categories remained stable over a 6-month period.
Our first-stage assessment suggests that the R-MHAP holds promise as a comprehensive assessment package for use in the refugee mental health field.
The research field of refugee and post-conflict mental health spans diverse areas of interest, including exposure to the traumatic events (TEs) of war, conditions of post-migration adversity, the broader psychosocial changes following conflict and forced migration, and a range of common mental disorders. There is a need for a comprehensive package of measures that incorporates key indices of relevance to researchers and practitioners working in the field worldwide. We report the design, development, and first-stage testing of the four modules comprising the Refugee-Mental Health Assessment Package (R-MHAP), drawing on data gathered from West Papuan refugees living in Papua New Guinea (PNG).
The establishment of a common package of assessment modules will facilitate closer evaluation of key questions that remain to be answered in the field, for example, regarding the relative roles of trauma exposure and ongoing adversity in determining mental health outcomes , the contributions of disruptions to the broader psychosocial context to mental health , and the range and salience of common mental disorder categories that arise from these experiences . An important challenge in devising measures is to integrate the universal experiences of refugees across contexts and the specific historical and cultural factors that are particular to each society. In devising a set of measures, it is important to provide guidance to future researchers concerning the broad procedures to follow in order to adapt the package to other cultures and contexts.
In devising our trauma event (TE) module, we draw on the legacy of research in the traumatology literature to include events commonly experienced by refugees worldwide, such as exposure to human rights violations (torture and other forms of interpersonal violence), witnessing murder and intentional harm to others, lack of access to emergency medical care, gross deprivations (shelter, food, water), and in many countries, natural disasters [4, 5]. In addition, we draw on qualitative data to supplement this list with the context-specific experiences relevant to the West Papuan community.
In relation to ongoing forms of adversity in the post-migration environment, past studies have identified a relatively consistent set of stressors across refugee populations [1, 2, 6–9], although the specific context influences the nature and salience of particular types of hardship . As with TEs, we therefore include both universal as well as context-specific stressors in our measure.
A novel aspect of our package is the inclusion of an index of the broader psychosocial changes resulting from conflict and displacement. In devising this module, we draw on the Adaptation and Development after Persecution and Trauma (ADAPT) model, a conceptual framework that identifies five interrelated psychosocial “pillars” or domains that are disrupted by mass conflict and displacement: safety and security; interpersonal bonds and networks; justice (in relation to past and ongoing human rights violations); roles and identities; and existential beliefs and systems of meaning [11, 12]. The ADAPT model assesses the longer-term changes in the eco-social context over time and place. In contrast, measures of TE exposure and adversity represent specific, proximate experiences that pose an immediate source of threat or stress.
Based on a growing consensus, we include the mental disorders and reactions that are relevant to the refugee experience, including but extending beyond post-traumatic stress disorder (PTSD) and depression [3, 13]. Constraints in time, resources and skills generally preclude the use of lengthy and complex diagnostic interviews in field studies amongst refugee populations. A further challenge is the extent to which measures reflect international diagnostic constructs and criteria as opposed to indigenous idioms of distress. We explicitly based our measure on contemporary international classification systems to ensure broad comparability with studies undertaken in other settings. The provisional taxonomy and diagnostic criteria are drawn from the Diagnostic and Statistical Manual for Mental Disorder, fourth (DSM-IV) and fifth (DSM-5) editions of the American Psychiatric Association [14, 15] and the International Classification of Diseases (ICD), both the tenth edition and forthcoming eleventh edition [16, 17]. We use qualitative techniques to ensure that the diagnostic categories applied are appropriate to the context and culture, drawing on feedback from psychiatrists working in the setting (PNG). In addition, we undertake individual interviews and focus groups to ensure that symptoms are meaningful and salient to West Papuans residing in the specific context. In future developments of the R-MHAP, we intend to focus on developing modules that assess indigenous or emic constructs of mental distress.
In designing the R-MHAP, we give particular attention to the overall structure of the package, particularly the sequencing and formatting of modules, the aim being to ensure ease of administration in a transcultural context where field workers have received limited training and where respondents may have low levels of literacy and education . In addition, we adopt a similar format across modules to maximize accuracy of administration, comprehensibility, and fidelity of responses amongst participants.
In summary, the objective of this report is to outline the sequence of qualitative and quantitative steps applied in the development and validation of the four modules (TEs, ongoing stressors, psychosocial impacts of conflict represented by the ADAPT index, and mental disorders/reactions) comprising the Refugee-Mental Health Assessment Package (R-MHAP), testing the properties of the modules amongst a sample of West Papuan refugees residing in Port Moresby, PNG.
Preparatory literature review
We used a rational expert-driven approach informed by existing reviews  and our extensive knowledge of the field to search for assessment packages that included at least two of the four intended modules. Based on existing reviews, we extended our search using the terms (alone and in combination) “mental health”, “mental disorder”, “psychiatric/psychological disorder”; “post-traumatic stress”, “depression”, “anxiety”, “panic”, “complicated and/or prolonged grief”, “adult separation anxiety”, “somatization and/or somatic symptoms”, “intermittent explosive disorder and/or explosive anger”, and “psychosis”, limiting results to adults (aged over 18 years), studies based on refugees and/or post-conflict populations, and articles published in English up to the present. Search results obtained from PsycInfo, PubMed, Medline and Scopus revealed only one widely used and cited package comprising the Harvard Trauma Questionnaire  and its companion measure, the Hopkins Symptoms Checklist . Our search therefore confirmed the need to develop an expanded package of measures.
Our sample (n = 230) of West Papuan refugees resided in Port Moresby, PNG. Because there are no general population census data identifying West Papuans in Port Moresby and this minority is dispersed within a larger population of PNG nationals, we applied a key informant, targeted sampling approach which involved two phases. First, we conducted an extensive process of location mapping and qualitative data gathering over three preparatory visits by the Australian-based team, each lasting approximately 3 weeks (May–December 2010).
Six settlements in Port Moresby, Hohola, Rainbow, Six-Mile, Eight Mile, Nine-Mile, and Tokarara/Waigani, were identified based on a desk top review of the relevant literature concerning West Papuans living in PNG and informant data provided by local government officials (the PNG Department of Health and Foreign Affairs) and the United Nations High Commissioner for Refugee (UNHCR) office which had undertaken an extensive mapping exercise of this community. Triangulation of available information indicated that the identified six settlements housed an estimated 90% of West Papuan refugees residing in the city. The settlements are geographically defined communities characterised by high density, makeshift housing, and few facilities. Based on all sources, we established a conservative estimate of 250 adult West Papuans (men and women) at the time of the survey.
Sociodemographic characteristics of the study sample (n = 230)
Country of birth
Papua New Guinea
A follow-up study was conducted based on a subsample selected from the baseline study comprising roughly equal numbers of persons who met criteria for at least one of the mental disorders (n = 42) assessed and the remainder who had no mental disorder (non-cases) (n = 57). Of the 120 participants in this stratified sample we approached at follow-up, 102 agreed to participate (85% response). The main reason for non-response was that we were not able to make contact with past participants because they were away or had relocated. The follow-up study was completed between September and October, 2012. The average time from baseline to follow-up interview was 6.5 months.
Ethical permission for the study was provided by the University of New South Wales Human Research Ethics Committee (HREC11273) and the Medical Research Council of PNG Ethics Committee (MRAC12.27). All respondents gave written or oral consent to participate in both the baseline and follow-up studies. Interviews were conducted in a private location or within the home of the respondent, depending on their preference.
Field personnel training
The in-country expatriate team operating in PNG consisted of two male researchers (one West Papuan research assistant, MK, who has extensive contacts with the community in Port Moresby, the other, AT, a bilingual Bahasa-speaking psychologist experienced in transcultural and refugee research). A team of five West Papuans employed as field workers received 3 weeks’ of interviewer training conducted by the expatriate team prior to the commencement of the community survey (January–May, 2013). The field interviewers included two female and three male West Papuans (mean age = 35) who were selected based on consultations with community leaders to ensure we included a broad representation of competent workers from different settlements with a spread of gender and ages. The interviewers worked in pairs over the course of the study. Prior to the follow-up study, we conducted an additional 1-week training with field workers to ensure their competence in applying the revised study protocol.
Translation and back-translation
Provisional modules were translated and back-translated from English to Bahasa Indonesian by three bilingual Bahasa-Indonesian speaking researchers and a psychologist following established guidelines [21, 22]. Our research personnel were proficient in the relevant local languages (Bahasa Indonesian, Pinyin), conducting interviews in the respondent’s preferred language throughout the study period.
Development of R-MHAP
The Refugee-Mental Health Assessment Package (R-MHAP) and recommended supplementary measures
Traumatic events (TEs)
This module comprises 23 items (rated as “not-experienced” or “experienced”) derived from a pool of 46 items. It assesses exposure to conflict-related TEs prior to, during, and following migration
This module comprises 26 items (rated as “a serious problem” or “not a serious problem”) developed based on the item pool derived from the Humanitarian Emergency Settings Perceived Needs (HESPER) scale 
Adaptation and Development After Persecution and Trauma (ADAPT) Index
This module comprises 22 items (rated on a 4-point Likert scale: 0 = none, 1 = moderately, 2 = very strongly, 3 = extremely) derived from a pool of 63 items, assessing five domains of psychosocial disruptions related to mass conflict: safety/security, bonds/networks, justice, roles/identities, and existential meaning
Loss of statehooda
This module comprises 10 items (rated on a 4-point Likert scale: 0 = none, 1 = moderately, 2 = very strongly, 3 = extremely) assessing refugees’ perception of statelessness and loss of statehood
Local idioms of distressa
This module comprises 45 items assessing locally understood syndromes amongst West Papuans
Mental health module
This module assesses nine current (last 12 months) and lifetime prevalence of mental disorders defined by the DSM-IV/V, Diagnostic and Statistical Manual of Mental Disorders (Fourth and Fifth Editions): Posttraumatic stress disorder, depression, generalized anxiety disorder, panic disorder, somatic symptom disorder, persistent complex bereavement related disorder, separation anxiety, psychosis, and intermittent explosive disorder
Alcohol and substance useb
This module comprises 5 items (rated as “no” and “yes”) based on the WHO Alcohol Use Disorders Inventory Test (AUDIT)  assessing excessive use of alcohol and other substances
This module comprises 12 items (rated on a 5-point Likert scale: 1 = none, 2 = mild, 3 = moderate, 4 = severe, 5 = extreme) based on the WHO Disability Assessment Schedule (WHODAS) 2.0  assessing disability related to health and mental health conditions in the previous 30-day period
General physical wellbeing
This module comprises 11 items (rated as “no” and “yes”) assessing general health status, health-related disability, help-seeking behaviour, and barriers to accessing health care in the last 12 months
Suicidal ideation and behaviour
This module comprises 12 items (rated as “no” and “yes”) assessing episodes of suicidal ideation and behaviour in the last 12 months
Exposure to Intimate Partner Violence (IPV)d
This module is based on the WHO Multi-Country Study on Women’s Health and Domestic Violence  assessing current and lifetime exposure to Intimate Partner Violence (IPV)
Desk and team-based development of preliminary modules of TEs, stressors, ADAPT index, and mental disorders/reactions
Our preliminary list of TEs and ongoing stressors were based on the general literature, our own prior research in the region and provisional “blueprint” lists derived from existing measures such as the Harvard Trauma Questionnaire (HTQ)  and the HESPER . Universal dimensions identified for TEs were conflict-related experiences, witnessing murder, traumatic losses, access to emergency medical care; and for ongoing stressors, extreme poverty, access to basic needs, health care, displacement, safety in the community [4, 8, 27, 28]. These lists were adapted and refined based on our prior knowledge of the history and conditions that prevailed amongst the West Papuan community and our qualitative work with community members.
The list of items measuring psychosocial adaptive changes was generated by consensus within the team based on our long familiarity with the five “pillars” that are considered at the core of the ADAPT model.
The mental disorder categories/reactions (defined by DSM-IV, DSM-5, ICD-10, and the proposed ICD-11 criteria) supported by the literature included PTSD [29–31], depression , generalized anxiety disorder , panic disorder [32, 33], somatic symptom disorder , complicated grief , separation anxiety disorder , psychotic symptoms [30, 31], and intermittent explosive disorder . We also included persistent complex bereavement disorder, a newly proposed but provisional DSM-5 diagnosis. Participants were asked to respond to all items irrespective of whether or not they screened positive on the initial screening items.
Contextual adaptation We consulted four experienced local psychiatrists (including the chief psychiatrist of PNG, the Mental Health Advisor at the Department of Health, and two academics at the Department of Psychiatry, Port Moresby General Hospital) over four formal consultation sessions to assess the face validity and practical utility of the nine selected mental health categories/reactions. The majority of expert informants were Melanesian in background, the broad cultural/ethnic grouping to which West Papuans belong. The focus of interviews was to assess whether the diagnostic categories we intended to include in our module had face validity and practical utility within the culture and context.
Contextual adaption of the modules was based on information obtained from five individual interviews and two focus groups (10 participants in each, mean age = 37 years) involving members of the West Papuan community. We purposively selected participants to ensure inclusion of persons including leaders and general members of the community with a range of ages and gender (five men and five women in each focus group).
Interviews and focus groups were conducted by a clinical psychologist fluent in Bahasa Indonesian, the lingua franca of the community, and a West Papuan research assistant. Facilitators of focus groups adopted an explicitly non-judgmental approach in which past experiences and current symptoms were contextualized according to the culture, history and current living circumstances of the community. Facilitators explicitly aimed to reduce any sense of stigma or shame related to revealing symptoms of distress by avoiding any inference that disclosures by participants indicated the presence of mentally illness, an approach that proved successful in encouraging sharing of individual experiences. Lack of familiarity with western diagnoses of mental disorder is likely to have reduced further any sense of stigma when West Papuans discussed their own experiences. However, where necessary, feedback sessions were undertaken in private with individual members to discuss and address personal issues of concern that may have arisen from information revealed in focus groups.
Quantitative data and statistical analysis
Psychometric and other quantitative analyses were conducted on data from the full sample at baseline and on the stratified sample at follow-up. TE and stressor items were analysed as dichotomous variables. Items of the ADAPT module were dichotomized by assigning a summary score of 1 for any item rated 2 (very strongly) or 3 (extremely) (see hereunder for details of scoring). Preliminary latent structures underlying the provisional lists of items comprising TEs, stressors, and the ADAPT module were specified according to theoretical considerations and the findings of prior research. The preliminary structures were then subjected to psychometric testing using Confirmatory Factor Analysis (CFA).
The Mean-adjusted Weighted Least Square method (WLSMV) was used in all analyses [39, 40], as an established statistical procedure in applying CFA to dichotomous variables, an approach applied extensively in past studies [41, 42]. Standardized factor loadings (based on probit regression coefficients) and R2 (the proportion of variance explained in the latent response variables underlying each CFA model) were calculated. Where we were able to demonstrate a good fit for a first-order factorial structure, we examined further for a second-order factor as a test of the unitary nature of the overall construct. Cross-loading items (with a factor loading ≥3.0) and items with overlapping content were eliminated sequentially until we achieved an adequate model fit. We judged model fit based on a suite of indicators including a non-significant Chi square test, the Root Mean Square Error of Approximation (RMSEA) (<0.05 indicates a good fit), the Tucker Lewis Index (TLI), and Comparative Fit Index (CFI) (>0.9 indicates a good fit for the latter two indicators) [43–45]. Internal reliability of relevant item pools was examined using the Kuder-Richardson (KR20) reliability index recommended for dichotomous items.
Convergent and discriminant validity were examined in a series of correlated uniqueness (CU) multitrait multimethod (MTMM) models. The correlated residuals are assumed to represent the effects of the measurement method [46, 47]. Convergent validity was assessed by estimated trait (factor) loadings and discriminant validity by cross-factorial correlations .
Descriptive statistics were calculated for mental disorder symptom scores (summation of all items for each disorder) to allow comparisons between baseline and follow-up amongst the structured subsample drawn from the baseline sample. Our analysis of concordance data comparing mental disorder diagnoses on the SCID with the field measure (described in full hereunder) applied the Area under the Curve (AUC), sensitivity, specificity, and positive and negative predictive values based on a Receiver Operating Characteristic (ROC) analysis [48, 49].
Qualitative findings examining transcultural validity of modules
Face validity of nine selected mental health categories/reactions
All psychiatrists endorsed the alignment between the categories of mental disorder selected and locally recognized mental health responses within the Melanesian context indicating that they applied all diagnoses in their routine practices (further details are provided in Additional file 1). The consensus achieved amongst these expert informants offered some evidence in support of the face and content validity as well as the clinical utility of these constructs within the context.
Our observations derived from individual interviews and focus groups supported the cultural acceptability of the R-MHAP modules amongst respondents. The focus group responses indicated that the translated modules demonstrated strong contextual validity. For example, our provisional list of conflict-related TEs, derived from both universal (torture, witnessing murders) and context-specific (displacement, sexual violence) were readily recognized as major events to which many West Papuans had been exposed. Symptoms of key mental disorder also were readily recognized and endorsed.
Linguistic equivalence (correct interpretation of items)
Respondents were asked whether translated items “made sense” and would be readily understood by the community. Items that participants found challenging in their meaning or in ease of interpretation were removed or reworded based on feedback from the focus groups.
Technical equivalence (appropriateness of response scale)
Respondents were asked to judge the ease of response to the scoring format for items. Based on recommendations from the focus groups, we changed the provisional frequency-based response scale to one of severity (for the ADAPT index: 0 = none, 1 = moderately, 2 = very strongly, 3 = extremely), which was uniformly regarded as easier to respond to compared to the previous format.
Completeness (equivalence of construct)
Completeness was examined by the extent to which items in each module reflected the underlying constructs we sought to study. This was achieved by iteratively inquiring whether participants could think of additional items or experiences that offered to expand the modules. Where there was a consensus on the addition, removal or alteration of items, modifications were made to the relevant modules.
Confirmatory Factor Analysis of the constituent items of the module of common Traumatic Events (TEs)
Factor 1: Conflict-related experiences
Forced to live in poor conditions due to ongoing violence
Direct experience of war for political reasons
Home intentionally destroyed
Lack of shelter because of conflict
Humiliated in front of other people
Forced to go into hiding during war
Involved in active combat as freedom fighters
Held captive or imprisoned
Abducted by members of other political groups
Factor 2: Traumatic losses
Disappearances of family members
Separated from family members
Forced to abandon family members during war
Multiple deaths of family members
Not being able to perform cultural ceremonies for the dead
Factor 3: Witnessing murders
Witnessing strangers tortured
Hearing about family members tortured and murdered
Witnessing dead bodies
Witnessing rape and sexual abuse
Factor 4: Childhood adversities
Witnessing violence at home
Physical abuse during childhood
Factor 5: Health-related stress
Not being able to access medical care for family members
Not being able to access medical care for self
Factor 1: Conflict-related experiences
Factor 2: Traumatic losses
Factor 3: Witnessing murders
Factor 4: Childhood adversities
Factor 5: Health-related stress
Confirmatory Factor Analysis of the constituent items of post-traumatic ongoing stressors
Factor 1: Basic survival needs
Lack of shelter/housing
Lack of access to toilets
Lack of access to clean water
Lack of food
Lack of hygiene
Lack of clothes and blankets
Factor 2: Access to health care and health problems
Lack of access to health care
Physical health problems
Factor 3: Displacement and separation from family members
Being displaced from home
Separation from family members
Factor 4: Safety issues in community
Safety or protection for women from violence in community
Law and justice in community
Support for vulnerable people in community
Factor 1: Basic survival needs
Factor 2: Access to health care and health problems
Factor 3: Displacement and separation
Factor 4: Safety issues in community
The ADAPT index of psychosocial impacts of conflict and displacement
Confirmatory Factor Analysis of the constituent items of the ADAPT index of psychosocial impacts of conflict based on the Adaptation and Development After Persecution and Trauma (ADAPT) model
Factor 1: Safety/security
Feeling very nervous about family members getting sick because unable to afford medical care
Serious worries that the future will be insecure
Serious concerns about my family might die of hunger
Serious concerns about whether family will survive
Feeling unsafe about visiting family or neighbours
Factor 2: Bonds/Networks
Feeling very upset about being separated from family in the homeland
Feeling very upset about not being able to return home
Feeling very upset because of not being able to perform traditional ceremonies for the dead
Serious concerns because of not being able to visit the graves of dead family members
Factor 3: Access to justice for human rights violations
Strong feelings of unfairness about the way me and family had been treated in the past
Thinking excessively about the unjust things that happened to me and family in the past
Lost trust in people because of the unjust things that happened to me and my family in the past
Become suspicious of authorities because of the unjust things that happened to me and my family in the past
Difficulties accepting the unjust things that happened to me and my family
Factor 4: Roles/identities
Difficulties trying to overcome cultural barriers so that I can have a place in society
Feeling frustrated because of not being able to contribute to the family in the way I used to
Feeling frustrated because I have to rely on others and cannot get things done on my own
Lost a sense of autonomy and control in my life
Factor 5: Existential meaning (related to expression of political aspirations)
Feeling worried about the strong influence of Indonesian culture in West Papua
Feeling worried about the future of my country
Feeling frustrated because I am not able to express my political aspirations
CFA yielded five dimensions corresponding to the ADAPT domains (χ2  = 227.20, P = 0.083, CFI = 0.998, TLI = 0.998, RMSEA = 0.025). Items constituting each domain demonstrated sound internal consistency as indicated by the KR20 coefficient: 0.88 (safety/security); 0.94 (bonds/networks), 0.92 (access to justice); 0.89 (roles/identities) to 0.84 (existential meaning/political aspirations). The CU model indicated a good fit (χ2  = 161.65, P = 0.56, CFI = 1, TLI = 1, RMSEA = 0.00) supported by sound convergent (factor loadings = 0.88–1.00) and discriminant validity (cross-factorial correlations = 0.56–0.78).
Diagnostic concordance in psychiatric case assignment by trained field workers
We tested diagnostic concordance between the lay-administered mental disorder assessment and the Structured Clinical Interview for DSM-IV (SCID) applied in a blinded manner by an expatriate psychologist (AT) . Since the SCID had not yet been modified for DSM-5, we added relevant items to allow diagnoses to be made according to that system as well as DSM-IV. As indicated, we drew a stratified subsample from the baseline study of 102 persons comprising 59 men (57.8%) and 43 women (42.1%) with a mean age of 42.51 (SD = 1.63), including approximately equal numbers with or without a mental disorder based on the baseline field module. No significant differences in gender, age, or length of settlement were detected between participants with and without a mental disorder.
The SCID and the field module concurred in assigning 42 (41.1%) participants as cases (presence of one or more current disorder), and 57 (55.8%) as non-cases. Of the 3 discordant case assignments, 2 were identified as cases by the field measure and 1 by the SCID respectively. The ROC analysis (cases versus non-cases) yielded an AUC of 0.93 (95% CI 0.87–0.98); a sensitivity of 0.98 (CI 0.88–1.00); a specificity of 0.97 (CI 0.88–1.00); a positive predictive power of 0.95 (CI 0.85–0.99) and negative predictive power of 0.98 (CI 0.91–1.00). The overall correct classification was 0.97 (CI 0.92–0.99), the intra-class correlation being 0.94 when comparing case assignments across the two methods.
Follow-up analysis of change in symptom scores of mental disorder categories/reactions between cases and non-cases at baseline and follow-up
Non-cases: mean differences between T1 & T2 (n = 40)
Cases: mean differences between T1 & T2 (n = 61)
Total (n = 101)
Non-cases (n = 40)
Cases (n = 61)
Subgroup diff, P
Total (n = 101)
Non-cases (n = 40)
Cases (n = 61)
Subgroup diff, P
Mean difference: (T1 − T2)
p-values (paired t-test)
Mean difference: (T1 − T2)
p-values (paired t-test)
To our knowledge, the R-MHAP is the only contemporary integrated package of measures in the field of refugee and post-conflict mental health that is specifically designed to incorporate key concepts and indices of relevance to researchers and practitioners worldwide. The conceptualization and development of the constituent modules was based on a thorough consideration of the relevant literature, drawing on the fields of psychiatric traumatology, clinical psychology, human rights, social sciences, and transcultural studies. Other established measures recommended to complement the R-MHAP are indicated in Table 2. Our qualitative and quantitative findings offer substantial support for the contextual appropriateness (in relation to West Papuan refugees) and psychometric properties of four core modules of the R-MHAP, including TEs, ongoing stressors, psychosocial impacts of conflict, and a range of mental disorders. Our findings provide support for the procedural validity of the R-MHAP in that trained lay interviewers with no background knowledge of international mental health constructs were able to administer the package effectively with a high level of adherence and completion by respondents.
Strengths of the study include the sound psychometric properties identified for the constituent modules, findings that set the stage for further testing of the R-MHAP amongst other populations of refugees. The design of the R-MHAP gives emphasis to the principle of coherence in the sequencing of topics, as well as ensuring procedural simplicity, consistency and ease of administration . We have outlined the steps that can be followed by future researchers to adapt key indices (particularly TEs and stressors) to each context.
Limitations of our inquiry include the sample size, a natural constraint that required us to aggregate mental disorders/disturbances in testing for convergence of case assignment against the SCID. The sample was heterogeneous, including refugees born in West Papua and offspring born in PNG. The advantage, however, was that we obtained a wide variation in responses which facilitated our statistical analyses. Future studies with larger samples will be able to investigate more closely the influence of demographic characteristics such as age, gender, and place of birth, by allowing application of more sophisticated statistical techniques such as multi-group factor analysis. Our testing of the modules is limited to West Papuan refugees, cautioning against generalizing our results to other refugee populations. Future studies are needed to test the contextual and psychometric properties of our modules across other refugee groups worldwide. The development of the module for mental disorders was based on international criteria, although qualitative data from our focus groups provided some support for the face validity of the mental disorder categories under investigation. Further in-depth interviews involving a diverse range of informants is needed for to validate the diagnostic module within this and other transcultural populations. In future developments of the R-MHAP, inclusion of modules for indigenous constructs of mental disorder will provide a more comprehensive account of the mental reactions relevant to each society under study .
The present study focused primarily on the core modules (covering trauma events, adversity, psychosocial effects of conflict, and common forms of mental disorder) of general importance to refugee mental health. The key steps we outlined in this study may be valuable in guiding field researchers and workers in the process of devising and adapting their own assessment tools.
We note that there invariably is a trade-off between cultural specificity/validity and comparability/reliability in developing mental health measures, particularly in transcultural contexts. Applying a strictly emic approach in which idioms of distress are derived de novo from the culture under study has the advantage of maximizing contextual validity but that approach inevitably forfeits comparability in relation to studies amongst other populations and settings. An approach limited to the translation and minimal adaptation of international measures such as the Composite International Diagnostic Interview (CIDI) has the advantage of increasing consistency and reliability of findings across cultures but at the potential expense of reducing the local validity of responses . We elected to take an intermediary approach in which the original structure and provisional symptom pool of our diagnostic measure was derived from international classification systems prior to being subjected to extensive qualitative testing and adaptation to ensure the relevance and comprehensibility of underlying constructs and constituent items. In so doing, we attempted to achieve a balance between local validity and the potential for the package to be adapted and used in other cultural settings. We further note that it may not always be feasible to apply the package as a whole especially in settings of mass conflict or disasters. Limiting the scope of assessment to the core indices (e.g., traumatic events, ongoing adversities, the psychosocial effects of conflict, and common forms of mental distress) outlined in our study may prove valuable in medium-term planning and development for psychosocial and mental health services following complex emergencies .
In relation to our substantive findings, our TE module identified five domains (conflict-related experiences, traumatic losses, witnessing murders, childhood adversities, and health stress) that match closely the known history of West Papuan refugees, offering indirect support for the validity of the module. For example, consistent with the history, direct exposure to political conflict and experiencing homes being burnt down yielded the highest factor loadings on “conflict-related experiences” (factor 1) . Our results are consistent with findings from previous studies amongst conflict-affected populations in the region, particularly neighbouring Timor-Leste, a territory also invaded and occupied by Indonesia [4, 8]. The five derived factors were predicted by a higher-order factor, supporting the use of either a composite score of TEs or individual scales in future studies.
Four dimensions of ongoing stressors were identified by our second-order CFA, comprising basic needs, access to health services, stress from prolonged displacement, and safety concerns in the community. Again, these domains assess experiences that are consistent with the contextual and historical background of the population, as identified in our preceding qualitative study . There was substantial endorsement of most stressor items, which was consistent with observations of the extreme conditions of deprivation in the settlements in Port Moresby where poverty, lack of services and endemic violence represent major challenges [58, 59].
In relation to our index of psychosocial impacts, CFA identified five dimensions consistent with the ADAPT model, namely safety and security; interpersonal bonds and networks; justice; the integrity of roles and identities; and existential meaning (related to expression of political aspirations). The establishment of this module offers the potential to test the ADAPT model in future studies to assess the impact of broader psychosocial disruptions on mental health outcomes in addition to the effects of TEs and ongoing stresses experienced by refugees.
In testing our mental health assessment module, there was a high level of convergence between the field measure and the SCID in identifying aggregated cases. The few discordant cases were equally divided across the two measures, indicating that the field interview was not over-sensitive in identifying persons with mental disorder compared to the SCID, a common problem encountered in past field studies using screening measures . The most likely reason for this finding is that our field measure followed a structured format that is consistent with a diagnostic interview conducted by clinicians. The predictive validity of the diagnostic module was supported by our findings that symptoms tended to persist over a 6 month period, suggesting that the mental disturbances identified were not simply manifestations of transient distress. Further support for the discriminant validity of our diagnostic module is that only a modest number of non-cases transitioned into becoming cases over six months, consistent with expectations.
The R-MHAP offers a comprehensive and integrated assessment package that covers a wide range of mental health and psychosocial indices of relevance to modern refugee research and practice. Our data offer support for the cultural validity and psychometric properties of the four modules that comprise the R-MHAP, as tested amongst West Papuans residing in Port Moresby. These outcomes were achieved by field workers previously unfamiliar with western mental health concepts who received limited training prior to the survey. Further testing of the R-MHAP is needed to support the validity, capacity for transcultural adaptation and general utility of the measure as a multidisciplinary research tool in the refugee and related fields. Electronic adaptation of the R-MHAP is in the early development phase and will be available for use in the near future.
AKT, SR, DS conceived study. AKT, MK performed the research. AKT performed and JC supervised the analysis. AKT, SR, DS, JC drafted and revised the manuscript. All authors read and approved the manuscript.
We thank the following contributors to this project: Mr Michael Kareth, Mr Paul Wandik, Mr Martinus Anari, Mr Freddy Waromi, Ms Dolly Songona, and Ms Olvianna Fonataba, Drs Goiba Tieneng (Chief Psychiatrist of PNG) and Uma Ambi (Principal Mental Health Advisor, Department of Health, PNG).
Compliance with ethical guidelines
Competing interests The authors declare that they have no competing interests.
NHMRC program number: RM08333; South Western Clinical School Faculty of Medicine Scholarship.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
- Miller KE, Kulkarni M, Kushner H. Beyond trauma-focused psychiatric epidemiology: Bridging research and practice with war-affected populations. Am J Orthopsychiatry. 2006;76(4):409–22.PubMedView ArticleGoogle Scholar
- Miller KE, Rasmussen A. War exposure, daily stressors, and mental health in conflict and post-conflict settings: bridging the divide between trauma-focused and psychosocial frameworks. Soc Sci Med. 2010;70(1):7–16.PubMedView ArticleGoogle Scholar
- Tol WA, Rees SJ, Silove DM. Broadening the scope of epidemiology in conflict-affected settings: opportunities for mental health prevention and promotion. Epidemiol Psychiatr Sci. 2013;22(3):197–203.PubMedView ArticleGoogle Scholar
- Silove D, Brooks R, Bateman CS, Steel Z, Amaral ZF, Rodger J, et al. Social and trauma-related pathways leading to psychological distress and functional limitations 4 years after the humanitarian emergency in Timor-Leste. J Trauma Stress. 2010;23(1):151–60.PubMedGoogle Scholar
- Mollica RF, Caspi-Yavin Y, Bollini P, Truong T, Tor S, Lavelle J. The Harvard Trauma Questionnaire: validating a cross-cultural instrument for measuring torture, trauma, and posttraumatic stress disorder in Indochinese refugees. J Nerv Ment Dis. 1992;180(2):111–6.PubMedView ArticleGoogle Scholar
- Silove D, Steel Z, Susljik I, Frommer N, Loneragan C, Chey T, et al. The impact of the refugee decision on the trajectory of PTSD, anxiety, and depressive symptoms among asylum seekers: a longitudinal study. Am J Disaster Med. 2007;2(6):321–9.PubMedGoogle Scholar
- Steel Z, Silove D, Bird K, McGorry P, Mohan P. Pathways from war trauma to posttraumatic stress symptoms among Tamil asylum seekers, refugees, and immigrants. J Trauma Stress. 1999;12(3):421–35.PubMedView ArticleGoogle Scholar
- Brooks R, Silove D, Steel Z, Steel CB, Rees S. Explosive anger in postconflict Timor Leste: Interaction of socio-economic disadvantage and past human rights-related trauma. J Affect Disord. 2011;131(1–3):268–76.PubMedView ArticleGoogle Scholar
- Jordans MJ, Semrau M, Thornicroft G, van Ommeren M. Role of current perceived needs in explaining the association between past trauma exposure and distress in humanitarian settings in Jordan and Nepal. Br J Psychiatry. 2012;201(4):276–81.PubMedView ArticleGoogle Scholar
- Silove D, Sinnerbrink I, Field A, Manicavasagar V, Steel Z. Anxiety, depression and PTSD in asylum-seekers: Associations with pre-migration trauma and post-migration stressors. Br J Psychiatry. 1997;170(APR):351–7.PubMedView ArticleGoogle Scholar
- Silove D. The psychosocial effects of torture, mass human rights violations, and refugee trauma: Toward an integrated conceptual framework. J Nerv Ment Dis. 1999;187(4):200–7.PubMedView ArticleGoogle Scholar
- Silove D. The ADAPT model: a conceptual framework for mental health and psychosocial programming in post conflict settings Intervention 2013, 11.Google Scholar
- Silove D, Bryant R. Rapid assessments of mental health needs after disasters. JAMA. 2006;296(5):576–8.PubMedView ArticleGoogle Scholar
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Forth ed. Washington, DC: American Psychiatric Association Press; 1994.Google Scholar
- American Psychiatric Association. Diagnostic and statistical manual of mental disorders. Fifth ed. Washington, DC: American Psychiatric Association Press; 2013.Google Scholar
- The ICD-10 Classification of Mental and Behavioural Disorders. Diagnostic Criteria for Research. The ICD-10 Classification of Mental and Behavioural Disorders 1992.Google Scholar
- Maercker A, Brewin CR, Bryant RA, Cloitre M, van Ommeren M, Jones LM, et al. Diagnosis and classification of disorders specifically associated with stress: proposals for ICD-11. World Psychiatry. 2013;12(3):198–206.PubMed CentralPubMedView ArticleGoogle Scholar
- Mollica RF. The trauma story: a phenomenological approach to the traumatic life experiences of refugee survivors. Psychiatry. 2001;64(1):60–3.PubMedGoogle Scholar
- Hollifield M, Warner TD, Lian N, Krakow B, Jenkins JH, Kesler J, et al. Measuring trauma and health status in refugees: a critical review. JAMA. 2002;288(5):611–21.PubMedView ArticleGoogle Scholar
- Mollica RF, Wyshak G, de Marneffe D, Khuon F, Lavelle J. Indochinese versions of the Hopkins Symptom Checklist-25: a screening instrument for the psychiatric care of refugees. Am J Psychiatry. 1987;144(4):497–500.PubMedView ArticleGoogle Scholar
- Malterud K. Qualitative research: standards, challenges, and guidelines. Lancet. 2001;358(9280):483–8.PubMedView ArticleGoogle Scholar
- Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care. 2007;19(6):349–57.PubMedView ArticleGoogle Scholar
- Semrau M, van Ommeren M, Blagescu M, Griekspoor A, Howard LM, Jordans M, et al. The development and psychometric properties of the Humanitarian Emergency Settings Perceived Needs (HESPER) Scale. Am J Public Health. 2012;102(10):e55–63.PubMed CentralPubMedView ArticleGoogle Scholar
- Babor TF, De la Fuente JR, Saunders J, Grant M: AUDIT—the alcohol use disorders identification test: guidelines for use in primary health care. AUDIT: The Alcohol Use Disorders Identification Test: Guidelines for Use in Primary Health Care 1989.Google Scholar
- Ustun TB, Chatterji S, Kostanjsek N, Rehm J, Kennedy C, Epping-Jordan J, et al. Developing the World Health Organization disability assessment schedule 2.0. Bull World Health Organ. 2010;88(11):815–23.PubMed CentralPubMedView ArticleGoogle Scholar
- Garcia-Moreno C, Watts C, Jansen H, Ellsberg M, Heise L. Responding to violence against women: WHO’s multicountry study on women’s health and domestic violence. Health Hum Rights. 2003;6(2):112–27.View ArticleGoogle Scholar
- Rees S, Silove D, Tay AK, Moses K. Human rights trauma and the mental health of West Papuan refugees resettled in Australia. Med J Aust. 2013;199(4):280.PubMedView ArticleGoogle Scholar
- Rees S, Silove D, Verdial T, Tam N, Savio E, Fonseca Z, et al. Intermittent explosive disorder amongst women in conflict affected Timor-Leste: associations with human rights trauma, ongoing violence, poverty, and injustice. PLoS One. 2013;8(8):e69207.PubMed CentralPubMedView ArticleGoogle Scholar
- Steel Z, Chey T, Silove D, Marnane C, Bryant RA, Van Ommeren M. Association of torture and other potentially traumatic events with mental health outcomes among populations exposed to mass conflict and displacement: A systematic review and meta-analysis. JAMA J Am Med Assoc. 2009;302(5):537–49.View ArticleGoogle Scholar
- Fazel M, Wheeler J, Danesh J. Prevalence of serious mental disorder in 7000 refugees resettled in western countries: a systematic review. Lancet. 2005;365(9467):1309–14.PubMedView ArticleGoogle Scholar
- Jones L, Asare JB, El Masri M, Mohanraj A, Sherief H, van Ommeren M. Severe mental disorders in complex emergencies. Lancet. 2009;374(9690):654–61.PubMedView ArticleGoogle Scholar
- Hinton D, Chau H, Nguyen L, Nguyen M, Pham T, Quinn S, et al. Panic disorder among Vietnamese refugees attending a psychiatric clinic: prevalence and subtypes. Gen Hosp Psychiatry. 2001;23(6):337–44.PubMed CentralPubMedView ArticleGoogle Scholar
- Beiser M, Fleming J. Meauring psychiatric disorder among Southeast Asian refugees. Psychol Med. 1986;16(3):627–39.PubMedView ArticleGoogle Scholar
- Van Ommeren M, Sharma B, Sharma GK, Komproe I, Cardeña E, De Jong JTVM. The relationship between somatic and PTSD symptoms among Bhutanese refugee torture survivors: examination of comorbidity with anxiety and depression. J Trauma Stress. 2002;15(5):415–21.PubMedView ArticleGoogle Scholar
- Momartin S, Silove D, Manicavasagar V, Steel Z. Complicated grief in Bosnian refugees: associations with posttraumatic stress disorder and depression. Compr Psychiatry. 2004;45(6):475–82.PubMedView ArticleGoogle Scholar
- Silove D, Momartin S, Marnane C, Steel Z, Manicavasagar V. Adult separation anxiety disorder among war-affected Bosnian refugees: comorbidity with PTSD and associations with dimensions of trauma. J Trauma Stress. 2010;23(1):169–72.PubMedGoogle Scholar
- Fincham D, Grimsrud A, Corrigall J, Williams DR, Seedat S, Stein DJ, et al. Intermittent explosive disorder in South Africa: prevalence, correlates and the role of traumatic exposures. Psychopathology. 2009;42(2):92–8.PubMed CentralPubMedView ArticleGoogle Scholar
- Fetters MD, Curry LA, Creswell JW. Achieving integration in mixed methods designs-principles and practices. Health Serv Res. 2013;48:2134.PubMed CentralPubMedView ArticleGoogle Scholar
- Muthen B. Contributions to factor analysis of dichotomous variables. Psychometrika. 1978;43:551–60.View ArticleGoogle Scholar
- Muthen B, Kaplan AS. A comparison of some methodologies for the factor analysis of non-normal Likert variables. Br J Math Stat Psychol. 1985;38:171–89.View ArticleGoogle Scholar
- Palmieri PA, Weathers FW, Difede J, King DW. Confirmatory factor analysis of the PTSD Checklist and the Clinician-Administered PTSD Scale in disaster workers exposed to the World Trade Center Ground Zero. J Abnorm Psychol. 2007;116(2):329–41.PubMedView ArticleGoogle Scholar
- Stein DJ, Rothbaum BO, Baldwin DS, Szumski A, Pedersen R, Davidson JR. A factor analysis of posttraumatic stress disorder symptoms using data pooled from two venlafaxine extended-release clinical trials. Brain Behav. 2013;3(6):738–46.PubMed CentralPubMedView ArticleGoogle Scholar
- Kline RB. Principles and practice of structural equation modeling. New York/London: The Guilford Press; 1998.Google Scholar
- Browne MW, Cudeck R. Alternative ways of assessing model fit. Soc Methods Res. 1993;21(2):230–58.View ArticleGoogle Scholar
- Barrett P. Structural equation modelling: Adjudging model fit. Pers Individ Dif. 2007;42:815–24.View ArticleGoogle Scholar
- Prigerson HG, Bridge J, Maciejewski PK, Beery LC, Rosenheck RA, Jacobs SC, et al. Influence of traumatic grief on suicidal ideation among young adults. Am J Psychiatry. 1999;156(12):1994–5.PubMedGoogle Scholar
- Prigerson HG, Shear MK, Jacobs SC, Reynolds Iii CF, Maciejewski PK, Davidson JRT. Consensus criteria for traumatic grief. A preliminary empirical test. Br J Psychiatry. 1999;174(JAN):67–73.PubMedView ArticleGoogle Scholar
- Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.PubMedView ArticleGoogle Scholar
- Shrout PE, Spitzer RL, Fleiss JL. Quantification of agreement in psychiatric diagnosis revisited. Arch Gen Psychiatry. 1987;44(2):172–7.PubMedView ArticleGoogle Scholar
- StataCorp. Stata Statistical Software: Release 13. College station: StataCorp LP; 2013.Google Scholar
- Muthén LK, Muthén BO. Mplus User’s Guide 1998. Sixth Edition. Los Angeles, CA: Muthén & MuthénGoogle Scholar
- Spitzer RL, Williams JBW, Gibbon M, First MB. The structured clinical interview for DSM-III-R (SCID): I: history, rationale, and description. Arch Gen Psychiatry. 1992;49(8):624–9.PubMedView ArticleGoogle Scholar
- DiCicco-Bloom B, Crabtree B. The qualitative research interview. Med Educ. 2006;40:314–21.PubMedView ArticleGoogle Scholar
- Rees S, Silove D. Sakit Hati: A state of chronic mental distress related to resentment and anger amongst West Papuan refugees exposed to persecution. Soc Sci Med. 2011;73(1):103–10.PubMedView ArticleGoogle Scholar
- Steel Z, Marnane C, Iranpour C, Chey T, Jackson JW, Patel V, et al. The global prevalence of common mental disorders: a systematic review and meta-analysis 1980–2013. Int J Epidemiol. 2014;43(2):476–93.PubMed CentralPubMedView ArticleGoogle Scholar
- Brundige E, King W, Vahali P, Vladek S, Yuan X. Indonesian Human Rights Abuses in West Papua: application of the Law of Genocide to the History of Indonesian Control. New Haven: Allard K. Lowenstein International Human Rights Clinic, Yale Law School; 2004.Google Scholar
- Rees S, Silove D, Kareth M. Dua sakit (double sick): Trauma and the settlement experiences of West Papuan refugees living in North Queensland. Australasian Psychiatry. 2009;17(SUPPL. 1):S128–32.PubMedView ArticleGoogle Scholar
- Griva M: Border Refugees in PNG. In.; 2009.Google Scholar
- Merdeka FWPP: Villages of West Papuan refugees burn on PNG border-humanitarian crsis developing. 2011.Google Scholar
- Silove D, Manicavasagar V, Mollica R, Thai M, Khiek D, Lavelle J, et al. Screening for depression and PTSD in a Cambodian population unaffected by war: comparing the Hopkins Symptom Checklist and Harvard Trauma Questionnaire with the structured clinical interview. J Nerv Ment Dis. 2007;195(2):152–7.PubMedView ArticleGoogle Scholar