The Delphi method
First developed to examine the impact of technology on warfare in 1950s, it has been defined as a type of consensus method, often using a non-face-to-face technique, for structuring a group communication process, allowing a group of individuals to deal with a complex problem [36, 37]. It has been proven to be a feasible method when developing culturally-adapted interventions in mental health for minority and diverse groups [38, 39]. This method involves questionnaires being sent out online to a group of experts, where responses are anonymous. The Delphi technique involves a number of iterations before consensus is reached. Feedback from the expert group as a whole is given in order to assist panellists to assess their ratings against the group feedback.
The development of these guidelines involved 8 steps: (1) framing a research question, (2) formation of the panel (3) determining the expert panel size, (4) constructing the questionnaire, (5) information provided to panel members to aid their judgements, (6) administering the questionnaire, (7) analysing rounds and providing feedback to the panel and (8) reporting results.
Research question
This study aimed to respond to the following questions: (1) what culturally appropriate mental health first aid strategies have been proposed for assisting Iraqi refugees in mental health crises or in the development of mental health problems? and (2) which of these are considered by experts in the field to be the most appropriate strategies for assisting this group?
Formation of the panel
The expert panel was composed of professionals who meet the following selection criteria: qualified as a psychologist, social worker, psychiatrist, general practitioner or mental health professional; and have worked in refugee mental health part-time or full-time for at least 4 years and have experience working with Iraqi refugees. Potential participants were considered to have sufficient expertise if they have authored material in Iraqi refugee mental health and/or were known as experienced professionals through different networks, such refugee health networks in Australia, professional associations or community involvement.
Potential participants were identified and selected to participate in this project through their involvement with professional colleges and associations, universities and research centres, refugee health networks and refugee health services in New South Wales, South Australia, Australian Capital Territory and Victoria, Australia. Expert panellists were contacted via multiple sources. The principal researcher (MGU) advertised the study in various professional associations such as The Australian Psychological Society, The Australian Association of Social Workers, and The Royal Australian and New Zealand College of Psychiatrists. In addition, an email invitation was circulated to those involved with key government and non-government refugee health organisations and university centres. In the invitation to participate, professionals were asked to share the study invitation with colleagues who they felt would be appropriate panel members. This research was granted human research ethics approval by the Western Sydney University Human Research Ethics Committee (H11054). Consent and participation information sheet forms were sent by email or post to participants. Signed consent forms were collected by the principal researcher (MGU) before the study commencement. Participants were reimbursed for their time with a book voucher of $50.
Determining expert panel size
A panel size of 23 has been found to yield stable results in a simulation study [40]. Our aim was for a minimum of 30 members in the panel, in order to allow for drop-outs. However, a total of 16 experts were recruited.
Constructing the questionnaire
A systematic literature search was carried out to identify available information about how to provide culturally-appropriate mental health first aid to Iraqi refugees experiencing mental health problems or crises. This search focused in three main sources. A comprehensive search of key terms was carried out using Google search engines (http://www.google.com.ca, http://www.google.com.uk, http://www.google.com.au, http://www.google.com). The searches included various combinations of search terms, for example, ‘mental health in Iraqi refugees’ AND ‘early mental health intervention for Iraqi refugees’ AND ‘first aid for Iraqi refugees’ AND ‘cultural considerations for Iraqi refugee mental health’. Lists of the first 50 websites for each set of terms were reviewed. Any links of interest appearing in the websites were followed, as appropriate. A second strategy was academic journal search using Medline, PsycINFO, CINAHL, PILOTS, Scopus and Cochrane databases to identify relevant published articles. A third strategy was searching for any printed books available in Australian-wide libraries using Trove Australia, Libraries Australia and Australian Libraries Getaway (ALG) search catalogues. Information from relevant mental health and refugee websites such as Transcultural Mental Health, Australian Psychological Society, Mental Health Australia and United Nations High Commissioner for Refugees (UNHCR) were also reviewed.
In order to develop the first round questionnaire, information obtained from the systematic literature review was divided into common sections or categories and were written as strategies or actions a first aider should follow. For example, in a report entitled ‘The Mental and Physical Health of Recent Iraqi refugees’ it was stated that ‘talking about mental health openly is often stigmatised by Iraqis’. This statement was included in the first round questionnaire under the category ‘stigma associated with mental health problems’ as a first aid action as follows: ‘The first aider should be aware that talking about mental health problems openly is often stigmatised by the Iraqi community’.
The process of phrasing and drafting statements into a questionnaire format involved a team of authors, who were experts in the Delphi method (AJ) or in transcultural mental health (MGU, SSY, YS). The team drafted the actions and attempted to remain as close to the original literature as possible. Statements were only modified in order to ensure format consistency or content comprehension. Several meetings were held to discuss the items before the final round 1 questionnaire was ready to be sent. In total, three questionnaires were created and presented to experts in a 3-round format.
Information provided to experts to aid their responses
Questionnaires were accompanied by definitions of key concepts that were thought to be relevant for experts when making their ratings. These definitions included the role of a ‘first aider’, the ‘person’ defined as an ‘Iraqi refugee’ and ‘people’ defined as ‘Iraqi people in general’, as well as definitions of ‘mental Illness’, ‘cross-cultural communication’ and ‘cultural awareness’.
Administering the questionnaire and analysing rounds
The majority of participants chose to complete the questionnaire (round 1) online and a link was sent hosted by http://www.SurveyMonkey.com to their nominated email addresses. A small number of experts preferred to complete a hard copy questionnaire. Experts responded to each statement by rating how essential the first aid action statements were to the development of guidelines on how to provide mental health first aid to Iraqi refugees experiencing mental health problems or crises. The questionnaire involved a 5-point Likert scale composed of ‘Essential’, ‘Important’, ‘Depends/don’t know’, ‘Unimportant’ and ‘Should not be included’ as response options.
Once the panel rating was completed, actions were categorized based of level of consensus following the procedure used in a previous Delphi study [23]:
-
1.
If between 90 and 100 % of panel members rated a statement as either ‘Essential’ or ‘Important’, the statement was endorsed as a guideline.
-
2.
If between 80 and 89 % of panel members rated a statement as either ‘Essential’ or ‘Important’, then the statement was entered into a second questionnaire to be rerated (second round).
-
3.
If neither of the above conditions were met then the statement was excluded from the guidelines.
In round 1, in addition to rating statements, participants were encouraged to provide feedback on any ambiguity in the statements presented and suggest any new first aid strategies that were not included in the content of the questionnaire. Submitted comments were drafted into statements and then presented to the working group, who tried to ensure comprehensibility and consistency. Those statements that were assessed by the working group to be original were included as new actions in a second round questionnaire for experts to rate.
Once categorisation was completed, participants were sent a report that included endorsed, rejected and new statements to be rated along with those statements that needed to be re-rated in the next round questionnaire. The statements to be re-rated were presented with the group percentages for each possible rating, and also with the individual’s response for the expert to compare their own rating to the group response. By providing this report to the experts, researchers aimed to aid participants’ re-rating.
In round 2, the same criteria were followed for endorsing, excluding and re-rating statements. However, those statements that failed to meet the criteria for endorsement in the second round were then excluded from the guidelines. Only those new statements that were introduced in round 2, and afterward fell into the re-rate category, were in a third and final round.