Skip to main content

Table 5 Evaluation data from face-to-face programs

From: Review of Australian initiatives to reduce stigma towards people with complex mental illness: what exists and what works?

Program name

Experimental design

Study sample

Sample size

Measures

Outcomes

Batyr [55]

RCT

N/R

N/R

N/R

In 2017, Macquarie University conducted a study into the effectiveness of the batyr@school program, looking at stigma reduction and help-seeking. The biggest two findings were

1. The program was successful in reducing stigma that young people had towards others experiencing mental health issues

2. The program lead to an increase in attitudes and intentions towards seeking help from professional sources for mental health issues and suicidal thoughts

The findings were maintained for at least 3 months after the program

BPD Community Information Nights

Post feedback

N/R

N/R

N/R

Usefulness of the event and information: 99% find them useful

Personal confidence and understanding: 83% said its better

Feeling more supported: 80% said yes

Help personal ability to build relationships: 92% yes

Do you expect to use knowledge gained: 97% said yes

BPD Community Family & Friends Group

N/R

N/R

N/R

N/R

From program authors: “A ‘formal’ evaluation occurred in 2017 which lead to the evolution of the program of today. Monthly evaluations of the program are conducted”

Collaborative Recovery Training Program (CRTP) [25]

Uncontrolled trial (pre/post)

Mental health workers from government and NGO organisations in eastern Australia

75 with data to analyse out of 103

Staff Attitudes to Recovery Scale (STARS; Crowe et al., 2006) assesses hopeful attitudes regarding consumers’ recovery possibilities. Therapeutic Optimism Scale assesses treatment expectancies

There was an improvement in STARS pre-post (d = 0.87) and therapeutic optimism scores pre-post (d = 0.78). MANOVA p = .02

Compeer (The Friendship Program) [32]

Survey only

Volunteers from the Compeer program

72 analysed

Social Distance Scale, Affect Scale, Dangerousness Scale, Match Bond (measures friendship strength)

A stronger relationship between the Compeer volunteer and friend was associated with lower levels of stigma: social distance (p = .001), Affect (p = .015), Dangerousness (p = .028). No relationship between time spent in relationship and stigma, suggesting it is quality of contact rather than length of contact that reduces stigma

Journey to Recovery [37]

Uncontrolled trial (pre/post)

Carers of person with psychosis

15

6 questions on perceived knowledge: understanding of psychosis, understanding of recovery, knowledge of medication, relapse prevention, understanding of links between substance use and psychosis, plus qualitative feedback

Significant improvements in perceived knowledge of psychosis (p = .001) and recovery (p = .008) pre to post. Qualitative feedback was that participants valued support, felt a reduced sense of isolation, felt a sense of collective experience, and appreciated the opportunity to ventilate and feel heard by peers

Journey to Recovery [35]

Qualitative interviews

(1) carers who continually attended; (2) carers who attended once only; (3) carers who never attended; (4) case managers and (5) early psychosis clinicians

10 carers, 8 clinicians

7 qualitative questions designed to illicit positive and critical information and suggestions for the future direction of the group

Carers reported Reduced isolation, sense of Collective Experience, Opportunity to vent and feel heard, Reduced stigma and shame, Increased knowledge about mental illness, Enhanced skills in supporting the person experiencing mental illness. The group enabled “helping us to communicate as a family again,” “learning how to communicate and describe what mental illness is to our children,” and “passing it on into the community to help others” (reduced stigma and shame)

Journey to Recovery (inpatient version) [36]

Qualitative interviews 6 months later

Carers of person with psychosis

27

14-item interview questionnaire on timeliness, correct people invited, sufficient time, useful information (written, oral, DVD, booklet, fact sheets), support offered, family use of information, follow-up in community, and improvement suggestions

The session and materials were perceived as helpful. Findings in the present study suggest that early psychosis carers are open to receiving psychoeducation at first contact with psychiatric services

Journey to Recovery [38]

Uncontrolled trial (pre/post)

Families of people with early psychosis

17

6 questions on perceived knowledge: understanding of psychosis, understanding of recovery, knowledge of medication, relapse prevention, understanding of links between substance use and psychosis, plus qualitative feedback

Significant improvements in perceived knowledge of psychosis and recovery pre to post (ps < .001). Qualitative feedback was that participants valued peer support and support from session facilitators, felt a reduction in a sense of isolation, felt a sense of collective or similar experiences and there was an appreciation of the opportunity to ventilate feelings and be heard by peers who understood the challenges faced

Kookaburra Kids camps and Activity Days

N/R

N/R

N/R

N/R

From program authors: “Evidence of impact; (changes in MHL and help seeking) currently continuing with published research to follow 2020”

Managing Mental Health Emergencies short course [27]

Repeated cross-sectional surveys (pre/post with some follow-up interviews 3-6mth)

Rural and remote healthcare providers (nurses, Aboriginal health workers, other allied health)

N = 456 at pre, N = 163 post workshop, N = 44 interviews

Survey: 7 questions ranking perceived skills. No information about interview guide

Perceived skills improved in differentiating between psychosis and substance intoxication (p < .001), assessing psychotic symptoms (p < .001), communicating effectively with people with mental health problem (p < .001), assessing suicide risk (p < .001). Almost all interview participants felt they had changed their attitude towards mental health clients as a result of the course, as many recognised that had been stereotyping and stigmatising clients. Participants talked about their increased patience when listening to acutely unwell clients

Mental Health 101 [42]

Controlled trial (pre/post). Comparison condition was non-participating schools

High school students

457

Two vignettes on stigma which were followed by four questions about their attitudes towards the person described in the vignette and four social distance questions. Multiple-choice questions and open-ended questions on knowledge of mental health and mental illness, and the General Intentions to Seek Help Questionnaire

The intervention group had lower mean stigma scores (p = .000) and greater knowledge on each of the knowledge questions (all p < .001), and increased help-seeking intentions (p = .000) compared to the control group at post-test. Further analysis revealed a significant effect of the intervention on reducing stigma after the effect of knowledge was removed (p < .001)

Qualitative responses revealed many students were deeply touched by the personal stories of presenters, that they were a powerful medium, and made the impact of mental illness tangible and encouraged the realisation that people with mental illness were just ‘ordinary people with extraordinary stories’

Mental Health 101 [41]

Qualitative interviews

Volunteer consumer educators

10

Semi-structured interview focused on the benefits and costs related to being in an advocacy/educator role and its impact on recovery from the experience of mental illness and treatment

Reports on the benefits and costs of being a lived experience educator in the MIE-ACT program. Benefits identified were the value of peer support where educators felt a unique sense of acceptance and understanding from their peers, gaining a sense of purpose and personal meaning from the personal satisfaction of educating others, and the impact and therapeutic effect broadcasting had in reducing self-stigma and assisting in positive identify development. Costs reported were feeling ‘raw’ or vulnerable during or after presenting and a fear of being stigmatised as a result of presenting

Mental Health 101 [42]

Post surveys

High school students (93.3%)

N/R, 90.7% of learners are surveyed after the program

Satisfaction ratings, perceived knowledge

89.7% of learners rated the program as either extremely of significantly informative

97.2% of learners state that the programs had increased their understanding of mental health

Mental Health Awareness

Post course evaluations of all programs

N/R

N/R

N/R

N/R

Mental Health First Aid [43]

RCT. Comparison condition was waitlist

Nursing students

181 (int = 92, control = 89)

Social Distance Scale, Personal Stigma Scale, Perceived Stigma Scale (all for depression vignette)

Outcomes are not relevant as not for schizophrenia/psychosis/bipolar disorder/personality disorder

Mental Health First Aid [44]

RCT. Comparison condition was waitlist

Adult members of community

178 (int = 90, con = 88)

Social Distance Scale, Personal Stigma Scale (depression and schizophrenia)

For schizophrenia, improvements pre-post in personal stigma (p < .001) and social distance (p < .001). Sig improvements at 6-mth FU: personal stigma (p < .001) and social distance (p < .01)

Mental Health First Aid [45]

RCT. Comparison condition was waitlist

High school teachers

423 (int = 283, con = 140)

Personal Stigma Scale for depression only

Outcomes are not relevant as not for schizophrenia/psychosis/bipolar disorder/personality disorder

Mental Health First Aid [46]

Uncontrolled trial (pre/post/6mth FU)

Adult members of community

246

Personal Stigma Scale and Perceived Stigma Scale (for depression and schizophrenia)

Improvements in beliefs about dangerousness (p = .005), unpredictability (p < .001), and willingness to disclose (p = .005) pre to post for schizophrenia. Changes in stigmatising attitudes about schizophrenia from pre-test to follow-up were only significant for disagreement about dangerousness (from 33.1% to 48.5%, p = 0.008). No significant change in perceived stigma

Mental Health First Aid [47]

Uncontrolled trial (pre/post)

Members of the Chinese community in Melbourne

108 (84 analysed)

Social Distance Scale (towards depression and schizophrenia vignettes)

Social distance for schizophrenia sig improved pre-post (p = .005)

Mental Health First Aid [48]

Uncontrolled trial (pre/post)

Members of the Vietnamese community in Melbourne

114

Personal Stigma Scale and Perceived Stigma Scale (for depression and schizophrenia)

Significant improvement in some personal stigma items for early schizophrenia (4 of 9) and chronic schizophrenia (3 of 9)

Mental Health First Aid [49]

Uncontrolled trial (pre/post/6mth FU)

Workers and volunteers of organisations working in multicultural communities

458

Social Distance Scale, Personal Stigma Scale, Perceived Stigma Scale (towards depression and schizophrenia vignettes)

Pre-post sig improvements in social distance (p < .001), personal stigma (p < .001) and perceived stigma (p < .001) for schizophrenia. Stigma data not collected at follow-up

Mental Health First Aid [50]

RCT. Comparison condition was Red Cross First Aid training

Australian parents of teenagers

384 (int = 201, con = 183)

Social Distance Scale, Personal Stigma Scale (Weak not sick, Dangerous/unpredictable) towards psychosis vignette

No significant changes in stigma outcomes in parents at 1-year and 2-year follow-up

Mental Health First Aid [53]

Controlled trial

Pharmacy students

272 (int = 60, con = 212)

Social Distance Scale for schizophrenia

Reduced social distance over time compared to control, p < .001

Mental Health First Aid [51]

RCT

Public servants

608 (int elearning = 199, int blended = 199, con = 210)

Social Distance Scale and Personal Stigma Scale (both for depression and PTSD)

Outcomes are not relevant as not for schizophrenia/psychosis/bipolar disorder/personality disorder

Mental Health First Aid [52]

Controlled trial (pre/post/3mthFU)

Chinese international students studying in Melbourne

202 (int = 102, con = 100)

Personal Attributes Scale, Social Distance Scale (both for depression and schizophrenia)

Significant improvements over time for social distance towards schizophrenia (p = .021). No sig change in perceived dangerousness or perceived dependency

Mental Health First Aid

Qualitative focus groups

Mental health first aid instructors, and members of the Aboriginal and Torres Strait Islander community

N/R

N/R

N/R

Mental Health Intervention Team (MHIT) training [28]

Controlled trial (pre/post/18 month FU). Comparison condition was officers who were not trained

NSW police officers, NSW health staff

260 (trained = 186, not trained = 74). Presurvey = 112, post = 32, FU = 42)

Levels of confidence, self-reported behaviour change,

The MHIT training led to an increase in confidence in dealing with jobs involving individuals with a mental health problem, or a drug induced psychosis at post and follow-up (ps < .001). Qualitative data supports the notion that the MHIT training led to an increase use of de-escalation techniques, with officers reporting that an increased understanding of mental health meant they were better able to deal with the situation. Qualitative data from NSW Health staff working specifically in mental health were uniform in their perception of an improved understanding about mental health amongst the police officers they engaged with when a scheduled consumer was delivered to their care, and noted the flow-on effect that officers ‘ increased understanding of mental health had on their engagement with consumers

Mental Health Intervention Team (MHIT) training (brief version) [30]

Controlled trial (post only). Comparison condition was those who have not completed the training

Emergency call operators (communications officers)

91 (trained = 18, not trained = 73)

Community Attitudes Towards Mental Illness (CAMI); Social Distance Scale

Findings showed no difference in stigma between those who had undergone CIT training and those who had not

My Recovery

Qualitative interviews

Lived experience adult members of the community

30

(Presurvey = 14, post = 16)

N/R

N/R

Recovery Camp [17]

Controlled trial (pre/post). Comparison condition was traditional nursing placements (inpatient and community mental health)

3rd year nursing students

50 (Recovery Camp = 23, comparison = 27)

Preplacement Survey, includes items on Negative stereotypes and Anxiety surrounding mental illness

Sig greater reduction in anxiety (p = .001) and negative stereotyping (.015) in intervention group compared to control. In particular, decreased endorsement of statements that describe mental illness sufferers as unpredictable, incapable and dangerous in the Recovery Camp group

Recovery Camp [15]

Controlled trial (pre/post). Comparison condition was traditional nursing placements (inpatient and community mental health)

3rd year nursing students

79 (Recovery Camp = 40, comparison = 39)

Social Distance Scale

Sig reductions in social distance in the Recovery Camp group pre to post, and pre to follow-up. No sig reduction in social distance in comparison group

Recovery Camp [16]

Qualitative analysis of written reflections

3rd year nursing students

20

4 critical reflections during their time at Recovery Camp

Students reported the placement was a unique, positive and educational mental health nursing placement. It allowed for the application of knowledge, consolidation of skills, experience of recovery-orientated care, development of therapeutic relationships and learning from people with a lived experience of mental illness about mental illness and related treatments. Recovery Camp was transformative in terms of learning the strengths of people with a lived experience of mental illness, acknowledging previously held fears and anxieties, and establishing future plans for practice

Recovery Camp [14]

Qualitative analysis of written reflections

3rd year nursing students

56 (28 students, 27 LE)

Content analysis of student reflective quotes

Reflective quotes of students’ experiences showed their understanding and empathy towards people with a mental illness increased, they developed practical skills, appreciated and learnt how to establish and maintain therapeutic relationships, and discovered the importance of lived experience

Recovery for mental health nursing practice [18]

Qualitative interviews

Nursing students

12

Asked to describe their views and experiences being taught by a person with LE, positives, negatives, and how their nursing practice would be influenced

Students were positive and reported an enhanced self-awareness and greater understanding of the person behind the diagnostic label and their experience. It encouraged them to question their attitudes and prejudices

Recovery for mental health nursing practice [19]

Controlled trial (pre/post). Comparison condition was traditional mental health nursing subject taught by nurse academic

Nursing students

171 (intervention = 110, comparison = 61)

Mental Health Consumer Participation Questionnaire

Both courses improved some aspects of attitudes towards consumer participation in mental health care

Recovery for mental health nursing practice [21]

Controlled trial (pre/post). Comparison condition was traditional mental health nursing subject taught by nurse academic

Nursing students

201 (intervention = 131, comparison = 70)

Scale measuring Anxiety surrounding mental illness and Negative stereotypes

The lived experience-led course showed sig decrease in negative stereotypes (p < .001). Reduction in anxiety was not sig (p = .04—p = .01 set as significance level). Reductions in comparison group were not significant (p = .02 for anxiety and p = .06 for stereotypes)

Recovery for mental health nursing practice [20]

Qualitative interviews

Lived experience educators

12

Not clear

Reports on the experience of being a lived experience educator in nursing programs. Themes identified were facing fear, demystifying mental illness and issues of power

Remind Training and Education [23]

Uncontrolled trial (pre/post/12 mth FU)

Pharmacy students

178

Questionnaire with 8 items on stigma towards schizophrenia, reported as individual items. Also focus groups with 11 participants

Significant decreases in stigma at 6-week post and follow-up for 5 out of 8 items relating to schizophrenia (p < .05) (unpredictable; have different feelings; are difficult to talk to; should pull themselves together; are not a danger to others; have themselves to blame). Focus groups showed that the intervention made mental illness more real to them and increased insight, enabled them to see consumers are able to lead a normal life despite their illness, removed some pre-conceived ideas they had about consumers, realised that pharmacists need to be non-judgemental in their interactions with consumers

Remind Training and Education [24]

Separate focus groups with students and consumers

Pharmacy students and consumer educators

23 (11 students, 12 consumer educators)

Impact of the training on students and goals, challenges and benefits of mental health consumer educators providing education to health professional students

All consumers nominated reducing stigma as a primary reason for becoming an educator. The contact the students had with the MHCE provided them with a greater insight into what it is like to suffer from psychotic symptoms and the challenges people face in managing their mental illness. Students reported a change in how they interacted with patients (pharmacy practice) and that their confidence had improved. Consumer educators felt empowered by their participation, reported improved confidence and public speaking skills, and enjoyed the social contact with other consumers. Some reported that fear of social situations was a challenge to fulfil their role

Remind Training and Education [22]

Controlled trial (pre/post). Comparison condition was film-based contact

Pharmacy students

244 (direct contact = 122, indirect contact = 122) were analysed

Social Distance Scale for mental illness [7items]; Attribution Questionnaire [6items]; 8 items on specific stigmatising beliefs towards schizophrenia

Both interventions showed similar reductions in Social Distance scores. The training had greater effect for 5 of 6 Attribution Questionnaire items and 5 of 8 stigma items. Both interventions showed reductions in stigma though

Richmond Fellowship Residential Accommodation

N/R

N/R

N/R

N/R

From program authors: “Ongoing evaluation including DREEM, feedback through the consumer advisory council, and ongoing feedback provided by consumers, families and friends”

Rotary mental health awareness forums [64]

Post program feedback forms

Attendees at the forums

6548

N/R

Perceptions of good understanding of mental illness increased from 63 to 76% following the forums 64% of attendees had a good to very good awareness of what can be done to reduce the stigma of mental illness following the forums

SPEAK UP! Stay ChatTY [56, 57]

Post-session feedback is collected from participants from the Schools Program, Sports Program, Community Presentation and Mitch’s lived experience story. Pre-post data (not linked) is also available for Schools Program

Athletes from sporting clubs in Tasmania (Sports program). Students, teachers, parents from participating schools (Schools Program)

1239 (Sports program). Approx 1750 students (Schools Program)

Perceived knowledge and attitudes

Sports Program: Before the session, 818 (66%) athletes reported they knew ‘a bit’ about mental health, whereas after the session, 896 (72%) athletes stated they now know ‘a lot’. Likewise, before the session 673 (54%) athletes reported they knew ‘a bit’ about stigmatising signs of mental illness, however, after the session 869 (70%) athletes knew ‘a lot’ about stigmatising signs of mental illness

Schools Program: Following the session, a majority (91.5%) felt more comfortable talking about mental health. There were also increases in perceived knowledge about mental health pre to post (A bit or a lot 81.6% to 97.0%) and perceived recognition of the signs of mental illness (A bit or a lot 63.0% to 96.6%)

The Dax Centre—Exhibition Program [61]

Post-feedback only

Exhibition visitors (86.4% were 16—17 year-old school students)

10,000

Response card with three statements with Likert scale response (Agree to Disagree) and brief written comments on any aspect of the person’s visit

Over 90% of respondents agreed that the exhibition helped them [1] gain a better understanding of mental illness, [2] gain a more sympathetic understanding of the suffering of people with mental illness; and [3] appreciate the ability and creativity of people with mental illness. These results were supported by the written feedback

The Station [31]

Qualitative interviews

Staff and members of a consumer-driven community mental health service

25

Interviews focused on The Station’s role in assisting recovery from mental illness, the limitations and strengths of the program, and relationships with the mental health system

Consumers reported feeling accepted and nurtured which increased feelings of empowerment and led to a greater belief in oneself from participating in the Station’s activities. Carers, consumers and volunteers all reported similarly of the positive impact of The Station on their lives. People who volunteer at The Station gain a sense of community and family, ‘time out’ and an opportunity to learn new skills and meet new people

  1. N/R Not Reported