How do you Measure Youth Empowerment?

PYE_Final_Impact2

You may be passionate about what you do, and you may be very familiar with the incredible results that your work has, but how do you communicate that to other people?

If you run an organization similar to PYE you will know that being able to communicate and quantify the impact you are having is very important. If you can come up with an accurate way of measuring and putting numbers to the work that you do then it helps you and everyone else to understand the benefits of the program. That can be a huge help when it comes to seeking funding. It also provides essential cues to suggest how you might improve on your work and expand your impact.

And yet – in the 15 years that I’ve been working in the youth development and non-profit sector the question remains – how do we effectively measure social impact? Is it with qualitative or quantitative evidence? How do you measure impact on beneficiaries over time? How do you isolate one particular piece of the impact from what are often a variety of other factors that influence social change?

Measuring impact is even more complicated for organizations like ours who are trying to measure abstract amounts such as the social impact of the arts. We are trying to measure intangible things like changes in attitude and behavior that often unfold over a long period of time – but that we know are fundamental in the process in laying the foundation for long-term social change.

There is a sizable body of research that proves the positive social benefits of participating in arts-based activities. These and other programs that use arts-based approaches develop key skills that people, particularly youth, need to survive in a fast-changing world – often called “21st Century Skills”. Most of the skills are connected to social and emotional intelligence – empathy, confidence, motivation just to name a few. I have included links to a few of the major studies and books below.

Uma Sekar works with our India partner organization Dream a Dream and has been responsible for collecting information about their impact. She says “Dream A Dream develops Life Skills in young people from vulnerable backgrounds using Football and Creative Arts as mediums. When we started developing an Impact Assessment framework, the challenge started with which Life Skills we could measure. We identified 5 observable and measurable Life Skills and developed a simple observational tool that can be easily implemented on the field by our Life Skills facilitators. We also developed a training module to overcome the observational bias that could arise from the facilitators. From our experience, once we identified the observable parameters from the field, the measurement became easier.”

Uma has some great advice for people who want to follow the same methods. “Define clear outcomes for the programs. Next, develop a simple tool, implementable with minimal training. We have found that a simple half-day training equips facilitators to measure impact according to our parameters. Always remember to identify and account for biases that are likely to arise from the tool too. For example, if it’s an observational tool, identify if there could be a bias in the measurement from the observer, and correct it through the training or re-look at the tool.”

At PYE, we decided to take impact measuring one step further and have just completed a year and a half long independent evaluation of our particular approach, an arts-based youth empowerment approach called the Creative Community Model. The evaluation was particularly complex for us given the international nature of our work. It took place in 8 countries, on 5 continents. Our evaluators used a mix of surveys, in-depth interviews, focus groups and observations. While the evaluation provides some really important learning, we know that these data collection methods only tell part of the story.

Download the PYE Evaluation Report here.

We express ourselves in a whole range of ways that may actually provide some of these missing pieces – an artistic performance, a short youth-produced video for example. But how do we weave this into an evaluation process that translates into concrete measures of impact?

Arts based evaluation is a growing field of practice that aims to incorporate arts practices into the evaluation process. Take a look at this fascinating downloadable toolkit which was produced by ArtReach Toronto and which includes some really useful advice and ideas.

We need to work collectively to improve our understanding of how to use arts-based evaluation methods so that we have another important dimension to the way we evaluate social change programs.

Have any of you faced similar difficulties? Have you come up with any innovative solutions? We would love to hear more from you, particularly if you are investigating arts-based evaluation models.

Resources:

Use or Ornament: the social impact of participation in the arts. This study helped to name and develop our understanding of the sector’s contribution to social change which still holds true today.

A recent UCLA study of over 25,000 youth over 12 years by James Katerall perform their peers in every category – academics and well as life skills. Check it out – there are some videos as well.

An IBM study of over 1,500 CEOs from around the world cited creativity as the most important factor for Future Success. These are exactly the kinds of skills that participation in the arts, especially from a young age, develops.

Another study that was carried out in Australia and that examined the social benefits of being involved with the arts.

For ideas and inspiration around how to document the impact of arts programs take a look at this study on the economic and social impact of the arts.

If you’re looking for a strong argument for moving away from a standard evaluation model and towards an arts-based approach then take a look at this report from Eleonora Belfiore and Oliver Bennett.

In this report you will find an argument for community music making as a tool for social change.

Can the arts really be used as a tool for evaluation? The authors of this study argue that using the creative arts in the evaluation process evokes different ways of knowing and understanding the values of a program.

 

 

Enjoyed that? You might be interested in these articles:


Screen shot 2013-09-03 at 17.34.45
What is a social artist?
A social artist is someone who is part artist, part social-change agent, part facilitator and part visionary. The social artist draws on artistic practice, group dynamics and creative ideas to facilitate change.
Screen shot 2013-12-20 at 15.34.18What drives a social artist?
Watch this video interview with PYE’s Senior International Trainer, Nadia Chaney to hear her thoughts on social artistry, what it is and why it is so important in today’s communities.
Screen shot 2013-12-27 at 11.23.00
How do you start a social art organization?
To find out more about setting up your own organization we spoke to Ed Wade Martins, co-director of MovingSounds.org, an organization that uses creativity to raise awareness of today’s social and environmental issues.

 

3 Responses to How do you Measure Youth Empowerment?

  • Im really happy to find this webpage, its like everything i have been feeling deep in my heart lately, writen in simple yet powerful words. I am doing my best around here in sao paulo.. seeing change in peoples face, but the city is still full of homeless kids, and we need global help for youth! People all crave for a more real life! Sending good vibes to you all around the world!
    Hugs, Peace, Love, Namaste!

  • Hi Manuela, your work sounds fascinating. You should consider joining us for our live web chat on Thursday at 6pm London time. We will be talking about issues relating to working with young people and it’s a great chance to meet and talk with other youth facilitators from our global community. The talk takes place here: https://www.facebook.com/groups/491719184185338/

  • It is very impressive to have creative arts in the process of evaluation.Yes we know the process of converting subjective data to variable indeed a big task.It is bound to have bias but we should see how best we can minimize.You may have to design more experimeents and collate the results for further fine tuning.Good attempt BY UMA

Leave a Reply

Your email address will not be published.