Tuesday, June 10, 2014

Using Creative Methods to Evaluate Youth Programs

By Kelly Washburn & Julie Carpineto 


We are the evaluators of an urban high school-based teen pregnancy prevention program in Everett, MA called “Empowering Youth in our Community”. Our partner, whom ICH has worked with for 9 years, is the Family Planning and Adolescent Community Services Division at the Cambridge Health Alliance.  

Each year, the program is required to conduct a site-specific evaluation. In previous years, we conducted focus groups with the high school students involved in the program to meet this requirement. However, two years ago - while reviewing the qualitative data from those groups - we realized the data was not that different than previous years. This realization led to discussions on how to conduct an evaluation to find new information, while continuing to ensure that the students could play an active, participatory role.  Together with the program partners, we developed a three-series collage project, drawing from the principles of Photovoice and other arts-based evaluation techniques. We aimed to engage participants in a discussion of a major theme addressed in the program: healthy relationships.

Developing the Evaluation Question
The first step in our process was to determine our evaluation question. We collaborated with the program coordinator to come up with “What does a healthy relationship look like to you?”  This question was chosen for a couple of reasons, the first being the timing of the evaluation with the program sessions. Students had recently completed a series of classroom sessions focused on healthy and unhealthy relationships, and we were interested in understanding how those sessions shaped their vision of healthy relationships. Furthermore, data from focus groups conducted in previous years revealed that the relationship sessions stuck out to students and had the most impact on them personally.  By asking this question, we were able to gain a more in-depth understanding of students’ opinions about this topic.

Collage Implementation
Over 90 high school students, in five separate classes, were asked to individually pull images from popular magazines that, from their perspective, represented healthy relationships. We left the type of relationship up to the individual students to think about, and many discussed a combination of romantic, friend, and family relationships. They were then led in large group discussions about why they chose their images.  Looking at their chosen images, students were asked to reflect on their thoughts, feelings, perceptions and experiences related to the different types of relationships. 

Afterwards, students were asked to work in small groups to combine their images to form a larger, collective collage poster. These posters served as visual representations of the students’ perceptions of healthy relationships.

Next Steps
The final phase of this project will include individual interviews with participating students to gather additional information about the collage process and their overall experience as program participants.

Lessons Learned:  
  • Magazine selection should be a part of initial planning. Examples of the magazines we used included: Glamour, People Magazine, Sports Illustrated, and Marie Claire.
  • Chose a diverse range of magazines. Make sure your selection of images is representative of your population!

Hot tip:
  • This project can be done with a small budget, limited time and with almost any participant population.
  • This project is a fun and creative way to engage young people in evaluation.

Cool Trick:
  •  If available, take notes from the large discussion on a white board. This serves as another visual representation and can help students develop their collages.  



ICH staff have expertise in a variety of program evaluation methods and strive to tailor each evaluation to meet the needs of individual clients and programs. Learn more about ICH's Program Evaluation services.



  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.