Tuesday, June 10, 2014

Using Creative Methods to Evaluate Youth Programs

By Kelly Washburn & Julie Carpineto 


We are the evaluators of an urban high school-based teen pregnancy prevention program in Everett, MA called “Empowering Youth in our Community”. Our partner, whom ICH has worked with for 9 years, is the Family Planning and Adolescent Community Services Division at the Cambridge Health Alliance.  

Each year, the program is required to conduct a site-specific evaluation. In previous years, we conducted focus groups with the high school students involved in the program to meet this requirement. However, two years ago - while reviewing the qualitative data from those groups - we realized the data was not that different than previous years. This realization led to discussions on how to conduct an evaluation to find new information, while continuing to ensure that the students could play an active, participatory role.  Together with the program partners, we developed a three-series collage project, drawing from the principles of Photovoice and other arts-based evaluation techniques. We aimed to engage participants in a discussion of a major theme addressed in the program: healthy relationships.

Developing the Evaluation Question
The first step in our process was to determine our evaluation question. We collaborated with the program coordinator to come up with “What does a healthy relationship look like to you?”  This question was chosen for a couple of reasons, the first being the timing of the evaluation with the program sessions. Students had recently completed a series of classroom sessions focused on healthy and unhealthy relationships, and we were interested in understanding how those sessions shaped their vision of healthy relationships. Furthermore, data from focus groups conducted in previous years revealed that the relationship sessions stuck out to students and had the most impact on them personally.  By asking this question, we were able to gain a more in-depth understanding of students’ opinions about this topic.

Collage Implementation
Over 90 high school students, in five separate classes, were asked to individually pull images from popular magazines that, from their perspective, represented healthy relationships. We left the type of relationship up to the individual students to think about, and many discussed a combination of romantic, friend, and family relationships. They were then led in large group discussions about why they chose their images.  Looking at their chosen images, students were asked to reflect on their thoughts, feelings, perceptions and experiences related to the different types of relationships. 

Afterwards, students were asked to work in small groups to combine their images to form a larger, collective collage poster. These posters served as visual representations of the students’ perceptions of healthy relationships.

Next Steps
The final phase of this project will include individual interviews with participating students to gather additional information about the collage process and their overall experience as program participants.

Lessons Learned:  
  • Magazine selection should be a part of initial planning. Examples of the magazines we used included: Glamour, People Magazine, Sports Illustrated, and Marie Claire.
  • Chose a diverse range of magazines. Make sure your selection of images is representative of your population!

Hot tip:
  • This project can be done with a small budget, limited time and with almost any participant population.
  • This project is a fun and creative way to engage young people in evaluation.

Cool Trick:
  •  If available, take notes from the large discussion on a white board. This serves as another visual representation and can help students develop their collages.  



ICH staff have expertise in a variety of program evaluation methods and strive to tailor each evaluation to meet the needs of individual clients and programs. Learn more about ICH's Program Evaluation services.



  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Tuesday, May 27, 2014

The Many Faces of Data: Qualitative and Quantitative Perspectives during My Internship at ICH

By Sami Hamdan


As an intern at the Institute for Community Health (ICH), I had the opportunity to study several different aspects of public health that fascinate me. However, my work at ICH also gave me the opportunity to explore health care and health reform through a lens that I had not previously considered: qualitative and quantitative data for evaluation.

As a Health Policy and Management concentrator at Boston University School of Public Health, I have been particularly interested in health reform and improving the delivery of care. I had taken many public health classes during my time as an undergraduate, and so I began my first full MPH year with a basic understanding of public health. As my classes continued to delve more deeply into health policy and management, I became very interested in gaining more practical experience with the health care system. Because the health care system is so complex, I wanted to gain deeper insight into the daily work of researching and supporting healthcare interventions. Because of this, I became interested in ICH’s work.

During my time at ICH, I worked as an intern for the Accountable Care Organization (ACO) and Patient-Centered Medical Home Transformation (PCMH) projects with the Cambridge Health Alliance, primarily with the results from a two year Workforce Survey. I assisted with quantitative and qualitative data analysis, report development, and several other small tasks that make a large project work. I became very familiar with Excel, especially the little short cuts that make long and complex data entry more efficient and accurate. I also learned how to use Excel for qualitative work and how to do basic qualitative coding. I realized that qualitative data offers a fascinating insight into people’s ideas and can give life and perspective to quantitative data and survey answers. I also learned that developing reports is not just about the information, but also about making a clean and engaging presentation (especially for graphs and charts) and using concise language. I was also able to attend several meetings with site leadership at the various Cambridge Health Alliance sites; learning how to present data back to clients was a very useful and important skill.

During my time at ICH, I had a wonderful experience working with very dedicated and supportive team members. I learned the value of working with a strong team, especially when reporting an immense amount of data concisely but accurately. I also learned the value of different perspectives in data: the quantitative and the qualitative. Because I spent so much time with the qualitative data on this project, I learned how powerful it can be, particularly when it is combined with quantitative data. Qualitative data provides nuanced feedback and gives a voice to the numbers, which are useful and incredibly important, but sometimes unclear. As is the case with so much else in health care, it takes multiple perspectives to form a clear picture.

Learn about the ICH Internship Program

  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Wednesday, April 23, 2014

Adapting Race & Ethnicity Categories to Fit your Community

By Jeff Desmarais & Ranjani Paradise


In honor of National Minority Health Month, ICH is exploring the complexities of race and ethnicity data collection. Any program, organization, or institution seeking to identify and address racial and ethnic health disparities must collect race/ethnicity data on the individuals they serve. However, in practice this proves challenging, as individuals often do not identify with the racial and ethnic categories provided.

ICH partners with the Cambridge Health Alliance (CHA) Zero Disparities Committee to ensure that race and ethnicity patient data are both accurate and meaningful for patients and the hospital. We assist CHA with patient demographic data collection and analysis, which helps CHA address disparities in service utilization and ensure that all patients receive linguistically and culturally appropriate care. We collect race data by asking patients to choose one or more of the categories defined by the federal Office of Management and Budget (OMB):
                                                                                                                                                                      
·        American Indian or Alaska Native
·         Asian
·         Black or African American
·         Native Hawaiian or Other Pacific Islander
·         White
·         Other

These OMB categories are required for state and national reporting and allow us to easily compare the race distribution of our patient population with other healthcare systems’ or communities’. However, CHA’s patient population is extraordinarily diverse, and the concept of race varies widely across cultures. Many of our patients simply do not relate to the OMB race definitions and categorizations. For example, Latino patients and Middle Eastern patients often express confusion when asked to select a race, as many do not see themselves fitting any of the given categories. And while it may seem clear that patients from India should fall under the Asian category, some Indians disagree - to them, Asian indicates East Asian and is distinct from South Asian. Overall, more than 25% of CHA’s patients do not identify with any of the OMB categories and choose “Other” as their race, which calls into question the usefulness of the OMB categories in our increasingly diverse community.

Because of these issues, we also collect detailed ethnicity data to better characterize our patients’ cultural backgrounds. Patients are asked to self-identify their ethnicity (or ethnicities) from a list of more than 150 options (e.g., Algerian, Bangladeshi, Egyptian, Greek, Nepalese, Syrian, etc.). We strive to make these ethnicity options specific enough that patients can select one or more with which they truly identify. Overall, we find that ethnicity data can be far more useful than race data for ensuring we provide culturally appropriate care to everyone we serve.


ICH also works as the evaluator of the Youth First Initiative in Holyoke and Springfield, MA. Youth First is a community-wide teen pregnancy prevention initiative, which was developed with grant funding from the Centers for Disease Control and Prevention. The Initiative is a collaboration between the Massachusetts Alliance on Teen Pregnancy (MATP), the YEAH! Network, and many stakeholders within the two communities, including schools, community organizations, and health clinics. Youth First began in 2010 and its goal is to reduce teen births by 10% by 2015 in Holyoke and Springfield, MA. While the overall teen birth rates in these communities are significantly higher than the state’s, there are also large racial/ethnic disparities within the communities.

Source: Births (Vital Records). Massachusetts Community Health Information Profile (MassCHIP). Version 3.0r328. Massachusetts Department of Public Health. Data downloaded May 6, 2013

Thus, collecting meaningful race/ethnicity data is critical to both improving and documenting improvement in teen pregnancy prevention efforts.

One of the key components of the Youth First initiative is implementing evidence-based sexual health education programs for adolescents. As a part of reporting requirements for funders, we collect race/ethnicity data from program participants via surveys. Collecting this data allows us to understand the populations we are reaching. In addition to the OMB race categories, the CDC requires data for the following ethnicity categories:

·         Hispanic or Latino
·         Not Hispanic or Latino

Cultural practices, beliefs, values and behaviors, all of which can reflect a person’s identity, can have an impact on a community’s health. Collecting data that taps these identities, if done meaningfully, can serve as an indicator to better understand, explain and ultimately address health inequities. With large Spanish-speaking communities in Springfield and Holyoke, it is critical to use measures that capture their unique and specific identities. Using the race/ethnicity categories often required by funding organizations can prohibit a comprehensive understanding of how communities might view themselves. For example, a Dominican or Puerto Rican teen might not identify with any of the OMB racial categories. Additionally, using the ethnicity category “Hispanic or Latino” can mask differences within Spanish-speaking communities. A teen that grew up in rural Mexico might have very different experiences, behaviors and values than a Puerto Rican teen that grew up in Holyoke. In this project, in order to ensure identity is accurately captured, we added an additional question that includes a much more comprehensive list of 29 different ethnicity categories (for example: Puerto Rican, Caribbean Islander, Dominican, Korean), thus expanding our understanding of identity among youth enrolled in evidence-based programs.

Capturing race and ethnicity data is essential not just for program improvement purposes, but also for ensuring that public health interventions are actively addressing health inequities. In developing alternative ways of measuring this data, it is critical to engage the community during the instrument development phase. It’s often necessary to go beyond the basic race/ethnicity reporting requirements, and delve deeper into communities’ understanding of race, ethnicity, and identity, in order for the data to be meaningful and actionable. 

  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Wednesday, March 26, 2014

ICH Supports Community-Based Approaches to Opioid Abuse Prevention

By Lynn Ibekwe

Although opioid abuse has been a growing problem over the past decade, public awareness has only recently peaked due to opioid deaths around the country reaching epidemic proportions and the deaths of celebrities like Phillip Seymour Hoffman and Cory Monteith. Opioids are highly addictive substances that include heroin and prescription pain medication such as morphine, codeine, oxycodone, and buprenoprhine. Characterized as an “urgent public health crisis” by U.S. Attorney General Eric Holder, rates of fatal heroin overdose, opioid drug trafficking, and prescription drug misuse and abuse have sharply increased in recent years. According to Drug Administration Enforcement (DEA) officials, the growing use of heroin stems from the corresponding growth in prescription painkiller abuse. In 2010, about 12 million Americans reported using prescription painkillers for non-medical purposes in the past year. Today, four out of five adults (ages 12 and older) who recently began using heroin had previously abused painkillers.


Past Month & Past Year Heroin Use among Americans Aged 12 or Older: 2002-2012 [1]  

Massachusetts communities have not been immune to the opioid epidemic. With many New England cities and towns now reporting dramatic increases in opioid overdoses and death, communities and stakeholders across MA are mobilizing to address this growing problem. ICH has been engaged in a number of these efforts over the past few years, including the current Massachusetts Opioid Abuse Prevention Collaborative (MOAPC) and the Partnerships for Success II (PFS II) initiatives.

MOAPC is a 
Massachusetts Department of Public Health Bureau of Substance Abuse Services (BSAS) funded project that aims to increase both the number and the capacity of communities across Massachusetts addressing opioid misuse and abuse. ICH works with the MOAPC Cambridge cluster, consisting of Cambridge, Everett, Somerville, and Watertown and lead by the Cambridge Public Health Department (CPHD). The mission of this collaborative is to employ systems-level approaches to prevent opioid misuse and reduce unintentional deaths and hospital events associated with opioid poisonings. Cluster members are currently strategizing around activities and initiatives to address key factors associated with opioid abuse, overdose, and death in their communities, such as access to opioids, low provider knowledge around prescription drug misuse, and misconceptions around the safety of non-medical use of prescription drugs.

ICH also collaborates with the cities of Quincy and Worcester on the three-year PFS II project funded through BSAS. Impact Quincy and the City of Worcester’s Division of Public Health (WDPH) were two of the many community-based prevention programs in MA awarded PFS II dollars to address prescription drug misuse and abuse in their communities. Both organizations have spent the past year identifying a number of prevention strategies to address misuse among youth, including:
  •  Increasing the number of locations for safe disposal of prescription drugs
  • Developing mass media and educational campaigns aimed at raising awareness among parents and community members
  • Conducting trainings and professional development workshops to educate physicians and other medical providers around the non-medical use of prescription drugs (NMUPD)
  • Developing social norms/marketing campaigns and a health curriculum for youth around misperceptions of peer use and safety of NMUPD

As the local evaluator for these initiatives, ICH assists these multi-sectoral groups in identifying evidence-based strategies for opioid abuse prevention, soliciting and analyzing community input, and documenting and understanding their impact. Our participatory approach values local knowledge and encourages capacity building so that our communities are well equipped to curb this disastrous epidemic. 



[1] Substance Abuse and Mental Health Services Administration, Results from the 2012 National Survey on Drug Use and Health: Summary of National Findings, NSDUH Series H-46, HHS Publication No. (SMA) 13-4795. Rockville, MD: Substance Abuse and Mental Health Services Administration, 2013.
  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Tuesday, February 11, 2014

ICH Volunteer Day

By Nazmim Bhuiya

On January 17th, ICH held our inaugural Volunteer Day at Cradles to Crayons (C2C) in Brighton. C2C is dedicated to providing children from birth to age 12, living in low-income and homeless situations, with the essential items they need to thrive - at home, at school and at play. At their Giving Factory, we sorted through and checked the quality of donated clothes, shoes, books, toys, and other necessary items and put together outfits for children. Together we were able to make a difference for almost 200 children across the state!

It was a great opportunity to roll up our sleeves and volunteer at Cradles to Crayons!

“I had a great time putting together outfits for baby boys in need - this was especially fun for me as I am expecting a baby boy in just a few weeks! I was very impressed with the care that Cradles to Crayons takes to make sure that children receive high-quality items, and I would love to volunteer there again in the future.” – Ranjani Paradise, ICH Research Associate II

“This was a great opportunity to step away from my desk for one morning and take a more hands on approach to making a difference in the community. I was amazed by how quickly time passed as I created outfits for young girls, and how many children we were collectively able to impact in such a short span of time.” – Reann Gibson, ICH Research Associate I


“We had a great time at the triage station where we sorted through all the donation items. It was good to work with colleagues outside of the office and also having an opportunity to give back to our community collectively as an organization.” – Nazmim Bhuiya, ICH Research Associate II

  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Thursday, January 23, 2014

Data Viz and Mixed Methods: How to Make Results Compelling

By Maeve Conlin & Molly Ryan

Here at ICH, we often use a mixed methods design for our research and evaluation projects. A mixed method approach acknowledges the limitations of only using one type of data source. It involves collecting and analyzing qualitative data, typically from interviews or surveys, and quantitative data, like mortality rates.  

Using multiple data sources helps to create a more complete and compelling picture of the data’s story. Mixed methods allow us to contextualize facts and figures, grounding them in the programs, projects, and communities they summarize. Perhaps most importantly, mixed methods facilitate a strength-based analysis, allowing for an exploration of opportunities as well as challenges.

Visualization Techniques to Connect Quantitative and Qualitative Data

Mixed methods are essential for much of our work at ICH, including all of our needs assessment projects. To conduct a needs assessment, we collect and analyze both quantitative data, like mortality causes, ED visits, and hospitalizations, along with community feedback on local health needs and solutions to health challenges. The result is A LOT of data! One strategy for helping your audience draw connections between your data is to include related quantitative and qualitative data side-by-side:


This same method of showing quantitative and qualitative data together can also be used for surveys, another tool we use frequently at ICH. For example, survey participants may be asked to rate their satisfaction with a variety of topics and explain their rating in a comment section. In this case, combining quantifiable participant satisfaction data with related quotes grounds the data and presents a fuller picture:


Making Qualitative Data Compelling
Within our qualitative data, we often look for ways to visually demonstrate similarities and differences across data points. As shown below, this can be done using a table format to display key themes. However, because this approach essentially quantifies qualitative data, we also include illustrative quotes so we do not to lose the interviewees’ voices or the richness of their comments.


Table 1: High-Risk Patient Definition by Site and Type of Respondent


Tailoring Data Visualization to Meet Unique Needs
Understanding data visualization processes and techniques helps us to present data that is not only eye-catching but easily understood. We can highlight important patterns and findings within a larger data set so that stakeholders can easily draw conclusions and make decisions.

However, having new and interesting ways to display data is not enough. Here at ICH, we work with a wide array of partners, including academia, hospitals, schools and community-based organizations. Exactly how and what data is presented, and to whom, are key considerations in the data visualization process. Most stakeholders or partners likely have different data needs, and it’s important to ensure you are presenting the information in a way that is comprehensible and useful for each unique project and audience!

  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.


Tuesday, October 22, 2013

Walking in the Workplace: ICH Staff Embrace Workplace Wellness

By Kelly Washburn, MPH

At the end of July 2013, the Cambridge Health Alliance’s (CHA) Take CHArge Wellness Program hosted a 6-week walking competition for all departments at CHA. Kelly O’Connor, Program Manager of Population Health, oversees the wellness program and describes the goal of the program as follows: “by developing policies, activities and a work environment that promotes and encourages wellness, the program seeks to create a healthy, engaged workforce who are role models for our patient community, working in a health conscious and supportive culture.”

The rules of the competition were simple: form a team, elect a team captain, and record team members’ steps each week. The team with the highest average number of steps wins! The ultimate prize for the winners was chair massages for the entire team. As somebody who sits at a computer much of the day, the chair massage was enticing enough for me to form a team at ICH. Within a day, 16 ICH staff members agreed to take on the challenge. Each team member received a welcome kit containing a pedometer, brochure, and a bracelet, which said “Goal: 10,000 steps a day”.
 
Initially, I thought the competition would be easy: “I walk plenty each day, how hard could this competition really be?” Oh, how wrong I was. It wasn't until I strapped on a pedometer that I realized how little activity I actually do throughout the day. The team definitely needed to get proactive about increasing activity levels throughout the workday, so we made efforts to ensure we were walking whenever possible at the office.  With the Gold Star Mothers Park located right next to ICH, it was the perfect place for walking meetings or even just grabbing a coworker for a short walk to take advantage of the beautiful summer weather. At the end of the six weeks, Team ICH had a combined total of 5,978,944 steps! All the teams’ steps were tallied and the results were in….ICH came in second place.

Even though we didn't win the ultimate prize, the competition was a great venue for ICH staff to remember the importance of taking a few minutes each day to go outside, walk and be active.  Additionally, it helped promote the concept of walking meetings instead of moving from one seat to the next. Changing office culture takes time, but this competition helped remind us all that as public health professionals we must take initiative and do the same activities we promote in the community.  






  ---------------------------------------------------------------------------------------------------------------------------------------
The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.