I have such interesting qualitative data! Now how do I analyze it?

Tips and Tricks for Qualitative Data Analysis (Part 3 in a 4 part series)

By Julie Carpineto, MFA & Eileen Dryden, PhD

Qualitative data analysis and reporting can seem like a mysterious process for those new to it – but  it doesn’t have to be.  Here are some tips and tricks to help you simplify the process of analyzing qualitative data. 

Getting to Know your Data: Focusing and Starting Your Analysis

Tip: Start thinking about your analysis from the moment you begin data collection! Reflect on and record themes, theories, and areas of interest throughout the data collection process.

• Trick: Schedule an extra ½ hour after focus groups for the facilitator, note taker and any other assistants to document preliminary themes, areas of interest and great quotes.

Tip:  Decide how you will approach your analysis based on your resources and with the research goals in mind.  Consider:

• What resources do you have? (personnel, time, money, skills)
• What level of detail and rigor do the people who will use the information need?     

 

Tip: Get to know your data by reading over notes and transcripts to assess the data’s quality, breadth and variability

• Trick:  Again, document preliminary thoughts on main themes and points of interest.

Tip: Focus, focus, focus!! 

• Trick: Keep your questions of interest at the forefront during all phases of your analysis and create a list of interesting ‘asides’ elsewhere that you may want to look into further at a later date. 

–        Review the reasons you wanted to collect qualitative data

–        Identify key questions you hope to answer or learn more about

 

Into the Thick of It: Developing a Codebook & Coding Your Data

After all your data is collected and you’ve identified some preliminary themes, the next step is to categorize your data into key themes, called “codes”.

To help organize this process, develop a “codebook” – or list of key themes – as a guide. This is essential for maintaining consistency if more than one analyst is coding, but can also be helpful for internal consistency and reporting transparency even if only one analyst is involved. Note: Your codebook will likely undergo changes. Emergent interests and insights may lead to adding or changing codes as you proceed.

Once you have developed a codebook, you can then start “coding” your data by labeling segments of text with the applicable themes/codes.

Tip: Use the focus group/interview guide to develop a preliminary codebook:

• What do you think are the biggest strengths in your community?
• Include the code “strengths"

• How do you think the intervention could leverage these strengths to increase its likelihood of success?
• Include the code “leveraging strengths”

• Can you think of other key players—organizations, agencies, individuals, etc.—who could help increase the intervention’s likelihood of success?
• Include the code “key players”

 

Tip: Have more than one person involved in analysis when possible. This increases reliability of findings.

o   Trick: Meet to review analysis periodically and resolve discrepancies in opinion.

Tip: Don’t rely on qualitative data analysis software to do the analysis for you. These software programs facilitate the analytical process by helping you manage large amounts of data – but you still have to do the analysis!

o   Trick: For smaller amounts of data you may find it’s easier to code ‘by-hand.’ When coding by hand, it can be helpful to use colors in MSWord or add extra code columns to MSExcel templates: for example:

How are you and your family getting along?	Code	What are you doing differently since the program?	Code
Yes I am more patient	Skill	Yes my son is talking to me and working on family therapy.	Comm., Ther
I feel better about myself	Conf	I listen more and calmed down to try to take more time for myself.	Li, Skill, SN
Same most of the time	NC	First I sit down with my kids than we talked about problems how to get rid of it	Comm.
Yes , not so different from their past and present	MISC	Not really	NC
I am happier so yes!	Feel pos	Meditate take space look @ problems vs solutions instead of just problems	SN, Skill
My husband and I are doing much better Its helpful to have the time without a child to reflect on parenting and on our own issues goals and needs	Sp, Rel, Intro, SN	think about son's perspective and the values we want to convey instead of focusing just on good behavior. For example, think about how to model respect for others in daily behavior.	Skill

 

Tip: Take analysis up a level: summarize categories about a topic or question.

o   Trick: Note what is interesting/relevant (e.g. variability within and across groups; relative strength/commonality of themes). This will help you determine what is worth reporting on, or highlighting in your reports.

 

These are just a few ideas for focusing and implementing your analysis.  There are many ways to approach qualitative data analysis and some really great resources for learning more. One book we particularly recommend is "Qualitative Data Analysis: An Expanded Sourcebook", 2nd ed. by M. Huberman and M.B. Miles, Sage Publications, 1994.

Now you’re ready for the final phase in this journey with qualitative data: interpreting your findings and communicating them to stakeholders.  Look out for our fourth and final blog in this series for tips and tricks on qualitative data reporting!

For more information about the Institute for Community Health, please visit our website http://icommunityhealth.org/. Examples of our work and collaborations involving qualitative research are described here.

Note: different projects require different methodological approaches. This is not an exhaustive list of qualitative analysis techniques. You may find other techniques fit your project’s aims more effectively and appropriately.

 ------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

 

 

What do I do with all of this qualitative data!?!

Tips and Tricks for Qualitative Data Management (Part 2 in 4 part series)

By Eileen Dryden, PhD & Julie Carpineto, MFA

You are sold on the benefits of using qualitative methods.  You have started using them.  Now you find yourself drowning in data. Does this sound familiar?  

People who are new to using qualitative methods may quickly get overwhelmed by the amount of data these methods can generate. Here are some tips and tricks to help you manage your qualitative data: 

Tip: Record data in a form that makes it amenable to analysis.

• Trick: For focus groups, create a diagram of the table around which participants are sitting, with identifying initials or a number for each person. Add that identifier next to each person’s comments when taking notes.

       

 

• Trick: Record participants’ responses near the questions to which they refer, not necessarily when they occurred during the interview/focus group.

• Trick: Use templates for recording data when possible.

          Examples:

• MS Excel template for focus group responses              

Participant	Q1: Barriers to Physical Activity
P1	Time & money
P2	Money
P3	No bike lanes
P4	Motivation
P5	Sidewalks; streetlights

 

• Data abstraction template (e.g. a table of information you want to make sure you collect) for transcript review

Unique ID	Family Situation	Issues Addressed with Case Manager
A0154	-Single mother with 1 son -Recently divorced -Son’s father not living in the country	-Trouble finding childcare -Cannot find full-time job

• Observation templates

Date	Time of Day	Site	# of Adults Present	# of Children Present	Activities
8/4/2013	10-11 AM	City Park A	2	6	Rollerblading, Biking

• Interview/focus group guide itself as a template (record notes in spaces following each question)

• Use an online tool like SurveyMonkey to distribute diary log ‘templates’ to respondents if looking for similar information over time from same people (minimizes unnecessary re-entry of data)

Tip: Always audio-tape if possible. Useful for:

–     Verbatim transcription

–     Writing detailed summary notes

–     Verifying you captured important points

–     More rigorous analysis at later date

Tip: If you are unable to audio-tape an interview for any reason (e.g. the participant is not comfortable being taped), although not typically done, it is helpful to have two research team members attend the interview: one responsible for facilitation and the other for note-taking. (Note: for focus groups, it is common and recommended practice to always have two team members present.)

Tip: Align the level of audio recording transcription with the overall purpose of the evaluation – it is not always necessary to transcribe interview or focus group content verbatim, and notes can often suffice. You can go back to the audio recording for clarification or specific quotes as needed.

       – Verbatim transcription = Expensive!
       – With a professional service:
          1 hour tape = ~4 hours transcription = ~$150-$200 (or more!)

Tip:  For large amounts of data (either length of transcripts or number of transcripts), it is helpful to use qualitative data management software.  Examples:

    – QSR NVivo
    – Atlas TI
    – EthnoNotes
    – Ethnograph
    – HyperResearch
    – Dedoose

Tip: Small amounts of data may be easier to manage/code/analyze by hand.

These are just a few ideas to help you organize and manage your qualitative data from the moment you collect it.  While this may make you feel a little more confident with qualitative data collection, even well organized data can feel overwhelming just by the sheer quantity of it!  The next post in this series will hopefully help you tackle the next step: qualitative data analysis.

For more information about the Institute for Community Health, please visit our website http://icommunityhealth.org/. Examples of our work and collaborations involving qualitative research are described here.

 Note: different projects require different methodological approaches. This is not an exhaustive list of data management techniques. You may find other techniques fit your project’s aims more effectively and appropriately.

------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Hello Qualitative Methods, It is Nice to Meet You!

Quick Introduction to Qualitative Methods for Community Partners 

Part 1 in a 4 part series

By Eileen Dryden, PhD & Julie Carpineto, MFA

Ever wonder why people engage in certain behaviors? Do you want to know more about how people make decisions about their behavior and how they feel about what they do? If so, you may be interested in using qualitative methods to explore your research or evaluation questions.

While many of us in the research and evaluation field are well versed in the benefits of using qualitative methods, some people seem to shy away from using them due to unfamiliarity and rely exclusively on quantitative methods instead.  This blog is intended to provide a quick introduction to qualitative methods: what they are, when they are most useful, and some guidelines for effectively using a few popular qualitative methods.

What are qualitative methods?

Qualitative methods collect descriptive data on phenomena.  They are often used to gain an in depth understanding of human behavior, investigating the why and how of behavior, not just the when, where and what.  They are useful for gaining insight into people’s attitudes, behaviors, experiences, feelings, concerns and motivations. Additionally, qualitative methods can provide context and depth to questions of interest and are especially useful for exploring a construct about which little is known.  The results of qualitative data can be quite compelling as responses are documented in the individuals’ own words.

However, there are limitations to consider when deciding whether or not to use qualitative methods, including the relatively large amount of resources needed for implementation and the related small sample sizes.  Qualitative methods are labor intensive – especially transcription and analysis!  Consequently, it is generally only feasible to collect data on small numbers of individuals. As a result, one must be strategic about choosing a sampling strategy and subsequent findings are often not generalizable.

Despite these limitations qualitative methods are commonly used to great effect!  Some popular qualitative methods are interviews, focus groups and observations. 

Popular qualitative methods:

1) Interviews

Let’s start with interviews. Qualitative interviews are one-on-one discussions with an investigator and a selected respondent. Interviews can be informal, more like conversations; unstructured, where particular general topics are covered; semi-structured, where there is an interview framework  that guides the discussion; and structured where respondents all answer the same questions in the same order.  Interviews are great for exploring a topic in depth with individuals and are an especially good method if the topic of interest is a sensitive one.

2) Focus Group

 

A focus group is a facilitated discussion with about 7 to 10 people on a particular topic.  This method is very useful when you’d like people to be truly discussing a topic, responding to each others ideas and opinions.  In contrast to interviews, focus groups are not recommended for sensitive topics and work best for topics that lend themselves to a friendly group discussion.  In order to stay focused and make the most of the time allotted, good facilitation skills are a must. 

3) Observation

The method of observation is especially useful to gain insight into experiences that may be hard to explain or put into words.  Observation is generally either participatory or ‘unobtrusive’.  During participant observation, the investigator is engaged in some way with the people he/she is observing (for example, taking part in the training that is being evaluated).  During unobtrusive observation activities, the investigator remains separated from the people he/she is observing – for example when a researcher observes the use of a fitness trail by sitting anonymously at the side of the trail and recording what he/she sees.

Those are few of the most popular qualitative methods. Hopefully this quick introduction has enticed you to explore the use of qualitative methods in your next investigation.  If you do start to use these methods you may wonder “what do I do with all this data!?!”  Stay tuned for the next blog in this four-part series: tips and tricks for qualitative data management.

Note: different projects require different methodological approaches. The methods presented here are examples of popular qualitative methods, but it is certainly not an exhaustive list. You may find other methods fit your project’s aims more effectively and appropriately.

 ------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Insurance issues as a barrier to HIV care: using a newsletter to raise awareness

By Kathleen Xu, MPH & Ranjani Paradise, PhD 

“Take the test. Take control.” – National HIV Testing Day slogan

In honor of National HIV Testing Day, we are sharing our experience working with Cambridge Health Alliance’s (CHA) two HIV clinics, The Zinberg Clinic and Somerville Hospital Primary Care, on a Continuous Quality Improvement (CQI) Team. Our team consists of a nurse manager, a social work manager, a nurse practitioner, a community health worker, a program support director, and three Institute for Community Health (ICH) staff members. We meet regularly to track quality measures and strategize how to help the HIV clinics sustain high quality services. One of the issues we have been tracking is the relationship between health insurance coverage and access to care for HIV patients. Insurance policy changes and gaps in coverage impact patients’ ability to maintain ongoing HIV treatment, which is extremely important for reducing HIV viral load and minimizing HIV drug resistance. With 63% of CHA’s HIV patients on public insurance, and a large proportion of them affected by unstable housing and employment situations, their insurance coverage often fluctuates. Therefore, the CQI team has made an effort to better understand the main insurance issues and ongoing policy changes that most affect these patients, so that CHA providers can better support patients through the process.

Over the past few months, our CQI team gathered information and feedback from the two clinics’ case managers about challenges they have experienced with insurance policy changes when trying to maintain coverage for their patients. We learned that as a result of many policy changes over the last 4 years, including those that resulted from the Massachusetts health care reform, the time it took case managers to ensure each patient received adequate coverage quadrupled.  Much of the added time was attributed to extra paperwork and delays in insurers’ response time.

One particular challenge case managers and patients have faced in recent years is stricter proof of residency requirements. Since many of CHA’s HIV patients are of low socioeconomic status and have unstable living situations, they often do not have the documents required to prove MA residency (e.g., mortgage papers, utility bills, leases). As a result of this policy change, patients experience unexpected coverage termination and coverage gaps. Case managers must continuously monitor patients’ social and living statuses on a month-to-month basis, as any income, dependency, address or job status changes could disqualify patients for some insurance policies. In such cases, case managers must help patients identify and apply for a new insurance policy depending on the status change and also help them determine which pharmacies accept the new insurance.

Overall, obtaining insurance has become a time-consuming and complicated process for patients and case managers. CHA is fortunate to have dedicated case managers to help patients navigate the complex insurance system, as well as providers who also work hard to keep patients in care and adhering to their medication regimens, even when faced with these insurance barriers.

In order to support our case managers, providers, and patients, the CQI team focused the April 2013 issue of our newsletter, Facts for Action, on insurance and HIV. This newsletter was disseminated to all staff and patients in order to raise awareness about the effects of insurance issues on HIV care.

The first page displays a timeline and comprehensive list of insurance issues that case managers have been working hard to overcome, while the second page includes a list of recommendations for providers and patients about how they can work together with their case managers to avoid insurance issues. Especially in the multidisciplinary setting of CHA’s HIV clinics, the newsletter served as an important reminder for all staff to collaborate as a team and communicate with each other and their patients to provide the best care for their patients.

National HIV Testing Day is an important annual event promoting HIV testing and encouraging all people to learn their HIV status. Getting tested is only the first step to managing HIV. As we reflect on our work with the CQI team, we are reminded that in addition to encouraging testing, we must ensure that those tested positive have access to the medical care that they need.

Special thanks to the HIV Continuous Quality Improvement (CQI) team and the CHA HIV clinics' case management staff for their help in compiling the Facts For Action newsletter!

------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Making Data Useful, Making Data Meaningful: Participatory Evaluation in the Youth First initiative

By Jeff Desmarais, MA, Maeve Conlin, MPH, Nazmim Bhuiya, MPH

May was teen pregnancy prevention month, which is a chance to highlight the Institute for Community Health’s (ICH) partnership focusing on teen pregnancy prevention. Massachusetts has made great strides in reducing the teen birth rate in the past decade, as evidenced by 2010 community-level teen birth rate data recently released by the Massachusetts Department of Public Health. In 2010, Massachusetts teen birth rate was 17.1 per 1,000 adolescent females aged 15-19, the lowest birthrate on record, continuing the state’s downward trend. While progress has been made, strong racial and geographic disparities in teen birth rates persist among Massachusetts’ communities. These disparities must be addressed through collaborative, multilevel efforts, such as Youth First, an initiative that aims to reduce the teen birth rate in Springfield and Holyoke, two communities with some of the highest teen birth rates in Massachusetts. In 2010, Holyoke had the highest teen birth rate in the state and Springfield had the third highest.

Youth First, a 5-year CDC-funded initiative, seeks to address teen birth rate disparities through a community-wide collaborative approach to enhance access to quality sexual health education and reproductive health services. This collaboration involves the lead organizations, Massachusetts Alliance on Teen Pregnancy (MATP) and the Youth Empowerment and Adolescent Health (YEAH!) Network, as well as a host of Springfield and Holyoke-based clinical and community partners. These partners aim to mobilize the community, empower youth through engagement and enhance the quality of clinical and health education services for sexual and reproductive health.

Working in partnership as the evaluator for this initiative, ICH’s role is to facilitate data collection and help our partners utilize this data to drive program improvements and decisions. Since ICH utilizes a participatory research approach to guide the evaluation process, we are simultaneously building our partners’ capacity to collect, analyze, and interpret data for their current and future work. This data allows our partners to think about their programs’ impact—to see how their hard work is making a difference in the lives of the youth they serve and the community more broadly—and informs their services in the future.

As we reflect on this important partnership and the value it brings to the health of populations, we have several critical lessons to share as evaluators:

First, in order for data to be useful for our community partners, it must be meaningful to them. As an evaluator, you can’t assume that all partners will have an understanding of how to interpret and use data to improve their communities. Building an understanding of what data actually tells us and how it can be used to inform programs’ strengths, gaps, needs, and target populations is an important step in any partnership. Having these conversations early on in the collaboration will help you identify which capacities must be improved so that data can be collected and used effectively.

Additionally, as evaluators we must help build communities’ data collection infrastructure to enable partners to collect and use data independently, so that teen pregnancy prevention efforts can be continuously improved and sustained even after grant funding ends. With a strong data collection system in place, the partners can better understand and improve their programs and in turn, their communities.

The Youth First initiative’s multifaceted approach to teen pregnancy prevention is what makes the program distinctive. Our work as evaluators is a small but very important piece of the initiative. Erica Fletcher, Prevention Director at MATP, describes our strong partnership as a unique project element and states that ICH has been instrumental to the initiative’s planning, implementation, and ongoing quality improvement efforts. Given that May was teen pregnancy prevention month, it is an opportune time to highlight strategies and share tips with other evaluators aiming to reduce teen pregnancy.  Strong partnerships with the community and a participatory evaluation approach are key steps towards recognizing opportunities to address social, behavioral and environmental factors that impact teen pregnancy.

Special thanks to Shalini Tendulkar, ScD, ScM, and Jessica Waggett, MPH at ICH and Erica Fletcher, Ed.M. Prevention Director at the Massachusetts Alliance on Teen Pregnancy for their support and guidance. 

 

------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.