What do these findings mean and how do I communicate them?

Tips and Tricks for Interpreting and Reporting Qualitative Data

Part 4 in a 4 part series

By Julie Carpineto, MFA & Eileen Dryden, PhD

Once qualitative data has been collected, coded and analyzed, it needs to be interpreted and packaged in a way that is meaningful to stakeholders.  Interpreting and communicating qualitative findings is essential for ensuring the results will be used – and this, really, is the ultimate goal!  Here are some tips and tricks for interpreting and reporting qualitative data.

Data Interpretation

Tip:  Begin by listing key points and themes:

• What have your qualitative findings confirmed? This is especially important to consider if your qualitative research was part of a mixed-methods effort to learn more about a particular topic, community problem, etc.
• What are the major lessons learned?
• What, if anything, can be applied to other settings, programs, or studies?

 

Tip: Establish criteria for deciding what is considered a “major” or “common” theme.

• A theme will usually be considered “major” or “common,” and therefore worth reporting or highlighting, if noted by at least 50% of a group or subgroup.
•  As appropriate, think about meaningful ways to categorize your themes (e.g. “suggestions for improvement”)
• Depending on your research question, it may also be important to note minor themes or the absence of an expected theme.

Tip:  Stakeholders (e.g. program staff, participants, community members) can provide valuable insight into qualitative findings! Work closely with stakeholders to review findings and determine their significance and relative importance. 

Data Reporting

Tip: Consider your audience(s) and determine the best report format and venue for communicating with them effectively.  Sometimes the best report is not a report at all!  Posters, videos, brochures, slide shows and oral presentations are all great options. Be creative!

Tip: If you do create a more traditional report, keep in mind that, in general, less is more. Again, consider your  audience(s) and stakeholders when determining appropriate report length.

 

Trick: Create an executive summary (‘1-pager’) that highlights main findings (qualitative reports can get very long!)

 

 

 

Tip: Use quotes and photos to illustrate themes throughout your report (whatever the format!).

Tip: Make the most of all the work you have done.  Qualitative findings can have multiple uses – think sustainability!  Compelling quotes and photos can be used for marketing initiatives, funding proposals, etc…

We’ve come to the end of this 4-part “Tips and Tricks” series for using qualitative methods.  While not all of these tips and tricks are appropriate for all your qualitative endeavors, we hope this series has given you a flavor of qualitative methods and encourages you to consider using them in your next research or evaluation project.

If you are interested in reading more about qualitative methods, there is a treasure trove of available books on the topic. One we highly recommend is Michael Quinn Patton’s book, “Qualitative Evaluation and Research Methods”, SAGE Publications, Inc: 3rd edition (October 2001). It is a great end-of-summer page-turner!

Please see our Qualitative Methods page for more information on ICH’s qualitative methods approach & expertise.

------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

I have such interesting qualitative data! Now how do I analyze it?

Tips and Tricks for Qualitative Data Analysis (Part 3 in a 4 part series)

By Julie Carpineto, MFA & Eileen Dryden, PhD

Qualitative data analysis and reporting can seem like a mysterious process for those new to it – but  it doesn’t have to be.  Here are some tips and tricks to help you simplify the process of analyzing qualitative data. 

Getting to Know your Data: Focusing and Starting Your Analysis

Tip: Start thinking about your analysis from the moment you begin data collection! Reflect on and record themes, theories, and areas of interest throughout the data collection process.

• Trick: Schedule an extra ½ hour after focus groups for the facilitator, note taker and any other assistants to document preliminary themes, areas of interest and great quotes.

Tip:  Decide how you will approach your analysis based on your resources and with the research goals in mind.  Consider:

• What resources do you have? (personnel, time, money, skills)
• What level of detail and rigor do the people who will use the information need?     

 

Tip: Get to know your data by reading over notes and transcripts to assess the data’s quality, breadth and variability

• Trick:  Again, document preliminary thoughts on main themes and points of interest.

Tip: Focus, focus, focus!! 

• Trick: Keep your questions of interest at the forefront during all phases of your analysis and create a list of interesting ‘asides’ elsewhere that you may want to look into further at a later date. 

–        Review the reasons you wanted to collect qualitative data

–        Identify key questions you hope to answer or learn more about

 

Into the Thick of It: Developing a Codebook & Coding Your Data

After all your data is collected and you’ve identified some preliminary themes, the next step is to categorize your data into key themes, called “codes”.

To help organize this process, develop a “codebook” – or list of key themes – as a guide. This is essential for maintaining consistency if more than one analyst is coding, but can also be helpful for internal consistency and reporting transparency even if only one analyst is involved. Note: Your codebook will likely undergo changes. Emergent interests and insights may lead to adding or changing codes as you proceed.

Once you have developed a codebook, you can then start “coding” your data by labeling segments of text with the applicable themes/codes.

Tip: Use the focus group/interview guide to develop a preliminary codebook:

• What do you think are the biggest strengths in your community?
• Include the code “strengths"

• How do you think the intervention could leverage these strengths to increase its likelihood of success?
• Include the code “leveraging strengths”

• Can you think of other key players—organizations, agencies, individuals, etc.—who could help increase the intervention’s likelihood of success?
• Include the code “key players”

 

Tip: Have more than one person involved in analysis when possible. This increases reliability of findings.

o   Trick: Meet to review analysis periodically and resolve discrepancies in opinion.

Tip: Don’t rely on qualitative data analysis software to do the analysis for you. These software programs facilitate the analytical process by helping you manage large amounts of data – but you still have to do the analysis!

o   Trick: For smaller amounts of data you may find it’s easier to code ‘by-hand.’ When coding by hand, it can be helpful to use colors in MSWord or add extra code columns to MSExcel templates: for example:

How are you and your family getting along?	Code	What are you doing differently since the program?	Code
Yes I am more patient	Skill	Yes my son is talking to me and working on family therapy.	Comm., Ther
I feel better about myself	Conf	I listen more and calmed down to try to take more time for myself.	Li, Skill, SN
Same most of the time	NC	First I sit down with my kids than we talked about problems how to get rid of it	Comm.
Yes , not so different from their past and present	MISC	Not really	NC
I am happier so yes!	Feel pos	Meditate take space look @ problems vs solutions instead of just problems	SN, Skill
My husband and I are doing much better Its helpful to have the time without a child to reflect on parenting and on our own issues goals and needs	Sp, Rel, Intro, SN	think about son's perspective and the values we want to convey instead of focusing just on good behavior. For example, think about how to model respect for others in daily behavior.	Skill

 

Tip: Take analysis up a level: summarize categories about a topic or question.

o   Trick: Note what is interesting/relevant (e.g. variability within and across groups; relative strength/commonality of themes). This will help you determine what is worth reporting on, or highlighting in your reports.

 

These are just a few ideas for focusing and implementing your analysis.  There are many ways to approach qualitative data analysis and some really great resources for learning more. One book we particularly recommend is "Qualitative Data Analysis: An Expanded Sourcebook", 2nd ed. by M. Huberman and M.B. Miles, Sage Publications, 1994.

Now you’re ready for the final phase in this journey with qualitative data: interpreting your findings and communicating them to stakeholders.  Look out for our fourth and final blog in this series for tips and tricks on qualitative data reporting!

For more information about the Institute for Community Health, please visit our website http://icommunityhealth.org/. Examples of our work and collaborations involving qualitative research are described here.

Note: different projects require different methodological approaches. This is not an exhaustive list of qualitative analysis techniques. You may find other techniques fit your project’s aims more effectively and appropriately.

 ------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

 

 

What do I do with all of this qualitative data!?!

Tips and Tricks for Qualitative Data Management (Part 2 in 4 part series)

By Eileen Dryden, PhD & Julie Carpineto, MFA

You are sold on the benefits of using qualitative methods.  You have started using them.  Now you find yourself drowning in data. Does this sound familiar?  

People who are new to using qualitative methods may quickly get overwhelmed by the amount of data these methods can generate. Here are some tips and tricks to help you manage your qualitative data: 

Tip: Record data in a form that makes it amenable to analysis.

• Trick: For focus groups, create a diagram of the table around which participants are sitting, with identifying initials or a number for each person. Add that identifier next to each person’s comments when taking notes.

       

 

• Trick: Record participants’ responses near the questions to which they refer, not necessarily when they occurred during the interview/focus group.

• Trick: Use templates for recording data when possible.

          Examples:

• MS Excel template for focus group responses              

Participant	Q1: Barriers to Physical Activity
P1	Time & money
P2	Money
P3	No bike lanes
P4	Motivation
P5	Sidewalks; streetlights

 

• Data abstraction template (e.g. a table of information you want to make sure you collect) for transcript review

Unique ID	Family Situation	Issues Addressed with Case Manager
A0154	-Single mother with 1 son -Recently divorced -Son’s father not living in the country	-Trouble finding childcare -Cannot find full-time job

• Observation templates

Date	Time of Day	Site	# of Adults Present	# of Children Present	Activities
8/4/2013	10-11 AM	City Park A	2	6	Rollerblading, Biking

• Interview/focus group guide itself as a template (record notes in spaces following each question)

• Use an online tool like SurveyMonkey to distribute diary log ‘templates’ to respondents if looking for similar information over time from same people (minimizes unnecessary re-entry of data)

Tip: Always audio-tape if possible. Useful for:

–     Verbatim transcription

–     Writing detailed summary notes

–     Verifying you captured important points

–     More rigorous analysis at later date

Tip: If you are unable to audio-tape an interview for any reason (e.g. the participant is not comfortable being taped), although not typically done, it is helpful to have two research team members attend the interview: one responsible for facilitation and the other for note-taking. (Note: for focus groups, it is common and recommended practice to always have two team members present.)

Tip: Align the level of audio recording transcription with the overall purpose of the evaluation – it is not always necessary to transcribe interview or focus group content verbatim, and notes can often suffice. You can go back to the audio recording for clarification or specific quotes as needed.

       – Verbatim transcription = Expensive!
       – With a professional service:
          1 hour tape = ~4 hours transcription = ~$150-$200 (or more!)

Tip:  For large amounts of data (either length of transcripts or number of transcripts), it is helpful to use qualitative data management software.  Examples:

    – QSR NVivo
    – Atlas TI
    – EthnoNotes
    – Ethnograph
    – HyperResearch
    – Dedoose

Tip: Small amounts of data may be easier to manage/code/analyze by hand.

These are just a few ideas to help you organize and manage your qualitative data from the moment you collect it.  While this may make you feel a little more confident with qualitative data collection, even well organized data can feel overwhelming just by the sheer quantity of it!  The next post in this series will hopefully help you tackle the next step: qualitative data analysis.

For more information about the Institute for Community Health, please visit our website http://icommunityhealth.org/. Examples of our work and collaborations involving qualitative research are described here.

 Note: different projects require different methodological approaches. This is not an exhaustive list of data management techniques. You may find other techniques fit your project’s aims more effectively and appropriately.

------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.

Hello Qualitative Methods, It is Nice to Meet You!

Quick Introduction to Qualitative Methods for Community Partners 

Part 1 in a 4 part series

By Eileen Dryden, PhD & Julie Carpineto, MFA

Ever wonder why people engage in certain behaviors? Do you want to know more about how people make decisions about their behavior and how they feel about what they do? If so, you may be interested in using qualitative methods to explore your research or evaluation questions.

While many of us in the research and evaluation field are well versed in the benefits of using qualitative methods, some people seem to shy away from using them due to unfamiliarity and rely exclusively on quantitative methods instead.  This blog is intended to provide a quick introduction to qualitative methods: what they are, when they are most useful, and some guidelines for effectively using a few popular qualitative methods.

What are qualitative methods?

Qualitative methods collect descriptive data on phenomena.  They are often used to gain an in depth understanding of human behavior, investigating the why and how of behavior, not just the when, where and what.  They are useful for gaining insight into people’s attitudes, behaviors, experiences, feelings, concerns and motivations. Additionally, qualitative methods can provide context and depth to questions of interest and are especially useful for exploring a construct about which little is known.  The results of qualitative data can be quite compelling as responses are documented in the individuals’ own words.

However, there are limitations to consider when deciding whether or not to use qualitative methods, including the relatively large amount of resources needed for implementation and the related small sample sizes.  Qualitative methods are labor intensive – especially transcription and analysis!  Consequently, it is generally only feasible to collect data on small numbers of individuals. As a result, one must be strategic about choosing a sampling strategy and subsequent findings are often not generalizable.

Despite these limitations qualitative methods are commonly used to great effect!  Some popular qualitative methods are interviews, focus groups and observations. 

Popular qualitative methods:

1) Interviews

Let’s start with interviews. Qualitative interviews are one-on-one discussions with an investigator and a selected respondent. Interviews can be informal, more like conversations; unstructured, where particular general topics are covered; semi-structured, where there is an interview framework  that guides the discussion; and structured where respondents all answer the same questions in the same order.  Interviews are great for exploring a topic in depth with individuals and are an especially good method if the topic of interest is a sensitive one.

2) Focus Group

 

A focus group is a facilitated discussion with about 7 to 10 people on a particular topic.  This method is very useful when you’d like people to be truly discussing a topic, responding to each others ideas and opinions.  In contrast to interviews, focus groups are not recommended for sensitive topics and work best for topics that lend themselves to a friendly group discussion.  In order to stay focused and make the most of the time allotted, good facilitation skills are a must. 

3) Observation

The method of observation is especially useful to gain insight into experiences that may be hard to explain or put into words.  Observation is generally either participatory or ‘unobtrusive’.  During participant observation, the investigator is engaged in some way with the people he/she is observing (for example, taking part in the training that is being evaluated).  During unobtrusive observation activities, the investigator remains separated from the people he/she is observing – for example when a researcher observes the use of a fitness trail by sitting anonymously at the side of the trail and recording what he/she sees.

Those are few of the most popular qualitative methods. Hopefully this quick introduction has enticed you to explore the use of qualitative methods in your next investigation.  If you do start to use these methods you may wonder “what do I do with all this data!?!”  Stay tuned for the next blog in this four-part series: tips and tricks for qualitative data management.

Note: different projects require different methodological approaches. The methods presented here are examples of popular qualitative methods, but it is certainly not an exhaustive list. You may find other methods fit your project’s aims more effectively and appropriately.

 ------------------------------------------------------------------------------------------------------------

The views expressed on the Institute for Community Health blog page are solely those of the blog post author(s), and do not necessarily reflect the views of ICH, the author’s employer or other organizations with which the author is associated.