People often focus on metrics such as contributor counts and engagement rates to measure the success of an engagement. However, effective community engagement must also include qualitative data.
Qualitative data is important because it can help you understand why participants choose a particular option or respond a certain way. This information can give you deeper insight into your community and help you make more informed decisions about your consultations.
Your analysis of qualitative data is equally important. With effective qualitative analysis:
Key questions are answered in depth.
You can understand why people answer a specific way, instead of just what they think.
Demographic input, trends, and relationships are considered, for example, to determine whether older people are more likely to feel a specific way or whether specific suburbs are more likely to have a specific opinion.
Qualitative data can be quantified, for example, using answer counts, analyzing the number of people who responded to themes, and analyzing the overall sentiment toward those themes.
How to Collect Qualitative Data
In EngagementHQ, you can collect qualitative data for each tool, except for Quick Polls. This includes:
Posts in the Ideas, Stories, and Guestbook tools.
Single line and Essay questions in Surveys and Places pins.
File submissions uploaded by participants in a survey.
Questions asked in the Q&A tool.
Comments on Forum discussions, Places pins, Ideas posts, Stories, and Newsfeed articles.
There are also several offline options, but remember to be consistent in framing your prompting questions to help consolidate your data later. Offline options include activities like mail submissions, conversations had in drop-in events or meetings, and workshops.
Analyzing Qualitative Data
Before analyzing your data, you should have a basic plan for creating your report. For example, if you’re using key themes or sentiments to create graphs and pull quotes, you should identify your themes before you start. If you know which questions you need to answer in your report, it will be easier to quantify your qualitative data and write your summary content.
With this done, you can start reading responses and using our Text Analysis reporting tool to:
Search for themes by keywords or phrases and tag responses accordingly. You can use the tags to code your data and help you understand why and how participants are responding to the theme.
Use the Sentiment analysis to confirm the tone of the response. In EngagementHQ, a sentiment is assigned when you turn sentiment analysis on, and admins can then read the responses and change or confirm the assigned tone.
Use demographic filtering to understand how specific demographic groups feel about your consultation issue. For example, it may be essential for your organization to understand how parents with school-aged children feel about the new road speed proposal or what the older population of your community needs in the new aged care policy. Demographic questions must be built into your engagement activities before you collect the data, either through registration or the activities.
With your data analyzed and coded, you can use the export option to download an Excel report of the responses, tags, and sentiment. You can then extrapolate the data to quantify and graph your qualitative analysis.
Tips for Thematic Coding
Coding your qualitative data refers to assigning themes to text-based responses. In EngagementHQ, you can do this using tags in Text Analysis. Coding helps you analyze your data because you can understand what participants are responding to and why.
Coding can seem like a difficult task; here are some tips to help you:
Choose how you want to code your data, will you use key themes, overall sentiment, tags for specific questions, or tags for related documents or plans?
Try to determine your code frame by reading a cross-section of your data for key repeated phrases and then working through the rest of your submissions using these are your key themes.
Create a brief legend for the code stating what each code phrase means and what it refers to, so your team is on the same page and can help you analyze your data.
Understand the limitation of relevance – is a theme relevant if 15 out of 100 people mentioned it? Does the baseline need to be at least one-fifth?
Don’t overcomplicate it. If you end up with too many themes, you risk having statistics and data that don’t tell you much of relevance. We generally recommend no more than 20 themes.
If you have too many themes, consider using subthemes to categorize as well. If you use subthemes, then you can first extrapolate using the overarching theme before analyzing the subthemes within it. However, to avoid overcomplicating things, we recommend using subthemes sparingly.
For big or long-term engagements, it may help you to start coding and analyzing your data before the consultation closes. This will also give you an idea of any missing data or demographics you aren’t hearing from, which you can target your communications towards.
Collate and Report
With your data coded and analyzed, you can start collating and reporting it. Here are some ideas for your reports:
Collect and list overarching themes before deep-diving into specifics using a summary and key (anonymized) quotes. For example, a survey question identified safer playgrounds as a key priority for Council and has a follow-up of “What does that mean to you?” In the reports, the key theme of “safer playgrounds” was summarized from the overall comments, and a range of anonymized quotes from participants of various demographics were included.
Create an Excel spreadsheet with columns to identify the data source, list the response, provide options for coding, and identify the demographic.
Use Excel to create charts and graphs, such as pivot tables, to compare codes with demographic groups.
Extrapolate your data into infographics that help summarize your findings.
While it is helpful to quantify your qualitative data, you also need to remember that the point was to find out why people may be responding to specific topics or themes. Your reports should reflect this, so don’t be afraid to use texts and graphics to contextualize your findings.