NOTE: This feature is currently in limited release. Please contact email@example.com if you are interested in learning more.
The ability to detect unique themes specific to your company, or a particular survey, is now available on engagement and other culture type surveys (e.g. Wellbeing, Values) surveys within the comments report.
We use machine learning techniques that can automatically detect groups of comments based on shared words or patterns of words they contain. These groups, or clusters, are highlighted as potential Emerging themes for you.
Different to our Topic analysis, where comments are tagged with one or two of our 18 pre-learnt Topics, our Emerging Themes algorithms can find themes that are unique to your business or comment set. For example, if people have commented about an office in a new city, the Emerging Themes algorithms are designed to pick up on this and will likely detect comments mentioning the city (even if the new city isn't well known).
The Emerging Theme algorithm works by finding themes within comments that have higher levels of similar word combinations and patterns. To make this more meaningful we do some pre-filtering to remove very common words and weight fairly common words less strongly. The algorithm itself works by iteratively building up groups of comments, assigning each comment to the best-known theme and repeatedly refining, We also use statistical analysis on the clusters as they are built up to determine the optimal number of Themes.
We know it is common for people to feel the need to read all of the comments. This is done for various reasons;
- to feel you have a better sense of what people are saying / feeling / needing
- to look for sensitive comments
- to identify possible opportunities, ideas or areas of focus
We know this process can be time-consuming and often there is time pressure to produce a report for executives or other stakeholders. We saw an opportunity to make this process a little faster and easier by using machine learning to pre-identify some potential key themes.
You can see the Emerging Themes feature by clicking into the comments report.
At the top of the screen you'll see two tabs;
1. Topics – This is the comments report you are used to seeing. It contains a topic frequency bubble chart, the comment filters with the topic tags and the sentiment analysis.
2. Emerging Themes – this is the new feature that detects unique themes specific to your business.
This first page contains potential themes our algorithms have detected.
- The bold words listed are the most frequent words in the Emerging Theme
- Below that are 3 example comments from the Emerging Theme using these words
- To the right you will find the overall sentiment associated with that Theme
- On the right you'll see the number of comments in that Theme and what percentage of total comments this Theme represents.
If you click into a specific Theme you can see a little more detail.
- A full list of the most common words found within that Theme
- A list of all the comments associated with that Theme
Emerging Themes for an entire survey – At the moment we only show Themes on questions that have a minimum comment count of 250. Going forward we want to enable Themes across an entire survey
Exporting Emerging Themes – Allows you to export the comments from any Theme
Standard filters – These filters are the standard filters you are use to seeing on the topics tab. Bringing them into the Themes tab will allow you to sort and filter within a cluster.
Show translated comments – At the moment our themes are built on all comments that our translation service can translate to English. We do not however show the translated version of the comment in the themes tab – these comments are only shown in their original language. Soon we will enable all the comments in English.
When is Emerging Themes available?
Upon survey close. If the survey was closed after January 2019, ET will automatically be triggered for you. If a survey closed prior to this date your CS coach or support team will be able to run this feature on that survey for you.
How soon is Emerging Themes available after a survey has closed?
About 2 hours after close
Who can view Emerging Themes?
Emerging Themes is available to survey admins within the comments report and anyone who can see a shared report where all comments are present. Emerging Themes is not shown when a filter has been applied.
Why is this feature only available for some questions?
We've tested our algorithms across many surveys and we've seen best results on questions with a minimum comment count of 250 comments or more. For this feature we have only run Emerging Themes on questions that meet this comment count.
Can I see Emerging Themes on an entire survey?
At this point no but it is coming next.
Can I see Emerging Themes on old surveys?
Yes this is possible. For any survey closed prior to Jan 2019 you will need to get your coach to run this for you.
Can I turn Emerging Themes off for shared reports?
Yes this is possible. Please contact your customer support rep to disable this feature.
Why do I see the Emerging Themes tab but no themes?
Emerging Themes is triggered upon survey close. It does take some time to run in the background, while this is happening you will see a message where Emerging Themes will appear. It may also be that questions in your survey didn't meet the minimum comment count for this feature to work. The messages shown on the tab should make it clear why you are not (or not yet) seeing Emerging Themes.
Can I filter on Emerging Themes?
Demographic filters have been turned off for MVP. These are still available in the topics tab.
Who can edit Emerging Themes labels?
All Survey Admins can edit the labels, these will override any previous edits
Any other questions please contact our support staff
REMINDER: at time of publication the release date for this feature is 15th July 2019