Question score calculation
Scores are reported as percentages showing the percentage of people who agreed/were neutral/disagreed to a question out of all people who answered the question. By default the favorable score is shown, meaning people who chose a favorable response like 'Strongly agree' or 'Agree'.
As it's possible for people to skip questions, we do not use the total count of participants when calculating scores. We only use the count of people who left a response to a question.
The neutral and unfavorable scores are also shown, often when hovering over an item.
Scores are rounded to the nearest whole number (using only one decimal place to decide the rounding), except for neutral scores, where neutral is calculated as 100% minus the favorable % and unfavorable %. This ensures that scores always add up to 100%, rather than have rounding cause a total of all 3 scores to be more than 100%.
Factor score calculation
A factor score is an average of the scores of each question within the factor, BUT only people who answer every question in the factor are used for the calculation.
If a person skips a question, then their responses for other questions in that factor are not included in the factor score calculation.
Beyond calculating the favorability, we also have to consider how benchmarks will be calculated. Never let losing a benchmark keep you from improving your survey! Below we outline how benchmarks are determined at both the question and factor level.
Question benchmark calculation
When determining if an item will receive an industry benchmark, we look for if the intent of the question is the same. For example, changing the word "manager" to "mentor" maintains the same intent if that is common lingo in your organization. However, often companies change the intent accidentally. For example, changing "We have enough autonomy to perform our jobs effectively" to "I have enough autonomy to perform my job effectively" can tap into social desirability bias. An individual will be more likely to say they have enough autonomy than if you ask about the collective autonomy within the organization. Please reach out with any customizations you've made and we'd be happy to let you know if the item can still be benchmarked.
Factor benchmark calculation
You will receive a factor level comparison if:
- No customized items have been added
Note: You may remove items and still have a factor comparison to the industry benchmark.
When comparing to your historical data, you will receive a factor level comparison if:
- The number of questions stay the same
- All questions are matched across the two surveys
- The factor name stays the same
If you're unsure how your benchmarks will be generated, please reach out to firstname.lastname@example.org.