Follow

Survey Methodology


A question we are often asked is "what is the right cadence for running surveys?". The answer is always ‘it depends’, but following is an approach that we are seeing work well for Culture Amp customers. We find it useful to think about four surveys over the period of a year (quarterly).

 

 

We describe these individual surveys under the survey types, specifically the baseline, trend and diagnostic surveys. We won't repeat these definitions here, but instead describe how they combine into a comprehensive People Intelligence methodology. 

Quarterly Structure

We break a year down into four surveys, one per quarter. Four surveys may seem like a lot, however, we've deliberately taken an approach that does not involve repeated, large surveys. Indeed, the second and final quarters are short, focused trend surveys that should only take 5 minutes. Whilst we do see this being successful in many contexts, you don't need to get fixated on quarters. This methodology provides a backbone and narrative that can be adapted to suit the needs of many organizational cultures. Part of the onboarding process for Culture Amp will be finding the right sequence and cadence to suit you.

Quarter 1, Baseline Survey

A baseline survey establishes a broad baseline for your culture. These surveys tend to be comprehensive, but we still recommend keeping to a maximum of 50 questions. An employee should still be able to complete this survey in 5-10 minutes. The baseline survey measures employee engagement as the primary outcome. This is the thing you want to understand and improve. We call this your "index". The baseline survey gives you the platform to iterate and take action.

Event-Based Surveys

Before we dive into your next organizational survey (Q2), it's worth touching on Event-Based surveys. Event-based surveys provide surveys that are continuous - aligned against particular events for your employees. Examples here include candidate, recruitment, training or on-boarding surveys. Many organizations implement these survey types first, depending on the most pressing need. However, it is also common to introduce these after the first baseline. The primary reason being the baseline provides context to interpret the data from the event-based surveys.

Quarter 2, Trend Survey

Trend surveys are a shorter survey intended to track progress on activities resulting from your baseline survey. Are you moving the needle in the way you expected? Some organizations repeat the baseline survey again, but we usually recommend going with a shorter survey that targets your driver and engagement questions. This keeps employees focused on your action areas and is a useful way of reminding employees of the activities you have underway. It reinforces your commitment to these actions and re-engages the employees in that. So, if your baseline survey was 50 questions, the trend should be 10-15 questions. Employees will be able to complete this in less than 5 minutes. In some cases you may be tempted to sample only a portion of your employees. However, we generally recommend against this as sampling can lead to results that are not representative of the organization or demographics within it. If your survey is short and targeted, your employees won't mind giving you 5 minutes to let you know how you're going anyway. Hopefully the trend survey shows the outcomes that you were expecting. However, the aim here shouldn't just be to validate your actions. This is an opportunity to refine and fine-tune your initiatives. Even with great action planning, circumstances change and you may need to change tack. So ask questions that best inform this. On a side note, we often get asked about the role of eNPS (Employee Net Promoter Score) questions and where they may fit in our methodology. We'll dedicate a different post to this, but trend surveys are good candidates for eNPS-type questions. These questions are a good indicator on how employees are feeling overall.

Quarter 3, Domain Diagnostic

The domain diagnostic is a bigger survey than the previous trend survey. We’d recommend 25 to 50 Questions, of which around half will be specific to the domain of interest, and the rest will track your drivers and engagement trend to ensure efforts are still on track. This survey should take 5 to 10 minutes to complete. Domain diagnostic surveys focus on something like Manager Effectiveness, Leadership, Values or Benefits. The sort of topics important enough to require some detailed understanding, but would be too difficult to get a full understanding of in your baseline survey. Being able to ask more questions on this topic lets you actually diagnose where an issue lies. So from a Benefits diagnostic, you won't just understand that there is an issue with benefits, you'll actually understand which Benefits are working for your employees and which ones aren't. Our classic example of a domain diagnostic is the Manager Effectiveness survey. This may not be a key driver of engagement, but as your managers are usually the real drivers of effective change in your business you want them to be self aware and readily equipped with all of the feedback that they need to really be effective. The Manager Effectiveness diagnostic will ask detailed series of question on how employees feel about their managers. This can then be used to provide detailed action planning for each manager - things like training, mentoring and cross-skilling. At the end of Q3 you now not only have hard data on improving engagement, but you're moving into a more sophisticated understanding of what makes your culture tick.

Quarter 4, Second Trend Survey

The most basic form of the Q4 Trend Survey is a repeat of your Q2 Trend Survey. We don't recommend a wholesale change to your trend survey - you should still track the engagement questions and drivers from previous surveys, but you may want to introduce some key concepts you've learnt throughout the year. In particular, you may want to tweak your questions based on the results of the Q3 Domain Diagnostic. One or two questions may be used to track initiatives resulting from this diagnostic and give some quick feedback on the impact they are having. In addition, this is a good time to prepare for your next baseline survey. In particular, you could consider an action measure - For example, "Have you seen positive change from previous surveys?" or "What is one other question should we be asking?". That said, it's important to keep to the intent of the trend survey. It should be a short, tracking survey. If in doubt, it's best to err towards a simpler survey and leave out anything that adds complexity. 

Putting it all together

Once you're done with the final trend survey, you've now completed four surveys over the course of a year. Not only have you gathered sophisticated understanding of your organization, but you've taken action, measured the results and drilled into key cultural concerns. This is a huge departure from the traditional annual survey that takes months to cascade to your employees and a year to get real data on if things are headed in the right direction. In my final blog on this topic, I'll put all this together and give an overview of how and why this all works. So why run four surveys when you could just run one?

  • The data is more relevant: Annual surveys are great for understanding your culture. However, you're doing all of your action planning and execution off a single data set. Waiting until the following year to see if you had a positive (or negative) effect is a lost opportunity. We don't do this with sales figures, company financials or project plans. So why are our people metrics any different? With any feedback system, your decision-making ability is either improved or confused by how "fresh" the data is that you’re working with. Running a survey every 3 months means you’re not left guessing if your efforts are having an impact. You can determine your next actions based on concrete data.
  • The data makes you more accountable: A lot happens in a year. Projects wrap up, new projects start, people leave, people join, people take new roles and companies restructure... you get the idea. If you only collect data once a year, that data can be susceptible to immediacy biases. This will confuse results and can be used as dismissive excuses. Have you ever heard "... we ran the survey at a really bad time for my team so my results aren't really representative"?. This thinking can result in companies only wanting to run surveys when things are "looking good". Regular surveys iron out these biases and remove the idea that there is a good or bad time to survey. It makes you accountable for your data, as the big rocks will be prominent in each survey.
  • The data empowers your people: Giving your people a simple way to provide feedback and visualize change as a result of this feedback makes them feel like they're part of the solution. Survey fatigue is not an issue. We continually hear it in our feedback - Employees don't mind contributing when they see real action as a result of their feedback. It's 'lack of action' that creates fatigue. 5 to 10 minutes for each employee, each quarter, is a very small cost for the potentially huge return where your employees feel that they have a real voice.

There are many benefits to this methodology, but this is a starting point only. The trick is to learn quickly. If you're doing it right you will be able to improve the questions you ask, improve the action you take, and ultimately figure out the approach that helps you amplify the positive impact that your people can have your company.



Was this article helpful?
0 out of 0 found this helpful
Have more questions? Submit a request

Comments