Making the right questions to collect useful data 

A survey is a quantitative research method that consists of a set of questions used to collect information from a representative sample of the target audience. It can be used to assess their preferences, attitudes, characteristics and opinions on a specific topic.  

Surveys can be conducted at any stage of the development process. They can be a really useful tool to provide input for the design process as they help to better define and understand the customers or user base of the product or service. They can also help mitigating the risk of designing a poor solution for users. 

Creating an effective survey 

To start building a survey there must be a clear understanding of the needs and information required to meet the project objectives. Then, based on the information that needs to be collected, a set of research questions is defined. Relevant questions and a good survey design are crucial to generate quality data and make sure participants don’t drop out.

How to write good questions?

• Questions should be relevant to survey’s goal. 
• Questions should be concrete, unambiguous and clear, as there’s no opportunity for the participant to clarify them.  
• The language used has to be easily understandable by everyone, so words that are not commonly known should be avoided. 
• If a technical term really needs to be mentioned then it should be clarified.
• Biases and too personal questions should be completely avoided. 
• Sentences shouldn’t be too long. 
• Each question should convey a single idea because if it conveys two ideas it’s possible that the person as a different opinion about each one. For example, instead of asking “Do you like pineapple and strawberries? Yes/No” it should be “Do you like pineapple? Yes/No. And strawberries? Yes/No”. 
• The survey should begin with an easy question, leaving the harder ones to the end as they are more likely to demotivate participants and eventually make them abandon it. 
• The survey shouldn’t mix two different topics.

The most effective and engaging surveys are kept short, focused on one topic, and contain a mix of closed and open-ended questions. While close-ended questions (rating scales, ranking, multiple choice, etc.) are good to gather proportions of feedback and priority, open-ended questions (open text fields) can provide more context into what is driving the feedback. However, open-ended questions should be restricted to the minimum and used only for comments as they are harder to evaluate quantitatively. 

Defining the right sample

Surveys are a cost- and time-effective way to understand a topic, if only the appropriate people fill it out. For example, if we want to know more about the use of a certain product, we will want real users of that product answering the questions. To make sure the survey doesn’t hit the wrong audience, we can set some questions that act as a screener. This will ensure we get the right data from the right people.

Besides choosing the right sample, it’s also important to define the sample size. In behavioural surveys the ideal is between 50 to 150 participants and to obtain statistical relevance it should be answered by more than 80 people. 

Making sure the survey hits the spot 

Before distributing the survey is a good idea to run a pilot test in order to validate the questions’ structure and detect possible errors. It should be carried out in the same context as the real test and ideally by 2 or 3 people who correspond to the profile being recruited. If a lot of mistakes are identified during the pilot, then they should be corrected and a new pilot run afterwards. 

Sending out the survey to the audience

The survey distribution can be done in-person or electronically through a survey tool, email or phone. The title or subject should be catchy and it helps a lot if it’s possible to offer some kind of compensation for taking the survey (like a gift card or discount code). It’s also important to include a small description of the main topic and the approximate time needed to complete it.