While it is possible to test your survey using the preview functionality, we strongly recommend that all survey authors pilot their surveys before releasing them to their target audience.
When you pilot your survey, you launch a test version of your survey and ask friends, colleagues and reviewers to complete it. You can also complete the pilot survey yourself. This is a more robust test of aspects such as routing and piping than the survey preview, and a pilot run gathers response data which you can view in, and export from, the Analyse page.
Here are a few reasons you should always pilot your survey:
To check that the results data is meaningful
This is one major aspect of your survey that you cannot test in the preview. Meaningful results data is the reason for running the survey in the first place. Piloting your survey may highlight areas where certain questions do not return the information that you need. For example, you may find that the flexibility of a free text question is great for your respondents, but you end up with the same answer reported in so many different ways that sorting the data usefully becomes incredibly difficult. Equally, you may find that a selection list question does not provide the answer options that respondents need, and they are constantly choosing “Other”. It is much better to find this out in a pilot and fix it, than gather thousands of genuine responses that you cannot usefully analyse.
To ensure that the survey displays correctly for all respondents
Your survey may look fine on your screen, but what about respondents using mobile phones, or those using assistive technology such as screen readers? It is worth testing your survey using a variety of devices and browsers to ensure that it displays as you expect for all respondents.
To find out whether respondents understand your questions
You are very familiar with your research topic, so your questions make perfect sense to you. However, someone with no background in your research area may not understand the terms you have used, or they may interpret a question differently from the way that you intended it to be interpreted. You may also find that respondents whose first language is different from yours interpret questions differently from you.
To run a second, third and fourth pair of eyes over it
Typos can be difficult to spot. Having several other people go through your survey will ensure that as many errors are spotted and resolved as possible.
To ensure that your survey complies with your ethics requirements
If responses must be anonymous or confidential, and/or if participants must be able to withdraw their responses at any time, running a pilot will allow you to confirm that your survey has been set up with the appropriate safeguards. You can, for example, check that you have put effective processes in place at your end to ensure that a request to withdraw a response can be honoured.
To discover and address any practical problems
There are many potential practical problems, so only a few examples will be provided here. Does your survey take too long to complete? Are there any firewall-related issues that prevent your respondents from being able to access the survey? Does your email client disable the links in your survey invitation emails?
To allow for expert review
Piloting your survey allows external experts who do not hold online surveys user accounts to test your survey and provide you with feedback.