Benchmarking

What is benchmarking?

A benchmark is a standard against which something can be measured.  Benchmarking in online surveys involves comparing your survey results against the results from other matching surveys, either over time or across institutions.

What are the advantages of benchmarking in online surveys?

We have 10+ years’ experience supporting national benchmarking surveys.  When you carry out benchmarking in online surveys you can:

• Create your own survey with questions that are directly relevant to your subject area.
• Set up your own benchmarking groups containing the surveys (and combinations of surveys) that you choose.
• Get quality support from the online surveys team.

How do I carry out benchmarking?

Online surveys offers two ways that you can carry out benchmarking:

1.  Run multiple copies of the same survey from your own account, and use the Compare functionality to see improvements/changes.

The Compare functionality within online surveys allows you to compare results from structurally or textually identical surveys, to identify similarities and differences between the survey results.  This could be used to, for example, benchmark employee satisfaction survey results within a single institution over a period of years.

See our Help page on comparing responses from several surveys for more information on how to use the Compare functionality in online surveys.

2.  Run the same survey across a group of institutions and compare your results against an anonymised aggregate.

Online surveys has been used to run UK-wide benchmarking surveys since 2002.  The general process for running a multi-institution benchmarking survey is as follows:

• A survey template is developed by one institution.  This institution manages the benchmarking survey, recruits the participating institutions, and acts as the central point of contact for (non-technical) queries about the survey.
• The institution managing the survey is responsible for obtaining agreement from each participating institution that they are willing to take part and have their data visible to other members of the group (even if this is purely in aggregate form).
• The online surveys team checks and locks the survey template so that the questions cannot be edited.  This is necessary to ensure that the comparison process works correctly – if questions are moved or changed the results cannot be compared.
• If desired, the survey can be set up so that each institution can edit one section of the survey to include their own questions.  These questions will not be included in the benchmarking data.
• The template is made available to the participating institutions for them to copy, add their own questions (if relevant) and distribute.
• The online surveys team provides technical support to the participating institutions on the use of online surveys to run the surveys.
• When all the surveys are closed, the online surveys team liaises with the managing institution to create the benchmarking groups.  These groups combine the surveys and aggregate the results data to produce the benchmarking reports.  It is possible to have several benchmarking groups – for example, a main group containing all of the participating institutions and then sub-groups for particular regional areas.  Benchmarking groups must contain a minimum of 5 surveys.
• Once the groups have been created, the participating institutions can access benchmarking reports which show them their results compared against an anonymised aggregate of the results from the surveys within the chosen benchmarking group.
• The managing institution is also given access to the benchmarking reports.
• In some cases, the total response data for all benchmarking surveys (excluding any institution-specific questions) is exported from online surveys and provided to the managing institution.

How much does benchmarking cost?

If you are making use of the Compare functionality for surveys run from your own online surveys account, there is no extra charge – this is part of the core functionality of online surveys.

If you wish to run a benchmarking survey across several institutions, there is an additional charge for the work carried out by the online surveys team in supporting the benchmarking survey and generating the benchmarking reports.  The cost depends on the complexity of the survey, the number of participating institutions and the extent of involvement by the online surveys team in the running of the survey and the benchmarking process.

Each participating institution will need to hold a Project or Organisation licence to access the benchmarking reports.  There is no additional charge, over and above the annual licence fee, for participating in a benchmarking survey or accessing the benchmarking reports.

Please contact us if you would like to discuss online surveys’ benchmarking capabilities further, and/or if you would like us to provide a quote for supporting your benchmarking survey.

What information is contained in a benchmarking report?

The report contains the following information:

1. The total number of responses across all surveys.
2. The number of surveys aggregated into the comparison, the names of the surveys and the institutions they belong to.
3. A table for each question, containing the following information for each answer option:
   • The responses from your survey.
   • The aggregated responses from all of the other surveys in the comparison group, excluding your survey.
   • The aggregated responses from all surveys in the comparison group, including your survey (where applicable).

The report can be filtered, downloaded as a PDF, and individual comparison tables can be exported in a variety of formats for more advanced analysis.

Free text questions and date and time questions do not contain data which can be compared across multiple surveys and are therefore excluded from the report.