Skip to Main Content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

User Testing: Web Scale Testing

Overview

Web Scale testing is a method used to collect feedback from a large number of users about different aspects of a website. A variety of tests come under the umbrella of web scale testing including:

  • Five second tests
  • First click tests, surveys
  • Preference tests. 

How Many Users?

By the nature of web scale testing, it can be conducted on large amounts of anonymous users. 

There are multiple websites to conduct these types of tests on, that provide users for you.

  • Usability Hub: https://usabilityhub.com/ - a free website that allows you to do all the above methods with some limitations, but has pricing options if you wish to do more detailed tests.
  • Survey Monkey: https://www.surveymonkey.com/- a subscription based service that allows user survey testing.
  • Optimal Workshop: https://www.optimalworkshop.com/- a free website that allows first click testing and user surveys. This website also has pricing options for better customization.

Usability Hub is an attractive choice, as all the test data is in the one central location, and it has the most testing methods.

Five Second Tests

In five second testing, users are presented with an image/page/design and five seconds to look at it. Afterwards they are given some questions to answer about what they saw. This method is a good way of getting a user’s first impression of a design and making sure they understand the intention/purpose of the design.

Questions may include:

  • What is the purpose of the page?
  • What stood out to you?
  • Who do you think the intended audience is?

 

When to do five second tests:

It is only necessary to use five second testing when you want a user’s first impression of a design.

Why do five second tests?

Five seconds may seem like a short amount of time to answer these questions, but studies have shown that users decide in the first 10 seconds of visiting a page whether they will stay on it or not.

First Click Tests

In first click tests, analysis tools allow us to view heat maps of the most common places users click, and also the time it took them to click.

There are two common ways to apply this type of testing.

  • Performing tasks- questions can be phrased such as “Click the link to view library open hours”
  • Finding out where users would expect to perform a task- questions could be phrased such as “Click where you would most likely expect to find library open hours”

When to do first click tests:

First click tests should be used to make sure the users understand the core navigation of a website.

Why do first click tests?

Jeff Sauro states that if users make the correct first click when performing a task, there is an 87% chance that they will go on to complete the task. On the other hand, an incorrect first click only has a 46% completion rate.

This testing method should be used to make sure users understand the navigation of your website.

Design Surveys

Design surveys can be used to ask a large number of users their opinions on a product design. A user is presented with a design and asked several questions. In contrast to a five second test, the design does not disappear whilst they answer questions. 

When to do design surveys:

Design surveys can be used when you want to get an idea of how a user feels about your website. Questions could be designed for controlled answers (yes/no, 0-5 rating), or more open-ended answers, where a user gives their opinion.  Whilst both types have their benefits, controlled answers are much easier to sort and quantify.

General tips:

  • Keep questions neutral- you don't want your opinion to sway the answer
  • Keep surveys short- research has shown that 52% of people doing surveys would not spend more than 3 minutes to complete one.

Questions can include:

  • What was your first impression of the website?
  • What do you think its purpose is?
  • Does anything frustrate you about this website?

Preference Testing

Preference testing is used when you have multiple design ideas and aren’t sure which one to use. The user is presented with all of them and asked which they prefer. Both qualitative and quantitative data can be gathered from preference testing.

Quantitative data includes the number of users that prefers each design. You can also gather the time it took the user to make their decision. A quick decision may indicate they feel strongly about their design choice, whereas a longer decision may indicate a tougher decision.

Qualitative data can be gathered by asking follow-up questions once they have made their choice, such as why they chose it.

This type of testing differs from a/b testing in that you are not trying to get the user to perform a task on different variations of a site, you just want their opinions on the different variations.

When to do preference tests:

When you can't decide on the best design variation.

 

Recommendations

  • First click tests best used soon after prototype creation to test navigation
  • Five second tests best used soon after prototype creation to test website message/purpose
  • Design surveys to be used at any time to gather opinions on design
  • Preference tests to be used at any time to gather opinion on which design to use

Charles Darwin University acknowledges the traditional custodians across the lands on which we live and work, and we pay our respects to Elders both past and present.
CRICOS Provider No: 00300K (NT/VIC) 03286A (NSW) RTO Provider No: 0373 Privacy StatementCopyright and DisclaimerLibrary Webmaster • ABN 54 093 513 649