A/B testing a banner design in a production environment is not a quick process. Days or weeks can pass before reaching a point of statistical confidence in the results, depending on the amount of traffic you get. This makes a quick sanity check with a preference test or five second test a worthwhile proposition if you’re designing and running a lot of banner advertising.
Testing banner ads on with UsabilityHub will give you a good steer on whether or not users understand what you’re trying to communicate, and what sort of visual design preferences resonate well with your message.
Using a combination of tests is recommended when analyzing banner designs, as each test type excels in finding out about particular attributes of your designs:
Preference tests give you an easy metric to determine a winner. A five second test or question test requires you to look at individual responses to find the insights but tagging the sentiment of responses with ‘positive’, ‘neutral’ and ‘negative’ labels can help you analyze the results of these tests.
The value of looking at responses can be seen in the sample test results. There’s a clear winner in the preference test section of the sample test — the banner with house image, rather than team or handshake — and there’s a gem in the responses too.
The first and third responses talk about the nuts and bolts of the design, but the second response articulates perfectly why the house image outperforms the other two images. It speaks directly to the aspiration of owning a home for these participants.
Feedback like this can give you clear guidance in what can be a confusing process, and help you iterate through designs rapidly in the right direction.