Eight user testing case studies that achieved amazing results

User testing is an integral part of web design as it gives impartial and quantifiable insights into the customer’s experience.

As discussed at our recent JUMP multichannel event, usability testing removes any ego or opinion from product design which helps improve customer satisfaction and ultimately increase revenue and growth.

There are a number of different methodologies for user testing, including A/B or multivariate tests, heatmapping, or videos of real customer journeys.

As these case studies show, each different methodology can identify small changes that lead to massive increases in CTR, conversions and sales.

For more information on this topic, checkout our new Conversion Rate Optimization Report 2012 in association with RedEye. 

The research looks at tools, strategies and processes employed for improving conversion rates, as well as examining different areas of best practice.

Sidelines increases landing page conversion from 5% to 55%

When sports website Sidelines first started out it found that, despite a steady flow of traffic, only 5% of visitors actually signed up to its service.

So to try and increase the conversion rate co-founder Vinay Kuruvila ran six A/B tests.

1. The original landing page showed three screenshots of the product and one sentence below each describing a benefit of using Sidelines.

Due to the poor conversion rate they experimented with a simplified version that just had a random picture from the 2012 Super Bowl and a couple of sentences about the product.

As a result conversions went from 5% to 17%. Kuruvila suggests this is because if a screenshot doesn’t immediately show a customer what your product does, then it’s better to go for a simple, eye-catching photo.

2. For the next test they changed the landing page so it had a large background picture that covered the entire screen and reduced the text to a single tagline and one sentence explaining the product.

This test saw conversions creep up to 25%.

3. After trying a number of different taglines, Sidelines found that the deliberately vague option of “Follow Sports Together” was the most successful.

The theory is that it intrigues people and convinces them to sign up so they can find out more.

4. Initially the landing page only highlighted the sharing functions of the site, but when they added the words “Follow your favourite teams” conversions jumped from 25% to 40%.

This is because it made new visitors aware of the brand’s value proposition, so they knew what they stood to gain from signing up to Sidelines.

5. Like many startups Sidelines originally required new users to signup using their Facebook or Twitter accounts.

However after adding an email sign up option the number of people signing up using their social account increased by 7%. Kuruvila says this is because email signup option increased trust for the brand.

6. Kuruvila initially thought that Sidelines would appeal mostly to men, but traffic from Facebook ads proved that assumption to be incorrect.

The highest conversion rate “by far” was women aged 26-40 with the messaging “share photos, videos, articles and opinions about your favourite teams.” In addition, these women were inviting 35% more friends to join Sidelines than men were.

Finally, men who received email invites from women converted about twice as well as men who received email invites from other men.

It was clear that women aged 26-40 loved to share, but Sidelines wanted to ensure the landing page message appealed to the vast majority of users who start off as pure content consumers.

The compromise was a message that started off with the consumer value proposition and then include the sharing aspect: “Follow your favourite teams, join the hottest discussions and share photos, videos and articles with other fans.” 

This helped landing page conversions increase to 55%.

DHL uses A/B testing to achieve a 98% uplift in conversion rate

DHL ran an A/B test to try and increase signups to a free Import Tool Kit. It created a new page that made the signup form the main focus by making it larger and moving it to the top right of the screen.

All the text remained the same, but the image was changed from a cityscape to a picture of a friendly courier.

The new form design

The test ran for around four weeks in July, and as a result DHL achieved a 98+% conversion rate increase in two countries at a 95% statistical confidence.

ASOS reduces abandonment rate by 50% by changing one CTA

ASOS ran A/B testing at the sign-in page of its checkout to try and reduce the number of abandoned baskets. The original page had a CTA for new customers saying ‘Create your account’ that it changed to simply ‘Continue’.

This simple change reduced basket abandonment by 50% at this stage as consumers associate creating an account with a long process of form filling.

The new checkout design

In fact ASOS left the checkout process for new customers largely unchanged and still required them to choose a password and create an account, but this simple change in copywriting had a massive impact on the customer’s perception of what they were being asked to do.

The old checkout design

Hyundai increased requests for test drive by 62% using multivariate testing

As with other case studies on this list, Hyundai set out to improve conversions on its product landing pages.

This actually included three different goals that it wanted to optimise:

• Primary goal was a brochure request and/or request for a test drive.
• Secondary goal was a clickthrough from the car page to the first step of the funnel.
• As a check, engagement (inverse of bounce rate) was also measured.

To test these different criteria Hyundai used multivariate testing, which is different to A/B testing as each change you make creates a new variation to be tested.

Original landing page

This allows you to track how each single change impacts conversion rates so you can find the best combination of factors. Using Traffic4U it tested eight different iterations by changing the following sections of the page:

• New (SEO friendly) text versus control text. The hypothesis was that if Hyundai changed its normal text to SEO friendly text and it didn’t impact conversion rate, it could permanently implement it for SEO benefits
• Extra call-to-action buttons versus no extra buttons. The thinking was that an extra call-to-action would encourage the user to complete the desired action.
• Large photo of the car versus thumbnails. Hyundai thought that larger photos would entice the visitor and also confirm to them that they are on the right page.

The results showed that the combination of SEO text, extra CTAs and larger images increased the conversion rate (request for test drive or brochure) by 62% and there was a 208% increase in CTR (step 1 to step 2).

New landing page

User testing videos help increase sales by 9.5%

Appliances Online commissioned 125 user testing videos to help identify potential improvements to its product pages, giving it 250 hours of footage of customers browsing the site before making a purchase.

The company used the videos in conjunction with other tools, such as Click Tale, which provided heatmaps showing which elements of product pages users were interacting with the most.

Thanks to this insight it uncovered several issues with its product pages. For example, 70% said that pages were too busy, 17% said service information needed to be clearer, while 13% thought the video experience could be improved.

As a result, Appliances Online made these changes to its product pages:

• Banners touting special offers were incorporated into the copy on the page, allowing the buy button to be moved above the fold.
• The buy button was changed from dark blue to a more eye-catching green colour, while the text was altered from ‘Buy’ to a more descriptive ‘Add to basket’.
• Previously product videos opened in a pop-up screen that took too long to load. The solution was to embed the video into the product pages, which is less interruptive, while consumers can also scan up and down the page looking at reviews and product specs, and the video stays still.
• Creative copywriters were employed to draft unique product descriptions.

These changes resulted in a 9.5% increase in sales, while 37% more visitors viewed the product videos. As viewers of these videos are 57% more likely to add items to the basket, this was a big improvement. 

In addition, the number of reviews left by customers increased by 11%, while there was a 33% reduction in calls about delivery, as the information was more clearly visible on the product page. 

Veeam changed a single word an increased CTR by 161%

In an effort to increase conversions Veeam Software asked all visitors to its product pages what other information they would like to see.

A number of visitors answered “pricing” however Veeam sells through partners so does not publish pricing information on its site. However it does have a ‘Request A Quote’ link that leads to a sales inquiry form.

Simply by changing the CTA from ‘Request A Quote’ to ‘Request Pricing’ Veeam achieved a 161.66% increase in CTR from 0.54% to 1.40% with 100% statistical confidence.

This is a great example of increasing conversions by using customer feedback to improve the website.

Amazon made $2.7bn by asking one simple question

Consumer reviews are a proven sales driver in e-commerce, with research showing that 63% of customers are more likely to make a purchase from a site that has user reviews.

Amazon receives hundreds of reviews so it was faced with the challenge of how to curate them into a useful order for its customers.

It struck upon the best system by adding the question: “Was this review helpful to you?” This allows users to endorse the best reviews, with the top three being featured on the product page.

Other touches, such as allowing users to easily view the best negative or positive reviews, as well as some handy charts that summarise review ratings, make the large number of reviews manageable for users and more useful for Amazon.

Displaying the most helpful reviews has increased sales in the media products category by 20%, meaning this feature was worth $2.7bn to Amazon.

VOIP telephone service increases quote requests by 262%

In this test an anonymous B2B VOIP telephone services company set about increasing its number of quote requests by performing an A/B test of different page designs.

The original signup page

The company redesigned the page to simplify the form and reduce friction in the sign up process. The number of fields that users had to fill in was reduced from six to three, and it removed any requests for personal information.

The new signup page

Other changes included:

• Changing the CTA from ‘Get Quote’ to ‘Show Me My Instant Quote’.
• Giving an interactive quote online rather than getting a salesperson to call the customer.
• Adding third-party security logos.
• Adding a chart comparing its service versus traditional phones.

The result was a 262% increase in conversion rate from 2.4% to 8.8%.

David Moth is a Senior Reporter at Econsultancy. You can follow him on Twitter or Google+. http://econsultancy.com/in/blog/10922-eight-user-testing-case-studies-that-achieved-amazing-results