An ebook designed for client can be accessed here VWO.com which is leader in industry providing best A/B testing tools.

2. INTRODUCTION
THE BACKGROUND AND NEED FOR OPTIMIZATION
WHAT CHANGES AND HYPOTHESES WERE TESTED?
NOT ALL TESTS SHOULD BE A/B TESTS
THE GOALS
THE TESTS
THE RESULTS
WHY DID COMBINATION 8 WIN?
WOULD COMBINATION 8 HAVE WON ANYWAY?
BEST-PRACTICES ARE NOT INFALLIBLE
LEARNING FROM THE TEST
WHAT IS VISUAL WEBSITE OPTIMIZER?
04
05
07
09
10
12
13
18
19
20
21
03
Contents

3. Introduction
03
98.3 of every 100 visitors to an automobile dealer’s
site leave without so much as downloading a brochure
or requesting a test drive.
Hyundai, the Korean automobile giant, was no
exception to this statistic. But great brands trust
themselves to beat great odds. Or, they trust the right
people to do it. For the Dutch market, Hyundai brought
on board Traffic4U, a VWOcertified international online
marketing agency, to help optimize its pages for better
conversion. For an automobile brand like Hyundai, a
lead is generated when a visitor arrives on one of the
landing pages dedicated to individual car models, and
downloads a brochure or requests a test drive.
For this case study, we caught up with Janco Klijnstra
from Traffic4U. He talks about the test that had eight
different page variations competing for the best
conversion rate. Eventually, the test went on to win the
silver award at the annual WhichTestWon awards.
Share this ebook:
According to Dataium, the
average conversion rate
for automobile websites is
a mere 1.7%.

4. The Background and Need
for Optimization
Hyundai has landing pages for each of its car
models where people can request test drives
or download brochure. Such pages serve to
convert visitors into leads, or prospects.
Lead generation pages achieve this
by:
• Attracting visitors by providing
more information about various
car models.
• Guiding visitors towards taking an action
(brochure-download or a test-drive request).
For an automobile manufacturer, such actions
help qualify a visitor as a lead. Hyundai can
then route the lead to a dealer and try to
close a sale.
Lead generation pages of Hyundai generally
get traffic from paid advertising campaigns,
but some of the footfall can be attributed
to SEO as well. Since such traffic is acquired
through considerable effort and at
a cost, the automobile brand
wished to make the most of it.
It is well known that design
has a direct influence on visitor
behavior on a web page.
A well-optimized landing page could
provide Hyundai more leads (therefore, more
business) from the same traffic.
It was with this idea in mind that Hyundai
reached out to Traffic4U to run A/B tests on
all of its car model pages.
04
Share this ebook:

5. What Changes And
Hypotheses Were Tested
Three areas for improvement were considered and the following elements were tested.
New SEO friendly text
versus control text:
The hypothesis was that
the SEO-friendly text will
not impede conversion rate,
and may also help the pages
rank better among search
engine results.
Extra Call-to-Action
buttons versus no extra
buttons:
Traffic4U hypothesized
that the extra call-to-action
button would highlight the
desired action expected of
a visitor.
Large photo of the car
versus thumbnails:
The large photo was meant
to g i ve v i s i to rs m o re
information regarding car
models. It was hypothesized
that a larger photo would
convert more visitors into
leads.
05
Share this ebook:

6. There was just one little
problem
If the original page were tested against a variation (an A/B test) that contained
all of the above changes, how could their individual effects on conversion rate be
understood? Perhaps, this point needs to be explained better.
Each of the proposed changes could potentially impact the conversion rate,
positively or negatively. But if the variation contained all of the changes,
the final conversion rate would be a reflection of the sum of the
individual contributions of each of the changes.
For instance, the extra call-to-action button could negatively
impact on-page conversion, while the other two changes
could increase the conversion rate by a higher measure,
resulting in an overall positive impact. With an A/B test, there
would be no way for Traffic4U or Hyundai to ascertain exactly
what combination of changes would increase conversion rate the
most. An A/B test, they realized, simply wasn’t enough.
Enter Multivariate Testing.
06
Share this ebook:

7. Not All Tests Should Be A/B
Tests
A/B tests often involve many different changes bundled together in the variation that is tested
against the control (original version of a page). An A/B test that tests multiple changes in a
variation, can help reveal if the variation converts better. But it gives little insight into exactly
which of those changes (or combination of changes) caused the maximum improvement to
conversion rate. Multivariate testing, on the other hand, creates a different page variation for
each combination of the different changes that are to be tested (see below).
Let’s consider a hypothetical case where three different elements on the page are to be tested.
Alternative versions for each of the three elements to be tested are created.
With these changes, Multivariate testing allows the user to generate eight different page
variations, as shown below.
07
...................................................................................................................................
...................................................................................................................................
...................................................................................................................................
...............................
...............................
Element A (Title)
Orange
Element A (CTA Text)
Fresh Pizza
Element A (CTA Color)
Control
Order
Bob’s Pizza
Buy Now
Green
Variation
Share this ebook:

8. To summarize, A/B testing allows a user to find out if a variation will perform
better than the control. Multivariate testing, on the other hand, helps
pinpoint exactly which of the changes, or combination of changes,
affect conversion and which ones don’t. Here’s more on multivariate
testing vs A/B testing.
Traffic4U needed to understand exactly which of the changes
would have the maximum impact on conversion rate. To that end,
they used multivariate testing with eight different variations of the
page.
An A/B test, on the other hand, would simply pit the Control (Number 1)
against the final variation (Number 8)
08
Share this ebook:

9. The Goals
There were several conversion optimization
goals that were included in this test:
PRIMARY GOAL :
Download request for brochure and/or request
for a test drive.
SECONDARY GOAL :
Click-through from the car page to the first
step of the funnel.
• As a check, engagement on the page was
also measured.
09
Share this ebook:

10. The Tests
CONTROL
10
Share this ebook:

11. WINNING VARIATION (COMBINATION 8)
11
Share this ebook:

12. The Results
To put that figure — 62% — in perspective,
on the winning page variation of Hyundai,
2.75 of every 100 visitors would convert into
leads, every time. Compared to the industry
average quoted above (1.7%), combination 8
would provide 1.08 more leads from every 100
visitors. In other words, if Hyundai.nl typically
attracted 10,000 visitors/month, combination
eight would give 275 leads a month, up from
170/month with the control, at no additional
cost. Needless to say, a higher number of leads
at the same cost could significantly strengthen
the bottom-line for the automobile brand.
The winning variation (combination) converted visitors into leads 62%
better, and got 208% more visitors to click through to the next stage
of the buying process.
12
Share this ebook:

13. Why Did Combination
8 Win?
Tounderstandwhythechangesincombination
8 worked, we’ll need to understand how it
influenced a website visitor.
Let’s begin by understanding why the visitor
landed on a Hyundai web page.
Any user who searches for Hyundai or one
of its car models, or clicks through from a
paid advertising campaign for one of its cars,
must be considering the purchase of a car.
Typically, the buying decision involves four
steps, beginning with Need Recognition,
Information Search, Alternative Evaluation,
and finally, Purchase Decision. A visitor to one
of the lead-generation pages would fit into
one of the last three stages.
How’s this information important? Because
understanding a visitor’s intentions helps us
tailor web pages to meet those expectations
(even exceed them). If a prospecting visitor
finds the page relevant, there is a good chance
that they will drop in a lead. According to a
study by Nielsen Norman Group, users often
leave webpages in ten to twenty seconds.
Couple that data with this report from The
Guardian on the waning attention span of
webpage visitors, and we realize how critical
a role landing pages play in quickly grabbing
attention and getting a positive response
from a visitor.
Let’s now look at why the changes to
Hyundai’s page influenced visitors positively
and delivered better conversion rates.
13
Share this ebook:

14. LARGER PRODUCT IMAGE
For most people, buying a car is a high-stakes
decision because of the cost involved, and the
constant, almost-daily interaction that they
have with cars.
A prospect, therefore, looks to obtain all the
relevant information before making a decision.
Thumbnail images, unfortunately, reveal very
little about a car’s design and aesthetics, and
could leave much to be desired. This could
cause the visitor to bounce-off, and find more
relevant images and information about the car
model elsewhere.
Online product portals present what is called
a touch-and-feel barrier to prospects. Since
actual physical interaction is impossible,
product pages need to go the extra mile
to lower the barrier as much as possible,
and give the user a virtual touch-and-feel
experience. This is the same reason why many
eCommerce product pages now let visitors
access an enlarged version of their products.
Larger product images have been shown to
inspire more trust in prospects, and is a widely
followed best-practice in eCommerce.
The larger image on Hyundai’s page
gave the visitor more relevant visual
information regardingthecarmodel.Thisway,
bymeetingthevisitor’sexpectations, Hyundai
could minimize the chances of a visitor leaving
the page without exploring it in detail.
Naturally, longer the time a visitor spends on
a page that he finds to be relevant, higher the
chances for an action to be taken and a lead
to be generated.
14
Share this ebook:

15. The most basic function of a Call-To-Action
(CTA) button is to highlight the desired action
and compel a visitor towards taking the
action.
Hyundai included two CTA buttons on
its landing pages, one each for ‘download
brochure’ and ‘request a test drive’.
Let’s understand the importance of these
buttons better. Consider a visitor who has
landed on the Hyundai landing page and has
consumed the information available in images
and text. Without the CTA buttons, the visitor
is left on his own to figure out what needs to
be done next.
• Should the visitor contact the dealer over
phone?
If yes, where can such details be found?
• Is the visitor supposed to share their
contact details to receive a call back?
If yes, how is that to be done?
• Where can the visitor find more information
about the car model?
EXTRA CALL-TO-ACTION BUTTONS
Hubspot, a leader in marketing automation, defines a CTA button this way:
A call-to-action (usually abbreviated as CTA) is an image or line of text
that prompts your visitors, leads, and customers to take action. It is,
quite literally, a “call” to take an “action.”
15
Share this ebook:

16. Buying intent is the strongest when the
visitor is right on the page, just after the
minimum relevant information required to
form an opinion is obtained. If the visitor
gets confused, without having a direction
to proceed, chances are that he will bounce
off the page looking for more information or
ways to contact the brand. But such barriers
— extra steps required to be undertaken by a
visitor — dampens the buying intent and the
probability of a visitor converting into a lead.
In combination 8, the extra call-to-action
buttons highlight specific actions that a
visitor is expected to complete to take their
buying process forward. Once the visitor
has consumed the information summary
presented on the landing page, they find an
option to download a brochure that contains
further details about the car. Or, if there is a
stronger intent, the visitor has the option to
request for a test drive right away.
At once, the new buttons inspired confidence
(by letting the visitor know exactly what
would happen if the CTAs are clicked upon)
and compelled the visitor to take an action (by
offering exactly what they’d expect to get).
Here’s Jeff Bullas on the five key elements for
creating a compelling call to action.
16
Share this ebook:

17. SEO-FRIENDLY TEXT
SEO-friendly text is simply, text that search engines can understand. It, in turn, dictates how
well a particular page ranks among search engine results. Historically, search engines depended
a lot on the availability of specific keywords on web pages to understand what the pages were
about. But, on a fundamental level, search engines exist to throw up results that its users will
benefit the most from. And today, search engine algorithms have advanced to the point that the
general consensus says, a site that can easily be understood and is of clear relevance to a visitor,
will naturally rank better among search results.
From the images of the Control, and
Combination 8, it can be seen that the
SEO-friendly text on combination 8 is more
structured, and broken down into smaller
sections with clear subheadings, compared to
the clunky text on the Control. Naturally, the
text on combination 8 was easy to consume
and understand.
The revamped text helped Hyundai in two
ways:
• It helped the pages rank better in search
engine results, driving more traffic.
• The easy-to-read text on the landing pages
attracted visitors’ attention leading to higher
conversions.
If you already have some experience with
conversion optimization, the changes
Traffic4U tested might seem as best practices
that were destined to succeed.
17
Share this ebook:

18. Would Combination 8 have
won anyway?
No. Combination 8 did not win because
the changes made intuitive sense. Nor did
it win because the changes were based
on ‘best practices’. Combination 8 worked
simply because it worked — the changes
therein could elicit the right responses
from the particular traffic that landed on
the automobile brand’s website. We can
rationalize test results using studies and past
observations of human psychology, and even
try finding patterns.
But such patterns should only act as a
reference point for framing good, plausible
hypotheses, and not be confused for one-
size-fits-all solutions.
Here’s why.
18
Share this ebook:

19. Best-Practices Are Not
Infallible
Vendio, an Alibaba company, did the unthinkable by
removing the sign-up form from its landing page.
If you are thinking why that is unthinkable, check out
this beautiful article from Hubspot on fifteen great
landing page design examples. Every one of those
landing pages contains a sign-up form.
So, what did Vendio do instead? A clickable button was
added to the page that would take interested visitors
to another page with the sign-up form. Cradling the
sign-up form within the landing page is considered
as best-practice, because it minimizes the navigation
required by a visitor to register on the form. But this
time, for Vendio, playing against the perceived best-
practice yielded a mammoth 60% increase in sign ups.
Here are six more examples of why intuitions are only
intuitions, and not always right.
There are even more shocking conversion-optimization
test results. And more, and more.
60 %INCREASE IN SIGN UPS
19
Share this ebook:

20. Learning From The Test
Traffic4U banked on its wide experience
in conversion optimization to zero in on
the changes to be tested. By going for a
multivariate test instead of a simpler A/B
test, they could validate best-practices and
discover the exact set of changes that created
the maximum impact. It certainly could have
happened that the SEO-friendly text impeded
conversions, while the other two changes
bolstered it, leading to an increased overall
conversion rate anyway. But that result,
as positive as it is, would have hidden the
treachery of SEO-friendly text, and Hyundai
would have had to settle for less.
Thanks to multivariate testing, the already
successful big brand benefitted in two ways:
• The test revealed a variation that could
convert 62% better
• Understand the individual impact of the
different changes
This knowledge, about how each change
helped refine user experience, will help
Hyundai and Traffic4U in conceiving more
such cleverly thought-out hypotheses, and
test those to optimize further for conversions.
When we asked Traffic4U about their comments on Visual Website Optimizer, here’s what they
had to say:
VisualWebsite Optimizer was very valuable, it made it possible for us as an
agency to build our own variations, test on a group of pages and measure
different goals at once. Also the Analytics plug-in was very helpful in the
analysis for segmentation purposes.
20
Share this ebook:

21. What is Visual Website
Optimizer?
It’s an easy-to-use A/B testing tool that will
help you get more conversions from your
websites or landing pages. It’s as simple as
this:
• Create a hypothesis - what change(s) to your
website will give you more conversions?
• Take a free trial of VWO and do a painless
1-minute integration that requires zero tech
know-how.
• Using VWO point-and-click editor, create
alternative version(s) of the page to be tested.
• Click RUN and sit back while VWO splits
the traffic to your website randomly among
the versions
Once the test runs on enough traffic to
achieve statistically significant results, you
will find out if the hypothesis was right or
not. If it turns out right, congratulations, you
have a winner — a site/landing page version
that will give you more conversions from the
traffic you’ve already acquired. Then, as with
21
Share this ebook:

22. all successful businesses, you go back to optimizing the page further. If the hypothesis is proved
wrong, you discover one more change that wouldn’t work. And you go back to testing again.
Visual Website Optimizer is also a flexible multivariate testing software (full factorial
methodology) and has a number of additional tools like behavioral targeting, heatmaps, usability
testing, etc. With the 100+ features in Visual Website Optimizer, you can be sure that all your
conversion rate optimization requirements will be covered byVWO. 4000+ enterprises and small
businesses in 75 countries are using our A/B testing software for landing page optimization,
increasing website sales, and improving conversion rates (see all case studies). Sign-up for a
FREE 30 day trial today!
22
Share this ebook:

23. 142 W. 36th St, 11th Floor New York, NY 10018, USA
14th Floor, KLJ Tower North, Netaji Subhash Place, Pitam Pura, Delhi 110034, India
+1 844-822-8378 (U.S.A) +44 800-088-5450 (U.K)
TALK TO A SALES REPRESENTATIVE
+1 415-349-0105 (Global)
Find Us On