101 Conversion Rate Optimization Definitions

Here is a glossary with 101 basic A/B Testing definitions.

A/B Testing

Also known as split testing. This is a proven, scientific testing method that’s been used successfully by direct response marketers since the 1930s. Now it’s used extensively on the Web as well as email, telemarketing, TV, postal mail, and other channels.

Online, the “A/B” refers to the way traffic is split so that every other person coming to a page sees a different version of the page being tested. You don’t have to split traffic 50/50 always, just split enough visitors to get statistically valid (aka conclusive) results for each test creative. So, if you had 100,000 visitors to a page, perhaps only 20,000 might be served a particular test version of that page. What’s critical is that all visitors come from the same general traffic source at about the same time. You don’t want to serve 80,000 people one version and then switch over and serve the following

20,000 the second version because time of day (among other external factors) can affect results.

To reach conclusive results about your changes, in A/B testing either you can test changing a single element on the page (while everything else remains the same) or you can test an entirely different page. The former gives you nice clean data about how much a particular element may affect response rates. That element might be price, button size, headline copy, form length, etc. The latter (changing multiple items on the page) will tell you which page did better overall. However you will have NO IDEA which of the elements on the page were responsible for the lift. (You need to expand to multivariate testing for that data if it’s worth the extra effort and cost to you.)

Sometimes you’ll see an “N” as in A/B/N testing. The “N” refers to “Nth name,” meaning you can run additional test panels at the same time – either each one completely different or each one varying in just one element versus the control -- if you have enough traffic to support them. For example, you might want to test two new prices against your control price. You’d have three pages being tested simultaneously, each to an “Nth” percent of incoming visitors equaling a valid sample size.

Abandonment Rate

Often used as a synonym for Bounce Rate, although bounces generally implied to occur on the entry page versus an exit/abandon can occur at any point during a longer visit. Nearly every page on your site has an abandonment rate, because some visitor has decided to leave the site entirely when viewing them. Abandons are not all bad, for example, a customer service Q&A page with high abandons may imply people found what they wanted and left the site rather than being forced to continue digging. However, you rarely want high abandons on entry pages or key conversion pages (carts, lead gen forms, etc.)

Abandonment rate is a useful number to fling about in meetings when you are trying to convince your superior (or anyone with power over the testing budget) that the site needs testing. Many non-analytics executives are stupendously uninformed about abandonment rates. I’ve interviewed many a C-level exec (even in marketing) who blithely told me they estimated their conversion rates were over 50%, even over 80%!

When you tell them, “95 out of 100 people who come to this page abandon, but with testing we can improve that,” their jaws will drop.

If I were to pick a page to optimize with testing, I’d look for pages with the lowest abandonment rates that either could make the most money for my company (by ecommerce or lead generation) or were direct doorways to these big money pages. Successful pages can always be made more successful and there’s money to be made by following where the money’s being made. Next, I might test pages with the highest abandonment rates because testing could cause a big swing in results.

Ad Optimization

The science of testing a variety of creative versions for an advertisement (often on-the-fly) to see which one gets the best response rate. Heavily used by big-budget banner advertisers. Unfortunately, many campaigns are optimized for click rate without regard to landing page results or ultimate conversion rates.

The types of ads and offers that get the highest clicks sometimes get the worst conversions.

Before you invest substantially in advertising, make sure you have first optimized your landing page and (this is critical) that you’ve made sure that your analytics tie through so the ad creative team can optimize ad creative to deliver clicks that convert instead of just a whole mess of clicks that waste server space and slide off. It’s scary how many ad agencies are kept in the dark, not allowed any access to client Web analytics data whatsoever.

No, you should not try to optimize both your ads and your landing page at the same time. The interplay of elements will confuse answers. Rule of thumb, do your page first and then your ad. This assumes you’ll do a small test run for the ad to get enough traffic for landing page testing purposes. Then you’ll turn on the ad click faucet once the page is at its best.

Lastly, don’t try optimizing your page for pre-existing traffic without ad traffic– existing traffic may be a very different demographic or psychographic than ad-driven traffic. Optimize for the type of visitor the page will be receiving!

Analytics

Data showing how visitors interact with a Website or landing page. Worthless without analysis. A bunch of numbers won’t help you unless you know how to

understand them and then apply what you’ve learned to make changes that improve the site’s results. Analytics systems can range in price from free (most famously Google Analytics) to tens of thousands of dollars a month plus more money for customized reports. How much should you spend?

First consider whether you have the manpower to set up and read reports? To make recommendations and act on them? I know plenty of overworked marketers who paid for big ticket analytics that wind up being stacks of reports getting dusty in a corner of their office. If you’re in this situation, I recommend you start with basic analytics (Google’s free Analytics may be good enough) to garner a baseline of information. Then schedule a project or two to prove to yourself and management that it’s worth the people investment to act on analytics-suggested site tests and improvements.

If you can then get the budget for more advanced analytics, your site or business is complex enough to require a fancy analytics package, and you have the staff-power to review and act on results, then by all means invest in it!

Anxiety

This is one of the factors that govern whether a visitor on your page will convert or abandon. Does your page include reassuring elements to help a visitor overcome the anxiety hump so he or she can move on to the next page or action? These elements can range from a keyword in the page headline (does it match the search keyword they came in on?) to trust icons and testimonials. In my experience marketers tend to overestimate how familiar with (and trusting of) their brand incoming visitors are. Anxiety-elimination tests often yield great results.

AOV

Average Order Value. One of the key metrics that ecommerce sites should optimize for. Do you want loads of small orders or would you prefer fewer, larger orders? The latter are more profitable in the short run. The former may bring in so many new customers that can be converted into return buyers that it’s more profitable in the long run. So, you can see it’s a fairly strategic decision.

If AOV is a key metric for your company, and you are testing pages prior to the final order conclusion, then you’ll need to make sure your testing technology can take AOV into account. You may do this using KPIs based on regression analysis of past high-value orders, or you may use complex testing technology to track how tests on one page affect results further down the conversion path.

B2B

Business-to-Business. You are a business that sells products or services to other businesses or to workers for job-related purposes. For some reason (in my experience) this term thoroughly confuses people who are not B2B specialists. They seem to think that because they are “in business,” they must ipso facto be with a B2B company. Annoying for a researcher covering the B2B marketing beat, but hardly earth shattering for anyone else.

B2B marketers often (but not always) are testing- challenged because they tend to market to a highly targeted, niche market. Conclusive test results require a certain amount of conversions, which in turn require a certain level of traffic. You might be able to run a simple A/B test with just a few hundred unique page visitors per month if the conversion measured is a very popular one, such as clicking on any link whatsoever to go further into the site instead of abandoning. If, however, you want to test improving lead generation form fills, depending on the quality of your form and your brand- name, you may need as many as 10,000 visitors to that page per month. Some B2B marketers are happy to get that much traffic in six months! This makes multivariate tests close to impossible and limits A/B/N split tests to only the most critical pages and elements.

B2C

Business-to-Consumer. This is anyone who is selling or marketing something to people who are not buying it for their job or for business-related purposes. I’ve found that generally only B2B marketers use the term “B2C.” B2C marketers just say they are “marketers”

because they consider themselves the mainstream and don’t need defining. Ironically there are far more B2B marketers in the USA and Canada than there are B2C marketers, even though B2B is considered a “niche.”

If you are a true “mass market” B2C marketer, testing can be a joy because you get loads of traffic. I’ve seen tests focused on slicing particular segments, niches, or personas of a mass market achieve extraordinary

results. So, just because your brand has mass appeal, it doesn’t mean you can’t get even higher Web results by building and testing pages for particular segments of the masses.

Big Idea Testing

The opposite of incremental testing. Instead of testing one clearly defined element on a page, you’re testing an entirely, perhaps radically, new page. As Josef Mandelbaum, CEO AmericanGreetings.com, (a man I consider a testing god) has said, you need to run both kinds of tests. Incremental testing will help you make big gains over time, with perhaps a 1%-5% lift for each test that can add up. Big idea testing can help you make a big gain all at once. However, big ideas can also be big failures. In fact, if all your big idea tests are winners, it’s a sign that something is terribly wrong.

Your site was either truly dreadful to start with, an external element is affecting results, or your analytics are broken.

You should test big ideas if your site or brand needs a massive overhaul. This is the quickest way to do it.

You should also test big ideas just to get out-of-the-box thinking applied to your site because you never know where the next breakthrough idea will come from. Top notch online marketers (often heavy-traffic ecommerce or b-to-c lead generation sites) schedule Big Idea tests to occur routinely one-to-six times a year, with a full plate of incremental tests on a weekly or biweekly basis in between. I’ve heard staffing-wise that the Big Idea thinkers are often different from the incrementally gifted, so don’t rely on a single person for all your testing ideas.

Important: Although big idea tests can be enormously powerful, don’t rely entirely on a big-idea test if the risk of failure is too large for your testing program to handle. For example, if this is your organization’s first test and the executive team is eyeing the process skeptically, perhaps you should pick a safer route. You need to prove testing can improve the bottom line before taking risks with it. The last thing you want to “prove” is that testing “doesn’t work.” Trust me, I’ve seen it happen.

Black Ops Testing

A term taken from military slang for covert operations. Much more prevalent in site testing than I’m comfortable with, but sometimes it’s the only way to get something done especially if the IT department is your barricade. Start with something others in the organization would consider inconsequential, no one’s personal stamp is on the page so they won’t be offended. Vigorously limit time and budget expended so you’re not on the hook for it later. Be very, very careful to include as many politically influential people such as legal, branding, finance, sales, etc., in your informal circle of decision- makers as possible so you have allies when it matters. Never test anything that your boss or his/her boss or the CEO is vehemently opposed to even if you are dead certain the results will prove worthwhile. It’s not worth losing your job over – many people do not want to be proven right or wrong, they want to be in control!

Bounce Rate

The percent of visitors who land on a page then leave without taking any other action whatsoever. Also known as the visitors who did not “stick” (referring to the old throw-spaghetti-against-the-wall test).

The first thing you need to determine is how quickly did they bounce? Are these people who clicked in, saw they were in the completely wrong (or unexpected) place, and got the heck out? In that case, you may have a traffic source problem. This is most worrisome when you’re paying for traffic. It is often due to allowing an automated ad-serving program such as Google AdWords, post your ad for inappropriate searches. For example, people searching for “Hotels in Las Vegas New Mexico” will bounce right off of landing pages for “Hotels in Las Vegas Nevada.”

The second group of bounces is people who stuck around for awhile (perhaps as long as 15 seconds or more), took a thorough look, and decided you were not right for them. In this case they were most likely a good prospect for your offer, but your page let them down.

Perhaps the path to conversion wasn’t clear. Perhaps you served them a fairly general page such as a home page or a category page when they were hoping for something extremely specific. Perhaps your page did not include enough trust elements to sooth their anxiety. Or any number of other reasons. These are factors you can improve with testing. These are visitors you might be able to win over if you invest in tests.

The nasty little secret about bounces is that most analytics systems don’t measure length of visit if a visitor only lands on a single page. So, you can’t run a report to discover what type of bounce problem you have (the wrong traffic or a problem page). I have heard programmers tell me there’s a simple work- around, but you have to bug them for it. Good luck.

Brand/Branding

Most site-testing folks have a love/hate relationship with brands and branding. Too often their experience with the brand is a list of rules made up by the graphics or marketing communications department about what they can and can’t do on a page, which can effectively limit testing pretty badly. (My pet peeve… brand rules that require all pages to include navigation as well as content completely unrelated to the conversion path

at hand. In that case, so-called “branding” turns into distraction and distractions can be the death knell of high conversion rates.)

On the other hand, loads of data shows the stronger the brand overall within the marketplace, the higher the conversion rate. If visitors liked and trusted your brand before they arrived at your site, they’re far more likely to convert. In my experience a strong brand with a crappy, badly optimized site will still out-perform a weak brand with a highly optimized site every day

of the week. Luckily, this also means you’ll get a nice bang for your buck when you optimize strong brand Web pages… as long as you can beat off the branding department’s dumber web design rules with a stick.

Can you improve brand perception with page testing? Absolutely. Especially with moderately-known and little-known brands or even with new users of well- known brands. The problem is measuring your branding test value to the company’s bottom line. It’s not as easy as saying, “We gained X% more leads

or sales from the test.” Measurement tactics include surveys for satisfaction, surveys for perception and awareness, offering brick-and-mortar coupons, reducing unhappy calls to your call center or tech support, focus groups, etc. Branding folks have been working on measurement in this field for decades. Ask for their help.

Call to Action

The element(s) on the page that tell a visitor what to do next in order to convert or move to the next stage of the conversion process. The actual element can be anything from a simple text hotlink to a big fat button, or it may be a chat icon, phone number, etc. Art and

offer copy surrounding the call to action is also included. What does your button actually say? What do the words next to it say? Is the call clear? Is there anything to induce a higher conversion rate such as deadline, exclusivity, or special offer? Lastly, is there any call to action at all? (A lot of consultants who say their blogs are marketing devices have no calls to action on their blogs whatsoever.)

Lot of marketers test buttons, including size and placement, which can help results. But I’ve found the copywriting and the offer next to the button is even more effective in raising results. It’s not just about a bigger button. It’s about a more compelling offer.

Cart Checkout

If you work for an ecommerce site, cart checkout optimization is, in my opinion, the very first test to invest in. After all, the people who’ve made it to your cart are far more willing, ready, and able to convert than anyone on any other site page! These people are the holy grail of visitors. If you can convert more of them, you’ll make more money right away; plus, you’ll probably make more money in the long run via repeat and referral customers.

Useful checkout tests include: reducing the number of pages, putting yellow “highlighter” on required fields, removing any need to “start an account” or “register” prior to checking out, changing the flow from left to right (i.e. shipping address on the left, payment on the right) to top to bottom, adding trust icons such as security and Better Business Bureau icons, showing the number of steps/not counting the number of steps left, offering Bill Me Later, etc. I’d love to say there are some best practices that you can plug in right away to reduce testing but aside from free shipping almost nothing seems to work for everyone all the time. Every store is different.

If you are using a third party cart that does not allow you to test and optimize the user interface at all, CHANGE CARTS! Yes, it’s that important. You’re leaving money on the table.

Clarity

Another element you can optimize your landing page and Website for. Do visitors easily understand where to go, what to do, the offers, your brand? If only in- house staff have developed and approved your site, it’s safe to say your site isn’t as clear as it needs to be. You’re too close to see potential problems. I’ve found usability testing can be very helpful in determining if there’s a problem. So is examining the internal search engine reports to see what terms befuddled visitors are entering. Also, check with customer service. You’ll probably discover that your customers and visitors’ questions reveal that they understand far less about your brand or your site than you assumed they did.

Clickstream

The list of pages a visitor clicked to during a single site visit. This can include the referral click (what the visitor clicked on to get to your page or site) and all clicks they made on your site. Although your own analytics package probably can’t tell you, external data providers such as Compete.com can give you an overview of the

destination clicks for your site (i.e. where people moved off to when they abandoned your site completely).

Clickstream information is different than page-path analysis in that you learn precisely which link a visitor clicked on as opposed to just which pages they traveled on the journey through your site.

Clickstream data is most useful if you can look at reports that show the upstream as well as the

downstream. If you’re spending money on traffic, you want to know about the clickstream patterns of visitors generated by that advertising and how it differs from other visitors. If you’re making money from your site

– either from lead generation or ecommerce – you also want to use regression analysis to look back upstream to see which clickstreams produced the most valuable converters. That way you can invest more attention on critical pages within those streams.

Clickthrough

When a Web user clicks on a hotlink. Annoyingly, the clickthrough numbers from the hotlink’s home (an email, online ad, referral link, whatever) will almost never match the “received clicks” number from the analytics reports at the click’s destination. That’s because some clicks get lost in the transition either due to human indecisiveness or technical errors. The difference should not be huge. If it’s more than a few percent, look into things.

Clickthrough Rate

Percent of people who viewed a hotlink and then clicked on it. Not as important as the conversion rate. If you’re buying advertising or traffic by the clickthrough, be sure you have conversion data reports set up beforehand together with the ability to alter campaign elements

(the landing page, the ad, the media buy) to make improvements. Otherwise you can buy a lot of clicks and end up with few conversions. If you think your clickthrough rate is too low, you can use advertising optimization to improve it. However, keep an eye on conversions throughout.

CMS

Content Management System. Behind-the-scenes machinery that serves up the content (text, videos, audio, etc.) for your site and/or landing page. CMS can be free, such as blogging platforms, or very expensive indeed. If you are planning to create and/or test multiple landing pages, you will want a content management system the testing and/or marketing department can use to easily create and alter the content on those pages. I’ve known many marketers who wanted to test but could not because their IT or internal Web department held total control of the CMS and were too busy to allow tests to occur (aka “get in line” syndrome.) To circumvent this, look for a testing platform that requires a little IT work as possible (often

IT only has to put a small snippet of code on the page to

be tested.) If It won’t cooperate at all, some marketers use separate systems to build and even host landing pages and microsites for testing purposes. Sometimes this is done as a black ops, but I recommend keeping IT in the loop so you don’t earn any enemies there.

Conclusive Results

Also known as statistically valid results, these are measured test results from which a formal conclusion may be safely drawn. You are hoping the results of one test version are significantly better (or worse) than the other so a winner may be declared. Testing platforms, including GWO and Omniture, help with the statistical math by showing you the “Chance to Beat Original” for each cell as the test runs. Most statisticians say they do not consider this data conclusive until that Chance reaches 95%.

(I’ve heard statisticians argue vehemently over this though. Some say they need to see a 98% probability before rolling out to the “winning” version. Others say no matter how accurately we think we measure, in some ways the numbers will always be fuzzy enough that 90% is strong enough if you can’t get any higher easily. As a civilian, I look for the 95% and cross my fingers. )

In order to achieve conclusive results, you need a certain volume of conversions – at minimum a few hundred - within a short time period – at maximum perhaps four-six weeks. Use your current conversion rate to calculate how much unique traffic you’ll need to the test for the estimated time period.

If you have an extremely high volume of conversions, you may also want to run tests for a longer time than the conversions alone might warrant, such as at least a week, including a non-holiday weekend. That’s because time of day, day of week, and many other external factors may cause an unusual blip in results. Your

goal in testing is not to find what works at this precise second, but rather what may be used as your new control until you resume testing again someday.

Test design also affects conclusiveness. For example, if you’re doing A/B testing and change two elements

at the same time on a single test page, you won’t be able to draw a conclusion as to how each one affected results. One element could lift response 5% and the other depress it 5%, thus appearing as though your changes had no affect at all!

Also, you can’t extend conclusive results to other circumstances (although this is very tempting). If you send a different type of traffic to that page (past buyers instead of newbies, clicks from a different PPC search term, or banner clicks from a different media buy) you’ll get different results. Time itself is another varying factor

– a page that was a red-hot-conversion-machine at one time may start to lag after a few months due to any

number of (mostly external) circumstances. This is why you need to create and test different controls for varying demographics and why you need to continue testing for better performing controls over time. For best results, testing is never completely done. No test is ultimately and definitely conclusive.

Lastly, beware of assuming your conclusions from one test will hold true on other site pages – or worst of all, on other sites. Testing experts all agree they’re frequently surprised by which test won. Prices,

graphics, hotlinks, copy, etc. that work gangbusters for one page or site may bomb on another. That’s why testing is never boring and why you have to test even if you built your site relying 100% on best practices.

Confidence Level

A measurement that shows the reliability of test results if you roll them out to a greater community. For example, if three visitors come to your landing page and all three convert, what’s your confidence level that the next three visitors would also convert at 100%?

Obviously it’s pretty low because three visitors isn’t a big enough population to draw conclusions from.

Confidence level is usually expressed as a percentage. Many marketers say they are aiming for a 95% confidence level. The higher the confidence level, the better, although you can never get to 100% because predicting every human’s behavior is impossible.

Confidence level also diminishes over time due to external factors and results degradation.

If you are mathematically minded and interested by these sorts of statistics, I highly recommend you read the book, “The Drunkard’s Walk: How Randomness Rules Our Lives,” by Leonard Mlodinow. It’s a horrifying and fascinating look at the math behind confidence level and probability statistics, luckily written in clear English that any marketer can pretty much comprehend.

Content Optimization

Content is anything on your page, including copy, images, graphics, videos, and audio. When you optimize content, you’re hoping to figure out which content will cause the best conversion rate. The most important content to test is anything a visitor sees within three-to-five seconds of arrival (i.e. the headline or whatever is in the top center of the page) and anything directly related to the conversion action itself (i.e. the offer, the form, and the button.) Another useful test

may be eliminating content. If you remove anything that might distract from the conversion such as “standard” navigation, unrelated copy, and even lengthy copy will the page convert better?

If the page you are testing is also an important page for search engine optimization, content optimization becomes a slightly tricky business. Any major copywriting changes, especially headline and hotlink/ navigation wording changes, must be run past the SEO team.

If the page you are testing is also a PPC landing page or a page used for any other type of ad click, you can’t optimize content in a vacuum because generally better performing pages match the creative of the ad the visitors clicked on. This may mean matching headline copy, sprinkling keywords throughout, and matching images when possible. You need to reassure visitors they are in the right place so they carry on converting.

Control

Also known as your Original, in A/B testing, the control is your original creative for which you presumably already have lots of data. The goal of the test is to see if any changes to the control – or completely different variations – will work better than the original control.

The control is considered the baseline that you want to beat. You should always run the control as one of your test panels to measure all other results against. (After all, control responses can change over time due to time of day, day of week, seasonality, competitor changes, etc. so you always want the latest data for comparative purposes.) If one of your test panels beats the control significantly, then it’s customary to start running this as your new control. The old control may be abandoned

(although not forgotten, it may be useful someday so keep a library of old controls on file.) Future tests then try to beat the new control, and so on. Lather, rinse, repeat.

Controlled Risk Testing

Testing, especially in a highly branded environment, can be terrifying. You may not want your public to see certain images, selling propositions, or prices unless you’re absolutely certain that it’s worth the risk of such a large brand-related change. Internal politics also come into play, sometimes the public is less concerned about your site’s changes than your CEO is.

You can control the risk of testing in two main ways. If the risk is perceived to be enormous, instead of changing any part of your live site, do a focus group/ usability lab combination to get immediate reactions from a cross-section of the public. The data will be qualitative rather than quantitative, but you may get a

strong enough impression to gain permission from the powers that be to move forward with a limited live test. If your problem is internal politics, be sure to videotape the lab and then cut a shorter results video to show around. Executives who stood firm on design ideas may crumble when confronted with video of an actual customer who hates or can’t figure out how to use the page.

To control risk for a live public test, do the math to figure out how little traffic you can get away with in order to get conclusive results. Depending on typical conversion goals and rates, you may be able to get away with as little as a few hundred or thousand visitors. Then split your traffic and only allow enough of a trickle to your test page to prove or disprove your test theory out.

Also, add code to the page so search engine bots don’t index or copy the content and make sure the “risky” page is retired from your servers afterward, so there’s no risk of the wrong page being copied or served.

Convert/Conversion

Super-critical word in testing and marketing. This is when a visitor takes an action that indicates he or she agrees to do whatever your page is asking them to do. Conversion activities can include (but are not limited to):

- Filling out and submitting a lead generation form

- Adding an item to a shopping cart

- Signing up for email and/or RSS

- Clicking on an advertiser’s offer

- Dialing your 800# or emailing you directly

- Posting a comment or review

- Starting a quiz or survey

- Forwarding the page to a friend (or Digg, or Twitter, etc.)

- Clicking on any navigational link to visit more pages on your site

Often the conversion won’t be complete on the page you’re testing. This particular page is just one in a longer process. Perhaps it’s a category page visitors need to click on to get to an individual product page and so on and so on. In general, the fewer pages required for a conversion the better. But not always. You’ve got to test that.

Annoyingly, often the conversion (complete or otherwise) won’t happen during the first visit. People may need to consider things, do more research, or check with friends, and then circle back to convert later. Delayed conversions can be as high as 60% of your total.

The big point is: you need to know precisely what conversion activity you are hoping for on your test page. Unless your page is a very simple landing page, this will be more complicated than you may expect. If there’s more than one place to click on a page, more than one activity that can take place there, you have conflicting conversion paths. You have to measure them all during the test and pick the ones you are optimizing for in order of importance and by audience type. For example, is your home page optimized for new visitors or for current customers? What actions do you want each of these groups to take on your home page?

What path of action do you want them to convert to?

Conversion Driver

A slight misnomer, this may be something that drives a consumer to convert but it also helps push visitors over their resistance hump into converting. For example, a consumer’s desire to own a Weber grill may drive him or her to convert, but your guarantee,

low pricing, free shipping, customer testimonials, Better Business Bureau icon, and Bill Me Later offer may be critical toward lowering their resistance hump so the conversion actually occurs.

When you’re optimizing a Web page, you’re manipulating the conversion driver elements so they work better. To do that, you must understand what elements drive conversions and aid visitors in overcoming their resistance hump. Given how low

most pages’ conversions are, I suspect marketers could do a much better job of this.

The nice thing is once you start considering conversion drivers, you realize some site elements (such as the color of your nav bar) that you may have spent endless committee hours debating, aren’t as important as you thought. Testing reveals which site elements truly drive conversions and which don’t matter much.

Conversion Optimization

The science of tweaking your site or page through statistically conclusive testing – sometimes a series of tests – to get the best possible conversion rate. Not to be confused with search engine optimization (SEO), which focuses on getting the best possible search engine ranking for the same page. When it comes to conversions, you’re trying to impress humans, instead of crawlers. The two goals can co-mingle as long as your copywriter is trained in the art of both and the search terms you’re optimizing for drive the type of people who are likely to convert. (All search engine traffic is not equal.)

A point of confusion – some marketers and “optimization” firms use this term when they are referring to qualitative testing such as usability tests, focus groups, intuitively-guided redesign, or persona- based design exercises. These are valuable activities, but they are not statistically conclusive or predictive.

They are more useful as tools to discover ideas that are worth testing. Do qualitative research, and then follow it up with quantitative, real-world analysis. Because

all that matters in the end is how well your marketing performs in the real world.

Conversion Rate

The percent of human visitors that arrive at a page and convert, instead of abandoning. If someone quotes

a conversion rate, always ask if they’re referring to final conversions or just page conversions. Final conversions would be people who went all the way through the process of converting (often involving multiple pages, a form fill, a cart, or offline activities). Page conversions are people who took a conversion action on the particular page you’re measuring.

If you are investing in a site or you count on a site (or a blog) to bring you business, you should know your conversion rate by heart. Sadly, in my experience, surprisingly few executives outside of the ecommerce industry know their site’s or landing pages’ conversion rates off the top of their heads. In particular B2B marketers who want to generate leads on their site

should be running numbers and considering conversion optimization tests.

Conversion Rate Optimization

The science of testing changes on your page and site to improve your conversion rate. Before you invest in a major external advertising campaign of any kind, first optimize your landing page and/or site. That way when the traffic you paid dearly for arrives, it’s not bouncing

off. Then, as much as you can, measure the conversion rate of each of the major traffic sources. Inevitably you will discover that traffic from some sources or keywords converts like crazy while other traffic sources, that appear equally good on the surface, don’t convert diddly squat. So, in the end, conversion rate optimization

may have as much or more to do with optimizing your external advertising and traffic source mix as it does with testing and optimizing your landing page. Both ends of the equation have to work.

CPC

Cost Per Click. Although this could also stand for Cost Per Conversion, most people don’t use it that way. Your cost per conversion will always be significantly higher than your cost per click. For example, if you pay $1 per click for 1,000 clicks and you have a 5% conversion rate, you’re actually paying $20 per conversion. Clicks that seemed cheap can suddenly get more expensive.

CPM

Cost Per Thousand (it’s the Roman numeral M). Most media buys, aside from pay for performance and PPC, are calculated this way by media buyers. Sites selling advertising, especially in niche and B2B arenas, don’t always show their prices this way. Instead the rate card may talk about length of time – such as cost per month. Smart media buyers always translate this into cost per thousand so they can spreadsheet an apples-to-apples comparison between CPMs for all the media bought for a campaign.

This doesn’t mean a lower CPM is your best media buy. In fact, your best media buy is the one at a reasonable- enough price with a great conversion rate. If you’re placing brand advertising, that conversion rate may not be a formal click or online conversion at all. Instead you may require a brand awareness and perception study to measure “conversion.”

Credibility

A key conversion driver. How credible does your page and offer appear to be? This is where strong pre- existing branding can be a powerful driver. If your brand is not well known to most visitors, then you’ll need to work harder, adding in elements that can make you appear more credible. Overall site design helps a great deal (does it look tacky or is it a classy site?) as do guarantees, testimonials, expertly (and honestly) copy length and style, images, and other classic credibility elements.

CRM

Acronym for Customer Relationship Management. It usually refers to the database system you are using to collect, organize, and track prospects and customer

accounts. If your site features any forms – email opt-in, lead generation, contact us, shopping cart, etc. – then you need to make sure they pour as directly as possible into your CRM system.

I’ve seen this be a real problem when simply-built sites send emails to an administrator with form fills, and then the admin has to manually input the info into the database, which winds up happening less frequently than one would hope. Another, perhaps bigger, problem is when various CRM systems are silo-ed so you have a series of databases all standing completely alone with little or no communication. Most frequently this happens when email opt-ins get sent to your email

database but not to your main CRM, or when sales reps keep personal lists separate from the main one.

As for testing and conversion analytics, you need to make sure not only that form fills are sent to the right place and the places all converge into a central location, but also that the CRM database ultimately informs your conversion data. Not all form fills – leads, email opt- ins, contacts – are equal in value. Some people may become incredibly valuable customers. Others are tire kickers. Unless you know which leads and ultimately accounts wind up being better ones, you can’t do any regression analysis to determine which page tests

were (at least somewhat) responsible for them. You want a landing page that converts the best possible customers… not a bunch of “leads” from students and competitors who clutter up your CRM system.

This is most critical for marketers with a long sales funnel who may have to track a lead for months or even years prior to purchase. Luckily Google and SalesForce (among other systems) have been working on tracking systems that integrate PPC and SEO traffic data with CRM data over the long haul.

CTA

Acronym for Cognitive Task Analysis. A fancy term for determining what your site visitors are thinking as they look at your page and decide how and why to move forward or to abandon. The easiest way to figure this out is simply by asking usability lab participants or by using a survey form. The results may not be perfectly quantitatively gorgeous because people who answer questions are never a dead-on representative sample of the whole, but the data is far better than nothing at all. I’ve also used eye tracking study heatmaps as a way to look into the user’s mind with (I think) some real success. If your page is getting really stinky results,

CTA is your first step toward forming a hypothesis that you can use to start A/B split tests and solve the problem.

Depress Conversion

When you formulate your hypotheses for page testing,

your goal is to try to figure out all the main factors that are depressing conversion (i.e. stopping visitors from converting) so you can test altering things to improve results. The frustrating thing about Web design – and perhaps the exciting thing about testing

– is that you can’t rely on best practices alone to increase conversions. There are too many unique, unpredictable factors in play. So, you give it your best shot, test, measure, analyze, and then keep tweaking.

Aside from “bad traffic” (i.e. people who click to your site who really aren’t the right prospects for you), conversion depressors can include:

- Extreme persona variations – When your prospects are such very different types of people that it’s very hard for one page to please and convert them all.

- Bad first impressions – Is your site a turnoff? Perhaps too cluttered or unprofessional at first glance?

- Low credibility – Lack of testimonials, trusted brand name(s), reviews, trust icons, guarantees, etc.

- Missing headline copy – Is your page headline unclear, hard to find, or does it lack the keyword terms that anyone who clicked there would be looking for? Very common problem with PPC marketing landing pages.

- Crappy call to action – Is your call to action either hidden (i.e. tiny, below the fold, jumbled with extraneous page clutter), confusing (especially in the case of multiple offers on a single page), or just not compelling enough to bother with?

- Typography – Is your body copy font size smaller than 10-12 points? Are paragraphs longer than 4 ½ lines? Are there too many font changes (ital,

caps, bold, hotlinks, colors) so the reader’s eye is confused?

- Distraction – Is there anything on the page (including the ‘standard’ navigation bar and any ads for other offers) that might take attention away from your hoped-for conversion path?

- Scary forms – How long is the form? Are you asking potentially intrusive questions such as phone number or income/budget? Is absolutely everything required? Is the form a list of vertical questions or must the user go left-to-right?

Distraction

Anything on the page that might distract the user from the conversion path you want them on. This might include navigation bars, multiple offers, advertisements, irrelevant copy/graphics, etc. In my experience, the worst distractions come from three places:

- The IT/Web department, which insists on using “standard” page layout for all pages even though you might do much better with a blank slate for landing pages or key conversion pages.

- The Branding department, which insists on adding a bunch of ‘about us’ information and images that are not relevant or helpful at this particular stage of the process.

- The Marketing department, which just can’t resist adding links and/or buttons for multiple offers “just in case” some visitors want to go off onto another conversion path than the one they’re currently on.

Dynamic Content

Refers to the way a page is coded. If it’s hard coded (aka ‘static’), then it never changes no matter who visits, when they visit, where they visit from, etc. If, however, the page dynamically pulls content from the server based on various rules (traffic source, traffic pattern, who the visitor is or what they are looking for) then it’s ‘dynamic content.’ For example, the server might switch out an image, copy, or offer on the standard page depending on where the visitor came from (such as a particular PPC ad) or based on how they reacted on a previous page (such as serving up in-house ads based on what topical content they’ve looked at on your site, what’s in their cart, what search terms they plugged

into your internal search box, etc.). Or you might pre- populate some form fields based on their cookies.

Some marketers worry that testing a dynamic content page will be tough due to the fact that you wind up with so many versions of the page that you won’t have enough traffic to any one of them to get valid results. The key to success lies in your testing hypotheses.

Start by looking at the page elements that don’t change (especially layout and offer). Also, review your traffic sources and incoming demographics. You may wind up requiring two or more significantly different pages to convert particular groups, instead of one page with a few dynamic elements to please them all.=

Ecommerce

Big ecommerce marketers are far ahead of all other industries in terms of testing. Most have integrated routine, ongoing testing into their schedules. Often the year-round schedule has a three-week rhythm: week one – pick a test, week two – run it, week three

– analyze results. Ecommerce marketers tend to schedule their riskiest tests (the big huge ideas) for their least busy sales months. That way, if a test bombs, no one has risked massive failure in an ROI-critical month. That’s why you’ll see so many tests running in June and July. Everyone gathers results, and then they roll out the winning element for fall and holiday sales.

In my opinion, the highest impact ecommerce site tests are: (in order of importance to your bottom line)

- Shopping cart optimization

- PPC landing page optimization to match keywords as closely as possible

- Internal search-engine-results-page optimization

- Product page optimization, including layout, typography, graphics, offer copy, button

- Category page optimization

- Home page optimization

- Typical entry pages for typical email newsletters or alerts sent to opt-ins

- Cross-sale/upsale and recommendations element optimization

- Receipt

- Blog optimization (if there is one)

Entry Page

The page on which a visitor enters your site. This is the top end of the conversion funnel on your site. (Of course, the marketing funnel actually starts prior to that

– wherever the visitor first heard about you and then where/when they decided to click to the page.) Because it’s the start of the site-funnel, any improvements you can make through testing will often pay off handsomely on pages further down. This doesn’t always mean it should be your first test (final conversion pages are often a quicker ROI hit), but it should be one of your top tests.

One problem: double-check which pages are your most popular entry pages prior to investing in tests. Due to

a wide variety of factors (emailed links, search engines, deeplinking, referral marketing, etc) the page you think is your main entry page may not be. And, you may have dozens or hundreds of entry pages.

One key for goal setting: if you have an expensive conversion process such as telemarketing and/or field sales reps, don’t assume the goal for your entry page is to increase volume of through-traffic to the next page. Pre-qualifying leads is critical for your budget. Your entry page may act as a screening device, enticing

the wrong sorts of people to abandon or shunted off into automated offers the sales team doesn’t fulfill directly. The entry page may also act as an educational platform, giving potential leads the information they need to be easier to convert in the long run. So, your testing should not only be for lead quality, but perhaps also for lead experience quality.

Exit Survey

A survey form which appears to visitors as they leave your site or page. Most often in testing it’s used to ask abandons why they left. Not everyone will answer your questions, but enough may answer that the results data will give you an insight that can turn your next test into a winner. It’s that “why” insight you may desperately need to turn things around.

The best exit surveys are fairly brief, perhaps only three to four questions. Your last question can be an open-ended question for people to type an answer of their own instead of choosing from your pre-selected

answers. Don’t do more than one open-ended question because many people don’t like to type much, so it can lower your responses. Plus open-ended questions are a lot more work to evaluate on the back-end analysis.

My favorite exit survey is free software from iPerceptions called 4Q. Chances are you’ll see it on my sites!

Experience Testing

Term coined by testing-guru Avinash Kaushik to describe testing the entire user visit to a site, instead of testing just one particular page. Although this sounds like a testing jackpot, in reality it’s extremely difficult

to do (both in implementation and in results analysis). My advice: if your Web analytics path analysis reports indicate that user paths are fractured (with people going all over the place, instead of down the path you’d like) or apparently confused (visit length will also help with this), you should do some usability testing to learn

what you can clear up. However, what you’ll probably see are a handful of pages along typical paths with unusually high abandonment rates. The easiest way to test your way to success here is to pick the page with the highest traffic and a pretty bad abandonment rate and A/B/N or multivariate test your way to better

conversions. Then pick the next worse page, and so on and so on.

Experiment

A term marketers, Web developers and some testing platforms such as Google Website Optimizer use when referring to tests.

I’d avoid using this term when referring to tests in some corporate settings because I think it can scare non-testing executives. An “experiment” by its nature is unproven. It sounds risky. A test, on the other hand, indicates caution and forethought. As a former copywriter who fought battles in corporate America to get a testing budget, I think taxonomy really matters.

External Factors

Everything outside of your site that might affect conversions, including things that affect the attitudes your visitors tend to walk in the door with. This, of course, is where ad agencies, customer referral programs, media buyers, affiliates, and brand marketers come into their glory. If they do their job well, it makes on-site conversions so very much easier.

Some external factors you can’t control, such as the competition, the economy, and today’s relevant news. That doesn’t mean you should ignore them. For example, if you’re running PPC ads in a heavily competitive area, much of your landing page testing hypotheses may be derived from reviewing the competitor’s landing pages to figure out superior positioning and offers.

Eye Flow

The lovely thing about human eyes is how they all tend to react in similar manners to Website layout and design. That can be formally measured with eye tracking. A page designer who uses the rules of

typical eye flow can make a tremendous difference in conversion rates. For example, text to the right of an online image is far more likely to be read than text to the left of an online image (in the Western world) due to eye flow. Therefore a smart designer will usually put text to the right of an image.

Key #1. Non-Western eyes don’t always react the same way. Eye flow in the Middle East and Asia obeys some different rules.

Key #2. Eye flow in other mediums, such as print, can be very different from online eye flow in some circumstances. For example, it’s easy to read two- column type in print and next-to-impossible online.

Key #3. Eye flow depends on screen-size and monitor resolution. A page that’s optimized for an iPhone may be laid out differently than a page that’s optimized for the gargantuan monitor most designers use to lay out Web pages.

Eye Tracking

The physical science of tracking precisely what human eyes are doing when they land on your page. Systems currently can measure the pattern (i.e. where people’s eyes go on the page and in what order) as well as the length of time the eye lingers on each element and where people actually click. I love that last part – linking the viewing activity to the click.

I’m fairly well known as a proponent of eye tracking tests because back when I ran MarketingSherpa, we were the first media company to run and publish research about its use in online marketing. The reasons I like eye tracking is because:

- It’s fairly cheap. You only need a dozen or so volunteers to get useful results and lab costs begin in the very low five-figures.

- It’s low-risk. You don’t need to go public with a live site or page – you can test a mock-up.

- It’s fabulous training for less experienced designers and copywriters. Once they see where visitors are

*really* looking, it blows their brains into a million tiny pieces. They tend to start writing/designing better.

- The colorful “heatmap” result graphics can make a big impression on testing naysayers (i.e. the executives who thought the site was perfect as it was). Great for internal presentations to get the budget to expand tests to quantitative platforms.

Factorial Test Design

A “factor” is any element of a Web page that might be affecting conversions. This would include the layout, images, headline, navigation, etc. If you run an A/B test looking for data on a particular factor, you are limited to changing just one factor on the page at a time. (You can also run an A/B test and test completely different pages, but you would only learn which page won, not how individual factors on the new page made the difference.)

Multivariate tests, on the other hand, allow you to test multiple factors on a page and get specific data on each one’s performance, all at the same time. There are two main type of multivariate tests.

Full-factorial tests are, in essence, multi-celled A/B tests, with a test panel for each of the factors and all of its permutations. These can require a lot of traffic to obtain conclusive results, but the data may be worth it. Fractional factorial tests, on the other hand, use complex (often Taguchi-based) mathematics

to extrapolate results from a smaller number of test panels. These tests may require less traffic, but some statisticians do not think the results are as conclusive as other types of tests might be. I’m not a statistician and will leave that argument up to the experts!

Fold

The spot on the page where a typical visitor’s screen ends. If they want to see more of the page, they will have to scroll down (or over). People are lazy. Roughly 80% of landing page visitors will never scroll down below the fold. They’ll glance at their screen and make a go/no-go decision entirely based on what’s above

the fold. Sometimes if what’s above the fold is extra- compelling and your screen design is well-done, they may continue down, scrolling to see more content to confirm their impression and gain the knowledge they seek prior to making a yes/no conversion decision.

High impact tests often involve factors above the fold. Sometimes this includes changing design so that items that were previously below the fold are now above it.

The nasty problem with the fold is that the darn thing isn’t stable. The fold changes depending on the monitor and screen resolution each individual user views the

page through. Someone looking at your site on a tiny mobile device may see a different fold than someone looking at it on a typical desktop monitor. Type-size can also affect the fold if you allow visitors to decide what size they want to view your copy in. The good news

is visitors who appreciate larger type often – but not always – are willing to scroll more than other people, partly because they can more easily read and become engrossed in your content. (By the way: This is a reason why designers who use tiny type in order to get as much copy as possible above the fold are probably doing you a disservice. Cut and rewrite your copy instead of making your point size too small to be easily read!)

I’ve noticed that designers often are “fold blind” because they tend to work on huge monitors set at a super-tiny screen resolution. Thus, they create design that only another designer could love. That’s why it’s important to preview design on a variety of monitors and resolutions commonly used by your site visitors. You can determine this from your own Web analytics reports as well as demographic profiles.

Funnel

Used in a variety of marketing and sales circumstances. It refers to the funnel shape you could diagram to show how most marketing and sales scenarios start with a large number of prospects and then wind up with a smaller number conversions. For example, a large universe of people may hear about your site. A smaller number of these actually visit. An even smaller number of these go to two or more pages when they visit.

Finally, an even smaller number of these convert to sales leads or direct sales. At the top end of the funnel you had perhaps thousands or even millions of people. At the bottom end (where the sales drip out) you may have wound up with just a handful.

Unfortunately, tracking the funnel on your site may not be as easy as you hope because all sorts of different pages can be entry pages (not just your home page or officially designated landing pages.)

Funnel Experiment

When considering which site tests to run first, it helps considerably if you diagram your site’s conversion funnel or typical conversion pathway, showing each step of the conversion process together with as many numbers as you can get. Then review each step, looking for the places where it’s worth investing in testing. Generally these are: (in order of importance)

1. The very end of the funnel – your conversion action page

2. The very start of the funnel – your entry page(s)

3. Any pages in between with stinkier than normal abandon rate

GIGO

Garbage In, Garbage Out. A term test experts stole from computer programmers because it’s unfortunately so darn applicable to many tests. Page and site testing tools are so widespread and somewhat easy to use these days, which encourages a lot of people to try testing. That’s great, except testing is sort of like typing. Just because you can type, it doesn’t make you a great writer.

In the case of testing, if you don’t know what the primary conversion goal of a page is and have analytics in place to measure it, then you can “test” all day long and not learn anything useful. Plus, you have to know which factors are worth testing, how to handle disparate traffic sources, and how to best design test pages to get conclusive results.

The most common cause of GIGO in testing is someone who changes more than a single variable on a particular A/B split test page and then expects to learn what affect each variable independently had on the result. It’s something you can learn from multivariable testing (which requires heavier duty technology, analytics, and traffic) but not from simple A/B testing.

Goals

To conduct a successful test you need two clearly defined goals. The first being what is the conversion goal of the page you’re testing? Often pages have multiple goals (especially if they are home pages

or category pages) so picking a clearly defined and measurable goal can be tough. Your second goal is what is the precise, primary goal of the test? Typical test goals can include:

- Increasing the conversion rate of the page and/or the value of a typical conversion

- Increasing user satisfaction (as measured by surveys, time spent on site, or decreased angry phone calls and emails)

- Proving that “testing works” so you can get budget and permission to continue testing

- Ending internal design debates with real-life data

GWO

Acronym for Google Website Optimizer. It’s a free testing tool for your landing page (or any page on your site that’s publicly available). I’m assuming Google launched this because they recognize how crappy most people’s PPC landing pages are (especially due to a lack of overt relevancy regarding the search term advertised under.) Google is hoping marketers will use this tool to test their way to more relevant pages and better conversion rates.

Even though it’s free, you do need to understand testing goals, best practices, and basic analytics to get a good result. Otherwise GIGO. A group of consultants and agencies have sprung up helping their clients use GWO.

Should you pay more for fancy Web analytics systems for testing, or is GWO good enough? In my experience GWO is enough to do the job for an awful lot of marketers and situations. The fancy stuff can do more and measure more… but will those measurements really result in having a better performing site, or will you just feel snowed under by data? If you have a fulltime analytics staff in-house and your sales funnel

is fairly complex (perhaps requiring multiple visits over time, multiple media, and/or multiple customer lifetime valuations) then you may need a fancier system.

Hawthorne Effect

Term invented in 1955 by Henry A. Landsberger

when he noticed that the results of worker productivity tests conducted at a factory called Hawthorne Works might not be as accurate as expected. Turned out that if workers knew they were being studied, their performance would often improve during that time.

This is one of the reasons why usability lab testing for Websites is not perfectly accurate, and why you should run (not walk) to fix problems uncovered. Users in your lab are probably trying even harder than a typical site visitor would be to find their way around. If they can’t figure something out, then your site has a real problem. (Note: Why isn’t this called the Landsberger affect?

Perhaps he was an unusually modest man? You can bet if I’d figured it out, it would be the Holland Effect in a heartbeat.)

Heatmap

The results document generated when you conduct an eye tracking test. Often very colorful, which makes a great presentation graphic when you’re trying to explain why a site sucks to senior executives. Eye tracking analysts have forever warned me not to leap to conclusions when I first see a heatmap. They say, “It’s not as obvious as it looks.” I don’t know. Looks

awfully obvious. By the way, not all heatmaps are alike. Ask for the version that includes the order in which eyes looked at things, typical time spent, plus lines showing how far people scrolled and where they clicked regardless of clickability. That data is just as fascinating as anything else. (You’ll be stunned to see how many clicks happen on non-clickable areas of the screen!)

Hits

Major pet peeve – I dislike this term because it’s been misused since the dawn of the Web. It refers to the number of objects that are pulled from a Web server when a page loads for a visitor. If the page has 10 objects on it (such as your logo, your headline, your button, etc.) then your hits report shows “10 hits”. If one person visits, then you got 10 hits. If 10 people visited, you got 100 hits. Yeah, that may not be what you were expecting.

Many people still use the term “hits” when they actually mean visitors. Or probably even “unique visitors.”

They’re saying, “Oh, look at all my lovely traffic, I’m so popular!” You may be popular, but you’re citing the wrong number.

Home Page

The Web page visitors see when they type your simple URL without any extensions. Officially, this page is supposedly the “front door” of your site and even brand. Unofficially, what with SEO, long tails, emailed links, deep linking, and whatnot, the vast majority of your

site traffic may come in through other doorways. This is why you probably shouldn’t spend endless hours in committee meetings discussing and approving home page design while leaving secondary (or tertiary) site pages an afterthought. The fact is you know the CEO is going to have a strong opinion about the home page, and he or she may not care as deeply about (or even ever look at) your email-sales-alert landing pages. So, the home page gets an unfair share of attention.

Despite the attention paid to them by the design committee, home pages often wind up badly designed because too many conflicting departments have

click and conversion goals for them. You’re serving everyone from customer service to HR to press relations, not to mention the sales department. This makes conducting home page tests very difficult as you need a primary conversion goal for a page in order to optimize for it. Start by analyzing the types of traffic that come to your home page. Chances are, unless you’re doing heavy offline brand advertising, most home page visitors are current customers who type in your URL directly and use the home page as more of a site map than a brand information viewing station. They don’t care about your latest promotion. They just want to get to the department they are looking for.

Hypotheses

Before you can conduct a page test, you must create a hypothesis that explains what you are testing. Just like great scientists, be prepared for many of your hypotheses to be proven wrong. You didn’t make

a mistake. You just entered a testing process and discovered the result. Now, you can build your next

hypothesis based on your results. No matter what the test answer is, you learned something that may prove invaluable in lifting conversions (even if it’s only that the factor, you thought was important, really isn’t at all). The closer you can get to the mind of your visitors and the more factors you can either improve or eliminate, the better your chances of raising conversion rates.

The art of great testing is not to have winning hypotheses all the time – in fact if you always win, then you’re playing things too safe. The art of great

testing is to test hypotheses that can lead to conversion breakthroughs as you expand upon your learning.

Plus, it’s always nice to end design arguments with,

“let’s just test it and find out.”

Incremental testing

The process of regularly and routinely testing varying factors on a page over time in order to continuously improve results. Incremental testers, including all of the direct marketing greats, have tested so much to eliminate really bad design from their sites that now they’re testing results in seemingly inconsequential tweaks that perhaps only raise response rates by a percentage point or two. (If you just started testing, your gains will almost certainly start in the double digits.) Of course, if you’re a huge company, a 1-2% gain is a lot of money! Plus, those incremental gains add up over time. By the end of 12 months you may have grown the bottom line by double digits.

The opposite of incremental testing is big idea testing, where you test against an entirely new, sometimes crazy and out-of-the-box page variation. Big idea testing can have big wins, but also big losses. Great testing teams schedule both types of tests routinely year-long. (Worth noting: the people who do best

as incremental testers are often blind to or scared of big ideas, while great big idea people are not detail orientated and/or easily bored to make great iterative testers. A dream testing team includes staff with both personalities.)

Indexed Pages

Site pages that have been indexed by search engine crawlers. You may not want your test pages indexed, either because it might look like you’re trying to spam search engines with mirror pages or because your tests should not be made permanently public pages for anyone to see. If the page you’re testing is already very highly ranked and it gets a half-decent conversion

rate, contact your SEO expert prior to implementing any major changes to that site. If your page is number one but your conversions are non-existent, there’s little to fear by testing; however, if your conversions are OK and your rank is great, test carefully with the input of your SEO team.

Intent

Describes what’s on the mind of a visitor when they come to a particular page. What’s their goal? If your page goal strays too far from the visitor’s intent, you’ll never get a good conversion rate no matter how much you test. Online, visitors are in control. You can inform, influence, and aid intent, but rarely radically alter it.

Many marketers forget that visitors are coming to the site with a fairly clear goal already in mind. If you know that intent and you can mesh your design to provide clear signposts along the way, your conversions will improve.

Interaction Design

Designing a page with the goal of interaction in mind. Often site designers treat pages more as shovelware than anything else – they’ve been given a bunch

of content and they have to shovel it onto the page somehow. They may see their job as “make the content fit on the page” or “make it look pretty” or in worse case scenarios “fill up the white space.” At the same time, branders are thinking, “make it fit our brand guidelines.”

All are reasonable, as long as they’re held in your organization’s and your visitors’ mutual overarching goals of actual interaction. The organization wants to sell something or gather something (perhaps

permission) or influence someone. The visitor wants to find something, research something, buy something, or perhaps enjoy something. These are all interactions.

Instead of giving your designer nothing but a bunch of content along with an approved wireframe, give him or her an intent map for each page of the site. What’s the organization’s goal for this page? What’s the visitor’s presumed goal? How can they design to make those intents a reality? Then, start testing against those goals.

Iterative Testing

Invented by software usability people to describe the fact that you can’t just test once in the development process and consider yourself done. The process

of software – and indeed Website – development is evolutionary. Pages the focus group loved may

confuse real-life visitors. Pages that worked last year may not work as well now. Pages that worked in a vacuum, may not work when they are one of several pages viewed in a row during a particular visit.

The key is that testing never stops. Sometimes this is because the world and fashion changes – sites that worked well and looked up-to-date in 2002 are now old fashioned looking and probably don’t get as good response rates. Also, your customers and frequent

visitors can become blind to your marketing ploys. You can lift their response rates just by shaking up design a little bit.

If your Website approval process assumes that at some point (perhaps after a bit of launch testing) the site

is “done,” then you’re in trouble. No launch date is a “done” date. It’s just a done date for that control. Next you start working on testing more improvements.

Kaizen Method

Kaizen is the Japanese term for “continuous improvement.” It’s been applied to everything from product development to personal workflow

management. In the site testing world, it’s less related to technology or statistical methodologies than it is to your overall testing strategy. The idea is no site (or landing page) ever reaches a static peak of perfection. You can always improve results. Plus, the world continually changes, so even to maintain current results you need to continue testing.

Whether you rely on A/B split testing or multivariate testing, or some other testing tactic, your goal should be to maintain testing as an ongoing routine… forever. For some organizations this may mean routinely testing a new tweak and/or big idea every week. For others it may mean running a series of site refreshment tests annually. It depends on how much you rely on your site for revenues or sales leads, how much traffic you receive, the competitive landscape, how much money you spend on traffic driving for which you’d like a higher ROI, and how complex your site is.

Testing should be built into your routine workload and routine budget. It’s not just for new launches or critical problems. It’s an ongoing part of doing business – just like accounting or HR.

By the way, although I’ve seen Kaizen used as a term in many purposes and places, our Sponsor WiderFunnel is the owner of the formal trademark for The Kaizen Method as it applies to site testing and optimization.

Keyword

A slightly confusing term because not every keyword is a word per se. Comes from search engine marketing where keyword is shorthand for “terms that people type into search engines.” Sometimes these are single word terms such as “car,” sometimes they

are brands such as “Ford,” sometimes they are phrases such as “Ford dealers in Rhode Island,” and sometimes they are technical IDs such as “F150 part gdehw432y6749985413.”

Keywords can affect conversion rates heavily. When a visitor lands on a page, the first thing his or her eye does is skitter around looking for signs of relevancy: “Am I in the right place?,” “Does this page have what I want?” The decision often happens horribly quickly, especially with the younger generation who can make an abandon decision after half a flicked glance. The two most important relevancy factors are words and images.

If your page prominently features the keyword the user was expecting or searching for, that word will catch their attention and they’ll stay a bit longer, maybe even long enough to convert. Marketers have tested adding keywords to headlines as well as scattering them

in body copy and even on call to action and submit buttons.

Keyword Density

A term from search engine optimization that indicates how often a particular keyword appears on a page in comparison with other words. If the word appears many times and there aren’t many other words, the page might be search engine spam. On the other hand, if the density is too low, the page might not be relevant for that word.

Keyword density matters to human visitors as well. In the conversion optimization world, often the higher your density, the lower your immediate abandon rate as long as the page doesn’t look spammy. You have to have valuable, compelling content beyond keywords. The problem, I’ve noticed, with many PPC ad landing pages is their utter lack of keyword density. The word the visitor searched for is nowhere to be found in copy.

Keyword Research

Although most people think keyword research is primarily for search marketing, in reality, it’s just as important for site testing. The words you put on the page can strongly affect conversion rates. That’s why copywriting taxonomy is more important than many marketers realize.

Interestingly, eye tracking studies have shown that visitors’ eyes rarely read entire headlines or sentences, let alone paragraphs. More often, the eye rests on the first few words of a headline or paragraph and then flickers elsewhere. Highly compelling keywords get more attention. For example, I’ve seen a study where a compelling keyword in the middle of a headline got heaps more attention than a word in that position normally would. Keyword research, in this case, is the art of discovering which particular words are highly compelling to the audience you’re targeting. Even

small changes, such as “bike” vs. “bicycle,” can make a difference.

Research these words by listening to prospects talking (or writing) about the subject of your page (or a topic related to it). Great listening sources include internal site-search-box-use reports, customer comments form- fills, customer emails, focus groups, as well as within external user-generated content such as message boards, blogs, twitter, etc.

KPI

Acronym for Key Performance Indicator. Very useful for marketers with highly complex sites, lengthy sales funnels, and/or lifetime customer value goals. When you test a single page on your site or a single landing page (as you must because running tests on multiple pages at the same time can make results murky),

the results data probably won’t give you the extent of conversion data you desire.

For example, you’ll learn immediately if someone clicked on a “submit” button for a lead generation form, but not if they will wind up becoming a paying customer days or months later. You won’t know how valuable the account will be to your company. Therefore in order

to evaluate whether a test really helped improve your bottom line, you need to first determine what your KPIs are. In this example, perhaps the percent of visitors who answer certain questions on the form in a certain way and click “submit” is the KPI you’re hoping the test will improve on.

Landing Page

The Web page that visitors arrive at (land on) when they respond to a marketing offer that has a special hotlink or URL that’s used just for that offer. For example, if I click through on any sort of online ad, the page I arrive at would be your landing page. As the first page in

the site sales funnel, this is a critical page to test and improve on. Most people click on landing pages with one specific desire in mind. They usually don’t give a site a “second chance” and poke around looking at

further pages if what they were expecting to find on the landing page isn’t there. If the landing page doesn’t have precisely what they expected (or a very clear pathway to get to what they wanted) they abandon.

This is why a landing page can’t serve many masters. Generally you should not point a wide variety of traffic from different advertisements or PPC search ads all to the exact same landing page, because people from different ads have different expectations. It’s also why sending clicks to a general “category” page on your site or a general home page often dooms you to lower conversions than you deserve.

For more information, see MarketingSherpa’s Landing Page Handbook, of which I am the proud author.

Landing Page Optimization

The science of testing your landing page in order to determine which tweaks and changes are optimal for high conversions. You may go further and define

optimal conversions as not so much a high number but more as a good number of the sorts of people who tend to make high quality leads or customers over the longer run. You may need to re-optimize your landing page, testing new changes and tweaks versus the control, when:

- Your traffic source changes (i.e. people are coming

from a different place).

- Your advertising has changed therefore clicks may be expecting something different.

- Your competitors are breathing down your neck with “me-too” or other aggressive campaigns therefore you need more marketplace differentiation for better conversions.

- You want to try new interactivity such as Twitter invites, online chat, embedded videos, etc.

- Someone powerful in your organization thinks a different page would be better.

- Your brand or other site pages have significantly

changed.

- A lot of time has passed, the world has turned, and your page needs to keep on turning with it.

Latent Conversions

Also known as delayed conversions. These are people who click to your landing page and sometimes even further into your site, but who do not convert

immediately during that visit. Instead, they may return to convert later… in the day, the week, the month, or the year! Data suggests latent conversions can be as high as 40%-60% for some brands depending on a number of factors such as how long cookies are left live and tracked.

Why are so many conversions latent? Sometimes it’s just bad timing. Perhaps the phone rang or the prospect was called away just as they got to your site. Sometimes the prospect needs more information

– perhaps asking a friend or researching competitors

– before making a yes/no decision. And, sometimes, they just need a few more “touches” to feel safe enough to continue with the conversion.

For decades, advertising research online and offline has shown that most prospects need to be “touched” multiple times (eight is sometimes an optimum number) before they convert. A “touch” can be any message they see or hear with your brand name involved. It could be a friend talking about you, a radio ad, a mention in

a newspaper story, one of your own advertisements, your brand name high up in search engine results, etc. Over time, the prospect both remembers and begins to trust your brand more than they did in the past. Then they’re more likely to convert when they see an offer or perhaps return to your site for a second or third time in order to convert.

Lead Generation

The business of generating sales leads. (Note: this does not mean “compiling” or “buying” leads.)

“Generation” implies the prospect has requested or agreed specifically to be added to your list. As for what a “lead” is… that is something you’ll have to decide internally after (hopefully) a great deal of pointed discussion with the people in your organization whose job it is to convert the leads into actual sales.

Once you’ve defined what a quality lead is, then you can begin to optimize your site to gather more of them. A winning test may actually give you fewer conversions

– fewer people filling out your lead generation form. However, those fewer people might be of such high quality in terms of their likelihood to convert that they become infinitely more valuable to your company than a big pile of resource wasting “leads.”

In my experience, B2B lead generation marketers are the least likely (out of all direct response marketers) to invest in page tests, while being the most likely to see an extremely high ROI from testing. The most

important thing to test first is your actual response form. Page images, copy, templates, etc. can help too, but the form is often a response killer unless you optimize through testing. So, test your form first and then work your way toward the creative stuff after that.

Lifetime Value

The value (to your company) of a customer account during the length of its lifetime. Most companies put a one- to three-year cap on “lifetime” because it’s more useful and possible to measure than a human’s lifetime. Value is generally determined by the amount of sales made to that customer in terms of total dollar amount or units or profitability or some calculation based on these, but some marketers also add in a “referral” factor. A one-time customer who then becomes a huge fan and refers 10 other customers may be more valuable to your bottom line than a two-time customer who never refers anybody.

If lifetime value is an important metric for your organization, you’ll want to identify some KPIs for individual pages you’re testing. KPIs that help you understand how changes on a particular page can influence the lifetime value in the long run.

Lift Analysis

Math analyzing how much of a “lift” (i.e. conversion increase) a change on a page creates as compared to the control. If you’re doing A/B testing and have changed more than one particular item, you can only analyze overall lift. You can’t tell which of the items was responsible for the lift, nor if any items actually

caused a decrease that was masked by another item’s lift. If you’re doing multivariate testing with adequate software, you’ve driven enough traffic to all varieties of the page you’re testing, then the results can conclude how each individual item affected lift.

The confusing thing about lift analysis is that it is not guaranteed to be pinpoint predictive. Just because you got a 15.82% conversion lift on a page by changing your headline copy, it doesn’t mean that three months from now the page will continue to perform 15.82% better than your old control page would have. Chances are very strong that the page will continue to perform somewhat better than the control, but the lift may have subsided a bit or even increased. There are too many external factors to consider (including time itself) to predict long-term lift with pinpoint accuracy.

List

A type of traffic source. Any database or collection of names that you can market or advertise to. Might be an email list, a lead list, a postal list, a publisher’s subscription list, etc. A “house list” is a list owned

in-house by the marketer or organization that owns the site in question. Quality of response varies tremendously from list to list.

In fact, if you send the exact same marketing offer to two different lists (even two demographically or

psychographically different segments of the same list) and responses land on the same landing page, the conversions from one list are nearly guaranteed to be significantly different from the other list. Same landing page, different results.

List, therefore, is the biggest factor to optimize. Test and pick the best lists for your organization and you’ll do better. It’s tough to run list tests at the same time you’re running page tests. For statistically valid results you must send the exact same sort of traffic (i.e. traffic from the exact same lists and segments within lists) to each version of the page you’re testing. Otherwise the natural differences in list quality will inevitably skew your page test results and you’ll never know what caused a lift or depression. Was it the page test or the lists?

Sophisticated marketers also find they must create and optimize separate pages for different major lists. In particular, your own house list will probably respond best to a landing page that’s been tested and optimized specifically for them. That same landing page might not work well for an outside list of people who have never heard of you and/or have no prior relation with your brand. If you tend to rent email lists or run ads on email newsletters, you may need to perform tests to optimize for these major external sources versus house marketing.

Marketing Funnel

A graphical representation of how prospects become your customers. Invented by people who like to think of life as something that can be flow-charted and made tidy and orderly. The big wide opening at the top represents everyone who might ever conceivably buy from you, and then the funnel narrows gracefully to a much thinner spout at the bottom out of which actual paying customers pour.

Although bottom is true to reality, everything from the top to the middle is far less graceful. You may not be accounting for the wide variety of entry pages and/or all the touchpoints along the way (especially word of mouth, which can pop up anywhere and dramatically alter results). And you’re certainly not accounting for the variations individuals go through in their buying decisions. Some people buy impulsively, others agonize.

The best places for testing in the marketing funnel are places where you’ll get the best immediate ROI.

These are near the very top of the funnel (i.e. at a landing page for your most expensive media – probably including PPC clicks), wherever lead generation or email opt-in activity could occur, and at the very bottom where shopping cart or lead qualification form fills take place. Once you’ve optimized top, opt-in, and bottom you should start analyzing the rest of the funnel looking for trigger points where large numbers of prospects are turned off or where they make a yes/no decision about your brand.

Last, but not least, examine the funnel for places that might spark word of mouth and then optimize for

that. Qualified prospects and customers are your best source of more people just like them. Friend referral campaigns tend to have outrageously high ROI. Optimize for it.

Marketing Optimization

The act of testing every significant aspect of your marketing – from individual tactics to overall strategy and mix – in order to determine what would work better. Far fewer marketing departments engage

in this than you might think. It’s far too easy to get caught up in launching campaigns, meeting deadlines, and managing to budget. Plus, a lot of marketing departments tend to rely heavily on legacy habits – we do something a particular way because that’s the way it was done last time.

I’ve seen two factors help with optimization. The first is new or shifted staff. When you’re in a new job, you tend to have a new perspective. The second is budgeting in tough times. If you have to fight for every dollar, you’re more likely to closely examine how to get more bang for your buck. Every marketing department would benefit from a marketing optimization audit every year. Think of it as a spring cleaning. You’ll often think of great testing ideas at this time.

But be careful about the way you relay news of results from improvements to upper management. I’ve seen this backfire politically dozens of times! When you

tell management something like, “We’ve changed the way we do things, now we’re spending much less per lead!,” management is likely to either: (a) expect you to keep up the pace of reducing costs beyond what’s reasonable, (b) reduce your budget because you “can do it now for less,” and/or (c) blame the old department for having done a bad job and “wasted” money in the past, which leads to general ongoing mistrust.

The fact is the old department didn’t do a bad job. They probably did the best job possible at that time. Now times have changed, tactics have changed, the marketplace has changed, and the old tactics don’t work as well anymore. It’s time to optimize for a new

day. Optimization never really stops, but if you keep on testing after immediate big gains, you’ll probably settle into a pattern of incremental gains. The first group of tests might raise response 40%. The tenth group of tests might only contribute 4%. Does that mean the tenth group isn’t worth doing? You have to do a time/ cost calculation. You also have to re-set management expectations for the long haul.

Microsite

What’s the difference between a landing page and a microsite? Simply put, a landing page is a single page. A microsite is a Website with multiple pages. A landing page can sit out on its own, with no other navigation beyond the response device. Or, it may be a page within your site that you believe serves as a good introductory or conversion page for people who click in from an ad or an offer you’ve promoted.

A microsite is a separate site from your main site. It can have as few as two pages or as many as hundreds or thousands. What’s important is that the microsite is developed, usually by marketing, as a destination site for a particular campaign, a particular product, or a particular segment of the marketplace. For example,

a cruise line with a main corporate site offering tickets and service for all of its operations might also have

a microsite that focuses entirely on one particular ship. Microsites work well when your main site has a multitude of information serving a variety of audiences which might distract, confuse, or otherwise lower conversions for a particular campaign or product line.

Microsites also work well when your IT department, legacy systems, or office politics are holding your company site hostage so you can’t make the changes or run the tests you need to, to be sure visitors convert. Instead you go off and build your own site for a marketing campaign. The site may link to the main company site through a logo, a site map, a copyright link, or an “About Us” page.

The danger is that any one company will end up with microsites sprouting like weeds, perhaps confusing the brand and/or the marketplace. So, management

should set best practice guidelines for microsites. Also, they should keep a list of site owners and maintenance schedules on an internal memo so no one forgets to keep a site updated or alive.

Microsites can and should be tested and optimized for better results. Because the home page is usually the landing page or main entry page, this page should be rigorously tested. Any pages that contain response devices used for conversions should also be tested.

Multivariable / Multivariate Testing

Also known as multivariable testing. Refers to the science of testing multiple variables for a particular Web page at the same time. In order to do this, the testers actually create a large number (as many as thousands) of different pages, each with a different version of

the variables. Then they open the visitor floodgates, showing just one test page to each individual visitor, and then tallying results across all pages and variations.

If you have enough traffic to support all the test pages, this type of testing can accomplish a great deal in a fairly short time. (I’ve been known to call it “A/B testing on steroids.”) You’ll get data back on how different tweaks to each variable on the page caused conversions to go up or down. So, perhaps a headline change improved results 7%, while an order form change lifted response by 12%. (Note: some of these lifts overlap because they affect the same individual

conversions, so changing your headline and order form would not give a total lift of 19%, although it would be higher than either one alone.)

Unlike A/B split testing, multivariable testing is new to marketing and advertising. (In fact, I first heard of it less than 10 years ago.) Also, the statistics involved can

be fairly complex. When multivariable became the hot marketing buzzword over the past few years, a lot of vendors and agencies jumped into the field, some of whom were not truly qualified to handle its complexity. As a result, I’d warn you to stay away from anyone who offers multivariable testing as an “add on” to general agency or site design offerings. You really want a specialist. In addition, don’t throw out your current control in favor of the new winning combination page without first running a follow-up A/B test – control vs.

multivariate winner – to make doubly sure the new page is best for you.

Nth Name

“N” stands for number, as in “seven” or “twelve.” So, an “nth” would be “seventh” or “twelfth.” The expression

is frequently used in Website testing to explain which of your incoming visitors will see a particular test panel. For example, if you have roughly 200,000 visitors per week and you want 10% of them to see a test version of a page; you’ll do an “nth” visitor select of 10%. The term originally came from the direct postal mail world where marketers would split lists for tests this way.

Why should you care about “nth”? Because it’s one

of the fairest ways to split up a list or traffic. Other methods – such as showing the test page to all visitors for 10% of the day – would skew results (in this case, people tend to convert differently at different times of the day).

Nth is not totally random. In my mind, it might be safer than random. But, I’m not a statistician by trade and some of them might disagree with me!

Non-Conclusive Results

“Non-conclusive” means you went through the effort of running a test but the statistics don’t show a significant conversion difference between the test versions. Either one might work as well as the other.

Some of site design decisions that get argued and anguished over in committee meetings (or just in your head) wind up showing non-conclusive results when you test them because, in the end, the factors you thought were critical, really aren’t. These factors may include (but are not limited to) your submit button color, the precise wording of your logo’s tagline, and in what order sidebar navigation options are listed. Testing firms with broad experience can save you money because they are able to spot which site elements are more likely to be worth testing than others.

Offer

Seems obvious, doesn’t it? This is whatever you are offering to get visitors to convert. Could range from a free email newsletter to a sweeps entry to an actual product visitors must add to cart. What’s surprising is how often non-ecommerce marketers make that offer tough to find, or fail to make any offer at all beyond perhaps a puny RSS feed icon.

If you’re not overtly asking for a conversion or you force me to click around looking for a a way to convert (where’s that contact form again?), I may not bother.

Pricing, gifts with order, free shipping, and acceptance deadlines also fall under the header of offer testing.

Offer copy and placement tests often show fairly

significant results. Definite bang for your testing buck.

Optimization

The science of testing different versions of a specific advertisement or Web page in order to discover which version will elicit the best response from prospects.

Anywhere from two to hundreds or even thousands

of versions of a particular ad are fielded. Then, as response rates come in, the winning versions become apparent through the glory of statistical analysis.

Optimization can be used for any type of advertising that generates a measureable response from a large- enough group of people that results may be analyzed reliably with stats. And where multiple creatives can be fielded simultaneously (or nearly so), including banner advertising, email broadcasts, direct response television, and Web pages such as landing pages.

If you’re investing in a major campaign, if possible, always run optimization tests beforehand to a limited (but statistically significant) group to determine which creative will work best. Only then should you invest in a full campaign roll-out. The difference in response rates can be quite astonishing. In the case of online campaign landing pages, I’ve routinely seen at least 40% response lifts due to optimization.

Original

Also known as a control. This should be a piece of already existing marketing (a landing page, a banner ad, etc.) with already existing results history that you’ll be testing new ideas against. Your goal is to find a new winner than performs better than the control. If you’ve never fielded any creative and everything is new, you can just designate your most conservative design (often one built to best practices and brand standards) as

your control for testing purposes. Your original should always be run as one of the live test cells during a test so you compare up-to-the-minute results data from it to its wanna-be competitors.

PPA

Acronym for Pay Per Action. Actually this is probably better expressed as Pay Per Conversion but there’s already a more famous meaning for PPC, so everyone settled for PPA. The “action” is whatever conversion the marketer has agreed to compensate advertisers, media, “partners,” or affiliates based on. Most often they get paid by either by the sale or by the lead generated.

This might be a flat rate or a percent commission.

PPA commissions can be very enticing because there’s a good bit of money involved (in fact it’s a billion dollar industry when you include affiliates). But, there’s a

lot more risk involved for the partner who sends traffic to the store or site hoping that traffic will convert into lots of lovely commissions. The problem is the loss of control. If a merchant pays per click for traffic, I can control (on my end) the activity I do to send clicks to the merchant. If the merchant only pays commissions on “actions” that happen on their site, however, my fortunes are tied to how well that site converts the traffic I send. I have zero control over those conversion activities. I send traffic, cross my fingers, and hope.

Since my time and traffic are expensive, the merchant

has to make me feel safe to keep me as a partner.

That’s why many merchants invest in heavy page and cart testing to improve conversions. It’s not just about converting more sales or leads. It’s also about impressing your PPA partners so they won’t cut off the traffic tap. Affiliates and partners love to hear you are testing – even when a test fails. They like to know you are investing in science to make their bottom line ultimately fatter.

PPC

Acronym for Pay Per Click. If you are a heavy PPC buyer, you absolutely MUST test and optimize your landing page for conversions before investing lots of money buying clicks. My tips:

o Keyword integration: Find a way (any way) to insert the actual search term used by the visitor into your landing page. If you can add a matching graphic too, that’s fine, but the actual word is most important. Heavy search marketers often do this

dynamically, creating one landing page template that is automatically altered for each keyword. Some copywrite a headline that nearly any search term can fit into. Some add a line at the top of the page that reads, “Search Results for: [insert keyword].”

Do some tests to see if keyword density will help conversions. You may raise responses by repeating the keyword in several places on the page.

o Demographic differences: If your brand or product appeals to very different demographics (for example, parents vs. teens), you should consider testing

and optimizing your landing page for the search terms that tend to be used by each segment. Don’t assume the same landing page will work equally well for both. If both demographics use the exact same term, you’ll need to design a landing page with separate areas of copy (and possibly graphics) for each group. Sometimes that’s as simple as giving each demographic their own navigation button (example: “Parents click here!” and “Kids, click here!”) I’ve also seen tests where each paragraph of body copy was designed specifically for a different persona.

o Conversion path position: If your products have an extended conversion path during which prospects might search for options several times (for example, people who want to read reviews first vs. people who want to buy now), you may need to test and optimize landing pages for each of these groups. Often their position is somewhat clear from keyword choice.

o Branding vs. PPC: Many advertisers like the PPC- media-buying model because they feel they’re “only paying for the ads that worked.” My feelings are mixed because it rules out the influence branding has on the ultimate conversion rates. Brand

advertisements – ads that are designed to raise

brand awareness and/or influence brand perception

– usually should not be designed or optimized for maximum clicks because that’s not their purpose. In every recession, marketers inevitably pull back on brand ads to refocus budget on PPC and direct

response because it’s “measureable.” Unfortunately, if your brand awareness or perception slips during this time, what you may measure is a lowered response rate both to the PPC ads and to their companion landing pages!

Page Rank

A term search marketers use when discussing how high a Website shows up on search results for a relevant keyword. A page rank of #1 means the site is in the

top organic (not paid) position for that term. If you’ve got a top five position for an important keyword to your business, you don’t want to do anything to risk losing it. At the same time, you need to make the most of the traffic coming from that rank. It’s a delicate balance. When you conduct optimization tests, be very careful to involve your SEO team so nothing is done that could adversely affect rank.

Path Analysis

Most Web analytics systems offer path analysis reports as a standard feature. The reports show you how visitors move through the site – which page they enter on, where they click-to next, and then where they leave. If your site is complex, path analysis reports can be overwhelming.

Instead of looking at all paths, do a form of regression analysis and focus on the paths of your “best” visitors

– people who converted. Start at that end-action and work your way backwards through the path they took to get there. What were the most common pages they visited? Were there any other commonalities? Next, list what appear to be the most important pages along their path to conversion. These are the pages to start

testing immediately. You already know these pages are converting some of your visitors, so it’s worth the testing investment to optimize them to convert even more people.

Performance-Based Advertising

Also known as Pay Per Performance (PPP) advertising.

Instead of paying for media space, you’re only paying for responses to your ad. Depending on your arrangement with the media owner, you might pay by the click (PPC), or by the action (PPA). Be sure to have conversion measurements in place prior to

launching this type of media buying so you learn how much your investment is really worth. Contract by a short test period at first to see if the responses from a source are any good. If you have a source that

tends to provide responses of outstanding quality, next consider investing in testing to optimize the page these responses land on. For best results, you may wind up with a slightly different landing page for responses from different major sources. If a source sends unusually low-quality responses, first check that they are doing everything on their end to create the best quality responses. Your landing page may not be to blame.

Persona Marketing

Rather unscientific but awfully useful. Personas are detailed personal profiles of an individual who is held as representative of demographic segment your site attracts. Usually these profiles are fictional, although the profile will include what looks like highly personal information for a real ‘named’ person. Personal information can include hobbies, age, job, what he or she loves/hates, what stresses him/her out, taxonomy, and even a photograph. The thinking behind personas is that if your Web designer, copywriters, and testing team can clearly visualize a “real” human being, their work will improve. I can tell you, it works!

Key: no site has one single “typical” visitor. When you develop personas, you’ll need to segment your typical traffic into groups by factors that make them different from each other. (Especially in the ways they evaluate and make conversion decisions relating to your type of product or service, as well as ways in which prospects self-define their own demographic.) Segments may be by demographic, psychographic, job function, company size, etc. You’ll probably end up with a dozen segments

that you can then carve down to four to seven by picking the most profitable ones to focus on. Then you’ll create a persona profile for each.

The actual profiles should be based on market research, rather than marketers’ gut and guesswork. (Unfortunately this doesn’t always happen.) Focus groups, surveys, and actual meetings with real-life prospects can help you. Also, always run your proposed personas by people in your organization who are on the front lines, interacting with actual prospects and/or customers daily. These are your customer service team, technical support, sales reps, store managers, etc. Their input will be invaluable.

Once you’ve got personas approved by the front lines, give copies to Web development, Web design, all advertising, PR, marketing agencies, execs, and (of course) your testing team. Some of the best big idea tests have sprung from persona review sessions. Persona-focused tests include navigation, images, copywriting, and offer wording tests.

Personalization

“What’s in it for me?” That’s the question every visitor thinks when they reach your site or evaluate a marketing offer. The more you can speak to each prospect individually, the higher your response rate.

Online, many marketers have tested personalized landing pages and microsites both for individual visitors and for organizations (for B2B accounts). The campaigns tend to do better overall than non- personalized ones. There’s something powerful about seeing your own name on a Web page and/or dynamically generated content that’s been selected specifically to appeal to you. There are costs to this approach and also risks. (Ever considered how a misspelled name can backfire?)

If you are sending out a personalized campaign

– perhaps a direct postal mail and/or direct email – to a list of qualified prospects with a strong offer, you should consider testing personalized landing pages. For example, pre-populating lead generation reply forms can help raise response rates considerably.

Predictive Analytics

Predicting future results (such as page conversions) based on your current data. Sadly, this is far less accurate than some people think. It’s not so much a math problem as a control problem. You can’t control all the factors that will affect conversions in the future. These include traffic source, competition, time of year, time of day, the growth (or lessening) of trust-in and recognition-of your brand, fashion trends, the economy, etc.

Plus, there’s also a mysterious “slide” factor where almost inevitably the response pop you see from a successful test slides downwards over a period of time, usually over some months. You may never hit as low as where you started out, but you probably will slide down, losing perhaps 25% of your gain over time. I don’t know why this happens, but I’ve seen it across marketing tests in every media (direct postal mail, web, email, TV, etc.) for years and years. This is why continuing testing is critical for long-term success.

Primary Sensitivity

This term was originally used in medical testing, but is extremely useful when you need to analyze why a Web page isn’t getting good enough conversions and what should be tested to fix things. The idea is

that each visitor is sensitive to elements on the page. Some elements (such as the headline, the offer, “trust” elements, etc.) are areas of primary sensitivity. If

you make changes to these, response rates can be significantly affected. Other areas, such as fine print in your page footer, are areas of very low sensitivity, so it may not be worth testing changes there.

When deciding which tests to invest in – or which to invest in first – pick the elements of primary sensitivity. Then work your way down the list. Big surprises in testing often happen because things you didn’t think were areas of primary sensitivity turn out to be. (For example, a copy test of bullet points vs. paragraph- form copy sometimes reveals a greater sensitivity than writers expected.)

Last but not least, remember primary sensitivity can be internal too. If a powerful member of your management team has a lot of sensitivity around particular page elements or design, be careful about testing these

things. You may decide to test another element first in order to win the exec over to the idea of testing before taking the risk of testing something that could prove his or her entrenched design ideas are wrong. Being

thoughtful about office politics mean you can live to test

another day.

Qualitative

Research or study results that are not based on statistical mathematics, yet still have some real value for the marketing team as they work to improve creative and results. These are often a human factor that numbers couldn’t quite measure properly such as a focus group or the participants in a small usability lab. Often these are very helpful for copywriting and product positioning breakthroughs, as well as pinpointing overall design flaws and areas of confusion. Sometimes the numbers tell you, you have a problem, but only the people can tell you why.

Quantitative

Numbers-based results and reporting on campaigns and activities. The data does the talking. Of course, numbers can be misinterpreted or twisted, which is why every marketing department needs at least one staffer (or consultant) who’s strong with statistics. Many of us got into marketing because we love graphics, creative thinking, or writing, so the growing importance of mathematics can feel a bit scary or annoying.

The biggest mistake I’ve noticed busy marketers make is not asking for precise definitions of the terminology in their quantitative analytics reports. Most analytics reports were created by techies and programmers who

are not wholly educated in marketing terminology or the importance of clarity for labels on reports. You may, in the end, have a number that you think says one thing when it’s actually mislabeled and means something entirely different. This is a more prevalent problem than most know. In fact, I’ve never in my life seen an analytics report where labels weren’t at least somewhat misleading (albeit not on purpose). It’s tough because marketers think they’re basing decisions on solid

quantitative data, when actually they’re making incorrect assumptions. So, always double check that every label on a report really means what you think it means.

Regression Analysis

A form of statistical analysis where instead of looking at your marketing results from the front end (the big incoming pool of prospects), you turn the marketing funnel upside down and start analyzing backwards from the smaller pool of successful conversions. The

goal is to back up through their actions and reactions to discover: Where did these people come from in the first place? What common elements do they share along their site path and visiting history? What elements helped them convert? Then you emphasize those factors to get more of those high quality customers.

In effect, the focus is on what worked and how I can get more of those people, instead of what didn’t work and how I can fix it. You’re not fixing a problem, but rather widening a channel to increase flow.

You need a fairly big pool of conversions to conduct regression analysis, so it’s most useful for marketers with massive volumes of conversions. You also need a competent statistician on staff. Wrong conclusions

could be more harmful to your company than just going on gut or no conclusions at all. Math matters.

Relevance

One of the biggest factors in landing- or entry-page abandon rates. Visitors have clicked to this page with a specific hope and expectation for what they’ll find there. If your page’s content meets these expectations in terms of content, then visitors will find it relevant and will be unlikely to abandon without carefully evaluating the page first. If your page’s content appears at first glance to not be relevant to expectations, then your abandon rate will be quite high.

Most marketers are too close to their brands and landing pages to truly see if the pages are relevant to new visitors or not. It really helps to get an opinion from a fresh eye. Tests that add relevancy to a page often have high success rates.

Results Degradation

Depressing but true, after you test pages and end up with new controls that get wonderful results, slowly over time those control pages probably won’t do as well.

Some marketers can track a results degradation curve for their newly tested pages and hot offers. This may happen because your visitors get bored with the new creative or offer, or your competitors start copycatting you, or the world simply continues to turn.

Degradation is why you must build testing into your budget and schedule on an ongoing basis. You can’t just optimize your site or landing page once and then be done with it. You have to keep floating new ideas. If the same ad worked well for years and years don’t you think advertisers would save money and never invest in new creative?

That said… change for the sake of change alone may not help at all. Always test major new ideas instead of just fielding them on a wing and a prayer. Get creative with ideas to test, but let the data drive your roll-out.

ROAS

Acronym for Return On Advertising Spending. Landing page testing is a proven, scientific method to improve ROAS because if you improve landing page conversions, you improve ad results. What bother me are marketers who spend a great deal of energy, budget, and time on their advertising creative and media buying. They’re investing and polishing their outgoing message. Often the landing page is an afterthought. I would not invest in a significant new outbound ad campaign unless I had tested and optimized my landing page first. Why spend all that

money on outgoing messages if the incoming clicks are going to take one glance and bounce off?

By the way, always make sure your agency and your media buyer have access to landing page data including conversions by campaign and (hopefully)

media source. For some reason, many marketers ask their agency to do the outbound work and the internal IT/Web team to handle the landing page. The twain don’t meet or talk. If your agency and media buyers can’t see how good or bad conversions are now, they can’t improve creative and media buys for the next go- round.

ROI

Acronym for Return on Investment. Only became a popular term in the marketing department since

2001 when so many were fighting for their jobs, not to mention their budgets. Testing can help you improve campaign ROI if you have enough traffic to run a viable test and you are testing for high-impact variables. In situations such as long sales cycles (where it’s hard

to assign a bottom line value to improved results for a particular page or campaign) you can use KPIs to make a decent stab at it.

Your first step is to make some baseline ROI measurements for current conversions on the page you intend to test. Do you know your conversion rate for this page already? Do you know what your competitor’s conversion rates are? (By the way, if your competitor has enough traffic to show up on the radar, you can

get custom conversion rate reports from services such as Compete.com.) Of those conversions, can any be tracked through to final sales? Alternatively, if you trace the origins of any of your big customers, did any of them first contact the company using this page or form? By showing the page does have an ROI of some sort, and that the conversion rate could be higher, you can make a business case for investing in testing.

Roll-out

The assumption of testing is that you are optimizing your landing page, site, or ad at the start of a campaign by trying different versions. Then, once you’ve got the best possible version, you “roll it out” so it’s the new control that all traffic sees. Roll-out implies a broader audience than just the test panel, so sometimes you will see some results degradation, but hopefully the new page is so much stronger than the old control, it still does significantly better than you would have before.

Sample Size

The number of visitors who saw your test page. This does not include people who were advertised to (or invited) but never visited one of the test versions. If you pay for traffic, or you just want to limit testing risk, you will want to limit sample size to just as much traffic

as your testing software needs to obtain conclusive results. This, in turn, depends on factors such as your typical conversion rate and the tests’ complexity.

SEM

Acronym for Search Engine Marketing. An umbrella term encompassing both search engine optimization (SEO) and search-based pay per click (PPC) advertising. If a significant portion of your traffic comes from search engines, you should be optimizing your most-used entry and landing pages for this type of traffic.

These may be visitors who are less likely to have trust- in and knowledge-about your brand than visitors who come through other paths. I’ve talked to marketers who told me search-driven traffic is more likely to

click on ‘about us’ pages and the ‘home’ button than other traffic. Search marketers call this activity “up periscope.” They use it to describe a visitor who lands deep within a site due to long-tail search results, who then clicks a general information page to figure out where they are and what the brand is anyway. If you notice that sort of activity in your Web analytics, it could be a sign you need to treat your interior site pages more as ‘home pages’ or ‘entry pages’ in their own right and include some ‘About us’ information and credentials as part of your standard page template. At least test it!

Segmentation

The act of slicing or dividing your total marketplace into groups – or segments -- of commonality in order to market to them more effectively. The more pointed, targeted, and relevant your messaging is (including taxonomy and benefit copy) for each visitor, the higher your conversion rates will be. Powerful segments tend to be by industry, by job function, by consumer demographic/psychographic, and of course by relationship with your brand (new customer, old loyal fan, etc.)

Landing pages developed for specific segments tend

to perform far better than landing pages developed for “everyone.” Sometimes you can use the same template with different copy for different groups. However, when

the segments come from really different cultures in terms of style, communication and/or online activity (think Japan vs. Norway, or parents vs. tweens), you may need a different template as well.

When you’re running landing page tests with your routine traffic prior to a massive outbound marketing push, bear in mind that visitors coming from the new marketing campaign are a slightly different segment than your routine traffic. Page test results may differ from roll-out reality. When possible, always test to a type of traffic as close to ongoing traffic as you can. Different populations behave differently!

SEO

Acronym for Search Engine Optimization. It’s the art and science of trying to get a Website noticed by search engine crawlers and ranked as high as possible on results pages for highly relevant keywords and search terms. Success factors include (but are not limited to) relevantly worded inbound text links from related sites, the ferocity of competition, the age, size, and structure of your site, the precise wording of your URL and page titles, as well as the precise wording of headlines and the first paragraph or more of body copy.

Page copywriting does double-duty as both an attractor and educator of crawlers and an educator and conversion tool for humans. This means any copy tweaks to improve conversions may affect your page ranking. Always work in coordination with your SEO team when testing pages critical for search engine traffic.

Session

Each particular visit from an individual visitor is called a “session.” Session length (i.e. visit length) is a metric many marketers and Web developers review to see how successful the design is. Shorter session length is often considered more successful for home pages and help areas because it may imply visitors found what they wanted and moved on quickly and easily, and thus they’re supposedly more satisfied with their site

experience. A longer session length might be indicative of success for an online library of canned webinars

or online tutorials, indicating the visitors dug in and enjoyed the content.

Unfortunately, if a visitor only viewed a single page and then abandoned (as 80% or more of typical landing- page visitors do), your analytics system probably

isn’t tracking the length of those sessions because it probably only tracks time for visitors who click at least once within the site or page. If you are concerned about landing page results, ask the analytics team for special programming to overcome this lack of session length information. By learning how much time typical visitors spend before they abandon, you can determine whether you have a big overall problem such as bad traffic and/or a page that appears to be irrelevant to visitors’ needs. You can see if you have a conversion problem whereby people like what they see but they just aren’t quite convinced by the offer or the details.

Split Testing

Another term commonly used for A/B testing. Comes from the tactic of splitting traffic to a page (or, in other media, splitting an actual email or direct mail list) into test cells each of which sees a particular version of the test creative. The split doesn’t have to be 50/50. In fact, a test cell can be any size you want as long as it gets enough traffic for statistically valid results. Otherwise, what’s the point? Biggest factor: make sure the visitor stream or list you’re splitting has nearly identical demographics. If you want to test different lists or

traffic sources, then you have to send them to the same identical landing page so you’re comparing apples to apples. If you want to test different landing pages, then the lists/traffic sources must be the stable, singular element. Pick one side or the other.

You can split testing by serving each cell a slightly different hotlink (although the visible hotlink where they click must be the same), or you can split at the server- level and show a different page to a portion of the incoming traffic. The latter is more common.

Taguchi

Mr. Genichi Taguchi is a (now elderly) Japanese engineer and statistician who invented new methods of statistical modeling and test (or experiment)

development in the 1950s to help factories improve the quality of manufactured goods. His statistical methods have never been completely accepted or proven by the Western statistical community, but they’ve been applied to everything from automotive R&D to advertising.

Taguchi testing tools are known as ‘fractional factorial’ because they extrapolate results based on fewer test cells than classic A/B testing would require for the same number of elements tested. If Taguchi methods are correct, it would be a great boon to marketers who don’t have enough traffic to sustain dozens or hundreds of A/B test cells per page. However, according to some statistical experts, the jury is still out on Taguchi and advertising.

The idea of multivariate testing for Websites first came from statisticians familiar with Taguchi’s methods. These days, however, only a few multivariate testing firms rely wholly on Taguchi’s calculations. Some have altered the calculations to account for the unique complexities of the online testing environment. Others, known as

‘full factorial’ tools, basically do A/B testing, but field so many test versions at a time that they feel they can describe their tools as as “multivariate.”

If you decide to use Taguchi-based methods for your page test, always run a confirmation A/B test afterward to compare the results from your control (original) page and the final Taguchi winning design. In my experience, sometimes the results will not be as expected. However,

this very much depends on the skill, tactics, software, and experience of the team running the Taguchi tests.

Test Cell

Also known as a test panel. A particular version of your test for a particular list and/or traffic source. If you are running a test with eight versions, you would have eight cells. If you are running a test with eight creative versions, each of which receives traffic from two sources (the results of which are tracked separately), you would have 16 cells.

I suspect this term came from direct response marketers who, in the past, used spreadsheets to prepare reports tracking the results of their tests. Each test you ran got its own results cell on the spreadsheet.

Thank You Page

The page that you send visitors to immediately after they convert. For an opt-in or lead generation form, this would be the page that shows up when the visitor clicks a submit-type button. For ecommerce, it’s the page you see when you check all-the-way out of the cart.

Outside of advanced ecommerce marketers, such as Amazon, very few marketers bother to test their Thank You pages, which I think is a huge mistake. As a vast generalization, (in my experience) as many as 40% of visitors who hit a “thank you page” can be persuaded to take another action, if you ask them to. This

additional action could be signing up for additional email newsletters (a co-registration page), filling out a brief survey, answering a few lead-qualification questions, referring others to your offer via a tell-a-friend form, clicking on cross-sell/upsell-offer hotlinks based on what they just purchased, and/or clicking on relevant- information hotlinks to continue interacting with your brand.

Visitors who have just converted are your Best Prospects for additional offers at that moment. This is golden traffic because it consists of both proven responders and those proven to be interested in your

offers. Why lead them through a conversion experience and then just dump them at a dead-end page on the other side? Instead, test offers and design layouts to keep the conversion experience going. In B2B and long-sales-cycle companies, the additional interaction could shorten your sales cycle. In ecommerce, you can gain from cross-sales. Anyone with an email program can benefit from either co-registrations or further email- list-segmentation offers.

The Thank You page is well worth testing.

Traffic

Visitors coming to your page or site. Not all traffic is equal or alike. As media buyers and affiliate managers know very well, different traffic sources send very different traffic even if the ad or link is identical on both sources! One site can send 1,000 visitors who convert at 5% and another site (using the exact same ad and offer) can send 1,000 visitors who convert at .01%.

You can guess, but never know, which traffic sources will be superior until you test them. In general, sites (or email lists, radio ads, or any other sources) that are highly relevant to your offer and serve individuals who are proven to be online-offer responders (such as past webinar attendees or multibuyers), are going to give you better performing traffic. Also, in general,

organic search engine links may give you slightly better converting traffic than PPC links (but that doesn’t mean you don’t need both in order to get all the traffic you deserve!).

The key to traffic and testing is that traffic source will

*always* powerfully affect test results. Traffic source is, in fact, often the factor that causes the biggest response variation. Therefore, you must consider traffic source carefully when setting up test cells. If you have traffic coming from several distinct sources, you may need test cells for each source or to extra-carefully distribute equal traffic from each source across all test cells so you control result variations caused by traffic vs. by page changes.

Also, if you are testing a page in order to optimize it before investing in a big traffic push, be aware that the conversion results of the page test and traffic push will vary because the traffic mix will inevitably be somewhat different. That doesn’t mean you should not test before a big push. To the contrary! But never make definitive promises to management concerning future results based on current tests because you will be proven wrong, one way or another. Instead, tell them results will be better due to testing, significantly better in fact... Just don’t detail a precise percentage.

Traffic Source

The place from which your traffic comes. It might be another Website, or a search engine, or an email, or a Twitter mention, or an offline campaign, etc. Traffic behaves very differently depending on source so it’s well worth tracking by source. If you are in a long or complex sales cycle, your official traffic source stats may not be completely helpful because they’ll only tell you where the visitor clicked from, but not what other advertising or messaging motivated that visitor click.

Perhaps the visitor had recently heard about you from a colleague, or seen a glowing press mention, or been aware of your persistent brand advertising elsewhere, or received a postal mailer, etc. The click at that moment may have been motivated by a confluence of factors, the least of which was the particular hotlink or ad they clicked on.

That’s why (when possible) you want to set up tracking to measure the other ways visitors have encountered your brand. Surveys, ad cookies, email tracking links, CRM systems, etc. all play an important role in this.

UI

Acronym for User Interface. This term springs from software programming design before the Web even existed. Programmers wanted to be sure the “interface” (the screen users looked at to use the program)

was easy to use and understand. Although most programmers are terribly intelligent and technically brilliant, the interfaces they design are not always optimum for other human beings to actually use. In fact, they often stink at that.

These days, Web page testing often centers around the UI. Small-seeming UI changes such as making the copy font more readable, forms less intimidating, layout more intuitive, etc. can often cause glorious leaps in conversion rates. My personal favorite – removing the “clear form” or “delete” button from lead generation forms is nearly always a guaranteed winner.

Unique Visitor

Each unique individual who comes to your page or site during a period of time (such as a month). Whether they came to your site 20 times in that month, or simply one time, they should be counted only once

as a “unique.” “Should be” being the operative term because if someone wipes their cookies, or doesn’t accept cookies, or enters your site when they are using a different computer or device, then inevitably they’ll

be counted twice or even more times as a different “unique.”

There was a great deal of fussing about this lack of preciseness in tracking uniques a few years back, especially by media buyers concerned about reach and frequency capping. In the end, everyone recognized that unique vs. non-unique counts will never be perfect and they are instead a fairly reliable indicator rather than an exquisitely-tuned precision instrument.

During site testing, you’ll generally want to cookie uniques to each of your variation pages so that they see the same variation if/when they return during the test run.

Urgency

A key element in whether visitors respond to an offer or not because it tells visitors why they should convert at this time rather than later. Urgency can be created by using deadlines (“Offer ends Friday!”) or by using scarcity (“Only 13 units left!”). The key is people have

to feel that if they leave the site now, without converting, by the time they remember to come back they may have missed out. A certain portion of the population will always want more time to consider a decision. Of these, some may be lifted over that consideration hump with urgency.

Usability Lab

A room in which usability testing is conducted. Sometimes these are very simple – merely a room with a desk or table and a computer. You may have several desks or cubes set up in the same room. You may even have video monitors and/or one-way glass. The nice thing is it doesn’t need to be very fancy or expensive. It must, however, be in person. You learn so much more by watching a lab rat (a human being asked to try out your site) use your site in person than you would from just an analytics report.

The term “usability lab” can also be extended to mean the entire usability test itself. If someone is talking about holding a lab or investing in a lab, they mean the whole test.

Lab tests often give you invaluable clues as to why a site or page isn’t getting the desired results. Labs can be a valuable prequel to testing because you get ideas of what to test from the lab. Labs are qualitative, not quantitative, because you probably won’t ever have more than a handful of people in each lab session.

Consider testing new pages or new designs suggested by lab outcomes to make sure findings hold true for the greater population.

Usability Testing

An important form of qualitative testing to determine if and why typical visitors may have problems actually using your site. In its most basic form, you may do

the “Mom” test (asking your mom to perform a task on your site) to see where confusion and barriers exist that you never noticed because you’re too close to the project. Many sites run usability labs when working on major redesigns (or launches) involving significant navigational and/or functionality changes.

Usability testing is not a replacement for other types of testing, but rather an earlier step in the design and redesign process. Among other things, you won’t gain quantitative evidence around which offer or specific headline copy would lead to the best conversions.

Value Per Lead

How much is each lead (sales lead or email opt-in) generated by the conversion worth to you? Marketers who live and die by the value of their leads often tie their campaign results right into their CRM system, following a lead’s quality over the length of a sales cycle. They also will include lead qualification activities, such as telemarketing and email autoresponders, in the marketing activities focused on leads post-generation.

However, I’ve found that marketers who don’t come from such strict measurement and accountability backgrounds often have no idea what their value per lead is. Do you know what percent of your email opt- ins then convert to being paying customers? And (as such) what the value of their account is? What about for sweeps entrants? Or ecoupon downloaders?

Using the simplest math (not including brand advertising, multiple touches, etc.) take your entire budget, divide it by the number of leads generated, and that is your cost per lead. Now, can you say the value you got was higher or not? If you can’t brag about value, it’s hard to pitch the next budget, test, or campaign to management.

Value Per Sale

The math behind profitability. How much money does each sale make after costs? Every marketer defines “costs” differently. Some include just the cost of the marketing campaign, others include the cost of goods, and still others dig down to the true bottom line including service, sales and management costs. To succeed in office politics and get marketing taken as seriously as possible, talk to the CFO as well as the CEO/President about what they would like to see accounted for in costs.

In addition, you may negotiate what is seen as the value from a sale. How is lifetime value accounted for including potential cross-sales, resales, and upsales? Are testimonials and word of mouth included? The latter rarely are, although they can make a huge difference in corporate profitability.

Value Proposition

This is the specific benefit (or group of benefits) that

a visitor will get if he or she converts on a page.

Marketers sometimes confuse benefits with features

– it’s easy for you to talk about features because that’s a list of facts about your product or service. It’s a bit harder to talk about benefits because benefits are in the eye of the beholder. Market research activities such as focus groups, surveys, persona research, and customer interviews can help you figure out true benefits.

Next, there’s a leap between knowing what your benefits and features are and actually expressing that in a concise, compelling way on your page. Acres of copywriting time and committee meetings have been dedicated to noodling around with value proposition wording. In the end, the best way to pick the right phrasing is often a test. Run an A/B test of your best ideas and see which one has more conversion power.

Although the branding people would rather have one set-in-stone value proposition for your product or service, tests often show that you need to be more elastic (within reason). Different customer and

prospect segments will respond best to different value propositions. And, what people respond best to may evolve over time (especially if you’re in a cutting edge arena).

Some marketers use the term “unique value proposition” instead, which is the same thing as a value proposition except that it takes the competition into account as well. If you are in a highly competitive field (especially one where prospects may evaluate your landing page vis-a-vis your competitors’ during the same research session) your landing page will need a unique value proposition to show off your positioning in the field.

Variable(s)

The elements that you are changing, and then testing. These elements may include (but are not limited to) navigation links, buttons, offer wording, offer type, the overall layout of the page, number of form fields, type size, trust elements, etc. The traffic source may also be a variable, although you generally don’t want to mix a traffic test with a page design test unless you create separate cells for each traffic source. Otherwise results will be inconclusive.

Variable Matching

Terminology used in multivariate testing where you may be testing multiple versions of dozens of variables on

a page. In order to determine conclusive lift results for each of the variables in the singular and as used

together on a particular page (which result in a different, often lower, total lift than would be seen individually), the software must be able to run and supervising statisticians must be able to analyze some complex figures for which I do not envy them.

Variation

Each version of your test page is considered a variation. You can field as many variations as your traffic, testing budget, and head of branding permit. Often tests

start with big variations just to get a feeling for a major direction to go in. Then they settle down into a string of incremental tests, each seeing if small tweaks to elements on the page make a difference.

Web Analytics Audit

Although most marketers have access to Web analytics reports, many don’t use the data routinely to help improve site results. That’s partly because some of

the numbers aren’t strictly useful from a marketing standpoint. You need results reports that help give direction about where the problem areas are (on the site) that need improvement, as well as where the winning areas are that you could build on. Data in-and- of itself may be interesting to review, but not inherently practical to use.

A Web analytics auditor will review all of your data looking for numbers you can act on. He or she will also suggest that you set up certain tracking reports for the future so you have more useful numbers coming from your system. Lastly, he or she may give you advice regarding which Web analytics system or software would be best for your site. Sometimes a free system such as Google Analytics is perfectly good. Sometimes you need a system with more bells and whistles.

Website Usability

Usability means ease of use from the visitor’s perspective. Is your Website intuitively easy to navigate and to use or does it confuse people? It’s next to impossible for a Web designer to see this – they are just too close to the site. That’s why I always suggest

a usability lab and perhaps eye tracking for major launches and redesigns.

If you are interested in website usability, I strongly recommend you buy a copy of Steve Krug’s book, ‘Don’t Make Me Think.’ It’s in trade paperback with lots of useful guidelines and illustrations. It’s highly entertaining not to mention revelatory for anyone who’s

ever designed a site or sat in design approval meetings. If you’re buying a used copy, be sure to get the second edition as it features a new chapter on low-cost/no-cost usability testing.

Website Optimization

Depends on whether you’re a search marketer or not. Search people optimize for crawlers first and people second. Your job as a marketer or site designer is to balance the demands of the crawlers and of the human visitors. A testing firm can help you optimize your site for human beings.

Frustratingly, you generally can’t test your entire Website at once (unless you’re doing a qualitative test such as a usability lab). That’s because people react to pages based on what they saw on the prior page

– if one element of their path changes, their ultimate conversion can be affected. If you change two pages along their path, you won’t know which one made

the ultimate big difference or how the two interacted. Unless you have a tiny site and are running multivariate testing with a master, only test one page at once!

Start with pages that will make the biggest, measureable difference in your bottom line. These would be pages at the end of the conversion such as a cart or lead-generation form, then your most successful pages (invest in success before you invest in the rest!); and then pages at the very start of the funnel such as landing pages and other popular entry pages. Then, move on to pages in the middle with high abandon rates to see if you can improve that as well.

Sponsored by: