13 Platinum Ideas for Testing

by Ray Jutkins

Testing is the "middle-name" of Direct Marketing.

Yet, countless programs are launched every year with no thought given to "learning". Learning what really works ... what really gains the largest number of responses, the most qualified leads, the most sales.

To go to market without a test is to leave money on the table. If not today, tomorrow.

The first of 13 ideas is . . .

Idea #1. Testing as a Concept

Testing is not an exact science. It doesn't need to be.

So, why test? Good question!

You test to learn about your marketplace. Before the tactics of any program fall into place, you must have a strategy. A plan of action. Which includes a plan for testing. A "concept" about what you need to learn.

Think. Plan. Organize. FIRST! Don't guess -- plan your test.

Idea #2. Research and Testing are NOT the same

What is research?

Research establishes pre or post buyers motivation for buying, their intention to buy, or not to buy.

Research techniques used most frequently are 3:

►	interviews
►	surveys
►	focus groups

Interviews and surveys can be face-to-face, over the phone, through the mail or fax. Or E-mail. Focus groups are just that ... a group of people brought together to "focus" on a topic or subject, and give their opinion.

Okay, what is testing? Testing determines if buyers behave as they say they will ... as they "intend".

Testing teaches you customer behavior. Not just what the prospect or customer says -- instead what they really do.

You test because you never really know what the marketplace will do, what might work ... or what does NOT work! You test to learn how they, your buyers, DO perform.

Testing is "live". So, ignore your instincts ... trust your customer, trust the marketplace, trust what really happens.

Idea #3. Why Test?

You test because no matter how well you plan, everything takes longer than it takes.

You test because no matter how well you plan, everything costs more than it costs.

You test because no matter what happened before, results this time will be different.

These "Murphy Laws" apply in marketing and sales. Testing will help you beat the odds and be a winner. Here are 8 things testing will help you do:

►	Design your total campaign and every project within it
►	Choose between different copy themes and graphic ideas
►	Zero in on your the prospects most likely to buy
►	Decide which segment is your greatest market potential
►	Decide which offer is your best offer
►	Determine the timing of your offer, by time of the year
►	Decide if you should expand geographically
►	Avoid unnecessary risks with your campaign

Idea #4. When do you NOT Test?

In theory, the answer is NEVER! You should always test.

Yet, in real life this is not always so.

If your numbers are too small you cannot test. If your audience is ONLY the presidents/owners of the 4,362 bus lines in the USA, you do not test. You simply DO whatever it is you do. And if it does not work, you do something else. The numbers are too small for a meaningful test.

And you do not test when the timing is wrong. Maybe you got a late start. Maybe the event is such the timing itself prevents a true test. For instance, Valentine's Day or Mother's Day. You either DO an event for those special occasions, or you do not. There is no time to test.

You do not want to test when you have unmatched or wrong elements. Meaning, for example, it is unwise to test ONLY a direct mail white standard business envelope with no teaser copy, vs. a jumbo 4-color envelope with windows front and back, and teaser all around. What could you possibly learn?

And you should not test if you have a pre-disposed position on what the results should be. i.e., you are looking for something specific, rather than learning what is happening.

Take your bias out -- make testing a part of your marketing program.

Idea #5. How to Test

The best way to test is to "think" a matrix.

You may prefer a pyramid style graph -- or a box chart. Really doesn't matter -- what matters is you determine before you begin what you need to learn. And plan your program to learn.

And, you must code. Why? So you have something to measure. Phone or fax numbers, special E-mail address, POBox, a persons name, a product serial number -- are all ways to code. Code every element in a direct mail package, several places in a space add, differently for broadcast. Code every time -- and code differently each time.

Also, do not expect 100% measurable results. It will NOT happen!

Idea #6. What Do You Test?

First, test 1 thing at a time ... OR test everything.

Next, look for a BIG GAIN, a true break-through result. Minutes or degrees may be fine once you have a sound, almost unbeatable control. As you begin testing reach for the sky -- settle for the moon.

Here are just a few of the things you can test:

►	Copy; style, approach, length
►	Graphics; format, layout, design, type, colors
►	Size, paper, photography vs. art
►	Audience; lists, segments within lists, activity of it,
►	Recency & frequency, age-groups, sex
►	Offers; such as advertising specialties & premiums
►	A limited time or limited number, or both
►	Sweeps, contests and other games
►	Product; packaging, name, and bundle with other services
►	Guarantee & Warranty; full or limited
►	Brochures & literature; lift letter, technical piece
►	Testimonial or case history flyer, Q & A
►	Business codes, company stats, such as size, location,
►	Financial info, job function and more
►	Media; direct mail, broadcast, print, electronic media
►	Telecommunications tools, and a mix of any/all.

Please remember, there are no failures ... only lessons. Test thoroughly.

Idea #7. When Do You Test?

The answer to this question is ALWAYS test!

As any time can be the right time. Sure, in your industry with your marketplace some times may be better than others. Yet, every time you go out the door you should be testing something. As you will learn something.

Test early -- test often.

Idea #8. What Do You Measure?

The answer is simple: Significant Things!

Significant things that aim you toward your objectives. To winning. To reaching your goals.

Things like . . .

►	Your total response
►	Response by various media
►	Response by phone, fax, e-mail, mail
►	Qualified response, and by media
►	Total sales, and by media
►	Cost per lead, and by media
►	Cost per sale, and by media
►	What product or service was bought
►	What is the average order / median order
►	How is payment made, check, credit card, PO, cash
►	How many new customers do you get
►	Which market segments offer greatest potential
►	How many contacts to make the first sale
►	And of course -- the most significant: L.T.V. (Life Time Value of the customer) That is, what will this customer bring to the table over the life of your business together.

Idea #9. Why Measure Your Test?

This is quick and easy: MONEY!

And a single 4-letter word we can all scream -- SOLD. For without a sale nothing happens. Of course, with a sale everything really begins to happen. Relationships are important long before anyone buys anything. Yet, it is after the sale that the promises and fulfillment meet.

Bottom line; did we make a sale ... did we make any money. That is "why" you measure.

Idea #10. How to Measure

There are formulas for measuring results.

Some are so easy to administer you do it with a large yellow tablet and a pencil. Others require a mini-computer. For most of us a hand-held calculator and some paper will do.

Rather than review specific formulas, let's talk about the purpose: The purpose of measuring your test is to learn what happened in the marketplace. Period!

Most often that involves numbers. And money. Know what your objectives are, what you are trying to learn -- and then decide how to measure.

Idea #11. What NOT to Measure

Bet you can guess what I'm going to say; Insignificant things!

Such as the day of the week the mail is delivered. You can't control this, so why measure?

Now, you CAN measure the day of the week an ad is in the newspaper, on radio or television. The day you change your offer on your WWW site. And in those instances it MAY be significant.

You will find measuring the "tilt" of your postage stamp on your direct mail a complete waste of test time. And the shade of blue is not nearly as important as color testing red vs. blue vs. green.

Price is an important element to test -- so make it significant. $19.95 vs. $29.95 says something. $99. vs. $139. is a test. Testing $19.99 vs. $19.88 will teach you nothing ... except how to leave 11 cents on the table.

Idea #12. When is a Test NOT a Test?

When you don't have enough results to draw a meaningful conclusion.

When your results tell you nothing significant.

And when you do not measure and evaluate your test.

Any of us can do "Mother-in-Law" research. Talk to our friends, neighbors, family and associates. And come up with any opinion we choose.

That is not the purpose of testing. Testing is to teach what the marketplace actually does. To help us learn how to market to the marketplace. Think. Plan. Organize. First! And then do it right.

Idea #13. Hindsight IS an exact science

Some would argue everyone has 20/20 hindsight. And with testing that is what you get.

You "see" what the marketplace has actually done. Live. In response to a marketing and sales proposal. You learn what you should do next time to accent the positive, downplay the negative.

Testing allows you to make informed, intelligent and profitable decisions.

Hindsight helps you evaluate your test 4 ways:

1. You learn What Works! In any test you are comparing "A" to "B". When you measure your test results you learn What Works! Instead of guessing you now know.
2. What Does NOT Work is equally important. So you do not repeat the mistake of what is not working. Testing helps you learn what is not working.
3. Why? Why what worked did work ... and why what did not work did not work. By knowing "Why" you can do more of the right thing and less of the wrong. Testing helps you learn "Why" customers do what they do -- or do not do.
4. What are you going to do with all this knowledge next time? Think. Plan. Organize. FIRST. From Idea #1. comes this same phrase. Equally applicable. Because to gain full benefit from testing you must PLAN to use what you learn.

-----------------------------------------------------------------------------------------------------------------------------------
About Ray Jutkins, October 3rd, 1936 — January 6th, 2005. Ray was one of the NMOA’s most generous contributors. Over the years Ray supplied the NMOA with hundreds of tips and articles for members. This is just one of many. Ray worked with B-2-B and Consumer clients throughout the world ... including USA, Canada, Mexico, Asia, the South Pacific, Europe, the Middle-East, Central & South America, Africa. Keep an eye out for more of Ray’s marketing tips and how-to articles in the pages of Direct Marketing Digest and the article archive on the NMOA website.