When an advertisement is placed in a professional journal,
marketers sometimes wonder if anyone will see it. If it is great, but no one
notices, what's the point? Will it break through the clutter of the other ads?
Will the signal cut through the noise?
This article attempts to offer a clear, standardized way to
tell. The Theory of Signal Detectability (TSD) suggests using the
“signal-to-noise” ratio to determine True Recognition, which is the percent who
say they have seen an ad that really have seen it, minus the percent who say
they have seen it who really have not. The measure takes into account both
accurate and false recognition: Hits and False Alarms.
There are also other elements of print ads worth knowing.
What does the advertisement communicate? Is its message extremely important? Is
it believable? New and different? Is it persuasive? What tone or image does the
Most useful would be to have a Norms base to set results
against, to be able to determine if an ad is above, below, or at norm with
respect to True Recognition and measures of these other elements.
To address these questions, and to test the practicality of
the Signal Strength technique, a study was conducted with 31 primary care physicians
(PCPs—defined as family/general practitioners and internists). The objectives
1. Measure True Recognition for four print journal test ads.
2. Obtain diagnostic rating and adjective checklist
information for each.
3. Begin developing TSD Norms by including three “control”
This method is intended to help discriminate between
different advertisements and to help make an informed decision about the uses
to which a given ad will be put.
The 31 PCPs were split into two groups, designated the Hit
group (16 doctors) and the False Alarm group (15).
Each respondent viewed online two lists of 30 different
journal ads. The lists were carefully constructed to allow for signal strength
measurement. Ads were always presented in random order.
In the Hits group, the second list showed 22 items not seen
before, and seven of the items that were repeats from the first list.
Four of the seven ads were test ads; the other three ads
were used as controls, to be included in future research so that test-retest
reliability could be established.
In the False Alarm group, the same seven ads were in the
second list. In fact, that list was exactly the same as the one the Hits group
saw. But the False Alarm group had not previously seen any of these seven ads.
When viewing the second list, for each ad, all doctors were
asked if they saw it in the first list or not (yes/no) and how certain they
were of the answer (very, somewhat, not at all).
Following this task, the doctors were asked specific
questions about each of the four test ads: They were asked to rate each on how
important, believable, new and different, and persuasive the information
Additionally, doctors were asked to check any of a set of 10
adjectives which might apply to each test advertisement. Finally, they were
asked about the main message of one test advertisement. Interviewing occurred
between April 2-4, 2007.
Findings: Signal Strength, True Recognition
The True Recognition percentages for the seven tested print
ads, derived from the Hits and False Alarms, are shown in Fig. 1. The Norm is
the simple average.
Surprisingly, given the low base size (N = 15), one of the
ads differs from the Norm. The Tylenol ad was shown to have a very strong
Signal, and is significantly (95% Confidence Level) above Norm.
The seven ads attained their Signal scores in different
ways. Specifically, the high scoring ads had many Hits and no False Alarms.
This suggests that they are distinctive, since no PCP said they had seen them
when they had not.
In contrast, the low scoring ads had a unusally high
percentage of False Alarms, suggesting they look like many other
advertisements—a high percentage of physicians who had not seen them before
said that they, in fact, had. For instance, the high False Alarms are the main
reason the Centocor ads and Crestor ads are not truly being recognized.
When the doctors were asked how sure they were about the
Hits, the degree of certainty roughly paralleled the percent of hits overall
(Fig. 2). They were most certain about Tylenol (the strongest Signal) and least
certain about Crestor (one of the weakest Signals).
In looking at the certainty around False Alarms (Fig. 3),
two things are evident. One is that there is a lot less certainty. This makes
sense, since these are errors—claims of seeing something not seen. The other
observation is that the level of certainty rises slightly with the level of
error, suggesting that those ads that look most like other ads truly confused
These PCPs rated each of the four test ads on four
attributes: Important, Believable, New and Different, Persuasive (Fig. 4).
- Importance—Zyvox is considered by far and away the most important ad. The
AndroGel and Centocor ads were least important. Zyvox is above the Norm, while
the two lowest ads are below it.
- Believability—Zyvox and Zocor are both believable. Centocor, while not
unbelievable, is relatively low on this measure. None of the ads vary from the Norm.
- New and
Different—The AndroGel ad is definitely the most new and different of the ads.
Zocor is least, with Zyvox and Centocor falling in the middle.
- Persuasive—The Zyvox ad is the most persuasive in the group, and the Centocor
ad is considered to be relatively unpersuasive. The other two fall in the
The PCPs were also given the opportunity to check off
adjectives that they thought applied to each of the Test ads.
The Zocor ad was the least involving and the most boring and
ordinary. The Zyvox ad was the most involving and least boring of this set of
four. The Centocor ad was most unique, being for enrollment in a clinical
trial; the whimsical AndroGel ad was “nicest.”
Since knowing the main takeaway is important when examining
any ad, PCPs were asked about the main message for one of the four Test
ads—Centocor. The ad shows a man drawing in the sand on a beach (what he is
drawing is unclear), and requests that doctors call an 800 number to learn of
investigator sites nearby. About half of these PCPs got the message. About
three in 10 did not know what the ad's message was at all.
Summary and Discussion
Tylenol control ad, with a signal strength of 81%, is significantly above the
norm of 57% (at a 95% Confidence Level). The other ads examined for signal
strength are at norm, ranging from a high of 69% (AndroGel) to a low of 36%
- Diagnostically, the ads with higher signal strengths had fewer False Alarms,
suggesting that they are more distinctive.
concerning Hits generally paralleled the percent of Hits overall. Certainty
concerning the False Alarms was much lower, and became slightly higher as False
Alarms increased, suggesting that the ads with the highest False Alarms create
a false sense of confidence that they had been seen before.
- The test
ad for Zyvox was the most Important, Believable, and Persuasive, likely because
it is for a product that treats a serious condition. It was moderately New and
- The test
ad for Centocor was the least Important, Believable, and Persuasive. It is to
recruit to a Phase II clinical trial, which may
explain the reaction. It is relatively low on “New and Different”, but not as
low as Zocor on that measure.
- The Zocor
ad was the least involving and the most boring and ordinary. The Zyvox ad was
the most involving and least boring of this set of four. The Centocor ad was
most unique. About half were able to play back the main message of the Centocor
ad. The whimsical AndroGel ad was the “nicest.”
Strengths and Limitations
The Signal strength technique presents an opportunity to
determine if a professional journal advertisement truly stands out or not, and
provides some diagnostic information to help determine why.
It is new, so the normative data are limited currently, but
time should resolve that issue. The most limiting aspect is that, for obvious
reasons, only one journal advertisement for the same product can be tested in
Therefore, to compare two or even three potential journal
ads, two or three tests would be needed. The saving grace for this limitation
is that, based on this test, a small base size does discriminate, so each test
can be cost-effective.
Stephen J. Hellebusch is president, Hellebusch Research
& Consulting, based in Cincinnati, OH