When an advertisement is placed in a professional journal, marketers sometimes wonder if anyone will see it. If it is great, but no one notices, what's the point? Will it break through the clutter of the other ads? Will the signal cut through the noise?

This article attempts to offer a clear, standardized way to tell. The Theory of Signal Detectability (TSD) suggests using the “signal-to-noise” ratio to determine True Recognition, which is the percent who say they have seen an ad that really have seen it, minus the percent who say they have seen it who really have not. The measure takes into account both accurate and false recognition: Hits and False Alarms.

There are also other elements of print ads worth knowing. What does the advertisement communicate? Is its message extremely important? Is it believable? New and different? Is it persuasive? What tone or image does the advertisement convey?

Most useful would be to have a Norms base to set results against, to be able to determine if an ad is above, below, or at norm with respect to True Recognition and measures of these other elements.

To address these questions, and to test the practicality of the Signal Strength technique, a study was conducted with 31 primary care physicians (PCPs—defined as family/general practitioners and internists). The objectives were to:

1. Measure True Recognition for four print journal test ads.

2. Obtain diagnostic rating and adjective checklist information for each.

3. Begin developing TSD Norms by including three “control” ads.

This method is intended to help discriminate between different advertisements and to help make an informed decision about the uses to which a given ad will be put.

The 31 PCPs were split into two groups, designated the Hit group (16 doctors) and the False Alarm group (15).

Each respondent viewed online two lists of 30 different journal ads. The lists were carefully constructed to allow for signal strength measurement. Ads were always presented in random order.

In the Hits group, the second list showed 22 items not seen before, and seven of the items that were repeats from the first list.

Four of the seven ads were test ads; the other three ads were used as controls, to be included in future research so that test-retest reliability could be established.

In the False Alarm group, the same seven ads were in the second list. In fact, that list was exactly the same as the one the Hits group saw. But the False Alarm group had not previously seen any of these seven ads.

When viewing the second list, for each ad, all doctors were asked if they saw it in the first list or not (yes/no) and how certain they were of the answer (very, somewhat, not at all).

Following this task, the doctors were asked specific questions about each of the four test ads: They were asked to rate each on how important, believable, new and different, and persuasive the information presented was.

Additionally, doctors were asked to check any of a set of 10 adjectives which might apply to each test advertisement. Finally, they were asked about the main message of one test advertisement. Interviewing occurred between April 2-4, 2007.

Findings: Signal Strength, True Recognition

The True Recognition percentages for the seven tested print ads, derived from the Hits and False Alarms, are shown in Fig. 1. The Norm is the simple average.

Surprisingly, given the low base size (N = 15), one of the ads differs from the Norm. The Tylenol ad was shown to have a very strong Signal, and is significantly (95% Confidence Level) above Norm.

The seven ads attained their Signal scores in different ways. Specifically, the high scoring ads had many Hits and no False Alarms. This suggests that they are distinctive, since no PCP said they had seen them when they had not.

In contrast, the low scoring ads had a unusally high percentage of False Alarms, suggesting they look like many other advertisements—a high percentage of physicians who had not seen them before said that they, in fact, had. For instance, the high False Alarms are the main reason the Centocor ads and Crestor ads are not truly being recognized.

Certainty

When the doctors were asked how sure they were about the Hits, the degree of certainty roughly paralleled the percent of hits overall (Fig. 2). They were most certain about Tylenol (the strongest Signal) and least certain about Crestor (one of the weakest Signals).

In looking at the certainty around False Alarms (Fig. 3), two things are evident. One is that there is a lot less certainty. This makes sense, since these are errors—claims of seeing something not seen. The other observation is that the level of certainty rises slightly with the level of error, suggesting that those ads that look most like other ads truly confused some PCPs.

Attribute Ratings

These PCPs rated each of the four test ads on four attributes: Important, Believable, New and Different, Persuasive (Fig. 4).

  • Importance—Zyvox is considered by far and away the most important ad. The AndroGel and Centocor ads were least important. Zyvox is above the Norm, while the two lowest ads are below it.
  • Believability—Zyvox and Zocor are both believable. Centocor, while not unbelievable, is relatively low on this measure. None of the ads vary from the Norm.
  • New and Different—The AndroGel ad is definitely the most new and different of the ads. Zocor is least, with Zyvox and Centocor falling in the middle.
  • Persuasive—The Zyvox ad is the most persuasive in the group, and the Centocor ad is considered to be relatively unpersuasive. The other two fall in the middle.

Adjective Checklist

The PCPs were also given the opportunity to check off adjectives that they thought applied to each of the Test ads.

The Zocor ad was the least involving and the most boring and ordinary. The Zyvox ad was the most involving and least boring of this set of four. The Centocor ad was most unique, being for enrollment in a clinical trial; the whimsical AndroGel ad was “nicest.”

Main Message

Since knowing the main takeaway is important when examining any ad, PCPs were asked about the main message for one of the four Test ads—Centocor. The ad shows a man drawing in the sand on a beach (what he is drawing is unclear), and requests that doctors call an 800 number to learn of investigator sites nearby. About half of these PCPs got the message. About three in 10 did not know what the ad's message was at all.

Summary and Discussion

  • The Tylenol control ad, with a signal strength of 81%, is significantly above the norm of 57% (at a 95% Confidence Level). The other ads examined for signal strength are at norm, ranging from a high of 69% (AndroGel) to a low of 36% (Crestor).
  • Diagnostically, the ads with higher signal strengths had fewer False Alarms, suggesting that they are more distinctive.
  • Certainty concerning Hits generally paralleled the percent of Hits overall. Certainty concerning the False Alarms was much lower, and became slightly higher as False Alarms increased, suggesting that the ads with the highest False Alarms create a false sense of confidence that they had been seen before.
  • The test ad for Zyvox was the most Important, Believable, and Persuasive, likely because it is for a product that treats a serious condition. It was moderately New and Different.
  • The test ad for Centocor was the least Important, Believable, and Persuasive. It is to recruit to a Phase II clinical trial, which             may explain the reaction. It is relatively low on “New and Different”, but not as low as Zocor on that measure.
  • The Zocor ad was the least involving and the most boring and ordinary. The Zyvox ad was the most involving and least boring of this set of four. The Centocor ad was most unique. About half were able to play back the main message of the Centocor ad. The whimsical AndroGel ad was the “nicest.”

Strengths and Limitations

The Signal strength technique presents an opportunity to determine if a professional journal advertisement truly stands out or not, and provides some diagnostic information to help determine why.

It is new, so the normative data are limited currently, but time should resolve that issue. The most limiting aspect is that, for obvious reasons, only one journal advertisement for the same product can be tested in one test.

Therefore, to compare two or even three potential journal ads, two or three tests would be needed. The saving grace for this limitation is that, based on this test, a small base size does discriminate, so each test can be cost-effective. 

Stephen J. Hellebusch is president, Hellebusch Research & Consulting, based in Cincinnati, OH