Quick and dirty polls need vetting, big research firms say
Online polls are increasingly playing foil to traditional marketing research. Just ask Daniel Palestrant, MD.
One recent morning the FDA issued warnings on a popular class of medicines. The CEO and founder of an online community for physicians recalled the immediacy with which members formed an opinion of the event:
“By 6 or 7 o'clock, a Sermo doc posted a warning into the community,” Palestrant said. “By 11 [am], 200 people had voted. “The results showed the physician community didn't think there was much merit to the warning.”
Sermo is the name of his Web site, and the FDA notice, which came last March, had to do with insomnia medications. But substitute TZDs or any class of drugs in the news lately, and it's probably being discussed, debated and voted on.
Pharmaceutical marketers, physicians and financial institutions have long sought ways to gauge the impact that new information will have on practice, or to forecast uses for new therapies. Informal polls are increasingly being used to answer these questions—should Merck's Januvia be a first-line treatment for type 2 diabetes? Twenty-nine percent of Sermo docs (12/41) said it should remain second-line; 22% (9/41) said it will eventually become first-line. But their lack of statistical rigor raises questions as to how much stock should be placed in them.
“We don't in any way, shape or form argue or portray that this research is a replacement for conventional scientific inquiry,” Palestrant said, referring to the scientific gold standard of double-blind, placebo-controlled trials.
That shouldn't dampen their clout in providing an instant read, though. “Right now the norm is to go out and sample physicians and survey them for days for a couple dozen answers. Now clients [post a question] and get an answer in real time,” he said. “The ability to do that faster, better, cheaper is a whole new approach.”
Clients who pay a subscription fee can ask questions and see results after a short delay. Members, on the other hand, must wait two weeks for polls to end. (Sermo said results stay hidden to prevent voting bias.)
Traditional research firms flagged some cautions.
“It whets peoples' appetites,” Karen Fender said of short-form online survey results. The SVP and general manager of pharmaceutical marketing research firm TVG said such polls are also a good way to retain panels, as physicians are more likely to participate if they gain insight into what their peers are thinking or doing.
However, she is concerned about whether the balloting contains a representative sample. “A poll is a poll, but it needs to be conducted with a representative sample to be projectable,” Fender said. “If it's an informal poll on a Web site, you can't project that sample to the rest of the universe of physicians. You don't know what the makeup of their panel is and haven't screened the respondents.”
Richard Vanderveer, PhD, CEO, GfK V2, agreed that online panel composition is often specious. “I would just need to know the demography of that group,” he said. “My guesstimate, based on working with physicians for the last few years around ‘e,' is it's a funny kind of doctor who uses his or her time to play with that.”
Robert Stern, CEO of medical news site MedPage Today, said he is aware of the methodological shortcomings. Nevertheless, “Many of our customers value our surveys as a kind of ‘heads-up update' that allows them to anticipate new trends in physician attitudes and practices,” he said. “Formal research is then used to validate these changes based on a large, statistically reliable sample.”
Stern said MedPage Today is increasingly eliciting reader reaction as a way to better focus content. Example: 55% of healthcare professionals in a one-week poll of 598 participants said they will no longer prescribe Avandia (rosiglitazone), the GlaxoSmithKline drug, for type 2 diabetes, and nearly one in four said the drug should be withdrawn from the market.
But clients need to view non-scientific studies in the proper light, said Vanderveer. He refuted the notion that they will use them as a mere jumping-off point to further study. “My concern is, after 35 years in pharma, that the client out there is prone to go off half-cocked. The initial results will prompt decision-making and not cause clients to take the second step of collecting more rigorous data.”
Although cyber surveys lack statistical rigor, that seems unlikely to dampen enthusiasm for them. As long as the pharmaceutical news keeps flowing, stakeholders will want quick takes on customer sentiment. News events may have an immediate impact on diagnosis and treatment, and polls often prove that where there's smoke, there's fire.
“We are striving to take our readers' temperature every chance we get,” Stern said.