“There are not enough data scientists in IBM Watson who know medicines,” one pharma executive, who asked to remain anonymous, told me recently. 

Yet the vision of the two working together remains powerful. In one scenario, an oncologist taps some information into an iPad, and within 30 seconds IBM’s Watson crunches the data and spits out a drug regimen tailored to the patient.

That dream, which seemed to be advancing steadily toward reality the last few years, took a step backwards with the February revelation that a partnership to enlist the IBM supercomputer’s artificial intelligence (AI) to offer care advice and match patients with clinical trials was put on hold by MD Anderson, as of late last year.

See also: How Watson for Oncology is advancing cancer care

The partnership, according to an audit, cost the hospital $62.1 million. Why it eroded isn’t entirely clear. Reports suggested a variety of reasons, from a failure to deliver, to bureaucratic misconduct, or to some combination of the two.

Yet IBM came to Watson’s defense, telling Forbes that the tool, known as Oncology Expert Advisor, gave recommendations that agreed with experts 90% of the time and “could have been deployed had MD Anderson chosen to take it forward.”

For their part, the University of Texas auditors stopped short of opining on “the scientific basis or functional capabilities of the system.”

But the incident, coming on the heels of what MM&M called a “backlash” by “Watson doubters” at last month’s SXSW Health, could portend more scrutiny. Investors pressed Deborah DiSanzo, IBM Watson Health’s general manager, for details on things like revenue, data stewardship, and data security after her talk at the JP Morgan Healthcare conference in January. None of their questions, reportedly, were answered in detail.

See also: Day 3 at JP Morgan: The move toward big data requires drugmakers to ask questions

One can see why investor patience may be waning. In Watson Health’s first 12 months, it had notched deals with such biopharma and medtech giants as Johnson & Johnson, Medtronic, Teva, and Novo Nordisk.  

But now, as the business unit turns 2 this month, and as pundits try to piece together what went wrong with the MD Anderson effort, a moment of reckoning in pharma may be at hand.

At JP Morgan, DiSanzo touted the unit’s 7,000 employees, including 1,000 data scientists. But do they know medicines? No, the above drug marketer told me, and for this reason, “We’ve had quite a few experiments fail with them.”

And what Watson doesn’t know about medicine could undermine these partnerships. To do a project properly with Watson requires embedding with the Watson team for about six months, my source explained, adding, “We’re not ready to do that as an organization.”

“Watson Health is still a really young company that doesn’t understand how to work with pharma,” the executive concluded.

See also: Challenges exist, even as excitement grows for big data market

Companies may find they need to partner on informatics capabilities while also building their own home-grown data-science teams.

Another lesson for industry: don’t be intimidated by vendors. “I always encourage people to ask ‘how does it work,’ and to keep asking that question until you understand it. And if the person presenting [the product] to you cannot explain it, don’t buy that product,” advised Hilary Mason, a well-known big data expert, at an AI-related event last year.

Despite the high-profile MD Anderson imbroglio, biopharma firms should not be deterred from moving further into data science. Proceed they must — since anything innovation-related will have a data component — but proceed with caution.





Marc Iskowitz is editor in chief of MM&M.