Here’s how often IBM’s Watson agrees with doctors on the best way to treat cancer

Hollis Johnson

We’re starting to get a better picture of how artificial intelligence could help doctors better treat cancer.

And in data presented at the American Society of Clinical Oncology meeting, IBM Watson Health gave a snapshot of how it’s playing out so far.

The studies looked at concordance rates, or how often Watson for Oncology reached the same course of treatment as the cancer doctors at different cancer centers around the world. At Manipal Comprehensive Cancer Center in India, for 112 cases of lung cancer, there was 96.4% concordance between Watson and the doctors. For 126 cases of colon cancer it was 81% of the time, and for 124 cases of rectal cancer cases were 92.7%.

The concordance was in line with what IBM expected in those cases: If Watson and the docs agreed all the time, there wouldn’t be much value for adding AI to the picture.

But it was a bit off when it came to Watson evaluating 185 cases of gastric cancer in South Korea. There, the concordance was 49%. Norden said that relates to the guidelines for gastric cancer being different in South Korea than at Memorial Sloan Kettering, the hospital where Watson for Oncology was trained.

The data presented at ASCO are among the first that IBM Watson Health has presented at a medical conference. And while it sets the stage for what artificial intelligence can do to help doctors treat patients with cancer, many still have lingering questions.

The biggest yet-to-be-answered question: Can using AI to determine cancer treatment actually extend patients’ lives compared to oncologists alone determining their treatments?

Andrew Norden, the deputy chief health officer at IBM Watson Health told Business Insider that the concordance data isn’t “the ultimate endpoint we’re interested in,” though it was the first they could get to relatively quickly.

To get to studies that evaluate overall survival (that is, finding out whether using AI-powered treatment plans can increase patients’ lives compared to traditional treatment plans) will take more time.

Get the latest IBM stock price here.