Innovation, it all starts with jeopardy. Is this a truism? Does all innovation start with jeopardy? There is little doubt that “the risk of loss, harm or failure,” which is the dictionary definition of jeopardy is a component part of why we innovate. The pace of change across most industries, driven by new technology and more open, collaborative and transparent ways of working, means we are all in potential jeopardy. We all need to innovate.
Now. One of my favourite quotes of the moment on this is “You can’t invest in the future, in the future,” we are making the bets today that will define our future selves, our future organisations and the future clients we serve. In another post for this site, How to Avoid the Iceberg, I explore some of the themes of exponential innovation, which carries great relevance here, after all we may often sense we are in jeopardy only when the time to do anything about it is long passed.
I am going to weave in the IBM Watson story, a very literal game of Jeopardy, to hopefully bring this to life. I think this serves as a great narrative for many aspects of our innovation trials and tribulations. If you are not aware of the history of IBM Watson, here is the quick backstory. In 1997 the IBM supercomputer, Deep Blue, famously defeated the then world chess champion, Garry Kasparov. In 2004, researchers chose Jeopardy! as its next conquest and began developing Watson in 2005. Watson was created as a question answering (QA) computing system that IBM built to apply advanced natural language processing (NLP), information retrieval, knowledge representation, automated reasoning, and machine learning technologies to the field of open domain question answering.
The goal was to build an AI supercomputer capable of defeating the two all-time Jeopardy! champions, a feat IBM managed to achieve in 2011 when their AI ‘machine’ the Watson computer system competed on Jeopardy! against former winners Brad Rutter and Ken Jennings, winning the first prize of $1 million.
Observation 1: Our innovation efforts can be great for PR
If we are honest, this is where many of our innovation efforts start and stop. We invest in the shiny stuff, great to showcase to new graduates, shareholders or customers. It might look great, but what value does it derive. Really? This was great PR for IBM, but so what?
The third era of computing?
The mid-1950s, the first era of computing, the tabulating era, brought forth the calculator which could perform basic arithmetic. The 1990’s gave rise to the programmatic era, where machines can process logical structures and “if/then” commands (such as “if not A then B”), but computer scientists must program the rules the computer follows. Watson straddles cognitive computing and programmable computing, harnessing and processing the more than 2.5 billion gigabytes of data generated each day. Its machine-learning capabilities simulate the human thought process, but is built to eliminate the biases and error out of decision-making. “The third era of computing, the cognitive era, starts to breakdown the rigidity inherent in that ‘if/then logic.” Cognitive computing is probabilistic computing, where the outcomes vary along a spectrum instead of being strictly “yes or no, right or wrong.” Sounds great, right? But still, so what?
“We were mainly interested in using Jeopardy! as a playing field upon which we could do some science,” Dr. Chris Welty later said about Watson’s appearance on the game show. “We wanted the ability to use questions that had not been designed for a computer to answer.”
Immediately after collecting the $1 million prize in 2011, IBM set off to apply Watson in real-world scenarios, but to date nothing has really been achieved of any material value.
Observation 2: From minimum viable product (MVP) to scaled product/service delivery is hard, even if you are IBM. Focus is often vital.
Critics say IBM executives overshot badly by allowing marketing messages to suggest that Watson’s Jeopardy! breakthrough meant it could break through on just about anything else. IBM executives are certainly guilty of setting up Watson for failure by throwing the net so wide in terms of application and focus whilst it was still in its infancy. Anyone who has built a first-generation prototype of anything will know the endless pitfalls of trying to rapidly scale off the back of your rudimentary prototype.
Anyone who has any experience of modern AI, NLP, ML applications will know that they are still a long way from being generally useful and scalable. We have made huge gains, but making sense of language remains one of the biggest challenges in artificial intelligence. Even though IBM must have been aware of the many and varied limitations, they quickly turned Watson into an umbrella brand promising a bewildering variety of bold new applications, from understanding the emotional tone of Tweets to scouring genomes for mutations. However, having dispatched Watson to many different fields, a standard set of challenges kept arising, and continue to arise. To get Watson from Jeopardy to oncology for example, there were three processes that the Watson team went through: content adaptation (getting the content to feed to the AI), training adaptation (testing and teaching the AI with questions), and functional adaptation (making any technical adjustments needed by tweaking taxonomies). The problem is that in many areas, typically the ones we need it the most, the data simply doesn’t exist in the right format, or in any form at all, or the data may be scattered throughout dozens of different systems, and difficult to work with. This is perhaps the biggest barrier to the true scaling and applicability of AI generally, whether Watson, Google’s DeepMind, Microsoft, Amazon or the multitude of other AIs entering the market.
Observation 3: It takes time. Progress is not linear. Failure is par for the course.
IBM have not seen the successes they would have liked following the hype of winning Jeopardy! They have invested heavily in this, buying several start-ups to buy in capability, competency and enhanced technology to allow Watson to scale. However, their biggest news story to-date is arguably the $60m failed, high-profile project, Watson Oncology.
A $60m failure (or the price to pay for progress)?
Immediately after winning Jeopardy! in 2011, Watson announced its first real-world applications would be in healthcare, as IBM began working with several companies and universities to develop Watson technologies that assist physicians and medical practitioners. In February 2013 at Memorial Sloan-Kettering Cancer Center, IBM and healthcare provider WellPoint commercialized Watson Oncology, a cognitive computing system that supports physicians as they make decisions on treatment. While Watson Oncology specializes in assisting with cancer diagnosis and treatment, Watson has been integrated into broader clinical decision support applications that can identify a wide range of illnesses.
In February 2017, a heavily promoted partnership between IBM and the MD Anderson cancer centre ended in failure, with the centre walking away from more than $62 million and four years spent on contracts promising a Watson system to help oncologists treat patients. The goal had been for Watson to read data about patient’s symptoms, gene sequence, and pathology reports, combine it with physicians’ notes on the patient and relevant journal articles, and then help doctors come up with diagnoses and treatments. But IBM and M.D. Anderson both overinflated expectations for the technology. IBM claimed in 2013 that “a new era of computing has emerged,” Forbes reported that Watson “now tackles clinical trials” and would be in use with patients in just a matter of months. In 2015, the Washington Post quoted an IBM Watson manager describing how Watson was busy establishing a “collective intelligence model between machine and man,” and the Post said that the computer system was “training alongside doctors to do what they can’t.” The reality is that after four years it had not produced a tool for use with patients that was ready to go beyond pilot tests and had overrun on the initial budget by over 20x.
And to conclude….
The challenge facing IBM is it has bet a lot on Watson and it needs the bet to start paying off. In July 2017, IBM’s quarterly results showed a 21st consecutive quarterly decline in revenues. CEO Ginny Rommety has made a habit of talking about Watson as a kind of saviour, and the company declared this week that this part of the business is growing, but it is far from clear that it is growing in a way to support the future of IBM. In fact, given overall revenues are in decline, we are safe to assume that Watson is still not a significant revenue contributor overall for IBM. Watson risks being overtaken by AIs from Google, Amazon, Microsoft and/or a whole host of high-profile start-ups. Having created the hype, IBM is now facing some heavy-weight competition. The pressure, and indeed the race, is well and truly on.
But we should celebrate Watson and applaud IBM. We should not all hail the mighty Watson as the saviour of the world, nor even the saviour of IBM….at least not yet. But as innovators we all know the difficulty of getting our organisations to take risks, give us time, investment and trust. Time will tell how this plays out, but the false expectations aside, set mostly by IBM’s over-zealous leaders and PR team, there is much to be optimistic about and we will all benefit in time from this innovation. Watch this space…..
