The Science And Technology Behind IBM Watson (Part 2 of 5 Series)

Series: <<1 >><<2>><<3>><<4 >> <<5 >>

Follow IBM Watson -

DeepQA is the underlying architecture that’s powering Watson. Actually DeepQA is software architecture for deep content analysis and evidence-based reasoning that embodies that philosophy. It represents a powerful capability that uses advanced natural language processing, semantic analysis, information retrieval, automated reasoning and machine learning. DeepQA deeply analyzes natural language input to better find, synthesize, deliver and organize relevant answers and their justifications from the wealth of knowledge available in a combination of existing natural language text and databases.

The DeepQA architecture views the problem of Automatic Question Answering as a massively parallel hypothesis generation and evaluation task. As a result DeepQA is not just about question-in/answer-out – rather it can be viewed as a system that performs differential diagnosis: it generates a wide range of possibilities and for each develops a level of confidence by gathering, analyzing and assessing evidence based on available data.

With a question, a topic, a case or a set of related questions, DeepQA finds the important concepts and relations in the input language, builds a representation of the user’s information need and then through search generates many possible responses. For each possible response it spawns independent and competing threads that gather, evaluate and combine different types of evidence from structured and unstructured sources. It can deliver a ranked list of responses each associated with an Evidence Profile describing the supporting evidence and how it was weighted by DeepQA’s internal algorithms.

“Natural Language Question Answering is a transformational benefit that will be adopted by the mainstream in the next 5-10 years.”
Source: Gartner July 2011 priority matrix for emerging technologies.

In short, powered by DeepQA, Watson is the combination of these capabilities -

Natural language processing by helping to understand the complexities of unstructured data, which makes up as much as 90 percent of the data in the world today

Hypothesis generation and evaluation by applying advanced analytics to weigh and evaluate a panel of responses based on only relevant evidence

Dynamic learning by helping to improve learning based on outcomes to get smarter with each iteration and interaction

Let us see what technological pieces make Watson a reality. The Jeopardy! Watson system configuration included 2880 processing cores, 16TB memory, 20TB disc and could process at 80 teraflops per second. Watson demonstrated its ability to sift through an equivalent of about 1 million books or roughly 200 million pages of data, and analyze this information and provide precise responses in less than three seconds. For Jeopardy, Watson held nearly twice the information contained in the US Library of Congress in instantly accessible RAM memory. On the software side, its information management capabilities combine to deliver deep content analysis and evidence based reasoning that connect widely disparate sources of information and make the kinds of connections that humans make.

Watson is 240 times stronger now in terms of processing capabilities after Jeopardy!. Capabilities like content analytics, business analytics, big data, database and data warehouses are areas where IBM has led the market for years and which form the backbone of Watson’s strength. 

For more details please download below eBook –

The Era of Cognitive Systems: An Inside Look at IBM Watson and How it Works
http://www.redbooks.ibm.com/abstracts/redp4955.html

Prev: What is IBM Watson? (Part 1 of 5 Series)
Next: Watson’s fitment for today’s challenges & use-cases (Part 3 of 5 Series)

 References -

Watson is ready for Business
http://www.ibmwatson.com/
IBM Watson
www.ibm.com/innovation/us/watson/index.shtml
Watson YouTube channel
http://www.youtube.com/user/IBMWatsonSolutions/feed
Watson in Google News
https://news.google.com/news/search?hl=en&gl=us&q=ibm+watson
IBM Journal - This is Watson
http://ieeexplore.ieee.org/xpl/tocresult.jsp?reload=true&isnumber=6177717