Enabling Content Discovery: Part 3 - Semantic Technology

Posted by Bill Ives on Sep 12, 2011 4:35:00 AM

This is the third of a six part series on enabling content discovery and combatting information overload. In this post we look at the benefits and short falls of semantic technology. Like search, semantic technology tries to have the computer perform tasks for people rather than with them. It has grown in capability in the last twenty years but still remains somewhat primitive and complex to implement. It is frequently used to dumb down tasks and automate some level of thinking on a question.  As Don Norman argues in Things That Make Us Smart it is time for us to adopt a more human-centered perspective and to insist that informational technologies enhance and complement human cognitive capacities rather than undermine them. The creation of an awareness engine was, in part, driven by frustrations with semantic technology. 

Semantic analysis requires human classification intervention and semantic model algorithms to infer and interpret meaning. Emerging Web 2.0 concepts such as folksonomy challenge these effectiveness of semantic technologies driven by predetermined taxonomies.  The challenge is rooted in the need for detection of new concepts, and the application of evolving analytic approaches for new business usage.  The current model for semantic technologies is robust but is rarely resilient because it relies on machine-driven inference of meaning. It locks down meaning in a way that limits the detection of novel concepts.

Technology can dumb down a task or smarten it up and we need more of the latter. Now there are some things that computers do better than people. For example, in legal eDiscovery, text analytics can reduce the amount of content for a person to scan by ruling out certain irrelevant documents and bringing forward those with relevant content with perhaps an acceptable level of proficiency. However, in the end a legal expert familiar with the case needs to be involved to make the final decisions. 

IBM’s Watson has demonstrated that you can build a machine to handle some level of cognition. However, this takes considerable effort. Watson was built and trained by a team of experts over a number of years. It uses math algorithms coupled with semantic analysis to allow it to understand a natural language question and determine the probability that its answer is correct. 

However, Watson is good for a very specific task and it is not perfect. The years of training may make it better than most, if not all, humans in playing Jeopardy. Although there is a debate as to whether it was really faster reaction times that caused its victory. Regardless, Watson will fail against humans in most of the other tasks we face every day as we just have too much flexibility in our processing power. 

We sometimes underestimate our abilities. Recent research by Martin Hilbert (USC) and Priscilla Lopez (Open University of California) noted that all the computers in the world combined have just recently reached the processing capacity of one person. They wrote, "the 6.4*1018 instructions per second that human kind can carry out on its general-purpose computers in 2007 are in the same ballpark area as the maximum number of nerve impulses executed by one human brain per second."

So the issue is not whether computers will outpace people but how the two can work together. Computers are very good at doing boring tedious, repetitive tasks that drive people crazy at a rate and scale far beyond what people can do. This frees people up to do the more complex and interesting tasks and supporting people with these more complex efforts is the challenge we tackle with our solution, an awareness engine. In our next post we will briefly compare semantic technology with human cognition.

Topics: Web Awareness, Information Overload, Content Discovery