A number of search engines have attempted to tackle the semantic web concept with varying degrees of success, some came out before web 2.0 emerged. In each of the cases that I am familiar with, they used complex search algorithms to attempt to make sense of unstructured data (aka text documents or communication). The search engines went part way and a person had to go the rest of the way. Often, a lot of work was required on the part of the person.

Expert System, an Italian-based company, has taken the concept an important step further. Instead of just turning the search algorithms loose, they provide significant help in the interpretation of text. They employed an army of linguists to create a semantic network of over 350,000 word definitions, 2.8 million word relationships and 3,500 linguistic rules to give the tool very concrete help, as we shall soon see. Last week I spoke with Brooke Aker, the new CEO of their U.S. subsidiary. Expert System has been operating successfully in Europe for 15 years. They started with tools like Italian spell checkers for Microsoft and have advanced in complexity and functionality since then. Now they are working to establish a larger U. S. presence through their semantic search tool, COGITO.

Key word search (aka Google and others) has been successful to a point. It has trouble when a word can have multiple meanings, when different words have the same meaning, and when different ones have similar meanings. COGITO can recognize the meaning of words thanks to its recognition of the context and the richness of its semantic network or set of rules. It uses four methods to uncover meaning. First there is a morphological analysis - what words are related. Then there is a grammatical analysis - what part of speech. Third, there is a logical analysis of how words relate to each other. Finally, there is a semantic analysis to understand the context of words.

Brooke showed me a concrete example. We looked at a sentence chosen to be potentially ambiguous; “My car eats gas when I step on the gas.” The system was able to correctly identify the parts of speech, the intended meaning of each word, and the fact it was a compound sentence. It even knew that the last use of gas was to indicate gas pedal, one of 12 definitions the semantic network had for the word. As a former cognitive psychologist I was impressed. So I asked about business applications.

In one business application, Expert System uses COGITO to monitor consumer sentiments in blogs, comment sections, message boards, and Web-based articles. Most tools uncover the amount of times a word appears in text and uses that as a basis of popularity/market trends. Expert System, however, detects actual, granular sentiments to identify actual consumer behaviors and trends. Right now, the biggest market the company is targeting is the automotive industry.

Expert System’s COGITO Monitor uncovers the types of cars that are purchased the most, the least and why; compares different models of cars and gives a consumer rating; and creates reports of anecdotal information about what consumers are actually saying about car models and manufacturers (such as why one chose a Nissan over a Hyundai).

Brooke showed me some live examples. The COGITO Monitor was picking up commenter sentiment from the blogs of several major auto magazines. You could compare cars, models, and features on a dial with a scale of 1 to 5 with many tick marks in between so it really is like 1 to 50. The comparisons we saw seemed to make sense and were easy to perform (e.g., people like the BMW engine a bit better than the Audi engine).

In another business example, they search the web to create situational awareness for one of the large international oil companies with operations in over 60 countries. What is the weather, the political climate, the price of oil, and other constantly changing factors that could have significant business impact on the firm? Instead of employing a group of people to manually monitor these situations, they now get a 24/7 situational analysis by monitoring these factors through the web with COGITO.

Expert System has also partnered with the Wikipedia to allow for natural language questions into the content within the Wikipedia, called AskWiki. I asked it “When was New Haven founded?” and it returned the correct answer and a link to the source article. Then I asked, “Who is the coach of the Boston Celtics?” Once again they got it right and linked to the source article. I even learned the real first name for Doc River’s - Glenn. After the answer, AskWiki asks you it if provided the right answer - yes, no, or uncertain - to give it feedback. It is still in the early stages of development but it was two for two on my questions.

Expert System also recently started a blog, COGITO, to better share their story. Today there is an increasing amount of user contributed content on the web to sort through, including blogs, wikis, and forums. I think Expert System may have given the semantic search concept just the help it needed to become practical.