Google killer over-hyped, or a silver bullet for web search?


10 Mar 2009

A London-based scientist has said he and his team will release a new search engine in two months that will understand people’s questions and answer them directly with the correct answer.

Codenamed Wolfram Alpha, the team at Wolfram Research, aims to do for search what its famed Mathematica technology did for math software.

In May, the new search system will allow people to ask a question and get the answer they are looking for in a single hit.

“Fifty years ago, when computers were young, people assumed that they’d quickly be able to handle all these kinds of things,” wrote Stephen Wolfram on his blog. “And that one would be able to ask a computer any factual question, and have it compute the answer.

“But it didn’t work out that way. Computers have been able to do many remarkable and unexpected things. But not that.”

Wolfram said there are billions of pages of text on the web, and while web search engines can efficiently find specific terms and phrases, they can’t compute from that. “We can look things up, but we can’t figure anything new out,” Wolfram noted.

Wolfram’s Mathematica software had two key ingredients – it had a symbolic language to represent anything, as well as the algorithmic power to do any kind of computation, while the research team’s NKS software had a paradigm for understanding how all sorts of complexity could arise from simple rules.

“Every different kind of method and model, and data, has its own special features and character. But, with a mixture of Mathematica and NKS automation, and a lot of human experts, I’m happy to say we’ve gotten a very long way.

“The way humans normally communicate is through natural language. And when one’s dealing with the whole spectrum of knowledge, I think that’s the only realistic option for communicating with computers too,” Wolfram said.

“Of course, getting computers to deal with natural language has turned out to be incredibly difficult. And, for example, we’re still very far away from having computers systematically understand large volumes of natural language text on the web.

“But, if one’s already made knowledge computable, one doesn’t need to do that kind of natural language understanding. All one needs to be able to do is to take questions people ask in natural language, and represent them in a precise form that fits into the computations one can do.”

Wolfram said that with a mixture of clever algorithms and heuristics, and lots of linguistic discovery and linguistic curation and theoretical breakthroughs, the Wolfram Alpha system is beginning to work.

“Pulling all of this together to create a true computational knowledge engine is a very difficult task. It’s certainly the most complex project I’ve ever undertaken. Involving far more kinds of expertise, and more moving parts. than I’ve ever had to assemble before. And, like Mathematica, or NKS, the project will never be finished.”

But what’s ready is a website, www.wolframalpha.com, which has a simple input field that gives access to a system with trillions of pieces of curated data and millions of lines of algorithms.

“We’re all working very hard right now to get Wolfram Alpha ready to go live. I think it’s going to be pretty exciting. A new paradigm for using computers and the web,” Wolfram said.

“That almost gets us to what people thought computers would be able to do 50 years ago!”

By John Kennedy