Natural Language Search

There’s an interesting article on Ars Technica today about search engine startup Powerset, which is reportedly proprietor of “the most sophisticated natural language technology known to man” (developed at Xerox PARC).

On his personal blog, [Powerset CEO Barney Pell] argues that keywords [the traditional search method] are the equivalent of studying a foreign language for a year: you can get around, but can’t communicate with much richness. Searching for “book by children” and “book about children” should produce greatly different results, but search engines today generally throw out the prepositions and treat both queries the same way.

The article goes on to talk about another startup currently laboring on a similar offering, called Hakia, which is already up and running. If you try the “book by children” / “book about children” combo on Hakia, you get similar results so one could say that Hakia isn’t quite there yet either. (Interestingly enough, neither of the two results pages Hakia returned were more useful than the ones Google returned.)

The implications of natural language search technology are only as significant as the number of people that adopt them, of course. In other words, if this catches on, it’ll change the face of the search industry. Why? Because SEO/SEM will be completely screwed. I have a little bit of SEO knowledge from watching our search team at syndeo::media work, and I’m always startled by the sheer number of keywords they have to manage. If a traditional SEO gig runs to a few hundred keywords/keyphrases, try to imagine what would happen if you had to optimize for keyword+preposition combinations as well (which is what natural-language search optimization implies). Your phrases would increase exponentially. In terms of volume, I think this will render SEO too complex to be handled by mere mortals, and SEO houses will eventually have to replace personnel with big proprietary algorithms.

Whether or not that’s better for the rest of the world is not for me to say, but if the search technology truly is headed in the direction of natural-language, it’s going to be a trick for the SEO industry to survive unscathed.