What’s the Big Deal About Deep Web Search?

What’s the big deal about deep web search and why it is creating a huge buzz in the Internet today?
According to Internet experts, while major search engines like Google, Yahoo, and MSN provide information that are retrieved from billions of web pages, still their search technology is not capable of returning results as exactly as it is wanted specifically when it comes to what we call the deep web search.
Above image shows Prof. Juliana Freire at the University of Utah, while working on DeepPeep, an ambitious effort to index every public database online.
While Google continues to add pages-in fact another trillion of pages in the last summer–to the list of web pages it knows about, still it is said to be these are just a tip of an iceberg. Beyond what major search engines know is even vaster collection of hidden data: financial information, shopping catalogs, flight schedules-among others which remain invisible to the eyes of major search engines.

So, to be able penetrate this data, data that is beyond the reach of major search engines, a new technology called ‘deep web search engine’ must be developed. A so called search engine technology that works to analyze and understand users’ query to give the most appropriate query results which present search engines are not capable of since they use a keyword-based technology.
Google is working for this so called deep web search conundrum but it is very much cautious in every step of the way particularly in making changes in its search technology for fear of alienating its users.
There are other search technologists which are trying to develop to unveil the portal of deep web search environment. And if the development of their system become successful, it could give the most-effective and efficient search query results ever that people, companies, would consider as their greatest search engine benefit.






Good article!
I just wanted to point out that “deep web search technology” has already been developed. Take a look at http://www.scienceresearch.com/, http://www.worldwidescience.org/, http://www.mednar.com/, and http://www.scitopia.org/. (I could go on). My company, http://www.deepwebtech.com/, is responsible for many such websites (public and private).
It’s actually quite difficult for a number of reasons, the biggest being that most deep web content is proprietary, requiring credentials or a subscription to access. Another component of deep web content is publicly available information that’s hidden behind a form. For example, to get at an article, you need to fill out a form. Automated search tools don’t necessarily know how to fill out a form, without basic assumptions.
This means, until ALL publishers create common interfaces to their information (as well as allow free searching, even if the content itself requires a subscription), the deep web will always be difficult to reach.
Larry.
Thanks for the useful info Larry!
Nice article. But please confirm that if that proposed technology “deep web search” is already implemented.