http://www.freepint.com/gary/direct.htm#top


大多數搜索引擎存在著非常大的問題，很多人已經意識到這個問題了。
現在的問題是，海量的網絡有一些通用搜索引擎——谷歌、百度抓取不到的“看不見的網頁”
這部分網頁比例是比較高的；
特別由于AJAX 以及RIA的大量應用，crawler面臨挑戰不小……
待續

There's a big problem with most search engines, and it's one many
people aren't even aware of. The problem is that vast expanses of the
Web are completely invisible to general purpose search engines like
AltaVista, HotBot and Google. Even worse, this "Invisible Web" is in
all likelihood growing significantly faster than the visible Web
you're familiar with.
So what is this Invisible Web and why aren't search engines indexing
it?  To answer this question, it's important to first define the
"visible" Web, and describe how search engines compile their indexes.