http://www.freepint.com/gary/direct.htm#top


大多數(shù)搜索引擎存在著非常大的問題，很多人已經(jīng)意識到這個問題了。
現(xiàn)在的問題是，海量的網(wǎng)絡(luò)有一些通用搜索引擎——谷歌、百度抓取不到的“看不見的網(wǎng)頁”
這部分網(wǎng)頁比例是比較高的；
特別由于AJAX 以及RIA的大量應(yīng)用，crawler面臨挑戰(zhàn)不小……
待續(xù)

There's a big problem with most search engines, and it's one many
people aren't even aware of. The problem is that vast expanses of the
Web are completely invisible to general purpose search engines like
AltaVista, HotBot and Google. Even worse, this "Invisible Web" is in
all likelihood growing significantly faster than the visible Web
you're familiar with.
So what is this Invisible Web and why aren't search engines indexing
it?  To answer this question, it's important to first define the
"visible" Web, and describe how search engines compile their indexes.