Dec 30 2007
Mimic GoogleBot
ave you ever visited a website that had the exact description in Google but once you went to it, it asked for a username and password? The reason for this is that there is a GoogleBot that is allowed to visit some sites unrestricted. This is allowed so the GoogleBot indexes the webpage in Google.
Now this trick is for FireFox.
In the URL bar, enter
about:config
and click Go.
Right click, select new > string and enter
general.useragent.override
For the string value, enter
Googlebot/2.1 (+http://www.googlebot.com/bot.html)
Also if you do not have FireFox try this website. This site allows you to search the internet as either the GoogleBot or the YahooBot.
