Thursday, September 24, 2009

Block all except adsense bot in robots.txt

I ran into a problem today when i was adding adsense to one of my
phpBB forums ( Short Jokes )

I had implemented a url rewrite mod in this forum, so in order to avoid duplicate content I had to block my old urls being indexed (such as viewtopic.php?*** ). I wrote a robots.txt to block all bots from accessing such pages.

All went fine until it was time to implement adsense. I was not getting relevant ads on the old style urls since its getting blocked.

After a bit of googling I got this piece of code which solved the problem.
Put this on top of all in my robots.txt file.

User-agent: Mediapartners-Google
Disallow:
Allow: /*


Now let me watch it for sometime and see if the quality of ads is improving.

Tuesday, September 22, 2009

Google Ignores keyword meta tag.

Well I used to believe keyword meta tag is important and used to put lot of efforts in it.


Below in the webmaster central video, Matt Cutts explains this is wrong. Some other search engine might use it, but not google.

Also he says google reads other meta tags such as description meta tags.