Google has recently added a new Search preferences setting in Blogger, in which one of the option is to edit Robots.txt file. Robots.txt file is one of the important file for SEO, it tells robots (spiders) to what content should it crawl, what it should not, where it can find contents, etc.
So if you want to take your blogging seriously and don’t want to move your blog from Blogger then this may help you in doing your blog Search Engine Optimization. To edit Robots.txt file
Note: – This post is only for those who knows a bit about robots.txt and SEO
- Go to Blogger in Draft dashboard
- From More option drop-down, select Settings
- Then click on Search preferences from left column
- Now edit and enable Custom robots.txt option
- Type your own or pate the below lines their (if you don’t know what should be their)
- Then click on Save change button
Sitemap: /feeds/posts/default?orderby=updated Sitemap: /atom.xml Sitemap: /atom.xml?redirect=false&start-index=1&max-results=500 Sitemap: /atom.xml?redirect=false&start-index=501&max-results=500 User-agent: * Disallow: /search* Disallow: /feeds/* Disallow: *?* Allow: /search/label/* Allow: / User-agent: Mediapartners-Google Disallow:
Now your robots.txt file has been updated, you can go and see by adding /robots.txt after your blog URL (e.g.: jajodia-saket.blogspot.com/robots.txt). Above lines will tell spiders where they can find sitemaps (if you are using the lines I mention above and have more than 1000 post in your blog then check this article on Blogger sitemap), not to follow search page, feeds and any URL which contain question mark (?), follow labels (or tags) and other pages and tells Google Ad bots to follow all pages. If you have any question or problem let me know.