| | |

Robots.txt for Bloggers using WordPress

When learning about SEO, the robots.txt file is likely to come up. This is a powerful file that ‘speaks’ directly to Google and Bing (& Other) search engine robots.

Is my Robots.txt Correct?

It is very important to understand what this file can do. The robots.txt file seems to generate a lot of opinions but you must truly be sure about your editing as this can start and stop Google from indexing your site!

  1. the order of the instructions matter
  2. Google wants to see everything (less is more)
  3. only disallow if there is a very specific reason for it (ie: affiliate links)

Disallowing crawling of Javascript or CSS files in your site’s robots.txt directly harms how well our algorithms render and index your content and can result in suboptimal rankings. (google webmaster)

What to do if your robots.txt file is wrong

If you’re quite sure that the robots file is wrong, check Yoast at this article for clear instructions on changing it. But I can’t not say it again: be careful! 

Similar Posts

Leave a Reply

Your email address will not be published.