Sunday, December 9, 2007

WordPress Robots.txt For SEO

The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation.

Implementing an effective SEO robots.txt file for WordPress will help your blog to rank higher in Search Engines, receive higher paying relevant Ads, and increase your blog traffic.

Here is my robots.txt files, which can further protect WordPress from this duplicate content issue.

User-agent: *
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /comments/


After you created the robots.txt file just upload it to your root directory and you are done!

7 comments:

  1. [...] The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. [...]

    ReplyDelete
  2. Well, I'll follow this for my site. thanks.

    ReplyDelete
  3. i put robots.txt in wordpress catalog or put it in the same of index.php

    ReplyDelete
  4. [...] therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. User-agent: *Disallow: /wp-Disallow: /feed/Disallow: /comments/feedDisallow: /trackback/ 5. Use [...]

    ReplyDelete
  5. Some people think that they are way too smart. They think their comments are wonders of the world! Dudes don’t be so self-confident.

    ReplyDelete
  6. [...] If you also are use Z-Blog system, might as well this paragraph of optimizes in the light of search engine Robots that consults I am written, will below code duplicates and put additionally for Robots.txt, below the list of rich guest root that puts you next can, if you use WordPress system, consult pleaseBe aimed at the Robots.txt of WordPress. [...]

    ReplyDelete
  7. very useful information!
    thx a lot!

    ReplyDelete