WordPress Robots.txt For SEO

Filed Under development |   

The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation.

Implementing an effective SEO robots.txt file for WordPress will help your blog to rank higher in Search Engines, receive higher paying relevant Ads, and increase your blog traffic.

Here is my robots.txt files, which can further protect WordPress from this duplicate content issue.

User-agent: *
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /comments/

After you created the robots.txt file just upload it to your root directory and you are done!

Related posts


Comments

7 Responses to “WordPress Robots.txt For SEO”

  1. Top Ten WordPress SEO Tips - Moonlight Blog on March 3rd, 2008 7:06 am

    […] The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. […]

  2. Batteries on March 5th, 2008 7:52 pm

    Well, I’ll follow this for my site. thanks.

  3. Sunxc on March 6th, 2008 6:17 pm

    i put robots.txt in wordpress catalog or put it in the same of index.php

  4. Best Battery Online » Blog Archive » Top Ten WordPress SEO Tips on March 11th, 2008 8:14 am

    […] therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. User-agent: *Disallow: /wp-Disallow: /feed/Disallow: /comments/feedDisallow: /trackback/ 5. Use […]

  5. 百度不支持nofollow的robots对策 | Frankboy on March 20th, 2008 12:28 pm

    […]   如果你也是使用Z-Blog系统,不妨参考一下我写的这段针对搜索引擎优化的robots,将下面代码复制并另存为robots.txt,然后放到你的博客根目录下即可,如果你使用WordPress系统,请参考一下针对WordPress的robots.txt。 […]

  6. Top Ten WordPress SEO Tips | Im Newer on March 31st, 2008 10:51 pm

    […] The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. […]

  7. Canvas Prints on April 10th, 2008 11:41 pm

    Some people think that they are way too smart. They think their comments are wonders of the world! Dudes don’t be so self-confident.

Leave a Reply




Close
E-mail It