Dec

10

WordPress Robots.txt For SEO

Filed Under development |   

The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation.

Implementing an effective SEO robots.txt file for WordPress will help your blog to rank higher in Search Engines, receive higher paying relevant Ads, and increase your blog traffic.

Here is my robots.txt files, which can further protect WordPress from this duplicate content issue.

User-agent: *
Disallow: /wp-
Disallow: /feed/
Disallow: /trackback/
Disallow: /comments/feed/
Disallow: /page/
Disallow: /comments/

After you created the robots.txt file just upload it to your root directory and you are done!

Related posts


Comments

14 Responses to “WordPress Robots.txt For SEO”

  1. Top Ten WordPress SEO Tips - Moonlight Blog on March 3rd, 2008 7:06 am

    [...] The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. [...]

  2. Batteries on March 5th, 2008 7:52 pm

    Well, I’ll follow this for my site. thanks.

  3. Sunxc on March 6th, 2008 6:17 pm

    i put robots.txt in wordpress catalog or put it in the same of index.php

  4. Best Battery Online » Blog Archive » Top Ten WordPress SEO Tips on March 11th, 2008 8:14 am

    [...] therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. User-agent: *Disallow: /wp-Disallow: /feed/Disallow: /comments/feedDisallow: /trackback/ 5. Use [...]

  5. Canvas Prints on April 10th, 2008 11:41 pm

    Some people think that they are way too smart. They think their comments are wonders of the world! Dudes don’t be so self-confident.

  6. Baidu does not support the Robots countermeasure of Nofollow : LOVE BLOG on September 19th, 2008 4:26 am

    [...] If you also are use Z-Blog system, might as well this paragraph of optimizes in the light of search engine Robots that consults I am written, will below code duplicates and put additionally for Robots.txt, below the list of rich guest root that puts you next can, if you use WordPress system, consult pleaseBe aimed at the Robots.txt of WordPress. [...]

  7. fan on October 25th, 2008 1:32 am

    very useful information!
    thx a lot!

  8. squeaky on November 5th, 2008 2:04 pm

    I find that the robots.txt file is one of the hardest ones to nail down properly. I have started working with the robots.txt file and have seen some pretty good results, so far.
    Google is visiting my blog much more often and I am getting much better search engine results for some of my keywords. I think that has a lot to do with less pages in the supplementals.

  9. samso on November 26th, 2008 12:53 pm

    The Robots.txt attacked at my blog post titles so many times. I noticed some tips from your post. I follow them to clear my problem. Thanks for useful information.

  10. buntelan on January 31st, 2009 8:13 am

    thanks for your tips!

  11. 小问 on February 16th, 2009 1:03 am

    very useful information!

  12. Label on May 24th, 2009 11:36 pm

    Thanks.

  13. SEO Your WordPress - Fun Tuner on May 29th, 2009 8:25 am

    [...] The robots.txt file is used to instruct search engine robots about what pages on your website should be crawled and consequently indexed. Most websites have files and folders that are not relevant for search engines (like images or admin files) therefore creating a robots.txt file can actually improve your website indexation. Here is an SEO optimized WordPress robots.txt file. [...]

  14. 伍少爷 on June 4th, 2009 1:23 am

    试一下。
    看看效果。

Leave a Reply