How do I disallow specific page from robots.txt

Daniel picture Daniel · Aug 15, 2010 · Viewed 46.5k times · Source

I am creating two pages on my site that are very similar but serve different purposes. One is to thank users for leaving a comment and the other is to encourage users to subscribe.

I don't want the duplicate content but I do want the pages to be available. Can I set the sitemap to hide one? Would I do this in the robots.txt file?

The disallow looks like this:

Disallow: /wp-admin

How would I customize to the a specific page like:

http://sweatingthebigstuff.com/thank-you-for-commenting

Answer

AlexanderMP picture AlexanderMP · Aug 15, 2010
Disallow: /thank-you-for-commenting

in robots.txt

Take a look at last.fm robots.txt file for inspiration.