Need to block subdomain using robots.txt which is on same directory level

Jalpesh Patel picture Jalpesh Patel · Sep 19, 2014 · Viewed 10.3k times · Source

I have one problem

I have domain name for example www.testing.com and new.testing.com so i do not want to new.testing.com display in any search engine. I have added one robots.txt to the new.testing.com. And both site has same parent directory

--httpdoc
----testing.com
----new.testing.com

So i want to know that can i handle both site using one robots.txt of testing.com is it possible???

Please suggest me solution if possible.

Answer

bmiljevic picture bmiljevic · Sep 19, 2014

The best thing you could do is to add separate robots.txt files. Put one in each directory. You should have

  • testing.com/robots.txt and
  • new.testing.com/robots.txt

After adding robots.txt file to new.testing.com, you should add following code to keep search engines away.

User-agent: *
Disallow: /