How to add `nofollow, noindex` all pages in robots.txt?

http robots.txt access-control
MeltingDog picture MeltingDog · Aug 21, 2017 · Viewed 11.6k times · Source

I want to add nofollow and noindex to my site whilst it's being built. The client has request I use these rules.

I am aware of

<meta name="robots" content="noindex,nofollow">

But I only have access to the robots.txt file.

Does anyone know the correct format I can use to apply noindex, nofollow rules via the robots.txt file?

Answer

Ankit Gujarati picture Ankit Gujarati · Aug 21, 2017

noindex and nofollow means you do not want your site to crawl in search engine.

so simply put code in robots.txt

User-agent: *
Disallow: /

it means noindex and nofollow.

Related questions

PUT vs. POST in REST

According to the HTTP/1.1 Spec: The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI in the Request-Line In other words, …

Read More
How are parameters sent in an HTTP POST request?

In an HTTP GET request, parameters are sent as a query string: http://example.com/page?parameter=value&also;=another In an HTTP POST request, the parameters are not sent along with the URI. Where are the values? In …

Read More
What exactly is RESTful programming?

What exactly is RESTful programming?

Read More