How to make a webpage unsearchable?

syzd picture syzd · Sep 4, 2016 · Viewed 7.4k times · Source

Is there a way to make a webpage made as a resume unsearchable by any search engine?

The aim is to only share the page with those who have the link.

Answer

pedrouan picture pedrouan · Sep 4, 2016

There are more points and I'd prefer to apply them all:

As Daniel stated in his answer:

1) In you website location http://example.com/robots.txt

User-agent: *
Disallow: /

This means: For all robots/crawlers, do not allow to index any URI of this location.

2) If robots.txt accidentally skipped or broken, in the HTML source code on all of your pages you could have:

<!DOCTYPE html>
<html>  
    ...
    <head>
        <meta name="robots" content="noindex, nofollow" />
    </head>
    ...
</html>

This means: If accessed this page, do not index this page, even do not crawl its links - in other words - stop and go away from here.

3) If your website is not dedicated to be public, crate a password in .htaccess, if using Apache for example. So as the user will try to enter the site, he'll need a username and password.

4) If your website has been already indexed in some way and you want to disable it to be searchable for public, you could remove the web URLS from Google's index using Google Webmaster Tools:

enter image description here