How to Prevent Web Pages from Appearing in Google Search Results
Believe it or not, there may be instances in which you would prefer elements of your website NOT to appear in Google, or another search engine’s results. This may include pages or information that are not meant to be seen by the public, information you would only choose to share with certain people, or a temporary page you need to create. By disallowing these elements to show up in results, you will be able to maintain your search engine ranking, while at the same time having these pages stored on your site so they remain accessible to you.
How to prevent a web page from appearing in search engine results
The two factors we are dealing with are index and follow. Index refers to the url itself and if you want the page to appear in search results. Follow tells the search engine bot whether or not to follow the links that are associated with the page.
There are 4 scenarios that direct a search engine bot on how to handle a particular page. In order for the tag to be understood correctly by the bot, it must be inserted into the header of the web page, between the <head> and </head> tags.
Index the URL and follow all links on the page:
<meta name="robots" content="index, follow">
This is the default behavior if no robots meta tag is specified.
Do not index the URL but follow all links on the page:
<meta name="robots" content="noindex, follow">
Index the page but do not follow any links on the page:
<meta name="robots" content="index, nofollow">
Do not index the page and do not follow any links on the page:
<meta name="robots" content="noindex, nofollow">
How to prevent an entire site from appearing in search engine results
In addition, you can also create a robots.txt file that directs a search engine bot as it crawls your website. This file is stored on your site and gives specific instructions regarding indexing and following links on your pages. The robots.txt file works in similar fashion to the index and follow tags above, but allows for addition options. The file allows you to block specific directories, images, files, and ads.  There is even an option to block your entire site from appearing in search results, making your site virtually invisible.
To block an entire site from appearing in search engine results, the following must be entered in the robots.txt file:
Disallow: /
Once that it entered, your site should no longer appear in search results. If it does, it will be removed the next time your site is indexed by the search engine bot.
2023
Leave a Reply
Want to join the discussion?Feel free to contribute!