1

I used to use only search console and Google analytics for website data. But recently I decided to try out Bing webmaster tools when I heard about Microsoft Clarity.

In the Site explorer, I noticed some strange links that Bing indexed (see the image below). These were peculiar pharmacy spam links created through the site's internal search.

Most of them were not indexed as such links are disallowed in robots, but some actually were indexed, maybe because there is 1 backlink to them, so the disallowed rule was ignored.

The Google's search console doesn't show them at all.

My question one is: Can this hurt the site?

Question two: What else can I do besides the robots.txt rule?

This is a WP site, with the Ivory Search plugin.

I thought of creating a rule in the htaccess file to add "noindex" to the header of such pages, but I am not that savvy to pull it off.

Spam links

Terry
  • 13
  • 2
  • 2
    Disallowing in robots.txt doesn't prevent indexing. It only prevents crawling. The rule is not "ignored," but search engines will sometimes index URLs even when they can't crawl them. – Stephen Ostermiller Feb 03 '21 at 11:00
  • Related: https://webmasters.stackexchange.com/questions/135976/how-to-stop-google-from-indexing-spammed-internal-search-results and https://webmasters.stackexchange.com/questions/107806/internal-search-results-noindex-or-robots-txt-blocking, and somewhat related https://webmasters.stackexchange.com/questions/86805/an-internal-search-page-is-ranking-higher-than-a-category-page-for-the-same-topi – Maximillian Laumeister Sep 03 '21 at 19:35

1 Answers1

1
  1. If it a search engine reaches it, then yes. It can hurt your site.
  2. You should add the robots meta tag with noindex nofollow and it should be okay.

Google (probably) understands that this is a search results page and therefore ignore it.

In addition to the steps above, I would recommend disavowing those links.

Tal
  • 386
  • 1
  • 7