2

I've many useful docs in my website and we know that Google has got smart enough already - that if the document serves visitor's purpose then it directly lists it in the SERP without listing the page from which it was linked.

That'd cause loss of revenue to us in case visitor does not visit our web pages.

Is there any way to prevent Google listing the documents directly in the SERP?

I see that there is X-Robots-Tag:noindex to prevent Google from indexing documents. But I do want Google to index the documents just to know their importance and also increase relevancy due to them.

Similar question: How to prevent a PDF file from being indexed by search engines?

Community
  • 1
AgA
  • 1,438
  • 3
  • 13
  • 29
  • A way of gaining that revenue back is to partially split the PDF files for example a few paragraphs in the PDF that Google discovered then at the end of paragraph within the PDF to read the FULL PDF file click here, which links to page that you want. This would benefit rankings and click through, PDF files have a good click rate vs a URL, so you can hide the full PDF, and then allow Google snippets of the real deal. – Simon Hayter Aug 12 '13 at 12:04

1 Answers1

3

To prevent Google to index your documents but let bots crawl your documents (for SEO purposes), you can put these following lines in your .htaccess (if you use Apache as a web server). It will give weight to tell to Google to index your webpages and not your documents.

<Files ~ "\.pdf$">
    Header set X-Robots-Tag "noindex"
</Files>

With these lines, you apply filter on all .pdf files for your entire website by using X-Robots-Tag in HTTP header. This is an article from Google to understand how it works.

Zistoloen
  • 10,036
  • 6
  • 35
  • 59
  • As I've already mentioned .. I do want Googlebot to read the documents though for SEO purpose .. Also can I disable indexing of all pdf's using *.pdf pattern? – AgA Aug 12 '13 at 08:58
  • I have edited my answer. – Zistoloen Aug 12 '13 at 09:17
  • I understand that crawl by Google will not make Google use it for ranking purpose. For that indexing will need to be allowed. So your answer will make it read the doc for new links etc.. But your answer is close enough, I'd accept it. – AgA Aug 12 '13 at 10:51
  • If you want to see your webpages indexed (webpages with documents), I think not allowing indexing of documents will help you. So visitors could find your webpages and thus your documents. – Zistoloen Aug 12 '13 at 11:03