How to tell Google to crawl distinct part of the webpage?

Question

What approach should I take to make the website Google-friendly, http://anuary.com/.

The problem is that when a specific content is being requested (eg. http://anuary.com/1/home/anuary) user is taken to the page using scrollbar, though the surrounding content is still being loaded. How to tell Google that for this URL http://anuary.com/36/portfolio/three-musketeers only the #36 entry should be crawled.

Is there a tag or other way to select the surrounding content as irrelevant for the URL?

You cannot do this. Google will more or less see what you see when you look at source code of that page. There is no way to tell Googlebot to not consider the structure of the page, you can only tell Googlebot to ignore entire pages/sections. — milo5b, Jul 12 '12 at 11:09
See also: Preventing robots from crawling specific part of a page — Stephen Ostermiller, Jan 21 '15 at 21:31

score 5 · Answer 1 · edited Apr 13 '17 at 12:33

It's not possible to tell GoogleBot to not/crawl portions of your documents. The bot takes it all in and they decide what they think is important. Only AdSense allows for section targeting.

As far as I can tell, GoogleBot isn't doing anything wrong. The problem is that your URLs aren't real in the first place. You have a giant document and you're using a fake sub-directory structure that you use to figure out how far down to scroll via something that seems to pretend to be Ajax. If you were actually using it, then you could follow the crawling guidelines

score -2 · Answer 2 · answered Jul 12 '12 at 11:20

-2

Google provides googleon/googleof tags that are just right for the job ( https://developers.google.com/search-appliance/documentation/46/admin_crawl/Preparing ). An explanation on how to use them is available at http://perishablepress.com/tell-google-to-not-index-certain-parts-of-your-page/.

Though, essentially it all comes to:

<!--googleoff: all-->
This content will not be indexed.
<!--googleon: all>

answered Jul 12 '12 at 11:20

Gajus

391
2
14

1

-1 That's for the search appliance. – Su' Jul 12 '12 at 16:31
googleoff and googleon only work with the Google search appliance and have no effect on normal Googlebot – Stephen Ostermiller Jan 21 '15 at 21:30

How to tell Google to crawl distinct part of the webpage?

2 Answers2

Related