0

When I wanted to fetch a certain page with htm extension I noticed an attachment after the htm extenstion http://www.example.com/IP_814.htm#.Uk1_MtKsh8E Google could not fetch this page it gave the 502 error code.

When I fetched the same page as http://www.example.com/IP_814.htm Google fetched it and gave me the green check.

When I created the page, I named it IP_814 with extension htm as this is a individual internal product page. I believe that the code or number behind the htm changes daily.

What should I do so Google crawls my individual product pages with the htm extension?

Stephen Ostermiller
  • 98,758
  • 18
  • 137
  • 361
Lexi
  • 1
  • 1

1 Answers1

1

A # and anything following in a URL is a fragment identifier. It instructs the browser to fetch the page from the server that is specified by the portion before the # and then show the user the portion of the page identified by the fragment. This is often implemented as scrolling the page to the appropriate section, but the JavaScript on the page may examine the fragment and take another action such as showing a specific DHTML tab.

When URLs are requested, the fragment identifier is never sent to the server. Browsers must strip this information from the URL before making the request. Googlebot should never be crawling URLs with fragments. It should be stripping the fragment off before crawling the URL.

Stephen Ostermiller
  • 98,758
  • 18
  • 137
  • 361
  • 1
    OH I think you answered my question, because I was the one who entered the whole link in webmaster google for the link to be fetched as google. The second time without knowing I stripped the fragment identifier, and that's why google was able to fetch my page. Thank you so much for explaining it to me! – Lexi Oct 03 '13 at 16:00
  • I would expect that even if you entered it into the "fetch as Googlebot" tool that it would strip it off before it fetched it, but they may have a bug in that tool caused by an oversight of that case. – Stephen Ostermiller Oct 03 '13 at 16:02