3

Possible Duplicate:
What is duplicate content and how can I avoid being penalized for it on my site?

I know that duplicate content needs to be addressed in some ways, but this problem is bugging me for some time I have no clue how to solve it.

I have a page say a product page with URL /product/1234/name-of-product that part of it is displaying a stream of news related to this product with a "more" button at the end, like Facebook stream.
This more button when clicked is intercepted by javascript code which will fetch the rest of stream through an AJAX request to URL /stream/product?keyword=name-of-product. However if the JS is disabled or for some reason the AJAX cannot be made the link works the normal way which will go to the URL above which will show only the stream.

Now I don't know how to tell search engines that this PART of the page is the same as the stream URL. The thing is I don't want the stream URL to rank higher than main product page.

Is it right to put a rel=canonical on the stream page that point to product page? If true the product page is only showing a portion of stream and for more it links back to stream page again.
I'm confused a little here. Thanks in advance and I'll provide further information if required to clear this up.

Community
  • 1
doctrey
  • 133
  • 3

2 Answers2

4

Since you only have the start of the content on the product page, the full stream page is not duplicate content (not by a long shot). It's no different from a blog showing the introduction to their posts on the home page - nothing to worry about.

However, from the sound of it the stream page is just a bunch of text without your design/branding surrounding it. In which case, you may not want users stumbling upon those pages in search results (and not knowing what site they are on), so you may consider blocking the stream pages from search engines using robots.txt.

An ideal solution in my opinion would be to add your design to the stream pages so that they fit within your site. Link to those from the product page, but set the AJAX to grab the basic version. You could call e.g. /stream/product?keyword=name-of-product&mode=ajax

DisgruntledGoat
  • 21,588
  • 5
  • 54
  • 101
  • Good idea. I think you have a point about the blog but I have seen this before on eCommerce sites and I think it is good to test. It seems counter intuitive but robots.txt won't necessarily stop the pages being crawled or the URL of the stream page appearing in the search engines (meta follow, no index will). Like the idea about including the site design on the stream pages =) – Sandy Lee Mar 08 '12 at 11:03
  • @DisgruntledGoat Actually I was thinking about excluding the streams with robot.txt. But I actually do want them to be indexed cause there are streams that are not directly associated to any product. And about the solution if I got you right it's actually what I'm doing right now. I show the first page and fetch the rest using AJAX from stream URL. – doctrey Mar 08 '12 at 11:52
  • @DisgruntledGoat I got you wrong in previous comment. Thanks it's a good idea. Actually before switching to this method, I would get the streams the same way but I didn't have SEO in mind and I thought if I load the page with the stream it could save a request to server. But now it seems like the better way. – doctrey Mar 08 '12 at 13:24
  • @doctrey sorry if my answer isn't so clear, but I am assuming that if you fetch a page (the stream) via AJAX, then it does not have your site design on it (otherwise you would get a design-within-a-design on the product page). Is that right? My idea is to link to a version with your design, so that users can enter from search results, but when they click the link on the product page, your AJAX instead fetches the version without the design. Hope that makes sense. – DisgruntledGoat Mar 08 '12 at 17:56
1

Wow what an interesting problem? It would be worth testing but I reckon over a large body of product pages that would be duplicate content and rel=canonical wouldn't cut it here.

Is there anyway you can put in a bit of JS that is executed as an onclick event when the "MORE" button is pushed that will append the ajax request url so the call can be made but otherwise wouldn@t be available until the "MORE" link is clicked?

That way when the search engine spider comes along it won't be able to execute the JS that causes the onclick event, the link will be empty, it won't be able to follow it and therefore can't be crawled.

You have the UX issue of someone who doesn't have JS enabled won't be able to follow the link but you can have an overlay that advises people to have JS enable to solve that one.

This pure conjecture by the way but it sounds like an interesting problem. Anyone got any other ideas?

Sandy Lee
  • 1,101
  • 5
  • 13
  • Thanks for the response. As far as I know it's a best practice to have links on the page as a valid ones. Mock links like just for the sake of JS is not advised. With valid links you solve UX problem and also let the search engines crawl the content. What you're saying is that I remove the links on more button and ask users to turn on their JS to make sure the AJAX request is sent. If that's the case I think I can mark the more button as nofollow. – doctrey Mar 08 '12 at 12:06