2

We've a WordPress blog and had disqus plugin in stalled for several months. Around late August this year, the plugin created a ton of URLs that linked to non-existent location on our website. For example -

Correct URL: domain.com/correct-URL/ Disqus created -

  1. domain.com/correct-URL/344322/ -> Throws 404
  2. domain.com/correct-URL/433466/ -> Throws 404

So essentially, Google found a LARGE number of broken links that pointed to unknown locations on our own domain.

As the count of those errors (404) rose, our site suffered massive drop in traffic and crawl rate dropped to 10% of what it was earlier.

I wish to know -

  1. Can large number of (we've over 99k of them) internal broken links cause rankings to drop?

  2. I've fixed the issue in one go by creating 301 redirects for each bad URL to correct URL and removing disqus. Google however drops the count by ~1000 daily, as I mark errors as 'fixed' in Google Webmaster Tools. Is there any way to speed this up?

  3. Should I setup custom crawl rate to 'Fast' in GWT to make Google crawl our website faster?

I'd appreciate your inputs and experience sharing.

TheBigK
  • 151
  • 1
  • This seems to be a common problem with Disqus: http://webmasters.stackexchange.com/questions/34229/suddenly-i-have-started-getting-404-wordpress-errors – MrWhite Dec 03 '12 at 10:57
  • 1
    Yes it is. Seoroundtable discussed it on their blog: http://www.seroundtable.com/disqus-google-errors-15663.html – TheBigK Dec 03 '12 at 14:36

1 Answers1

2

Can large number of (we've over 99k of them) internal broken links cause rankings to drop?

A large number of broken links would indicate that a site is low quality would probably cause it to be affected by the Panda algorithm. That would cause ranking drops and pages being removed from Google's results.

I've fixed the issue in one go by creating 301 redirects for each bad URL to correct URL and removing disqus. Google however drops the count by ~1000 daily, as I mark errors as 'fixed' in Google Webmaster Tools. Is there any way to speed this up?

No.

Should I setup custom crawl rate to 'Fast' in GWT to make Google crawl our website faster?

This is only a suggestion and won't result in Google speeding up the crawl rate for your site.

John Conde
  • 86,255
  • 27
  • 146
  • 241
  • Thanks. I'm sure it's not 'Panda', because the traffic drop and error count going up happened on 4-5 September, and no Panda update was published at that time. Plus, the site is of reasonably good quality that has survived panda and penguin updates. Just confirming that this indeed can cause SEO drop. – TheBigK Dec 02 '12 at 21:22
  • 3
    There doesn't have to be an update for it to affect your site. – John Conde Dec 02 '12 at 21:24
  • Yes, Google pushes several updates every day, but not every update is Panda. I'm wondering why does Google take a long time to recognize that all those errors have gone? It just drops them at 1000/day. – TheBigK Dec 02 '12 at 21:25
  • They have priorities when it comes to crawling sites. This site is crawled constantly. Yours is probably crawled every few days or even less frequently then that. It depends on lots of factors. – John Conde Dec 02 '12 at 21:27
  • I've been noticing that indeed. Isn't it true that Google crawls websites every day? What does the crawl rate indicate in GWT? It seems to get updated on daily basis. My reports indicate that until end of August, Google used to crawl ~70k pages per day. – TheBigK Dec 02 '12 at 21:29
  • Google does not crawl most websites every day. In fact most websites are not. Crawl rate is used to make suggestions to Google for crawl rate. It primarily only affects slowing down their crawl. It typically doesn't make it go faster. – John Conde Dec 02 '12 at 21:31
  • Is there any way to find out whether Google's crawling website on daily basis? Do server logs help? – TheBigK Dec 02 '12 at 21:32
  • Yes, your server logs will tell you when Google crawls your site. – John Conde Dec 02 '12 at 21:33
  • I'm on it. Thanks a lot, John. I really appreciate your quick responses. I'll look forward to experiences other webmasters have had with similar issues. :-) – TheBigK Dec 02 '12 at 21:39
  • 1
    @TheBigK Panda updates don't just affect sites right there and then and never again. If you do something against their guidelines you are penalised when you do it, not when an update rolls around. Also FYI in Google Webmaster Tools it shows you how frequently Google is crawling your site. – DisgruntledGoat Dec 04 '12 at 10:13