Massive template cache cleaning tasks always get stuck

Question

We have a site where we use lots of {% cache %} tags.

Now we almost constantly have a "cleaning template cache" task running, and they often get stuck.

Manual DB intervention and resetting the stuck task to status pending, will sometimes make it move again.

I've tried the various solution here, but nothing helps: DeleteStaleTemplateCaches job getting stuck

Also - there are no error entries in craft.log or phperrors.log when it gets stuck.

Looking at the craft_templatecaches table I find lots of duplicate looking entries:

select id,locale, cacheKey, expiryDate, path from craft_templatecaches;

...
| 685607 | nb     | zDzBdD8bc1y4pf9Sx2HFyL3i24YypSp2sAq5 | 2015-09-08 08:23:21 | site:kompetansetilbud/ny-i-hr-rollen-i-trondheim-10013                                   |
| 685610 | nb     | TMz4eh7FPiINSUixwkpXb0SlkHnZ3b5TSgXm | 2015-09-08 08:23:22 | site:kompetansetilbud/ny-i-hr-rollen-i-trondheim-10013                                   |
| 685611 | nb     | HgLIHHgf56nM1ggOdTmaLlaylu5JeOeM1dNm | 2015-09-08 08:23:23 | site:kompetansetilbud/ny-i-hr-rollen-i-trondheim-10013                                   |
| 685612 | nb     | SLHUKfUz2fF5Jmpy0kWPpvMxFBHH8PvlVJEt | 2015-09-08 08:23:25 | site:kompetansetilbud/ny-i-hr-rollen-i-trondheim-10013                                   |
| 685614 | nb     | oEU6fe1tGc7MdI8WSQIXsf2wZZd9qmXG8P9H | 2015-09-08 08:23:26 | site:kompetansetilbud/ny-i-hr-rollen-i-trondheim-10013                                   |
...

Is this normal?

Anything else I can do to debug why the task gets stuck?

Update:

Using keyed and global caching reduced the size of the craft_templatecaches table from ~20k rows to ~300. However, this did not stop the task getting stuck.

You might want to use a key on those cache tags. It looks like it's always creating new caches: http://craftcms.stackexchange.com/questions/7096/cache-always-creates-new-cache-data-never-uses-existing-cached-data — Fred Carlsen, Sep 08 '15 at 12:35
Also take a look at http://buildwithcraft.com/help/stuck-tasks — Luke Holder, Sep 08 '15 at 15:29

score 1 · Answer 1 · edited Apr 13 '17 at 12:51

As suggested by Fred, using the {% cache globally using key craft.request.path %} trick from https://craftcms.stackexchange.com/a/8335/342 gets rid of almost all duplicates!

But for the last few I notice that the duplicates all expire within seconds of each other, maybe this is an instance of https://en.wikipedia.org/wiki/Cache_stampede ?

I.e. several clients request the pages "simultaneously" and the page is rendered several times, and each version is cached?

Massive template cache cleaning tasks always get stuck

1 Answers1

Linked