17

I'm having a very odd issue pop up wherein if I put all 75 of my account testMethods in one class, I get less than stellar coverage (maybe 70%) but if I split those testMethods up into 8-10 smaller classes (same 75, but split) then I get higher coverage (upwards of 90-95%)

This is in the sandbox

Is this just a glitch in the way they calculate code coverage in test? Or is this actually something I need to pay attention to and not allow more than x testMethods per class?

Nathan Williams
  • 3,445
  • 1
  • 29
  • 49
  • Are you by chance getting those total coverage figures from Run All Tests or from the Developer Console or org estimation? – Brian Mansfield Mar 30 '15 at 21:50
  • Developer Console... I have more tests than I'm about to deploy, so I choose just the ones I am about to deploy to make sure I have apples-to-apples with what will be in production. I run them using the Apex Test Execution page and then monitor the % through Developer Console... – Nathan Williams Mar 30 '15 at 21:53
  • 2
    I would open a case by salesforce for sure. It is pure coverage bug. – Sergej Utko Mar 30 '15 at 21:54
  • Thanks. I wasn't sure if I was missing some statement buried deep in their developer forums that you can't have more than 20 (for ex) testMethods per class before they start ignoring :) I'll file an issue shortly... – Nathan Williams Mar 30 '15 at 21:55
  • I don't think it's technically a bug. I've observed this in my org before. It seems to have a correlation with any classes that have been edited since last test run - our SFDC architect attempted to explain by saying that Salesforce attempts to "cache" test coverage by selectively examining classes. The only way to guarantee an accurate coverage representation is to run all tests. I would be willing to bet that if you tried your method 1 and method 2 and did exhaustive Run All Tests on both, the coverage would be the same – Brian Mansfield Mar 30 '15 at 21:58
  • Also, storing all results versus only aggregate results might affect the total. – sfdcfox Mar 31 '15 at 01:16
  • @sfdcfox can you elaborate? I store all results... Is there some benefit to only storing aggregate? The system itself says it should only be used sparingly... – Nathan Williams Mar 31 '15 at 07:39
  • 1
    That's good. The aggregate result mode is usually less accurate. – sfdcfox Mar 31 '15 at 15:00
  • I have seen many times that Salesforce has trouble dealing with tests that take a long time (more than a minute or so). The tests pass, but the code coverage does not reflect that. The solution is to break up the long test classes into smaller ones. – Sander de Jong Apr 14 '15 at 07:42
  • 2
    Have you tried Test > Clear Test Data in the dev console? I've noticed that it sometimes caches results, and they're reported incorrectly. – powlo Aug 26 '15 at 11:49
  • I logged a case with salesforce a while ago, related to code coverage issue and they suggested the following 1)Go to Apex test execution 2) clear test history 3) under options - check 'Disable Parallel Apex Testing' and uncheck 'Store Only Aggregated Code Coverage' 4) Compile all classes and then 4) Run all test – C0DEPirate Sep 14 '15 at 12:20
  • We had some similar testing issues in the past. This post might be helpful to you. http://salesforce.stackexchange.com/questions/70300/develop-console-code-coverage-issues-winter-15-release-production-vs-all-t – rodneyaz Sep 27 '15 at 16:45
  • 1
    Do the both classes have the same version? Did you solve this issue? – Patlatus Mar 08 '16 at 12:18
  • I am very OCD about keeping all my classes the same version now, and have not had this issue since. However, it did seem to be a short term salesforce issue post-release that eventually resolved itself. The most helpful thing during that time period was @powlo comment from Aug 26 '15 and NickSFDC from Sept 14 '15 – Nathan Williams Mar 08 '16 at 17:09

4 Answers4

1

Distilling the comments:

  1. Clear out all the test coverage and try running it all from scratch. There can be caching issues with previous test runs. See How to eliminate bad code coverage data for apex classes ?
  2. Avoid the “Store Only Aggregated Code Coverage” option for test runs unless your volume of tests requires it.
  3. Consider disabling parallel testing. Also accessible via the Apex Test Execution Options page.
  4. Compile all the classes before running the tests.
  5. Check that the API versions for the test class and the class being tested are the same.

Apex Test Execution Options

If those steps fail you, raise a support case with Salesforce.

See also:

Daniel Ballinger
  • 102,288
  • 39
  • 270
  • 594
0

That's really odd, although the test coverage % results are really just estimates. That might be what you're running into, where salesforce is calculating it differently because it's in 1 big class vs separate classes. Doesn't make a lot of sense to me, but if you're really concern you're best bet is to submit a support case. You might also want to try the developer boards.

Salesforce Wizard
  • 3,529
  • 4
  • 27
  • 32
0

It does happen sometimes. I guess the most likely reason is about test data which will be different between all tests and separate tests.

Allen Li
  • 860
  • 2
  • 8
  • 19
  • This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post – cropredy Apr 30 '16 at 17:15
0

I am not 100% sure but If your class has if else blocks it would cover either if or else even though you have data for both the conditions. This happens when your class is being called by trigger and you do not call it directly in the test class. And in such a case you are needed to do it in separate test methods.

yGade
  • 51
  • 3