5

When SCRUM teams develop components individually, components which will be integrated to form a solution, my understanding is that there are 3 levels of testing.

  • Unit testing: Test how the programmer sees the system
  • Component testing: Test how user sees this component. So mock all its dependencies.
  • Integration testing: Test how the user sees this solution with all the integrated components

Is this a valid separation of testing concerns? Recently a team member questioned the need for component testing, if the component that I intend to mock is a library and not a service dependency. Is this a fair request?

Krishter
  • 153
  • 3

2 Answers2

6

Point of testing units and components before whole system is to localize bugs BEFORE they damaged some unrelated part of the system, causing confusing red herring error. If you mock the surroundings of a component, you eliminated moving parts - and localized the bug.

Of course you need to weight the effort to mock: because it is code, it will have bugs and will have to be maintained too, if parts of the system it mocks are in flux.

As always in engineering, it is about finding the right compromise between competing interests within limits you are facing.

Decision to mock such externality (or not) depends on many factors:

  • Does external module adds time delay? If so, mocking it with canned answer might improve speed of unit testing. If not, and module has strong unit tests, you may want to keep real thing.
  • Does calling external interface changes global status, like database changes? Will you keep the changes, or restore database (which takes time)?
  • Was your system designed to mock certain external interface? If not, is it worth to add complexity, possibly breaking working code?
  • If interface is to outside vendor who is very cooperative and has special QA subsystem for you to test, even keeping such externality might make sense, even with delay. Or crudely mock such service for developers, and use real thing for higher tests.

You need to pick your battles, use limited resources wisely. Where do you have more spare resources:

  • developers, to create advanced mock,
  • sysadmin, to create complicated multilayer system, with different levels of mocking, and deploying right code in right layer
  • QA team time and focus, manually analyzing logs and tracebacks for errors
  • QA developing custom tools fit your exact process (we use this approach a lot, low-hanging fruits because all is so custom)
  • money to buy more servers to run continuous integration tests in close-to-real-production environment, even if slower (and with risk of red herring error message). Servers are cheap (compared with cost of sysadmin's time needed to administer complicated multilayer system) - it is single-time decision.

You will NOT be able to cover all bugs by single approach. Instead, think about using multi-layer defense like Swiss cheese with holes: one slice has holes in it, but if you place multiple layers of cheese, and every layer has holes in different area, together they will cover sandwich without holes.

Peter M. - stands for Monica
  • 12,316
  • 26
  • 43
  • Well put on developing mocks. May I ask, if the component that another team is developing is a software library, as a jar which my team will use, does it still make sense mocking the jar? I fail to see the difference between integration testing and component testing if I do not mock here. – Krishter Mar 10 '15 at 15:27
  • Depends how good unit tests are, and if you care about the delay calling it adds to your other tests. If little delay and no persistent changes, it might make more sense to invest development time in better unit tests and use real thing instead of mocking it. – Peter M. - stands for Monica Mar 10 '15 at 16:01
0

Usually component testing is before integration testing. I believe first doing component testing and then moving to integration testing will not only save you time but you will be able to find bugs quite early before the integration takes place.

Component testing is done with different things in mind and integration testing has a different purpose than component testing. These cannot be substituted for each other. They are both essential to be on the quality path.

Sachin
  • 324
  • 2
  • 8