10

In my free time, I am working on a small web based system which collects crash reports (but not other, non-crashing bug reports) that are sent from Delphi Windows applications.

For troubleshooting, users would love to have a data-mining feature to find relationships between hardware or operating system versions and the specific bug and/or crash.

As an example how this should work:

  • for every crash there is a report in the database, which has a fingerprint / hash code of the stack trace (call stack) at the moment of the crash to identify duplicates
  • the algorithm checks if all duplicates of a bug report also have some other common attributes, for example a missing service pack of the operating system
  • the analysis result lists all properties which the bug reports have in common

Let's assume these automatic bug reports contain all key information like the names of all processes which are currently running, file names, version information of loaded DLLs, etc.

How can I find correlations between repeated crashes and the environment? Are there specific algorithms or statistical methods that would help?

  • 2
    Correlation finding is a hefty task. Have you seen this tool at work ? It might help you find the proper way of organizing your data. –  Oct 20 '11 at 17:39
  • So you want to do the correlation computation in Delphi, too? Then take a look at this list of Delphi math functions: it covers a lot and also gives explanations and links to the libraries. –  Oct 20 '11 at 21:47
  • @DaveBall Thank you for the interesting link, however while the bug reports come from Delphi, the web service actually is implemented in Java - maybe I can use the algorithms as a starting point, or in an 'offline' stats tool. –  Oct 21 '11 at 04:42
  • Market basket analysis is a potential way forward if this question is still active. – Michelle Feb 05 '12 at 20:27
  • 1
    You might want to look into "discrete hazard models." This seems like a case where you want to regress failure events (crashes) on fixed characteristics (environmental variables) and the time since the last crash. If you have enough observations (crash reports), you will be able to add interactions between the environmental variables to the model. These will then point you in the direction of system conflicts. Unfortunately, I don't know how easy it is to implement such a model in Java. – jmtroos Apr 05 '12 at 14:46
  • 1
    The kinds of analyses you could conceivably employ depend on the size of the problem i.e. How many distinct kinds of crashes are there? How many possible attributes are there? – Sameer Jun 25 '12 at 04:34
  • @mjn what approach was finally taken? seems a interesting problem. please share for others benefit. – Srikar Appalaraju Jan 24 '15 at 18:11

1 Answers1

1

Could you sample your user's [non-crashed] machines for the same info as you get in a crash report? Because then you could use logistic regression to model those attributes (and interactions) to the probability of getting a crash.

zzk
  • 717