What are some differences between confirmatory analysis and exploratory analysis?

Question

In confirmatory analysis do you basically just test hypotheses? Then in exploratory analysis you try to generate hypotheses? In general, I know that you can first do exploratory analysis to form hypotheses and then confirmatory analysis to test them. But can you first do confirmatory analysis and then do exploratory analysis?

score 2 · Accepted Answer · edited Jun 01 '16 at 11:20

First EDA will be done on the data set to understand the data & prepare the hypothesis, then confirmatory analysis is done. In EDA, most of the time we do visual analysis. Whereas in Confirmatory analysis we take probability models into consideration.

Comparison from here:

Confirmatory Analysis
- Inferential Statistics - Deductive Approach
  - Heavy reliance on probability models
  - Must accept untestable assumptions
  - Look for definite answers to specific questions
  - Emphasis on numerical calculations
  - Hypotheses determined at outset
  - Hypothesis tests and formal confidence interval estimation
- Advantages
  - Provide precise information in the right circumstances
  - Well-established theory and methods
- Disadvantages
  - Misleading impression of precision in less than ideal circumstances
  - Analysis driven by preconceived ideas
  - Difficult to notice unexpected results
Exploratory Analysis
- Descriptive Statistics - Inductive Approach
  - Look for flexible ways to examine data without preconceptions
  - Attempt to evaluate validity of assumptions
  - Heavy reliance on graphical displays
  - Let data suggest questions
  - Focus on indications and approximate error magnitudes
- Advantages
  - Flexible ways to generate hypotheses
  - More realistic statements of accuracy
  - Does not require more than data can support
  - Promotes deeper understanding of processes
  - Statistical learning
- Disadvantages
  - Usually does not provide definitive answers
  - Difficult to avoid optimistic bias produced by overfitting
  - Requires judgement and artistry - can't be cookbooked

For further reading read this.

Some precautions apply to the use of exploratory and confirmatory methods on the same data. I'm not sure EDA is usually done visually. Probability models can be useful in exploratory analysis (e.g., parallel analysis). I'll say more in an answer of my own if I can find the time to finish what I've started so far... — Nick Stauner, Mar 10 '14 at 06:50
@nick The answer is too descriptive and in all probability the question should address difference in exploratory factor analysis and confirmatory factor analysis. — , Jun 01 '16 at 12:07

score 1 · Answer 2 · answered Mar 10 '14 at 04:32

1

I don't think there is a set recipe for when to perform which. You have to use the tools required for the task, whether they are most useful for an exploratory analysis or testing hypotheses. It is likely you will begin with hypotheses (that's why you collected this data in the first place right?) and then test them. Your results may not be what you expect. Then you go back to exploring the data and generate new hypotheses. This is just how the scientific method works.

answered Mar 10 '14 at 04:32

bill_e

2,831
2
23
33

The task can begin with exploratory analysis. Data collection serves more than hypothesis testing, and the scientific method does not necessarily begin with a hypothesis...not that you necessarily intended to imply the opposite of these points. – Nick Stauner Mar 10 '14 at 06:46

What are some differences between confirmatory analysis and exploratory analysis?

2 Answers2