'Large data' refers to situations where the number of observations (data points) is so large that it necessitates changes in the way the data analyst thinks about or conducts the analysis. (Not to be confused with 'high dimensionality'.)
A sufficiently large number of observations for an analysis may require changes in the way the data analysis proceeds, or in the way it is understood.
Some examples where the process may need to be adapted are:
- special strategies may be required if there are more data than can fit in a computer's memory,
- the analyst may need to pay attention to the computational efficiency of different optimization algorithms,
- consideration needs to be given to how to effectively visualize the data, if standard plots (e.g., a scatterplot) would just display a large black spot due to overlapping points.
A common example of a case where analysts conceptualize an aspect of the process differently concerns statistical significance. With sufficient data, any difference, no matter how trivial, will be 'significant'. This fact leads many analysts to view findings of significance differently than when smaller data sets are available.
