Number of significant digits to report

Question

Is there a more scientific way of determining the number of significant digits to report for a mean or a confidence interval in a situation which is fairly standard - e.g. first year class at college.

I have seen Number of significant figures to put in a table, Why don't we use significant digits and Number of significant figures in a chi square fit, but these don't seem to put their finger on the problem.

In my classes I try to explain to my students that it is a waste of ink to report 15 significant digits when they have such a wide standard error in their results - my gut feeling was that it should be rounded to about somewhere of the order of $0.25\sigma$. This is not too different from what is said by ASTM - Reporting Test Results referring to E29 where they say it should be between $0.05\sigma$ and $0.5\sigma$.

EDIT:

When I have a set of numbers like x below, how many digits should I use to print the mean and standard deviation?

set.seed(123)
x <- rnorm(30) # default mean=0, sd=1
# R defaults to 7 digits of precision options(digits=7)
mean(x) # -0.04710376 - not far off theoretical 0
sd(x) # 0.9810307 - not far from theoretical 1
sd(x)/sqrt(length(x)) # standard error of mean 0.1791109

QUESTION: Spell out in detail what the precision is (when there is a vector of double precision numbers) for mean and standard deviation in this and write a simple R pedagogical function which will print the mean and standard deviation to the significant number of digits that is reflected in the vector x.

Answer 1

The Guide to Uncertainty in Measurement (GUM) recommends that the uncertainty be reported with no more than 2 digits and that the result be reported with the number of significant digits needed to make it consistent with the uncertainty. See Section 7.2.2 below

http://www.bipm.org/utils/common/documents/jcgm/JCGM_100_2008_E.pdf

The following code was my attempt to implement this recommendation in R. Noe that R can be uncooperative with attempts to retain trailing zeros in output, even if they are significant.

gumr <- function(x.n,x.u) {
  z2 <- trunc(log10(x.u))+1
  z1 <- round(x.u/(10^z2),2)
  y1 <- round(x.n*10^(-z2),2)
  list(value=y1*10^z2,uncert=z1*10^z2)
}

x.val <- 8165.666
x.unc <- 338.9741
gumr(x.val,x.unc)

Answer 2

If you show the confidence interval as well as the value of the statistic, then there is no problem with giving as many significant figures as you wish, as in that case a large number of significant figures does not imply spurious precision as the confidence interval gives an indication of the likely actual precision (a credible interval would be better). It is then essentially a matter of making the table neat, concise and readable, so essentially there is unlikely to be a simple rule that suits all occasions.

Replicability is important in scientific studies, so ideally it should be possible to reproduce the results to any number of siginifcant figures (whether they are of practical significance or not). Rounding to a small number of significant figures could reduce confidence in a replication of a study as errors could be masked by the rounding of the results, so there is a possible downside to rounding in some circumstances.

Another reason not to round too far is that it can make it impossible for others to extend your study without actually repeating it. For instance I might publish a paper that compares various machine learning algorithms using the Friedman test, which depends on the rankings of the different algorithms on a set of benchmark datasets. If the statistics for individual classifiers on each dataset are given to a number of significant figures depending on their standard errors, this will undoubtedly create many apparent ties in the rankings. This means that (i) a reader/reviewer of the paper will be unable to replicate the Friedman test from the results given in the paper and (ii) someone else would then be unable to evaluate their algorithm on the benchmark datasets and use the Friedman test to put it into the context of the results from my study. Generally using sufficient signfiicant figures to avoid introducing spurious ties doesn't cause a problem for the reader.

Answer 3

Surely any decision, made objectively or subjectively, would strongly depend on what you are measuring, and how precise your instrument of measurement is. The latter is just one part of the observed variation, and not always easy to discern or find existing evidence for. Thus I strongly suspect there is no objective, universally-applicable decision. You just have to use your brain and make the best judgement in each situation.