ANOVA Puzzlement: huge Significance Shifts with Variable Type Conversions

Question

My rowdata in R, as revealed by str() shows that: Variable A is of type character (chr), and variable B is of type numeric.

Variable A has 4 levels representing solutions A, B, C, and D. I used as.factor() to convert A into a factor.

Variable B has 3 levels representing different concentrations of 25, 50, and 75 mmol/L, I used as.factor() to convert B to factor.

The data is normally distributed with homogeneity of variances, I therefore used 2-way ANOVA analysis to assess the influences of variable AB on y (independent).

However, there are 2 strange results on Df and significant after converting types of factor:

there are the codes I used in R.

I first check if they are factor (False) and calculate with 2-way-anova, NO significant.

Secondly, I convert to factor and check again (True). same calculation again: p<0.05!! Df also changed.

is.factor(sheet10$Concentration) #False
is.factor(sheet10$Solution) #False
a1 = aov(AD1 ~ Solution * Concentration, data = sheet10)
summary(a1)

This has left me confused about which analysis approach to trust/use. May please help me to explain?

supplements on experiment and data:

• The experiment involved an adsorption test of 5 different samples (S1, S2, S3,S4, S5), each representing a different material, across 4 solutions of A, B, C, and D.

• each solution has 3 concentrations of 25, 50, and 75 mmol/l.

• replication 2, n=2. resulting in a total of 120 data points.

design of my experiment. each sample has the same design, therefore, there are 120 data points.

How I list my data and imported to R:

Answer 1

The first model, with only 1 df devoted to concentration, implicitly assumes that the outcome is linearly related to Concentration. That's how these functions interpret a numerical predictor, unless you specify otherwise. Furthermore, even though you didn't specify Solution as a factor before building that model, R will interpret a character-valued predictor as a factor if it can. It did.

A strict linear association between a continuous predictor and outcome is seldom going to hold. See Chapter 2 of Frank Harrell's Regression Modeling Strategies for flexible ways to evaluate associations of continuous predictors with outcome in the general case.

The second model, with Concentration treated as a factor, makes no assumptions about linearity between outcome and Concentration. Each Concentration level is allowed to have its own association with outcome and its own interaction with Solution. As it removes the highly restrictive linearity assumption of the first model, this second model makes a lot more sense. (For future reference, if there were more than 3 levels of Concentration you might consider a different approach from treating it as a multi-level factor, as discussed in the Harrell reference.)

A couple of warnings, however.

First, although the basic R aov() works OK with a perfectly balanced design like yours, it can lead to problems when there isn't balance. Learn about other ways to work with more general data sets.

Second, your models don't take into account that there only seem to be 5 separate samples, each tested in duplicate under all the conditions. The multiple observations on the same sample are likely to be correlated in some way (e.g., much lower overall in S5), so your standard-error estimates will be incorrect unless you use a modeling method that accounts for the correlations (e.g., include sample as another fixed effect in your ANOVA, or as a random effect in a mixed model).

Third, it looks like your outcome levels are restricted to values under 100; perhaps they are something like a percentage absorption. In that case a simple ANOVA might not be the best choice, although sometimes it can work OK if the residuals are close enough to normally distributed.