0

I am currently running a randomized, placebo-controlled, cross-over design for a new type of intervention. This intervention is compared to a placebo and another more conventional type of treatment. Each participant is allocated in a randomized order to each of the three interventions (i.e., they receive all three interventions with wash-out periods of 1-week+), a repeated measures design.

The dependent variable was not normal distributed (Shapiro-Wilk), and I therefore decided to run a Friedman's test. Following testing of the specific variable, a significant difference in the distribution of ranks was found for the three treatments. Using SPSS' new legacy Friedman test, multiple comparisons with Bonferroni correction, however, showed no significant differences between the three pairs. After reading up on the procedure, it seems that the new legacy Friedman's test runs the test on all data as a whole, compared to multiple Wilcoxon signed-rank tests, which naturally would indicate different methods and therefore also results. I am, however, not entirely sure of the impact this would have. I am not willing to compromise the sound approach to statistical analysis, and start digging around with multiple different tests to see what give the best result. Therefore, I thought I would kindly ask for advice, and the reason why I get a significant distribution of ranks difference but nothing in the multiple comparisons, and how (if this is correct) to report it.

gung - Reinstate Monica
  • 145,122
D.B
  • 1
  • The DV need not be normal, only the residuals. Did you test the DV itself, or the residuals of a model? How much data do you have? How far from normality were they? – gung - Reinstate Monica Mar 05 '15 at 17:27
  • Thank you for your comment, gung.

    I am sorry for not stating this. I used the SAVE option for predicted and unstandardized/standardized residuals in the GLM (repeated measures), and ran Shapiro-Wilk test on the residuals. Only one of the residual variables showed normal distribution, whereas the remaining were severely non-normal distributed. I should mention that my variable is ordinal (0-10 scale).

    The analysis is performed on 13 different descriptors (DVs) with treatment (A, B, or C) as independent variable. I have included 30 participants for the study.

     – D.B Mar 05 '15 at 18:17
  • continued.. Each participant rate the 13 descriptors on a scale from 0-10 - yielding 13 descriptors per participant per session = 13303 = 1170 datapoints. – D.B Mar 05 '15 at 18:22
  • I haven't studied the algorithm doc on the multiple comparisons done by SPSS after Friedman. However you should take into account that Friedman is not comparable with Wilcoxon. – ttnphns Mar 05 '15 at 18:23
  • Thank you for the input, ttnphns. I am aware of the differences broadly speaking, however, I am still unaware of the impact that a significant overall Friedman's test on a dependent variable means, when the subsequent multiple comparisons give no indication of any significant differences among the distributions of ranks. I do believe, since I am testing differences between rating of each of the 13 DVs within each treatment paradigm (well-knowing that these are ranked in Friedman's test), that Friedman's test is justified, but I may be wrong. – D.B Mar 05 '15 at 18:26
  • Consider for example ANOVA and corresponding pairwise t-tests (done properly, with variance estimated from the enrire sample). If ANOVA is significant then it means that almost sure there be at least one t-test significant, not corrected by Bonferroni or Sidek or something alike. Is your situation similar to this? – ttnphns Mar 05 '15 at 18:39
  • Yes. I do believe the new legacy Friedman test in SPSS v. 22 automatically applies Bonferroni correction to the multiple comparisons. To test your hypothesized scenario, I ran a Wilcoxon signed-rank test on the descriptor (DV) that showed a significant different distribution of ranks between the three treatments. This showed a significant difference for treatment B, but not treatment A and C (even when applying BC). However, when testing the same DV in the Friedman's test Multiple comparison which automatically applies the Bonferroni correction, the same statistical significance is not seen. – D.B Mar 05 '15 at 18:43
  • I understand the problem in describing Friedman's as a direct extension of Wilcoxon, however, this is just to describe the confusion I am having with these tests atm. – D.B Mar 05 '15 at 18:49

0 Answers0