1

I have 1 dependent variable (DV) measured in binary (Exam, pass/fail), 3 independent variables (IV-1 is continuous (age, in years), IV-2 is binary (Country, Canada vs. US), and IV-3 is nominal, but repeated measure (Subject, Math/English/Science/History).

Participants are "repeated" for the IV-3 as they took all of the 4 subject exams, and each exam they received Pass or fail.

I want to test what predicts pass/fail the most. Is it the age, country, or subject? Mixed-design ANOVA will not work because my DV is nominal and will not be normally distributed. I would like to know which statistical model I can use to test this.

I am in Psychology. It would be great if I can run the analysis using SPSS or JASP, but if necessary, I can try R (but I prefer SPSS/JASP).

Monika Wat
  • 11
  • Are you familiar with logistic regression? That can be combined with a mixed model, treating individuals as random effects. Thousands of questions on this site discuss logistic regression in general, and nearly 200 discuss mixed models in logistic regression. This is implemented in most statistical software. – EdM Jul 17 '20 at 19:19

1 Answers1

0

You can use the Chi-2 test for $k$ independent samples to test whether your classifiers (your models for prediction of pass/fail) yield the same fractions of misclassified prediction. If $H_0$ is rejected and there are differences between the model performance estimates, you can use pairwise Chi-2 tests to test whether one prediction model with a specific variable subset performs significantly better than the others.

The general test design is a $k \times 2\; $ Chi-2 test where one outcome is correctly predicted cases and the other outcome is number of wrongly predicted cases (simple counts) in each of the $k$ rows. Each of these $k$ models use:

  1. Model 1: IV-1
  2. Model 2: IV-2
  3. Model 3: IV-3
  4. Model 4: IV-1 and IV-2
  5. Model 5: IV-1 and IV-3

… etc.

N. Model N: IV-1 and IV-2 and IV-3

In the second testing round you compare IV-1 with IV-2, IV-1 with IV-3, IV-2 with IV-3, IV-1 with IV-1 and IV-2, IV-1 and IV-2 with IV-1 and IV-3, etc. in a series of pairwise Chi-2 tests. Should give you the significant most well-performing variable subset when you order them based Chi-2 significance probabilities.

You have only $3$ prediction variables so a Bonferroni correction for multiple testing should not be necessary. With larger variable sets to begin with the number of pairwise comparisons grow fast, and Bonferroni will be recommended.

SPSS or another software package can certainly perform the Chi-2 testing for you.

I recommend as reference book: S. Siegel, N J. Castellan, Nonparametric statistics for the behavioral sciences, McGraw-Hill.

Match Maker EE
  • 1,871