2

I am trying to carry out a Monte Carlo simulation that results in N pairings of N p-values from Student's t-test and N values of Cohen's d, such that the resulting vectors of p-values and d-values can be joined in a data.frame I can subset to create a data frame of d-values that are paired with p-values that are < .05. I am trying to illustrate the "winners curse" described by Button et al. (2013, ). Presently, I get a vector of p-values and a vector of d-values, but they are not paired. Plainly, I am missing at least one step.

Button, K., Ioannidis, J., Mokrysz, C. et al. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nat Rev Neurosci 14, 365–376. https://doi.org/10.1038/nrn3475

Here is the code I have used.

set.seed=1
library(MBESS); options(scipen=999)
pv = c()
cd = c()

for (i in 1:10^3)
x<-rnorm(20,20.45,1.5)
y<-rnorm(20,20,1.5)
pv[i]<-as.vector(t.test(x , y,var.equal=T)$p.val)
cd[i]<-smd(x, y)
cor(pv,cd)  # r is essentially zero whereas it should be strongly negative
Eoin
  • 8,997
Seth Lindstromberg
  • 21

1 Answers1

1

There were several syntax errors in your R code. This works as expected with corrected code.

set.seed(1) # Changed from set.seed = 1
library(MBESS); options(scipen=999)
pv = c()
cd = c()

for (i in 1:10^3){ # Missing parentheses
  x<-rnorm(20,20.45,1.5)
  y<-rnorm(20,20,1.5)
  pv[i]<-as.vector(t.test(x , y,var.equal=T)$p.val)
  cd[i]<-smd(x, y)
} # Missing parentheses
cor(pv,cd)  # Correct result
Eoin
  • 8,997