0

I am currently running multilevel analyses in R to analyze data for a daily diary study where observations are nested within people. I am interested in identifying how much of the total variance in my variables of interest is at the between-person and within-person level. I am trying to determine this with the ICC(1) in R and the residual ICC (1-ICC), but I was wondering if anyone might know how to find the associated p-values for the ICC and residual ICC? I have tried using the icc() function in the performance package and the multilevel.icc() function in the misty package, but they don't seem to provide p-values in the output.

Thank you!

ccoul
  • 31
  • Hi, could you please elaborate on the model design? It will be very important when thinking about the question. FYI, p-values for ICCs are not commonplace. It may be possible to construct confidence intervals. (also, check out https://stats.stackexchange.com/questions/113577/interpreting-the-random-effect-in-a-mixed-effect-model?noredirect=1&lq=1) – pep Jun 02 '22 at 15:59
  • Hi, thank you so much! I'm essentially hoping to see if there is enough within-person variance in a certain variable (DB_results) to be considered statistically significant. The same participants filled out the same scale every day for 10 consecutive days. I ran the following code:

    DB_results <- lmer(DB_score~1+(1|MID), data=analysisData, REML=FALSE)
    icc_DB <- performance::icc(DB_results)

    And then, to look at the residual ICC, I did the following: icc_DB <- 0.756 residual_icc_DB <- 1 - icc_DB

     – ccoul Jun 02 '22 at 16:19
  • The outcome variable is DB_score, the model is DB_results, and the variable that corresponds to the ICC is MID. The model doesn't have any between-subjects factors, so the only specified variability would be at the within-person level. If you're interested in the reliability of the diary measures, maybe start with https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4913118/. I'd also suggest taking a look at https://www.tandfonline.com/doi/pdf/10.1080/15427600902911189 and https://www.sciencedirect.com/science/article/pii/S009265661630068X – pep Jun 02 '22 at 16:29

0 Answers0