Compare p-values of equal sample sizes: What does it mean if a p-value of kaplan-meier log rank test is larger than the other?

Question

I'm analysing detection rates of carcasses for birds and mammals in either open fields or in forests, using kaplan-meier survival functions. The p-value is determined by log rank test.

I found that birds detected carcasses in open fields significantly faster than in forests (n = 34, chi^2 = 11.11, df = 1, p = 0.00086), whereas mammals detected carcasses in forests significantly faster than in open fields (n = 34, chi^2 = 8.01, df = 1, p = 0.0047).

My question is whether or not I can say anything about the fact that the relation of habitat (open/forest) is stronger for birds than it is for mammals, given the statistics mentioned above? And if so, what?

On the internet I read different opinions on this matter. Some say it does not make sense to compare two p-values between each other, some argue otherwise. However, everybody talks about comparing not necessarily the p-values, but the effect sizes. But how do I calculate the effect sizes for log rank tests?

Here (What sense does it make to compare p-values to each other?) it is stated that if the sample size is fixed (which is the case here: both are based on the same 34 carcasses), then p-values are monotonically related to Cohen's d. Does that mean that (because the sample sizes are equal) I can compare the p-values?

If I could compare the p-values with each other, what could I say about it? Is the relation between habitat and detection rates of birds 5.5 times stronger / more evident / clearer / ... than for mammals?

Answer 1

The page to which you linked refers to a situation with t-tests, which aren't appropriate for your survival data. Also, your effective sample size isn't just the number of carcasses but also involves the number of birds or mammals that you evaluated; it's the number of "events" (detections of carcasses) that's the critical sample size in survival analysis. So you can't count on a proportional relation between p-value and effect size, as on that page.

An effect size for a difference is the ratio of the point estimate of a difference to a variability estimate, like a standard deviation. For your situation, you could use mean survival (if the carcass is always found eventually) or a restricted mean survival (up to some end point in time) as a measure of speed of detection. Software for Kaplan-Meier studies can report those mean values and corresponding variances (squares of standard deviations). If the field and forest tests were done independently, the variance of the (restricted) mean-survival difference between field and forest for each Class (bird, mammal) is the sum of those variances. The square root of that variance of the difference provides the standard deviation of the difference. The ratio of the difference in (restricted) mean survival to its standard deviation would be a type of effect size.

A potentially more elegant way to proceed would be to move away from separate Kaplan-Meier analyses for each Class and combine all the data into a single survival regression model (semi-parametric Cox, or fully parametric regression like a Weibull or an accelerated-failure-time model), with Class, habitat (field/forest) and their interaction included as predictors. The result would provide estimates and covariances of coefficients that could be combined however you want to examine the significance of differences between the Classes, based on the formula for the variance of a weighted sum of variables.