9

I am studying disturbance caused by ship traffic to a small seabird. I observed focal animals for a set amount of time and record whether or not they fly from the water during the observation. This particular bird does not fly at high probabilities when not disturbed (about 10% of the time). Post hoc, I have appended the distance to the nearest ship to every observation (ships of interest had GPS locators recording a point every 5 sec).

I have plotted the cumulative distribution function for ALL observations and for observations where the bird flew from the water as a function of distance to nearest ship. As expected, the majority of the observations in which the bird flew were observed when the ship was close.

ecdf plot, fly, did not fly, all obs

Can I use the Kolmogorov-Smirnov test to test if there is a statistical difference in the distribution of the flight observations and the total observations? My thought is if these two distributions are different it would suggest that ship distance has an influence on flight. I worry as these distribution functions are not independent as the flight observations are a subset of the total observations.

Thoughts?

Having read a bit further on this site I think I can test the distribution of the observations in which flight occurred (F) against the distribution of the observations in which it did not (NF) as these are independent. If these distributions are the same F=NF, then we can assume that the distribution of (F) and (TOT = all observations) are the same as we know that the distribution of (F) is equal to itself and (F)+(T) = (TOT). Right?

UPDATE: 2/12/14

Following the suggestions from @Scortchi I investigated the relationship of the incidence of flight vs distance to the nearest ship in a logistic regression framework. There was slight relationship present (negative slope) but the p-value was non-significant suggesting that the true slope could be zero. Based on the desciptive statistics (including the ecdf plots) I suspected that the effect of close ships was being drowned out by the many observations when the ship was not affecting behavior. I then used the R package segmented (http://cran.r-project.org/web/packages/segmented/segmented.pdf) to try and find a break-point in the model. The program found that breaking the data at 2.6 km from the ship and fitting two separate coefficients was better than the single coefficient model. The coefficient for the slope of the close ship approaches was negative and suggests that ships affect the flight response until about 2.6 km (p-value < 0.001). The coefficient for the second slope was slightly positive but the p-value was non-significant at the 0.05 alpha level (p-value = 0.11). So in summary, the segmented regression line was able to detect a threshold difference at which flight probability increases. The estimate for the probability of flight when the ship is further than 2.6 km is 0.11. Fittingly, I observed 79 birds in when no ships were even in the study bay (> 100 km away) and 9 of these individuals flew from the water during the observation (0.113 probability).

Thanks for all the suggestions. I hope this question along with the suggestions and answers helps others.

Community
  • 1
marcellt
  • 521
  • Why not logistic regression with flew / didn't fly as the response, & distance to nearest ship as the predictor? – Scortchi - Reinstate Monica Jan 29 '14 at 21:38
  • I have tried that. The intercept is significant but the slope is not. There is too much noise (flight) in the data at distances not biologically relevant. This is the natural flight that does occur in this species. I am trying to pinpoint the distance at which the ship affects flight so that I can group observations as "in the presence" or "in the absence" of ships. I think max distance or point at which the D statistic is calculated might get me closer to that goal. – marcellt Jan 29 '14 at 22:07
  • 2
    Bad idea - see here. And using the response variable to decide where to break a continuous variable is especially bad, biasing the coefficient estimates. Did you check for curvature in the relation of the logit of the response to the predictor? – Scortchi - Reinstate Monica Jan 29 '14 at 22:14
  • And what about birds that have already flown? If I understood right you're sampling just those birds still in the water. – Scortchi - Reinstate Monica Jan 29 '14 at 22:24
  • Birds were chosen at random and were observed from a land based observation point. Observations took place around the clock without regard to distance to ship. This variable was appended later in ArcMap from geospatial data collected on board the ship – marcellt Jan 29 '14 at 22:28
  • Each bird was observed for 10 minutes. Behavioral data was collected in order to calculate time activity budgets (t.a.b.). Three categories were defined 1. foraging, 2. flying, 3. resting. These were mutually exclusive and cumulatively inclusive. The study objective was to test if t.a.b were different in the presence vs the absence of ships. I am using the ecdf and KS test to define presence vs absence rather than pick an arbitrary distance (as other studies tend to do). Control vs treatment as defined by an observation experiment in a way. Do you still see fault in this? – marcellt Jan 29 '14 at 22:34
  • 1
    Nothing wrong with your test (F vs NF), or with determining the distance at which the ECDFs are furthest apart; but (a) using your data to define control & treatment groups, then testing/modelling the same data on those groups is invalid; (b) unless you really think there's a distance beyond which birds are unaffected & within which they're affected, that's a bad way to model it even for future data; (c) given the evident differences between the ECDFs, an appropriately specified logistic regression should be showing a significant effect of ship distance on flight probability. – Scortchi - Reinstate Monica Jan 29 '14 at 22:52
  • 1
    (i) You really don't want to compare (A vs A + B) because you lose independence of the two and any test will have to be adjusted for that (which will be tricky in many cases). Test A vs B and keep the independence. None of the standard tests will work otherwise. (ii) you want to avoid discretizing the predictors if it can be avoided. – Glen_b Jan 29 '14 at 23:13

1 Answers1

1

Interesting problem. I have two thoughts, one general and one about how to characterize your data...

First, with respect to comparing distributions I agree with @Glen_b and @Scortchi that you do not want to compare Fly vs All as shown in your chart (but nice idea to overlay the plot of the D statistic). Because you have a strong belief about where the distributions are likely to be different, and not just that they are different, you might want to consider comparing quantiles of the two distributions. There is a nice blog post on the subject which works through R code to develop the testing method. And there is an R package, WRS, which implements quantile-based testing methods.

Second, I'd consider dropping the use of a formal comparison test altogether and instead use Weight of Evidence (WOE). This approach is commonly used in industries that need decision frameworks dealing with different levels of risk across various predictors. Examples include insurance underwriting, credit evaluation, and clinical trials.

In your setting there is a baseline "risk" of flight (you said 10%), but the odds of flight seem to increase greatly in the presence of ships at certain distances. Using the WOE approach you can convey the change in odds of flight as a function of a ships distance, which is easy to understand for lay audiences (well, at least easier than understanding p-values associated with test statistics). Note that this is closely related to @Scortchi's suggestion to use logistic regression, but with WOE you are not trying to fit a regression model.

There is nice documentation on Statistica's website for applying the method, but the best introduction I have found is in a book Credit Scoring, Response Modeling, and Insurance Rating: A Practical Guide to Forecasting Consumer Behavior. If you search on the term "WOE" you'll find multiple sections discussing the idea, and section 5.1 walks through a complete example of calculating WOE (it's pretty easy) and evaluating the results for decision-making. Finally, note that there is a stackoverflow post on this topic, which is not very developed, but there is a link to PDF walking through another example in the context of SAS coding.

Community
  • 1
Josh Hemann
  • 3,414