Treatment length in differences-in-differences

Question

I have a question regarding how long the treatment has to be in a causal study? In my case I have a random shock that just occurred in one day for a treatment group (or just an event that exposed one group) and no exposure for a control group and I am planning to use a difference-in-difference (DD) approach. Is this sufficient enough for a causal study? People might ask if that shock was good enough to change behaviours. How do I overcome this problem in the experimental design if a longer time period is required. Any suggestions?

Answer 1

To my knowledge, I do not know of any scholarship suggesting the treatment/intervention must be a specific length. Treatments (e.g., social welfare policies, economic recessions, police crackdowns, etc.) come in all shapes and sizes. Some treatments commence at different times for different entities, varying greatly in duration or even intensity. Some entities may even experience multiple treatments/exposures over a long time horizon. The possibilities are manifold.

The question you must grapple with is how long does it take for your treatment/exposure to influence your outcome. Does theory suggest it takes a few days for individuals/entities to perceive the treatment? If theory suggests a shift in behavior is instantaneous, then you should be fine. However, a meaningful behavioral response may manifest with a lag. Suppose it takes 72-hours for the treatment to take effect. By then, you've moved into a 'treatment withdrawal' phase. Are effects observed in the epoch where treatment is removed the effect of a perceptual delay, or is some other unobserved shock outside of the event window also influencing behavior? Again, this is for you to justify to a reader.

In the comments you noted that you will be discarding all observations after the shock. Does it make sense? Yes. Do I advise it? No. Remember, the days beyond the intervention serve as another source of variation you could exploit. Once units are no longer exposed to the intervention, how does individual behavior change? You could most certainly investigate this in a difference-in-differences equation. Review my answer here for more information regarding how to set this up in practice.

To offer some perspective, I've evaluated crime policies at the district level in major American cities. In practice, I often find the policies quite short (i.e., 30–90 days). Moreover, I typically acquire monthly crime outcomes at the district level to facilitate the evaluation. But that leaves me with, at best, three post-treatment time periods. In this setting, I often try to exploit the timing of the intervention. Suppose I do discover program effects, I might ask myself when do they concentrate? In difference-in-differences applications, effects shouldn't concentrate in the days/weeks/months before the exposure of interest. Any strong non-zero effects in a period before the treatment could be interpreted as selection bias. On the other hand, it is possible to observe anticipatory effects in the periods close to the treatment's start date. Suppose a new law or policy is publicly announced weeks before its effective date. Behavior may start to change in anticipation of the event. This is also something for you to consider.

As a final recommendation, I would graphically inspect the group trends before and after the exposure. By "group" I mean separate trend lines for your treatment and control groups; they should be reasonably parallel before treatment. Do you observe a shift in trend for treated entities on the day of the shock? Is the shift persistent? And if so, does theory suggest behavioral change would manifest even beyond conclusion of the intervention? I do not know the particulars of your treatment, but be prepared to answer some of the foregoing concerns.