1

I have three treatments, and I'm trying to calculate the ATTs. The literature in my field seems to deal with this one of two ways. The first way involves conducting separate matching procedures for each treatment and then calculating the ATT. Each matching procedure only includes observations from the treatment group of interest and the controls. Observations from the other treatment groups are excluded. See option 1 below. This would then be repeated for Treatment B and Treatment C. A second approach involves doing a single matching procedure with a dataset that includes all treatments and then calculating the ATT. With this option, observations that received any of the treatments (A, B, or C) are part of the treatment group. Is one approach preferred over another?

Option 1

#create a data frame with only observations from Treatment A and Untreated Groups
dfA<-df
dfA$treatmentA<- ifelse(dfA$treat == "B", -9999,
                 ifelse(dfA$treat == "c", -9999, 1,0))
dfA$treatmentA[dfA$treatmentA==-9999] <-NA 
dfA<-dfA[complete.cases(dfA), ]

#Match
match1<- matchit(treatmentA ~ var1 + var2 + var3, method = "nearest", data=dfA)

#Calculate ATT with a Regression
olsA<-lm(outcome~treatmentA + var1 + var2 + var3, data=dfA_matched)

Option 2 A dummy variable (dummy treat) indicates whether the observation was treated (Treatment A, B, or C) or not. In the regression, the variable (treatgroups) indicates whether the observation received Treatment A, Treatment B, Treatment C, or No Treatment.

#Match
match2<- matchit(dummytreat ~ var1 + var2 + var3, method = "nearest", data=df)

#Calculate ATT with a Regression
ols<-lm(outcome~treatgroups + var1 + var2 + var3, data=df_matched)

I've tried both with my data. I get similar treatment effects for Groups B and C but not for Group A. It makes sense that the ATTs would not be the same since the matched datasets from Option 1 to Option 2 are different, but I'm not sure how to deal with the difference in ATTs for Group A.

BlauMond
  • 13
  • Welcome to the site BlauMond. Please define ATTs before using acronyms. – André.B Sep 18 '19 at 22:25
  • Thanks Andre. ATT is average treatment effect on the treated. ATT[a,0] = E(Y[a] − Y0 | T[a] = 1), where T[a] =1 if the observation received treatment A. I then repeat for each treatment group. ATT[b,0] = E(Y[b] − Y0 | T[b] = 1), where T[b] =1 if the observation received treatment B. ATT[c,0] = E(Y[c] − Y0 | T[c] = 1), where T[c] =1 if the observation received treatment C. Is this what you meant by define ATT? – BlauMond Sep 19 '19 at 10:22

2 Answers2

1

Option 1 is correct. To estimate three ATTs, you need three separate analysis. This is because, for treatment A, you need to estimate the counterfactual mean under treatment A for those who received treatment A using the (matched) controls. You then need to do the same with the other two treatments. There is no way to, in a single matching specification, match the controls to treatments A, B, and C simultaneously. The matched controls must resemble those either in treatment A, treatment B, or treatment C. The problem with the second method is that it involves finding a set of matched controls that resembles all those who received any treatment, but they won't resemble those in any specific treatment group.

Note that this could be avoided if you were to choose the control group as the focal group and estimate the three ATCs instead of three ATTs. In this case, you would create matched sets from each of the three treatment groups to resemble the control group. Each effect would be interpretable as the effect of the corresponding treatment for those who received control. See my answer here for a description of the method and R code to perform the matching and estimate the effects.

Noah
  • 33,180
  • 3
  • 47
  • 105
0

Based on the snippets of code, e.g. "ols<-lm(outcome~treatgroups + var1 + var2 + var3, data=df_matched)", it looks like you are calculating the main treatment effects for groups {A, B, C} conditional on the covariates {var1, var2, var3}. These are conditional effects, not ATT estimates.

However, that the two different approaches have yielded different conditional estimates for group A is informative. If the estimates for group A differ meaningfully, i.e. it's not just noise from changing the matched cohort, it suggests you have something interesting going on in group A.

One possibility is an interaction between A and one or more of var1, var2, var3. You would want to look for the interactions in the cohort of A+B+C matched to controls (your "df_matched"). You could also look for it in an unmatched cohort ("df") provided covariate overlap was decent. As used in your code, the matching is being used as a pruning or preprocessing tool to make the data good for estimating the conditional effects (Ho, Imai, King, Stuart 2007). If the covariate overlap is good to begin with, the matching isn't essential to estimating those conditional effects. Moreover, while the matching can help blunt the impacts of model misspecification, it doesn't eliminate them. If you are missing a key interaction in the model, your results are going to be sensitive to the distribution of the covariates in the cohort.

Another way to get a hint at what is different with group A is to look at how the covariate distributions of the two matched cohorts differ. The "dfA_matched" cohort is selecting a cohort well designed for estimating the conditional effect of A. It is ensuring good covariate overlap between A and the controls. The "df_matched" cohort is only ensuring good overlap for groups A+B+C combined and the controls. There may be poor covariate overlap for group A somewhere within that cohort.

Robert Alan Greevy Jr PhD
  • 1,227