Matchit - Distance between 2 Propensity scores for findings specific pairs

Question

I'm trying to find "spare" controls in case of drop-out in my primary control. I took my primary control sample (n=29), and ran Optimal match for find secondary match sample (N=350 optional controls)

s <- matchit(
    Primary ~ Retreat_Days + Regualr_Practice_Total_Hours + Group_age,
    data = secondary_match,
    method = "optimal",
    distance = "glm",
    ratio = 1)

I'm not sure it's the best way to find match at the subject/pairs levels. I don't care here about balance at the group level.

1. Does the distance column is the propensity score?
2. the distance column is the same when I'm running "nearest", but the subclasess/pairs are different and seemed better in "nearest" at the subject level, there is advantage to use nearest in my case of looking of specific spair control for specific primary control who droped out?
3. Minimal difference between propensity scores is the ideal match always?
4. I need to find replacement for one of the primary controls, and then maybe also the "spair" one will drop out, and then I'll have to find a third option (and fourth and so on..)- what's the best method, and "how low" I can go? Can I define a rule by a specific maximal difference between 2 propensity scores? I'm looking for a conventional method that I can also report in a paper - but I couldn't find an example for it.

Thanks

Answer 1

1. The distance row in the summary() output corresponds to the propensity score. This terminology is a bit confusing, but I explain it in this post. The average difference between pairs is in the Std. pair dist. column.

2. This is consistent with previous research finding that there is not a big difference in bias between using nearest neighbor and optimal matching because they tend to use the same control units as matches, though not necessarily matched to the same treated units. I'm not sure how you're judging which method yields better pairs, but it's not uncommon for the default use of method = "optimal" to yield pairs with a higher average pairwise propensity score difference than method = "nearest". This is because optimal matching relies on a tolerance value; the lower the tolerance, the closer the resulting pairs will be (but the longer the matching will take). Set tols = 1e-7 when using method = "optimal" and the pairwise difference should be smaller.

3. No, the ideal match is covariate balance if you are trying to form similar groups. Closer propensity scores facilitate finding balanced samples, but for a given pair of units, you would not expect the units to be similar with respect to any given covariate even if they had very similar propensity scores. In fact, this is exactly the reason King and Nielsen (2019) recommend against using propensity scores for matching. I discuss this finding a bit here. If you want pairs that are matched closely on specific covariates, you need to match directly on them, e.g., using the Mahalanobis distance or using genetic matching.

4. It sounds like what you want is a caliper, which can be specified using the caliper argument. When you set a caliper to be, e.g., X, units can only be paired if the difference between their propensity scores is less than X. Any units that don't have a match due to the caliper are dropped. This is a very common method of matching. You can also place calipers on the covariates directly rather than on the propensity scores. All this is explained in the matchit() documentation, and there is an example of using calipers directly on covariate at ?method_nearest. Note that calipers are not allowed with optimal matching.