Calculating the privacy that Renyi ($\epsilon, \delta$) Differential Privacy satisfies

Question

I add differential privacy (DP) to my machine learning models by using PyTorch-DP. PyTorch-DP supplies me with the values: $\epsilon$ and $\delta $. I know that the $\epsilon$ tells us something about the probability $\rho$ that the privacy of individuals in the dataset (DP) is broken. So, a $\rho$ of 0.5 might correspond to an $\epsilon$ of 2 (depending on the statistics that are released).

So now, my question is: With the implementation that PyTorch-DP uses (Renyi DP calculated by using the Moments Accountant method), to what $\rho$ does the $\epsilon$ correspond to? If I do not know this, how can I say something "meaningful" about the level of privacy $\rho$ that $\epsilon$ satisfies?

Answer 1

I don't think your interpretation of $\varepsilon$ in DP is correct. You can find a decent explanation of this parameter in this crypto.SE question and links therein.

The best intuition I have for Rényi privacy is that it bounds the average of the privacy loss) $L(O)$: instead of being a worst-case property, like DP ("for all outputs $O$, $L(O)$ is lower than $\varepsilon$"), Rényi privacy is an average-case property ("when we average $L(O)$ across all possible outputs $O$, the result is lower than $\varepsilon$"). The $\alpha$ parameter determines which kind of average we're using: used: $\alpha=1$ bounds the arithmetic mean of $L(O)$, $\alpha=2$ bounds the arithmetic mean of $e^{L(O)}$, $\alpha=3$ bounds the quadratic mean of $e^L(O)$, etc.