I want to know how my ML model is performing by customer race/ethnicity. The issue is that we do not know our customer's race.
We, however, do know zipcode level demographic information. ie) A customer live in an area where X% of residents are A, Y% are B, etc ... What I want to know is if the model is more accurate for A than for B, etc ... with some confidence as well as model accuracy by race.
You could try some regression model. You did not tell us how you measure performance, but let $Z_i$ be your performance measure for zipcode $i$, let $j$ index the races, and $w_{ij}$ be population fraction of race $j$ in zipcode $i$. Note then that $$ \sum_j w_{ij} =1 $$ so the predictor variables $w_j=(w_{1j}, w_{2j}, w_{3j}, \dotsc )^T$ will be linearly dependent. But we can solve that simply by omitting the intercept from the model, so $$ z_i = \sum_j \beta_j w_{ij} + \epsilon_i $$ is your regression model. Then the estimated coefficients should be estimates of performance by race, and from the model you can get confidence intervals for them.
Note that this model simply represent the zipcode performance scores as a weighted mean of the race performance scores $\beta_j$ with weights the population fractions $w_{ij}$.