Any book or paper on missing data analysis, e.g. Little and Rubin 1988, should work fine as a reference, if only to point out that what a variable represents has no bearing whatesoever on whether it can be imputed. That means there will be no discussion of this 'issue'.
Maybe think of it this way: if you would consider using a variable as a dependent variable in another study, then you could also think about imputing it. This is because an imputation is just a draw from the conditional distribution provided by such a model. People certainly do use regression models to predict age, e.g. from consumer choices.
What you advisor may be thinking of is that demographic variables like age are causally prior to most other variables. That's true, but irrelevant for imputation. For example, even if age causes attitudes, it's obviously statistically unproblematic to predict either attitudes from age or age from attitudes (or both). And since that's statistically unproblematic, it's equally unproblematic to use these models to impute in either direction too. Even if the goal of the final model is to make some causal inference, the goal of the imputation model is only to provide a reasonable range of possible values for missing data using whatever information is available. This is a different task.
From all this there are two consequences: first, imputations of a variable are ok whenever it would have been ok to run a regression model with that variable as the dependent. In that sense your intuition is correct. But second, the 'information available' that is summarised in the predictions of an imputation model is not well represented by one draw from it (one imputation). Doing that over-estimates how much information all those other variables actually had about the missing demographic (you're saying that the imputation is just as certain as if you had seen it for real, which is pretty strong). This is why people do multiple imputation by taking several draws that create separate filled in data sets, then analyse them separately and combine the results. If you're going to impute, you should really do this too.
As a side note: it might be that you made your imputation without an explicit model, e.g. by pulling the most recent value forward, or putting in a modal value or an average. If you did this, the reasoning about 'imputation models' above still holds. Your 'model' is just implicit in the procedure you used rather than explicit.
In summary: you're right - imputation is not causal inference but associational modeling, so any variables will work and there's nothing special about demographics. But you should be doing multiple imputation if you want to avoid the criticism of the next reader.