how to find contribution of feature in accuracy

Question

Given a set of features, how do I find what all features are contributing how much to the accuracy / prediction ?

Answer 1

When using one of the gold standards, the maximum likelihood, we have an ideal optimality criterion and associated proper accuracy scoring rule (deviance/log likelihood/measures derived from these such as $R^2$). This means that you have available

1. The combined influence of all predictors, from for example the model likelihood ratio $\chi^2$ statistic
2. Partial contributions of predictors such as partial likelihood ratio $\chi^2$ test statistics or approximations of them such as partial Wald tests (in the simplest case the square of a single regression coefficient divided by the square of its standard error). Partial $R^2$ can be derived from these.

One nice and fast way to present things while allowing for combination of multiple terms per predictor is the proportion of $\chi^2$ explained by each predictor. See Relative importance of variables in Cox regression and the comment about bootstrapping ranks of predictors, which exposes the true difficulty of this task.