Creating a scoring model when different variables have higher or lower as better

Question

I created a scoring model based on 4 metrics. Currently I have them weighted as such: 50%, 20%, 20% and 10%. After calculating and ranking I realized this may not work. The issue I'm facing is that for metric 1 higher is better and metrics 2-4 lower is better. Any suggestions on a better way to go about calculating this?

Answer 1

This is a common problem when creating a rating system. Typically, the first step is to get all component variables onto the same scale. In some contexts, there is an a-priori reason to think the scales are already commensurate, for example, if they are all likert-type items using the same responses (say, $1$ - $7$, anchored by "strongly disagree" to "strongly agree"). In other cases, you cannot assume this, so you can standardize all the variables by turning them into $z$-scores. At that point, we run into the problem you have: the relationships are not all going in the same direction. You need to reverse score some of the variables so that they are all pointing in the same direction. How you do this depends on how your variables are constructed, but for example, if you had $1$ - $7$ likert items, you would subtract those you need to reverse score from $8$ (that turns $1$s into $7$s, e.g.). On the other hand, if you have $z$-scores, you can multiply them by $-1$, etc.