1

A very well validated score exists. This score predicts mortality after disease A when patients present from country W . The score is a 4 point score. If a subject presents from the community with disease A and his score is 1, mortality is x1. If a subject presents from the community with disease A and his score is 2, mortality is x2 etc...

I have a population with disease A. Some present from country W and some from country M. each subject has a clear classfication= mortality ie dead or alive.

The score when applied to those from country W seems to perform as expected for each category. meaning individuals with score 1 seems to have a mortality very close to x1 and those with score 2 close to x2 etc...

However for population M, that is not the case. score 1 mortality is far from x1 and score 2 far from x2 etc....

1- is there a statistic that proves that for my sample population (from country W) the concordance between the expected mortality at each score category and mortality is very good? 2- is there a statistic that shows that for the sample from country M this is not the case ?

In other words, for a validated score, how can I show that it expectedly performs well in the target population but not so in another.

enter image description here

Ali Mansour
  • 21

1 Answers1

-1

My suggestion, an exploratory analysis:

Step 1: Normalize the data. For example, from the population of scores by country, sample (with replacement) 12 scores from the same country and compute their average score. Repeat the process to generate a set of approximately normalized score values by country. Basis, the average of 12 random deviates, even from a Uniform distribution, will approximate a normal distribution (source, see discussion/graphs here).

Step 2: Apply Factor Analysis. I would expect if a country is distinct, it may be isolated into a separate factor. If so, create a dummy variable for that country (or countries) and proceed with your data modeling.

[EDIT] If you have data access issues, like only the categories percent distributions, try employing a Kolmogorov–Smirnov statistic against a selective reference grouping. See Wikipedia discussion here.

AJKOER
  • 2,180
  • thank you but really the point is that the score is a validated score and works fine as seen above for the population it is designed for ( country W). all i am looking for is a statistical metric that says hey the percentages observed in country M are not consistent with what the score predicts. does such a think exist? I don't have the raw data to model and explore – Ali Mansour May 24 '20 at 19:31
  • Try a Kolmogorov-Smirnov test, see https://en.wikipedia.org/wiki/Kolmogorov%E2%80%93Smirnov_test . – AJKOER May 25 '20 at 02:01