0

I'm working on a paper where we trained a statistical model and tested it on a dataset. We achieved some accuracy score.

There's another study that has a different accuracy score but in the same domain. But is on a different test dataset. We need to communicate the comparison somehow.

How should we communicate a comparison between our study and the other study in the paper? Our study achieved higher performance than the other study.

I know it's not a totally perfect comparison but it still has some merit. Does anyone know have any tips on how to communicate this?

It doesn't make sense to say: "Our method is superior because we outperformed them on the test set".

Warrior213212
  • 1
  • Welcome to Cross Validated! What’s wrong with saying that you got an accuracy of $x$ and others working in the field have tended to get lower accuracy scores around $x-\epsilon?$ // If your rivals only published accuracy values, then that’s all you have unless you ask them for more information or repeat their work, but I would be remiss not to mention the issues with using accuracy. – Dave Jun 09 '22 at 23:46
  • @Dave Thanks for your comment. The issue is that they are two different test sets. While we carefully constructed our test set, it's tough to compare it to another test set that might be constructed a different way.

    So, I'm wondering whether we can confidently say "Our method is better".

     – Warrior213212 Jun 10 '22 at 00:05
  • We picked apples better than they picked oranges? I wouldn't say that because ti doesn't prove anything. Our accuracy was such and theirs was not that good using different data doesn't say much either. Can you measure noise in the two data sets, somehow? Perhaps you can repeat their methods using your data? – Carl Jun 10 '22 at 00:28
  • @Carl Thanks for your comment. Their dataset isn't publicly available. – Warrior213212 Jun 10 '22 at 19:56
  • Yes but you have your data set--don't you? – Carl Jun 11 '22 at 05:36

0 Answers0