Decision Mechanics

Insight. Applied.

  • Services
    • Decision analysis
    • Big data analysis
    • Software development
  • Articles
  • Blog
  • Privacy
  • Hire us

Accuracy vs precision

March 13, 2023 By editor

I recently conducted an inter-rater reliability study for a client. There was some confusion about what this measures.

Inter-rater reliability measures agreement. It’s a measure of precision, not accuracy. As anyone who’s been on social media knows, it’s possible for everyone to be in complete agreement, yet utterly wrong.

The following diagram summarises the difference between precision and accuracy.

Precision indicates whether the assessments are similar. If multiple people access the same service, and give the same ratings, your survey is precise. There isn’t a lot of variability in precise ratings.

However, it doesn’t mean they are accurate. Systemic bias leads to high precision, but low accuracy. Racism, sexism, domain specialism, etc. can cause biased, but precise, scores. Accuracy measures whether assessments are correct on average. It’s harder to gauge in complex domains, as it’s not always clear what is being assessed. However, it’s more important than precision.

We’d rather be generally correct than specifically wrong. Of course, we want ratings that are both precise and accurate. Inter-rater reliability only addresses the less important half of the problem.

Print Friendly, PDF & Email

Share this:

  • Email
  • Twitter
  • LinkedIn
  • Facebook

Filed Under: Decision science Tagged With: accuracy, inter-rater reliability, precision

Search

Subscribe to blog via e-mail

Subscribe via RSS

Recent posts

  • Accuracy vs precision
  • It’s not because we have insufficient data…
  • Large Language Models
  • Spreadsheet disasters
  • 10 ways to mislead with data visualization

Copyright © 2023 · Decision Mechanics Limited · info@decisionmechanics.com