I recently conducted an inter-rater reliability study for a client. There was some confusion about what this measures. Inter-rater reliability measures agreement. It's a measure of precision, not accuracy. As anyone who's been on social media knows, it's possible for everyone to be in complete agreement, yet utterly wrong. The following diagram summarises the difference between precision and … [Read more...]
The Forer effect
Psychologist Bertram Forer gave 39 of his students a personality test. Each was given a personalised profile based on their answers. Except they were all given the same profile...taken from an astrology book. When asked to assess how well it described them, on a 0-5 scale, the students reported an average of 4.3. Wikipedia describes the Forer effect as ...a common psychological phenomenon … [Read more...]
Human brains hate probability
There was a great article published last month discussing risk communication. Statistician Regina Nuzzo is quoted as saying Human brains hate probability, they hate ambiguity, they hate the uncertainty. We’re just not wired to deal with this sort of thing very well. Science journalist Tara Haelle explains that it's difficult to separate reporting from influence. She notes that covering polls … [Read more...]
Decision fatigue
This Economist has an article this week on the dangers of decision fatigue. Research suggests that people fall back into making "default" decisions when they are tired. Examples are cited from finance, law and medicine. One thing that isn't discussed is the obvious benefits of automated decision-making---computers don't suffer from exhaustion. The more we can have computers advise … [Read more...]
Why is everything so complicated?
Ever expressed frustration that stuff is unnecessarily complicated? If so, a new paper published in Nature might have an explanation. Apparently, when solving problems, we have a natural inclination to add things. For instance, when asked to make a design symmetrical, people are more likely to do so by adding features, rather than removing them. You can see evidence of this in a range of technical … [Read more...]