Insight. Applied.
A statistical note published in the BMJ (formally the British Medical Journal) reminds us that
…trials that do not show a significant difference between the treatments being compared are often called "negative." This term wrongly implies that the study has shown that there is no difference, whereas usually all that has been shown is an absence of evidence of a difference. These are quite different statements.
Great quote on the use of hypothesis testing by @JPdeRuiter.
Reject a specific null, and then argue for an arbitrary alternative. It’s indeed pretty remarkable that so few people see how absurd this procedure is.
The Wall Street Journal ran an intriguing story last month about Netflix’s management overriding recommendations coming from the company’s algorithms.
Analysis showed that promotions for the comedy “Grace and Frankie” were more successful when they only featured one of the two stars of the show.
Apparently fearful of alienating one of their stars, Netflix’s management decided to include both in promotions—even though that would produce a sub-optimal response.
The subtitle of the article mentions
overriding the metrics
However, this isn’t how I see it.
Data science produces inputs to the decision-making process—not recommendations to be followed slavishly. Netflix’s management presumably considered all the information at their disposal and made a decision that they believed would maximise their long-term rewards.
This is as it should be…even at Netflix.
The formal analysis could have been extended to include information on the excluded star’s contract, longevity as an asset, propensity to be offended, etc.
Maybe game theory could have been applied…and some bright Netflix quant could have developed a “diva scale”. But, this would have complicated the analysis considerably and compromised its accuracy.
Looks like data and judgement might have been combined effectively in this case.
I tend to listen more to opinions when people have placed money on their espoused outcome—having “skin in the game”, as Nicholas Taleb calls it in his latest book.
However, this robotics developer seems to have gone one better. He literally has skin in the game.
Good scientists aren’t supposed to experiment on themselves, but I will feel better if the first people to use autonomous cars are those who built the guidance systems.
At least it’s more impressive than Test-Driven Development.
Two surveys run by the University of Helsinki found that the majority of respondents found it was morally acceptable—as long as you are single.
One of the major barriers to the wider use of robots (such as autonomous vehicles) is the willingness of society to mingle with them. Seems like that is becoming less and less of a problem.