Black Mirror’s “Nosedive” episode portrays a future society, frighteningly like our own, in which people rate each other as a consequence of all kinds of trivial social interactions. Your overall rating is public and determines your job prospects, housing options, social invitations, etc—causing people to obsess over improving them.

As in most public policy decisions you control behavior by tweaking the incentives.

This terrifies me—because I can see it happening. I’ve since been informed that Uber pretty much operates along similar lines.

Now, it has to be said, I’m not a huge fan of social media. And, I’ve railed against the pointless tyranny of personal ratings in the past. To say the least, the world portrayed in the show isn’t my kind of thing.

So, imagine my horror to read in the Wall Street Journal that

Beijing wants to give every citizen a score based on behavior such as spending habits, turnstile violations and filial piety, which can blacklist citizens from loans, jobs, air travel

My concern is that we know that data science is a bit of an art form. False positives appear all the time when profiling potential terrorists. Recommendation systems run the gamut from bloody obvious to downright bizarre. Many corporations can’t begin to make sense of their own data lakes. Basically, it’s a work in progress.

Yet, here we are…on the verge of disenfranchising people on the basis of scores that, I can guarantee you, will be fundamentally flawed.