Choosing the right type of chart is an essential part of producing an effective data visualization. It's pointless adding bells and whistles to something that's fundamentally unsuited to the message you are trying to convey. The Financial Times Visual Journalism team have a Visual Vocabulary tool that helps them choose the correct chart for a story. It's basically a catalog of charts indexed by … [Read more...]
Data organization in spreadsheets
Karl Broman and Kara Woo offer some good advice on organizing data in spreadsheets. They advocate confining the use of spreadsheets to data entry and storage---moving calculations and visualizations to other tools. This certainly avoids some of the biggest problems with using spreadsheets. However, spreadsheets don't enforce any discipline. It's up to the user to be vigilant and do the right … [Read more...]
Do you really need machine learning?
I have a lot of sympathy for the view expressed in the following tweet Good CS expert says: Most firms that thinks they want advanced AI/ML really just need linear regression on cleaned-up data.— robin hanson (@robinhanson) November 28, 2016 Many organizations who dive into machine learning haven't even started to extract value from their data. I understand why they want to get on the … [Read more...]
UK’s National Health Service loses data—old skool
Over half a million UK National Health Service (NHS) records went missing between 2011 and 2016. When I first skimmed the headline, I was thinking "Another data breach...[yawn]." But, turns out it was all paper. 500,000 paper records went missing. A private mail direction company sent the records to a warehouse. They had either been incorrectly addressed or the associated patients had moved … [Read more...]
Personal rating dystopias
Black Mirror's "Nosedive" episode portrays a future society, frighteningly like our own, in which people rate each other as a consequence of all kinds of trivial social interactions. Your overall rating is public and determines your job prospects, housing options, social invitations, etc---causing people to obsess over improving them. As in most public policy decisions you control behavior by … [Read more...]