Learning Tree published my article on using Principal Component Analysis to reduce the dimensionality of data.
Worst passwords of 2015
SplashData, a purveyor of password managers, has produced its annual list of the year’s worst passwords.
The top ten are
- 123456
- password
- 12345
- 12345678
- qwerty
- 123456789
- 1234
- baseball
- dragon
- football
I guess we should all be shocked at how poor these passwords are. However, there’s no breakdown of which sites these passwords came from.
If they are all from bank accounts then, yes—OMG! However, if the bulk of them are from “sign up to read this article” sites then, meh. If I create an account just to download a free PDF, I don’t care in the slightest if my account is hacked.
To draw conclusions about behavior we need to play close attention to context.
Don’t trust the polls
2016 is the year of the US presidential election. Prepare to be besieged by polls. The embarrassment of the 2015 UK parliamentary election predictions is a distant memory. We get to start over.
However, Mona Chalabi reminds us, via the Guardian’s Datablog, of the challenges facing pollsters. She lists five:
- The media are fallible. They follow fashion and make the news.
- Journalists are fallible. They are biased—just like the rest of us.
- Predicting the future is hard. Context shifts.
- It’s difficult to reach people. There’s no longer a household landline.
- People don’t want to be polled. The few who do probably have something in common.
That’s a pretty damning list—and it’s far from exhaustive.
Of course, if we don’t have poll results we’ll have to have news stories about the issues…and who’s got time for that?
Election polling errors blamed on bias

A report has concluded that the spectacular failure of pollsters to predict the result of the 2015 UK Parliamentary elections was largely due to
systematic over-representation of Labour voters and under-representation of Conservative voters
The report, compiled by a panel of academics and statisticians, was commissioned by the polling industry to determine why they had predicted a “photo-finish” in an election where Conservatives outpolled Labour by 36.9% to 30.4%—a crushing defeat for Labour that lead to the resignation of their leader.
Pollsters apparently used collection methods that were more likely to be used by young (Labour-leaning) voters than older (Conservative-leaning) voters. Frankly, not realising that online surveys are going to under-represent the over 70s is a shocking oversight.
While betting markets also under-estimated the extent of the win, they did better than the polling industry—without the expense.
Data science with Microsoft
Jan Mulkens recently published an article on Microsoft’s recent rush to enhance it’s data science offering. And, as he illustrates, they have been very busy this year.
He highlights a number of their flagship data science initiatives.
- Azure Machine Learning
- Power BI
- Cortana Analytics Suite
- Acquisition of Datazen & Revolution Analytics
- Integration of R in SQL Server
Other significant data science activities at Microsoft this year, in my opinion, include:
- Prajna
- The rise of F# as a data science platform
- The introduction of the Data Science Virtual Machine on Azure
I’m looking forward to what they come up with in 2016.