Decision Mechanics

Insight. Applied.

Adults struggle to solve puzzle kids solve with ease

By

A National Geographic puzzle (described below) that 80% of children can solve flummoxes most adults.

As we get older we develop a whole range of skills that allow us to operate more efficiently. The problem is that these optimizations result in blind spots and lowering of creativity.

Now, don’t get me wrong—the tradeoff is worthwhile. You really don’t want a six-year-old in charge of your data analysis strategy. Creativity is over emphasised in modern business writing. However, clearly we should try to eliminate some our harmless biases.

This is where decision and data science help. Mathematical models, formal decision making processes and real-world data can challenge our preconceptions. When confronted with a gut-feeling square peg that won’t go into the round hole of a formal model we are forced to confront our biases and re-frame our understanding of our environment.

Alternatively, we can angrily toss out the model and data for disagreeing with our cherished theories—but that’s politics for you.

The puzzle

The puzzle is to work out which way the bus shown at the top of the post is traveling.

Data science and statistics

By

Prolific R developer Hadley Wickham provided an interesting perspective on data science and statistics in a recent Priceonomics article.

There are definitely some academic statisticians who just don’t understand why what I do is statistics, but basically I think they are all wrong. What I do is fundamentally statistics. The fact that data science exists as a field is a colossal failure of statistics. To me, that is what statistics is all about. It is gaining insight from data using modelling and visualization. Data munging and manipulation is hard and statistics has just said that’s not our domain.

This insight is at the heart of why the only way to get good at data science is to do it. Obtaining and preparing data prior to analysis is the bulk of a data scientist’s work. But, it’s not a simple concept that you can tie up with a nice neat bow. It’s a messy, convoluted process involving

  • trial and error
  • multiple, incompatible tools
  • missing information
  • organizational silos
  • quality issues
  • etc

It’s very difficult to cover this kind of stuff in a book chapter, or a traditional lecture. It’s like trying to teach automotive maintenance without putting on overalls—all makes perfect sense until you attempt to change the pistons.

R Consortium

By

The R Consortium has just been announced – with Microsoft as a founding Platinum member. Google, HP and Oracle are Silver members.

According to their website, the consortium

is a group of businesses organized under an open source governance and foundation model to provide support to the R community, the R Foundation and groups and individuals, using, maintaining and distributing R software.

What does all this mean for R users? One goal is to make R more accessible. As a open source project, getting started can be a little daunting.

A related challenge is the sprawling number of packages. These are of variable quality and, in many cases, have overlapping functionality. There is also no consistency across them. All of this adds to R’s learning curve. The foundation will facilitate collaboration and communication within the community which should help to mitigate some of these problems.

Microsoft’s participation as a Platinum member reinforces their recent commitment to R. It follows their purchase of Revolution Analytics, inclusion of R in Azure Machine Learning and plans for in-database R in SQL Server 2016.

Commenting on the foundation, Joseph Sirosh, Corporate Vice President of Machine Learning at Microsoft, said

Our efforts to build R into more Microsoft products and services, combined with our contribution to the R Consortium as a Platinum Member, gives me confidence that we’re helping today’s data scientists and business leaders to drive innovation and advances in the field of data science with R.

What next? R.NET?

R at Microsoft BUILD conference

By

R was prominent at last month’s BUILD—Microsoft’s conference for developers.

Corporate Vice President for Machine Learning, Joseph Sirosh, said,

if there is a single language that you choose to learn today…let it be R

His keynote demonstrated a genomic data analysis using Revolution R Enterprise, recently acquired by Microsoft, running on a 1600 core Azure Hadoop cluster.

Microsoft are clearly taking this whole data science thing seriously.