Decision Mechanics

Insight. Applied.

Supporting data users

By

stress photo

A critical, but overlooked, area of data science is linking analytics to action. Decision-makers must

  • Understand the information being presented to them
  • Have confidence in the analysis
  • Be able to place information in the context of the decisions they are required to take
  • Understand how to apply the information to make better, more informed, decisions

Too much focus has been placed on the needs of the analysts—and this is stifling the overall effectiveness of data science initiatives.

Brian Williams has produced an article—Creating an analytic culture through data interaction—discussing this topic. He has some interesting things to say.

He defines the problem as follows.

The complexity of information environments is challenging most organizations. Not all decision makers are ready to become more analytic, and as we focus on visualization, data mining, and predictive analytics, decision makers will become even more reliant on analysts and may not perceive they have the abilities to keep up with the quants.

One area of data science I’ve found to be weak is visualization. Many data scientists assume that charts are visualization tools that can be presented to decision-makers. Wrong. Charts are tools for data scientists. Managers may not have the time and/or the skills to draw conclusions from a chart.

The article has this to say on visualization

When analysts design visuals without user interaction in mind, their charts and infographics simply are a static sharing of their own analysis. […] Design needs to be sufficient enough to be a starting point for a decision maker that invites interaction.

Tooling clearly has a major part to play in bringing visuals to life.

Managers who are clued-up about data science—and its strengths and limitations—will be able to direct analysts to produce more actionable insights. There’s a constant cry for this sort of guidance from data scientists who feel they have the tools, but not the questions.

If we are to realize the substantial potential of data-driven decision-making we need to develop suppliers and consumers in lockstep. The current focus on supply-side tooling and training will result in disillusionment and throwing the baby out with the bathwater.

Visualizing election results

By

US flag in the shape of a map of the US

The New York Times published an article today on how it has mapped election results over the years. It illustrates the challenges of trying to present complex information succinctly to a lay, and possibly hostile, audience.

As they note, simply shading the states of the US based on the party that it voted for makes the country look decidedly Republican.

A timely reminder, if we needed it, of the challenges data scientists face in their goal of providing objective, unbiased information.

Flag image by DrRandomFactor.

Tufte in R

By

minimal line plot using ggplot2

Lukasz Piwek maintains a resource that shows how the visualization practices developed by Edward Tufte can be replicated in R.

His neat Tufte in R site currently contains examples of the following visualizations

  • minimal line plot
  • range-frame (or quartile-frame) scatterplot
  • dot-dash (or rug) scatterplot
  • minimal boxplot
  • minimal barchart
  • slopegraph
  • sparklines
  • stem-and-leaf display

It’s a living resource, so he’s adding to the collection over time.

Example code is given for R base graphics, lattice and ggplot2.

What Nathan Yau uses to visualize data

By

Nathan Yau of FlowingData has published an up-to-date list of what he uses to turn raw data into his impressive visualizations.

What’s always striking about the tools he uses is that there’s no “quick fix”. He utilizes a range of industry standard data manipulation (e.g. R) and graphic design (e.g. Adobe Illustrator) tools in his work.

I particularly liked his comments on processing and formatting data

Assuming I have the data I want (big assumption), this is a stage of tedium. Solutions typically reflect my state of I-want-this-to-be-done-already, and I use whatever tool is closest. I would use a hammer if I could.

Common Core echo chambers

By

common core twitter groups

Aankit Patel, a data consultant at NYC Department of Education, has published an interesting article on the Common Core State Standards (CCSS).

The CCSS are nationwide US academic standards for English and mathematics. They were introduced in response to:

  • American students’ declining scores on international tests
  • differing standards being applied by state education departments
  • the prospects for low-skilled workers in the modern workforce

It’s proved to be controversial, with much of the debate being driven by politics.

Patel analyzed 41,000 CCSS-related tweets from 21,000 users during November 2014. He ran a number of network analyses on the data to obtain insights into the nature of the debate.

One of the interesting results was the diagram shown above. This is based on a “who follows who” analysis. It shows liberal (red) and conservative (blue/green) groups fiercely debating in their own echo chambers. There’s very little cross-pollination in the debate. While the debate isn’t a simple one of liberal supporters against conservative opponents—liberals are more split—there’s seem to be little evidence that stakeholders are receptive to dissenting views.

If this analysis of tweets reflects the wider debate, it’s difficult to see anything constructive emerging from it.