Andrew Tait presented at the International Executive Seminar in Political Management and Governance hosted by George Washington University’s Graduate School of Political Management. The theme of the conference was "governing in turbulent times". Andrew's talk covered stakeholder management (including Confrontation Analysis) and the development of political strategy.

The conference was attended by senior politicians, civil servants and representatives from NGOs.

Districts where residents have the highest average educational achievement tend to be the smaller ones. Staying true to the tradition of blogging, we’re stating this without having conducted any research whatsoever. Still, we’re confident in the assertion.

Oh, and did we mention that districts where residents have the lowest average educational achievement tend to be the smaller ones. Yep. That’s right.

Eh? How does that work? Well, it’s a consequence of the higher variability of averages in smaller groups.

In a city like London, the average IQ will be close to the UK national average (probably about 100). Granted, for London, it may be a fraction higher, due to the likelihood that an international city is a draw for talent—but it won’t be much above the average.

However, imagine a picturesque hamlet where all nine residents have average IQs. A successful entrepreneur with a genuis-level IQ (say 160) decides to build his dream house there and move in. Suddenly the average IQ of the hamlet is now 106. If the entrepreneur had moved to London the average IQ of the city would have changed imperceptibly.

Relying on averages alone is misleading. We need also to consider sample size.

Let’s take another example. Imagine there’s a software development project that has three components—a database, a server application and an iPhone application. All three components are essential parts of the overall system.

Each component is assigned to a separate team and all are asked for estimates of how long their projects will take to complete. For the sake of simplicity, we’ll assume that they all say eight weeks—which we’ll interpret as being a 50% chance that the component will be completed within eight weeks. So, there’s a 50% chance that the entire project will be completed within eight weeks, right?

Well, that’s how it would probably be reported by many project managers, but it’s wrong. In fact, there’s only a 12.5% chance that the project will be completed with eight weeks. All three sub-projects have to go well for the project to deliver within eight weeks. So, it’s highly likely that the project will miss its deadline. It’s impossible to say by how much, as the sub-project estimates are single-point estimates, as opposed to (more realistic) distrubtions—but that’s a topic for another time.

Clearly, these are fairly simple examples. But this kind of “average” thinking is going on every day. And, as the importance of data as a decision-making tool grows, sloppy analysis is going to increasingly undermine the value of good data.

Big data is big news. Companies like Amazon, Google and major supermarkets are delivering new services and competitive advantage through analysing their massive datasets. The Economist reports that 30% of Amazon’s sales are through its “you may also like” recommendations.

Organizations everywhere want to make similar use of their own data. Articles and conferences on "predictive analytics" and "data science" are popping up everywhere. Even the New York Times has been promoting careers in statistics as "cool".

But, are we learning the right lessons from the successes of Amazon et al? Should decision scientists be focusing their efforts on helping organizations make sense of the data they have?

Massive datasets are a byproduct of something the showcase "big data" companies do that is, arguably, more important—they crowdsource data in real-time. Both crowdsourcing and real-time data collection are valuable. Together they are dynamite.

Making one small team within the organization responsible for collecting and "cleaning" the "official" data limits the volume of data that can be collected and increases the possibility of bias. Crowdsourcing mitigates those problems—and is cheaper.

The quality of data decays over time. Different industries experience different decay rates, but basing decisions on old data risks missing fundamental changes. Obviously, lagging data is almost useless when responding to a crisis. Real-time data collection means decisions can take the immediate situation into account.

Before you ask how you can draw insights from your existing databases, it may be advantageous to ask how you can build higher quality databases in the first place.

Tools to assist decision-makers are increasingly drawing on existing data and then combining it with the decision-makers’ assumptions and beliefs to predict outcomes and suggest action. These assumptions and beliefs are then often discarded once the decision has been made. However, these are real-time insights from the front-line. Capturing and storing them would allow decision-makers to tap into the current views of their peers—and monitor shifts in these views over time.

In addition to designing decision-making tools to produce insights, we also need to design them to collect insights. The latter activity may be the real innovation.

Privacy

Of course, there are potential privacy implications to be considered in crowdsourcing data. However, collecting data about an organization (as opposed to individuals), in the course of paid employment, and with full disclosure, raises few privacy issues. It is similar to writing and publishing a business report.