Science News published a great article on the use and abuse of statistics this week.

“Odds Are, It’s Wrong”, by Tom Siegfried, highlights some of the problems associated with testing hypotheses using statistical methods. These problems are well known within the statistics community, with “hundreds” of papers having been written on the subject. As Siegfried pithily observes, “if you believe what you read in the scientific literature, you shouldn’t believe what you read in the scientific literature”.

Statistical analysis is often presented in support of decisions. It’s rarely challenged. Yes, it’s sometimes ignored if it contradicts the views of key stakeholders. But it’s rarely challenged. This, we would suggest, is a consequence of:

• statistical analyses being considered to be objective; and
• decision-makers not being confident enough to dig around in the details.

Combine the reluctance to challenge statistical analyses with their widespread abuse and you clearly have cause for concern.

The article points out that, even among scientists, statistical literacy leaves much to be desired. For example:

• Statisical significance (e.g. p < 0.05) is often presented as a "black or white" binary concept. However, the use of 0.05 (…or 0.01…or 0.001) is completely arbitrary. In fact, with a p value of 0.05, there's a 1 in 20 chance that the observed result is a fluke. Is that acceptable given the decision you have to make?
• Statistical significance at the 0.05 level is commonly equated to 95% certainty that the result could not have occurred by chance. This isn’t the case. You can’t draw conclusions about the likelihood of the hypothesis being correct based on its statistical significance. The correct interpretation, given a p value of 0.05, is that there is only a 5% chance of getting the observed result if no real effect is present.
• Studies also tend to equate statistical significance with practical significance. An example given in the article is that an expensive new drug may be statistically significantly better than an old one, but, if it only provides one new cure for every 1000 patients, that’s not of much practical use.

A final interesting observation made by Siegfried concerns random trials. Selecting groups at random provides no guarantee that they exhibit random traits with respect to the phenomenon of interest. Let’s say we walk out onto the street right now and select two groups of five professional strangers. How likely is it that both groups would express similar political views, for example? When discussing (as Siegfried does) drug trials, there are countless dimensions on which patients can vary.

Humorist Evan Esar was clearly onto something when he defined statistics as “the science of producing unreliable facts from reliable figures”…

With industry panels claiming that unless companies have an iPhone app they "don’t exist" it’s hardly surprising that senior managers are asking the question, "Does my organization need an iPhone app?"

Information is increasingly accessed via smartphones. That’s in no doubt. But does this mean that you need to rush out and build a smartphone version of your corporate accounting system?

There are clearly some situtations where the arguments for developing a smartphone app are compelling.

• If your customers are consuming your content in their leisure time, then it probably makes sense for you to provide them with a smartphone application. Sports scores, articles, event listings, products reviews, etc. are all likely to be increasingly accessed via mobile devices.
• If you’re selling lower priced, “impulse-purchase” items then you may benefit from developing a smartphone app. It’s unlikely that you’d need to allow customers to purchase a car or house over a smartphone (although you’d probably want to let them research such purchases via this medium – e.g. list cars on the lot and book a test-drive).
• If you are providing real-time services to customers who aren’t desk-bound, then it’s worth considering the development of a smartphone app. Providing medical professionals with access to patient data may fall under this category.

There’s no doubt that a wide range of organizations could (and will) benefit from the introduction of a smartphone app. However, this doesn’t mean that every application fits this mold.

Some tasks—like writing a lengthy report, buying a car, doing your company accounts, etc.—would seem unlikely to be well-supported via a “mobile experience”. These are better undertaken using a “traditional” desktop or web-based application.

In fact, the value of developing a smartphone app for a task may be inversely proportional to the importance, value or complexity of that task.

If your customers are likely to set blocks of time aside specifically to perform the tasks that require your products, then the chances are they will be able to arrange access to a more traditional computer (e.g. netbook, desktop) when performing those tasks. And, there’s no doubt that, with current technology, the smartphone experience still lags the traditional computer experience for most tasks.

The smartphone revolution is upon us. And, for many organizations, not having a smartphone app in place is nothing short of corporate suicide. However, that does not mean that all organizations have to dive in…yet.