Decision Mechanics

Insight. Applied.

Subjectivity in data science

By

An article recently published in Nature reinforces the fact that the real challenge in data science is not mastery of the technical tools, but the ability to understand and define the problem.

Researchers posed the question of whether the color of a soccer player’s skin is a factor in how many red cards (serious reprimands) he receives. Seems like a pretty straightforward analysis.

The authors sent a data set to 29 research teams and asked them to answer the question. 20 research teams found there was a difference. Nine said there was no relationship between skin color and number of red cards received. Of the 20 who found a difference, two reported that dark-skinned players were less likely to receive red cards.

So, what explains this wide variation in results? Bias? Incompetence? No. It’s down to things like choice of what data is important, selection of analysis methods, etc.

If you have the resources, it seems like the best thing to do is crowd-source your answers. Have multiple researchers do the analysis, compare/contrast the results, share the insights across the teams, redo the analyses and then accept the majority answer. Of course, if you can only afford to employ one team, you need to be aware that data science isn’t an exact science…

Analyzing an Isle of Man TT legend

By

The BBC has an article on using a sensor array to determine what makes 23-time Isle of Man TT winner John McGuinness so quick.

Motorcycle riders have been tackling the 38-mile street circuit for over a hundred years. As it’s run on (closed) public roads, it’s an incredibly dangerous race. Riders average 212kph (132mpg) round the course—often coming within inches of stone walls and buildings.

John McGuinness is one of the most successful riders ever to tackle the course and data analysis company EMC decided to find out why he’s so quick. They fitted 50 sensors to him and the bike—collecting data on speed, acceleration, lean angle, throttle use, braking, body position, heart rate, etc.

One key finding is that he’s as cool as a cucumber, recording a heart rate of a mere 120 beats per minute when travelling at over 300kph on country roads. That lets him conserve energy and concentrate more than less skilled riders.

Applying machine learning techniques to the data revealed that only 14 variables were influential in his performance. McGuinness was just a little bit better in most areas than other riders, but each of these small advantages added up to a significant difference in lap times.

F# for Scala Developers

By

Scala has gained prominence in the data science community due to the rapid growth of Spark. Functional languages are well-suited to data science work so, if you find Scala productive, you might be interested in looking at F#. There’s a project devoted to promoting F# as an effective data science language—FsLab.

Those with Scala experience who wish to dip a toe into the F# waters should have a look a Alfonso Garcia-Carot’s F# for Scala Developers presentation.

He even used F# to create the presentation. That’s dedication.

Spark 1.5.0 released

By

Spark 1.5.0 has now been released—and it’s a significant one for the data science community. Databricks, in their announcement blog post, state

Another major theme of this release is data science: Spark 1.5 ships several new machine learning algorithms and utilities, and extends Spark’s new R API.

Improvements of note include better coverage for the pipeline API and an MLlib API for SparkR.

Apple steps up recruitment of machine learning experts

By

Apple is looking to recruit another 86 artificial intelligence experts, according to an article on VentureBeat.

The recruitment drive is due to concerns that they are falling behind Google, Amazon, Facebook and Microsoft in the area of machine learning. Competitors seem to be stealing a march on Apple by developing services that can anticipate users’ requirements—services that rely on sophisticated machine learning capabilities.

One of the challenges facing Apple is their privacy policy. Machine learning is data hungry and researchers crave lots of high quality data. As Apple leaves a substantial proportion of data on users’ devices—rather than storing it in the cloud—they have less data to play with than their peers. Less data means less effective machine learning solutions and, critically, also makes Apple a less attractive employer for ambitious PhDs.