Machine learning at scale on HDInsight using Microsoft R Server at Spark

Microsoft have published an article on how to conduct a decision tree analysis using Microsoft R Server and Spark on Azure HDInsight.

Using four 8-core 28Gb RAM (D4) worker nodes they were able to process 170 million rows (37GB) in around 5 minutes. This was 20% faster than using Spark’s own MLLib libraries—although there’s no comparison with Spark’s newer ML libraries.