Decision Mechanics

Insight. Applied.

Status of Spark MLlib wrappers in SparkR

By

Wrappers for Spark’s MLlib machine learning library in SparkR have been slow to arrive. However, the future looks bright.

The imminent 2.0 release will bring k-means support to SparkR and the 2.1 release is scheduled to include wrappers for the following machine learning stalwarts

  • Alternating Least Squares (ALS)
  • Decision Trees
  • Gaussian Mixture Models
  • Isotonic Regression
  • Latent Dirichlet Allocation (LDA)
  • Multilayer Perceptron Classifiers
  • Random Forests

The Mathematics of Machine Learning

By

Formulae with handwritten annotations

Wale Akinfaderin, a research scientist intern at IBM Research, has written an article on the mathematical background required to fully understand machine learning.

He includes a list of educational resources for those who wish to brush up in certain areas.

Wale correctly stresses that you don’t need this level of knowledge to start making use of machine learning. There are many packaged solutions. However, if you wish to master the topic, some mathematical prowess is inescapable.

Machine learning algorithm cheat sheet

By

The recent explosion of interest in machine learning has resulted in a profusion of algorithms. It can be difficult to know which one is most suited to your problem.

Recognizing this challenge Microsoft have produced a machine learning algorithm cheat sheet. It’s designed to allow you to choose between the algorithms available in Microsoft’s Azure Machine Learning Studio, but, as many of the algorithms are generic, it’s applicable to other machine learning toolkits.

By following a path from your general task, and desirable features of the model, you end up at a suggested algorithm. For example, if you are looking to predict values and need accuracy and fast training, then a decision forest regression is suggested.

Of course, selection of an appropriate machine learning algorithm is a non-trivial task. A cheat-sheet is no replacement for a trained data scientist. And, the cheat sheet has obvious weaknesses. For example, a neural network regression is classified as an accurate approach to predicting values, but with a long training time. Given that, why would I ever pick a neural network over a decision forest when faced with a regression problem?!

However, if you are new to machine learning, and are bewildered by the choices, it’s a good place to start.

ScaleR package now available as part of free Microsoft R Client

By

The ScaleR package provides functions for performing scalable and extremely high performance data management, analysis, and visualization in R. It was only available to those who had a Microsoft R Server license—until now.

With the introduction of the free Microsoft R Client for Windows tool you can now work with the full set of ScaleR functions without having to part with a cent.

Of course, there’s a catch—there are constraints. Specifically

[…] the data to be processed must fit in local memory, and processing is limited up to two threads for ScaleR functions.

However, this allows you to prototype your analyses using the free client and push them to SQL Server or Hadoop using Microsoft R Server when you need to scale.