Decision Mechanics

Insight. Applied.

The 5 most downloaded R packages

By

DataCamp have published an article on the five R packages with the most (direct) downloads. This is based on their leaderboard.

Packages 3-5 are currently swapping positions. As I write this (18 November 2016) the top five are

  • dplyr
  • devtools
  • ggplot2
  • cluster
  • foreign

It’s notable that the list of the most popular packages is heavily weighted towards the manipulation and display of data. This is the bulk of the work done by data scientists.

The highest ranking analytical package is for performing cluster analysis. No surprise, really, as looking for groups in data is a very common requirement.

Public data sources

By

ethernet cables

Data science requires data. Yep. Insightful.

Unless you work at a data-rich organization, data can be hard to obtain. You may want to try out a new technique or tool. Alternatively, you may need additional data to fuse with your own limited in-house data. In either case, Nathan Yau’s updated list of public data sources might help.

He lists sources for the following types of data

  • demographic
  • health
  • geographic
  • news
  • sports
  • general purpose

RStudio 1.0 released

By

RStudio have released version 1.0 of their eponymous R IDE. They are calling it their

…biggest [release] ever!

It certainly has a number of very significant features.

Integrated support for Spark

Spark and R are core tools for data scientists. While Spark has an R API, support for the machine learning libraries is lagging.

So, it’s great to hear that RStudio now has integrated support for Spark and the sparklyr package. sparklyr provides extensive access to Spark’s Machine Learning Library (MLlib) and, through the rsparkling extension package, access to H2O’s distributed machine learning algorithms.

RStudio can be used to manage connections to Spark and run R functions on data held in the cluster. Data is read and transformed using Hadley Wickham’s excellent dplyr data manipulation package.

R Notebooks

R Notebooks allow the creation of documents where computation can be interspersed with narrative. Code can be executed interactively and the document updated accordingly. Readers of an R Notebook can modify the code in-place, execute it and see the new output—e.g. an updated chart. This is a particularly powerful tool for teaching R and data science.

Code profiling

I’ve used the profvis package many times to rescue clients from an analysis tool that takes hours to run. profviz provides an interactive graphical display of where you R code is spending time or eating memory.

This has now been integrated into RStudio, so you can select a block of code, click a menu option and see a visual representation of your code’s performance characteristics.

What are you waiting for?

RStudio 1.0 is free and available now on Linux, OS X and Windows. Why are you still reading this? Go and download it.

Machine learning algorithm cheat sheet

By

The recent explosion of interest in machine learning has resulted in a profusion of algorithms. It can be difficult to know which one is most suited to your problem.

Recognizing this challenge Microsoft have produced a machine learning algorithm cheat sheet. It’s designed to allow you to choose between the algorithms available in Microsoft’s Azure Machine Learning Studio, but, as many of the algorithms are generic, it’s applicable to other machine learning toolkits.

By following a path from your general task, and desirable features of the model, you end up at a suggested algorithm. For example, if you are looking to predict values and need accuracy and fast training, then a decision forest regression is suggested.

Of course, selection of an appropriate machine learning algorithm is a non-trivial task. A cheat-sheet is no replacement for a trained data scientist. And, the cheat sheet has obvious weaknesses. For example, a neural network regression is classified as an accurate approach to predicting values, but with a long training time. Given that, why would I ever pick a neural network over a decision forest when faced with a regression problem?!

However, if you are new to machine learning, and are bewildered by the choices, it’s a good place to start.