Decision Mechanics

Insight. Applied.

Jupyter Notebooks—love ’em or hate ’em?

By

Jupyter Notebooks are popular with data scientists. Microsoft even offers a free, hosted, “no-install” service for Python, R and F#.

However, there are some downsides to notebooks—mostly to do with software engineering best practices.

Joel Grus gave a provocative talk at JupyterCon 2018 entitled “I Don’t Like Notebooks”. Yihui Xie then followed up with a response to Grus’ talk.

Both authors make a good case and have interesting points. As ever, the truth is that notebooks are good in some situations and not so good in others.

Personally, I use both. Notebooks for smaller, exploratory, data science projects and IDEs (Visual Studio Code, PyCharm and RStudio) for everything else.

Python tops programming language list

By

Python has topped the IEEE Spectrum list of top programming languages again this year—extending its lead in the process.

The sources used to compiled the list cover

contexts that include social chatter, open-source code production, and job postings.

Obviously that list of sources isn’t an accurate reflection of what developers are doing day-to-day in organisations. Any list of top programming languages that puts R (#7) above JavaScript (#8) clearly has some methodological challenges. My belief is that the list reflects the current buzz around data science.

However, interest in Python clearly remains high. As it does in R—#7 is impressive for a domain-specific language.

The 5 most downloaded R packages

By

DataCamp have published an article on the five R packages with the most (direct) downloads. This is based on their leaderboard.

Packages 3-5 are currently swapping positions. As I write this (18 November 2016) the top five are

  • dplyr
  • devtools
  • ggplot2
  • cluster
  • foreign

It’s notable that the list of the most popular packages is heavily weighted towards the manipulation and display of data. This is the bulk of the work done by data scientists.

The highest ranking analytical package is for performing cluster analysis. No surprise, really, as looking for groups in data is a very common requirement.

Sharing R code using R-Fiddle

By

If you want to share a snippet of R code with others—e.g. for teaching or to get help on Stack Overflow—consider using R-Fiddle.

While gists are good for basic code sharing, R-Fiddle allows others to execute the code in place. You can even embed the code together with a working R console in blog posts (as an iframe).

RStudio 1.0 released

By

RStudio have released version 1.0 of their eponymous R IDE. They are calling it their

…biggest [release] ever!

It certainly has a number of very significant features.

Integrated support for Spark

Spark and R are core tools for data scientists. While Spark has an R API, support for the machine learning libraries is lagging.

So, it’s great to hear that RStudio now has integrated support for Spark and the sparklyr package. sparklyr provides extensive access to Spark’s Machine Learning Library (MLlib) and, through the rsparkling extension package, access to H2O’s distributed machine learning algorithms.

RStudio can be used to manage connections to Spark and run R functions on data held in the cluster. Data is read and transformed using Hadley Wickham’s excellent dplyr data manipulation package.

R Notebooks

R Notebooks allow the creation of documents where computation can be interspersed with narrative. Code can be executed interactively and the document updated accordingly. Readers of an R Notebook can modify the code in-place, execute it and see the new output—e.g. an updated chart. This is a particularly powerful tool for teaching R and data science.

Code profiling

I’ve used the profvis package many times to rescue clients from an analysis tool that takes hours to run. profviz provides an interactive graphical display of where you R code is spending time or eating memory.

This has now been integrated into RStudio, so you can select a block of code, click a menu option and see a visual representation of your code’s performance characteristics.

What are you waiting for?

RStudio 1.0 is free and available now on Linux, OS X and Windows. Why are you still reading this? Go and download it.