Decision Mechanics

Insight. Applied.

R analysis of frequency of swear words in Irish English by gender and age

By

profanity comparison charts by gender and age

Linguist Martin Schweinberger has used base R to perform a sociolinguistic analysis of swear word use in Irish English. The data analyzed is from the Irish component of the International Corpus of English.

I’m particularly tickled by the fact that the script contains a set of regular expressions that define swear words. For instance

search.pattern2 <- c (" [A|a]rse [s|d]{0 ,1} ")

Turns out that while young women swear less than young men—the latter seem to go through a period of more intense swearing—by the time people get beyond 34, there’s gender harmony in the use of profanity.

Real-world datasets for learning data science in R

By

R comes with a range datasets that can be used when learning the basics or trying out a new approach/package. mtcars and weather` are popular choices.

However, most of the common datasets are “toy” examples. They are great for practising basic techniques, but are useless when it comes to realistically simulating data science tasks.

The dslabs package provides datasets more suited to exploring data science techniques. It includes

  • admissions Gender bias among graduate school admissions to UC Berkeley (example of Simpson’s paradox)
  • divorce_margarine Divorce rate and margarine consumption data (illustrates spurious causation)
  • gapminder Health and income outcomes
  • heights Self-reported heights
  • murders US gun murders by state for 2010
  • na_example Randomly generated count data with missing values
  • outlier_example Randomly generated adult male heights (in feet) with outliers
  • polls_us_election_2016 Fivethirtyeight 2016 Poll Data
  • research_funding_rates Gender bias in research funding in the Netherlands
  • rfalling_object Simulate falling object data
  • take_poll Models results from taking a poll

Visual Vocabulary tool

By

Financial Times Visual Vocabulary tool

Choosing the right type of chart is an essential part of producing an effective data visualization. It’s pointless adding bells and whistles to something that’s fundamentally unsuited to the
message you are trying to convey.

The Financial Times Visual Journalism team have a Visual Vocabulary tool that helps them choose the correct chart for a story. It’s basically a catalog of charts indexed by the following data relationships

  • Deviation
  • Correlation
  • Change over time
  • Ranking
  • Distribution
  • Part to whole
  • Magnitude
  • Spatial
  • Flow

There’s also a PDF poster available.