• 4.0 Basic data operators
  • (Tidy) data frames, glimpse, rename and select, mutate, recode
  • Plain text, read_lines, stringr functions
  • JSON with jsonlite (or NDJSON)
  • XML/HTML trees with xml2
• 4.1 Imports and Exports
  • Files
  • Databases
  • Files from Outer Space: Downloading a Google Spreadsheet
  • Files from Outer Space: Web Scraping with httr and rvest (and xm2)
• 4.2. Reshapes and aggregates (N.B. things below are really case studies)
  • Split-Apply-Combine and tidiness
  • Reshaping: long and wide (always prefer long)
  • Aggregating (aggregate, group_by, summarise), slicing (apply, slice etc.)
  • Coercing to, and binding, data frames
    • possible example: Facebook data
• 4.3. Practice
  • Currently: Estimates of Congressional ideology
  • >> MOVED from 4.2. Visualizing the U.S. housing market by city (Case Schiller Index)
  • >> MOVED from 4.2. Visualizing U.S. homicide trends by weapon type

TODO:

• Include Twitter API example.

Notes:

• Complex data object structures should be covered in Section 02 'Objects'

cartography + sf: http://rgeomatic.hypotheses.org/1149
linemaps: https://rgeomatic.hypotheses.org/1156

• Keep an introductory part with simple choropleth maps in ggplot2
  • possible example
• Keep geocoding example with ggmap + Google Maps
  • possible example
• Also show Leaflet
  • include mention of leaflet.extras
  • include mention of leaflet.esri
• Practice:
  • full-fledged analysis (clustering + mapping)

The list above was way more detailed, but it seems that my hard work a few days ago on detailing this issue got lost in the ether…

In brief, also use many of the posts by

• https://ikashnitsky.github.io/
• https://callumgwtaylor.github.io/

Map projections:

https://xkcd.com/977/

Excellent multi-part tutorial by Bhaskar V. Karambelkar:

GeoSpatial Data Visualization in R

Some steps match those of Robin Lovelace:

https://github.com/Robinlovelace/Creating-maps-in-R

Bookmarks:

• https://pinboard.in/u:phnk/t:viz:maps/
• https://pinboard.in/u:phnk/t:stats:spatial/

Cite:

• http://www.afterthemap.info/
• http://www.radicalcartography.net/
• https://www.citylab.com/equity/2015/06/incredibly-detailed-map-europes-population-shifts/396497/

Cite, French:

• https://visionscarto.net/
• http://www.metropolitiques.eu/

• Thomas Rahlf, Data Visualization in R. 100 Examples (2017)

• Soc Sci Quant

• http://www.radstats.org.uk/

• http://www.penombre.org/ (French)

The text online references data/grades.csv, whereas the file is called grades.2012.csv in the data directory.

• Tutorial by Matt Dancho

https://www.mathjax.org/

The link to version 2.0 is now dead.

What license is this tutorial under?
I would like to reuse it for teaching purposes, and I am unclear if I can, e.g., leave out some of the examples in the forked version.

Thanks for clarification

Dear Matt Dancho:

I am running your Customer Churn code, but am getting the following error, and cannot fix it. Can you help me please? Thank you.

Run explain() on explainer

explanation <- lime::explain(

• x_test_tbl[1:10,],
• explainer = explainer,
• n_labels = 1,
• n_features = 4,
• kernel_width = 0.5
• )
  Error in dimnames<-.data.frame(*tmp*, value = list(n)) :
  invalid 'dimnames' given for data frame

This is one of the missing sections mentioned in #4.

Business analytics perspective:

• k-means

• PCA

• cosine similarity/dendogram/tree/network viz

• Use ggforce to send PCA results straight to ggplot2

See #17.

• 3.0 Functions, e.g.
  • Math
  • Data
  • Models – introduce formula notation
  • quickly mention plots, HTTP calls, whatever
• 3.1 Control flow
  • The Human Narrative: # comments
  • The Computer Narrative: %>% pipe operator
  • Conditions: if/else, else_if, mutate_if, when
  • Sanity Checks: stopifnot – quickly mention unit tests for packages
• 3.2 Iteration (with mentions of parallelization)
  • Loops: for, while – mention parallelization with foreach
  • Vectorization: sapply, lapply, mapply, map_* – parallelization with mcapply
  • Need for Speed? mention benchmarking
  • Reproducible Code:
    • Reproducibility = 10% code, 90% human documentation
    • Writing pseudo-makefiles for a data analysis folder
    • Makefiles + cron (with demo scraper)
• Practice
• Currently: Computing the Herfindhal-Hirschman Index
• Add: Examples of calls using Stan, Julia, Python, shell?
• Add: Game of Life example to show iteration and matrix computations (Petr Keil)

• Use bookdown
• Transfer wiki content to 'Extra Readings' section

Either way thanks for sharing this, easily the most interesting data analysis course I've found, great teaching!!

Let's assume that v1.0 was the Jan 2013 release, and that the Aug 2013 release was v1.1.

• v1.2 -- just run the existing code properly
• v1.3 -- start improvements
• …
• v2.0 -- new major 2017 release?

TODO_FIRST: determine whether this section should be

1. about modelling per se (in which case, show many models),
2. or about general model classes (linear, nonlinear, hierarchical/multilevel, temporal/spatial effects and SE clustering, bootstrapped, Bayesian), plus tips and tricks (e.g. ggfortify, Zelig)

I'm slowly drifting towards Option 2, covering only the basic modelling stuff, and citing examples of text models (topic models), network models (ERGM, SOAM), etc.

• 8.0. Linear models
  • Current example: Markus Gesmann's prediction of London Olympics 100m men's sprint results
• 8.1. Linear correlation
  • Visualizing linear relationships
  • Measuring linear correlations
  • Correlation matrixes
  • Scatterplot matrixes
• 8.2. Linear equations (changed title; also, not yet sub-sectioned)
  • Ordinary Least Squares (Legendre published the method of least squares in 1805.)
  • Results:
    • residuals
    • fitted values
  • Generalization, e.g.
    • to add dummies (show that)
    • or lagged values (leave it to Section on 'Time Series')
  • Presenting results:
    • Tables: texreg
    • Marginal FX plots (margins)
• 8.3. Advanced Modelling (leave anything to do with 'Times Series' or 'Networks')
  • Nonlinear equations
  • Corrected standard errors
    • Robust SEs (jackknife, sandwich), FE, RE
    • Bootstrapped SEs
  • Quick word on a few model 'classes'
    • Spatial / Gravity
    • Econometrics: 2SLS, DiD, Oaxaca decomposition
    • Lasso, regularization
    • Machine Learning, random forests, neural networks…
  • Bayesian models with Stan
    • Gelman's blog, e.g.
    • A few links from Pinboard bookmarks, e.g. http://metrumrg.com/opencourses.html

Note: Section 8.3. really should be a collection of examples.

References:

• Hastie, Tibshirani, Friedman, The Elements of Statistical Learning
• Shalizi, ADAEPoV

• Install tidyverse packages
• Use dplyr to manipulate tidy data frames

Teach the tidyverse first: http://varianceexplained.org/r/teach-tidyverse/ (David Robinson)

Not entirely sure that I want to drop loops. The main opponent is brackets.