Getting (re)-acquainted with R, RStudio, data wrangling, ggplot2, and plotly

class: center, middle, inverse, title-slide

# Getting (re)-acquainted with R, RStudio, data wrangling, ggplot2, and plotly
### Carson Sievert
### Slides: <a href="https://bit.ly/plotcon17workshop" class="uri">https://bit.ly/plotcon17workshop</a> <br /> <br /> Slides released under <a href='https://github.com/cpsievert/talks/blob/gh-pages/LICENSE'>Creative Commons</a>

---

class: inverse, center
background-image: url(../your-turn.jpeg)
background-size: contain

## Your Turn

(1) Do you have the required software (i.e., could you run `devtools::install_github('cpsievert/plotcon17')` without error)?

(2) Share (with your neighbor) 3 things you're hoping to get from this workshop (share them with me, via [Slack](https://plotconrworkshop.slack.com) if you like!)

.footnote[
PS. remember this background image -- it means I want something from you!
]

---

### About me

* PhD in statistics from Iowa State (defended 4 months ago!)
* Maintainer of plotly's R package (for nearly 2 years!)
    * Before that: animint, LDAvis, pitchRx, rdom

### About the workshop

* Focus on things that are hard to learn from documentation alone.
* Today is mainly about core ideas (& lots of mapping examples).
* Tomorrow is a bit more advanced: animation, linked views, & shiny.
* I'm hoping to save time today/tomorrow for a Q&A session.
  * Feel free to stop me at any time.
  * Feel free to ask me about personal projects during down time.
  
.footnote[
Did anyone attend my talk on Tuesday?
]
  
---
## About the attendees

<https://plot.ly/ggplot2/geom_density/>

---
## About the attendees, another look

<https://plot.ly/r/parallel-coordinates-plot/>

---

## R wisdom

> Everything that **exists** is an **object**.
> 
> Everything that _happens_ is a _function call_. 
>
> -- John Chambers

---

## R wisdom

> Everything that **exists** is an **object**.
> 
> Everything that _happens_ is a _function call_. 
>
> -- John Chambers

---
## Universal truths

Use the `str()` function to inspect any object (`View()` in RStudio is a nice interactive alternative).

```r
str(mtcars)
#> 'data.frame':	32 obs. of  11 variables:
#>  $ mpg : num  21 21 22.8 21.4 18.7 18.1 14.3 24.4 22.8 19.2 ...
#>  $ cyl : num  6 6 4 6 8 6 8 4 4 6 ...
#>  $ disp: num  160 160 108 258 360 ...
#>  $ hp  : num  110 110 93 110 175 105 245 62 95 123 ...
#>  $ drat: num  3.9 3.9 3.85 3.08 3.15 2.76 3.21 3.69 3.92 3.92 ...
#>  $ wt  : num  2.62 2.88 2.32 3.21 3.44 ...
#>  $ qsec: num  16.5 17 18.6 19.4 17 ...
#>  $ vs  : num  0 0 1 1 0 1 0 1 1 1 ...
#>  $ am  : num  1 1 1 0 0 0 0 0 0 0 ...
#>  $ gear: num  4 4 4 3 3 3 3 4 4 4 ...
#>  $ carb: num  4 4 1 1 2 1 4 2 2 4 ...
```

Use `<-` to assign value(s) to a name

```r
nms <- names(mtcars)
nms
#>  [1] "mpg"  "cyl"  "disp" "hp"   "drat" "wt"   "qsec" "vs"   "am"   "gear"
#> [11] "carb"
```

---
## The pipe operator (`%>%`)

Takes object on LHS and inserts into function on RHS.

```r
library(magrittr)
mtcars %>% names()
#>  [1] "mpg"  "cyl"  "disp" "hp"   "drat" "wt"   "qsec" "vs"   "am"   "gear"
#> [11] "carb"
```

Makes function composition more readable

```r
# read left-to-right
mtcars %>% names() %>% length()
#> [1] 11

# not inside out
length(names(mtcars))
#> [1] 11
```

---
## R's basic data structures

.footnote[
Read Hadley Wickham's brilliant chapter on data structures http://adv-r.had.co.nz/Data-structures.html
]

---
## A data frame holds (homo or hetero!) 1d vectors.

.footnote[
Read Hadley Wickham's brilliant chapter on data structures http://adv-r.had.co.nz/Data-structures.html
]

---
## More than a table

.footnote[
Watch Jenny Bryan's brilliant talk https://www.youtube.com/watch?v=4MfUCX_KpdE
]

---
class: inverse, middle, center
background-image: url(https://media.giphy.com/media/h4Z6RfuQycdiM/giphy.gif)
background-size: contain

# How is this useful?

---

# What data goes into drawing this map?

---

```r
library(albersusa)
usa <- usa_sf("laea")

library(dplyr)
usa %>%
 select(name, pop_2010, geometry) %>%
 View()
```

```r
usa_sf("laea") %>% select(pop_2010, geometry) %>% plot()
```

![](index_files/figure-html/unnamed-chunk-8-1.png)

---
class: inverse, middle, center

# Rows should represent the unit of interest!!!

Hadley Wickham (probably)

---

## An aside on dplyr

The R package **dplyr** makes common SQL-like operations fast and easy<sup>1</sup>

Important single table operations:

* `select()`
* `mutate()`
* `filter()`
* `arrange()`
* `distinct()`
* `summarise()`

I will use dplyr sporadically through the workshop...please stop me if anything needs more explaining

.footnote[
[1]: It will even perform SQL queries for you -- using the same interface!
]

---
class: inverse, center
background-image: url(../your-turn.jpeg)
background-size: contain

## Your turn

See `help(geom_sf, package = "ggplot2")`. Can you plot population by state using `geom_sf()` and `usa_sf("laea")`?

**Bonus:** Use `plotly::ggplotly()` to convert it to an interactive version!

.footnote[
Solution is [here](01-your-turn.R)

PS. Someone commented "I would really like to learn more about working with shapefiles". Hopefully `sf::st_read()` *just works* for you!
]

---

```r
library(plotly)
usa_sf <- mutate(
  usa_sf("laea"), txt = paste("The state of", name, "had \n", pop_2010, "people in 2010")
)
p <- ggplot(usa_sf) + 
  geom_sf(aes(fill = pop_2010, text = txt)) 
ggplotly(p, tooltip = "text")
```

---
class: inverse, center, middle

# More compelling examples

Tooltips & zooming are cool -- but we can do more!

---
class: center, middle

---
## [Add 2 lines](https://gist.github.com/cpsievert/7dd28a478b4c051180d802321353259d), & voila!

---
class: center, middle

---

See `demo("highlight-epl", package = "plotly")`

---
class: inverse, center, middle

# Disclaimer

I'd say ~80% of the ggplot2 API is correctly translated by `ggplotly()`.

I'm aiming for ~99% before the end of the year.

Regardless, knowing how it all works helps to workaround limitations & specify additional features not supported by the ggplot2 API

---
class: inverse, center
background-image: url(magic.gif)
background-size: contain

# How does it work?

---

## ggplotly returns a plotly htmlwidget

```r
class(p)
#> [1] "gg"     "ggplot"
gg <- ggplotly(p, tooltip = "text")
class(gg)
#> [1] "plotly"     "htmlwidget"
```

The [htmlwidgets](http://www.htmlwidgets.org/) framework guarantees things *just work* in any context.<sup>1</sup>

The htmlwidgets gallery has 85 registered widgets to date! http://gallery.htmlwidgets.org/

.footnote[
[1]: For example, at your R prompt, inside RStudio, rmarkdown, or shiny apps
]

---
class: inverse, center
background-image: url(../your-turn.jpeg)
background-size: contain

## Your Turn

Embed the `gg` map in an rmarkdown document.

**Bonus**: get the plot to print in an [r notebook](http://rmarkdown.rstudio.com/r_notebooks.html)

---

## What happens when you print a plotly htmlwidget?

.footnote[
All htmlwidgets take this same (R list -> JSON -> JavaScript -> HTML) approach!

Every htmlwidget is defined through an R list. Any R list maps to JSON through **jsonlite** package
]

---

## Mapping R list to JSON

```r
barchart <- list(
  data = list(list(
    x = c("a", "b", "c"),
    y = c(1, 2, 3),
    type = "bar"
  ))
)
plotly:::to_JSON(barchart, pretty = TRUE)
#> {
#>   "data": [
#>     {
#>       "x": ["a", "b", "c"],
#>       "y": [1, 2, 3],
#>       "type": "bar"
#>     }
#>   ]
#> }
```

.footnote[
**Pro tip:** Did you know `:::` can access *any* object from *any* package (exported or not)?
]

---
## Indexing/subsetting in R

Grab a list element with `$` or `[[`

```r
str(barchart$data)
#> List of 1
#>  $ :List of 3
#>   ..$ x   : chr [1:3] "a" "b" "c"
#>   ..$ y   : num [1:3] 1 2 3
#>   ..$ type: chr "bar"
identical(barchart$data, barchart[["data"]])
#> [1] TRUE
```

There is also `[`, which _always_ returns the "container"!

```r
str(barchart["data"])
#> List of 1
#>  $ data:List of 1
#>   ..$ :List of 3
#>   .. ..$ x   : chr [1:3] "a" "b" "c"
#>   .. ..$ y   : num [1:3] 1 2 3
#>   .. ..$ type: chr "bar"
```

---

```r
str(mtcars["vs"])
#> 'data.frame':	32 obs. of  1 variable:
#>  $ vs: num  0 0 1 1 0 1 0 1 1 1 ...
str(mtcars[["vs"]])
#>  num [1:32] 0 0 1 1 0 1 0 1 1 1 ...
```

---
## Mapping R list to plotly

```r
library(plotly)
as_widget(barchart)
```

<div id="c11a58d33f79" style="width:720px;height:288px;" class="plotly html-widget"></div>
<script type="application/json" data-for="c11a58d33f79">{"x":{"data":[{"x":["a","b","c"],"y":[1,2,3],"type":"bar","xaxis":"x","yaxis":"y","frame":null}],"layout":{"xaxis":{"domain":[0,1],"type":"category","categoryorder":"array","categoryarray":["a","b","c"]},"yaxis":{"domain":[0,1]},"hovermode":"closest","showlegend":false},"attrs":[],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1}},"base_url":"https://plot.ly"},"evals":[],"jsHooks":{"render":[{"code":"function(el, x) { var ctConfig = crosstalk.var('plotlyCrosstalkOpts').set({\"on\":\"plotly_click\",\"persistent\":false,\"dynamic\":false,\"selectize\":false,\"opacityDim\":0.2,\"selected\":{\"opacity\":1}}); }","data":null}]}}</script>

---
## PSA: use `plot_ly()` over `as_widget()`

```r
# plot_ly() adds some useful abstractions that we'll get to later
plot_ly() %>%
  add_bars(
    x =  c("a", "b", "c"),
    y = c(1, 2, 3), 
    unsupported = "nonsense"
  )
#> Warning: 'bar' objects don't have these attributes: 'unsupported'
#> Valid attributes include:
#> 'type', 'visible', 'showlegend', 'legendgroup', 'opacity', 'name', 'uid', 'hoverinfo', 'stream', 'x', 'x0', 'dx', 'y', 'y0', 'dy', 'text', 'hovertext', 'textposition', 'textfont', 'insidetextfont', 'outsidetextfont', 'orientation', 'base', 'offset', 'width', 'marker', 'r', 't', 'error_y', 'error_x', '_deprecated', 'xaxis', 'yaxis', 'xcalendar', 'ycalendar', 'xsrc', 'ysrc', 'textsrc', 'hovertextsrc', 'textpositionsrc', 'basesrc', 'offsetsrc', 'widthsrc', 'rsrc', 'tsrc', 'key', 'set', 'frame', 'transforms', '_isNestedKey', '_isSimpleKey', '_isGraticule'
```

<div id="c11a155f9824" style="width:720px;height:288px;" class="plotly html-widget"></div>
<script type="application/json" data-for="c11a155f9824">{"x":{"visdat":{"c11a17217461":["function () ","plotlyVisDat"]},"cur_data":"c11a17217461","attrs":{"c11a17217461":{"alpha":1,"sizes":[10,100],"x":["a","b","c"],"y":[1,2,3],"type":"bar","unsupported":"nonsense"}},"layout":{"margin":{"b":40,"l":60,"t":25,"r":10},"xaxis":{"domain":[0,1],"type":"category","categoryorder":"array","categoryarray":["a","b","c"]},"yaxis":{"domain":[0,1]},"hovermode":"closest","showlegend":false},"source":"A","config":{"modeBarButtonsToAdd":[{"name":"Collaborate","icon":{"width":1000,"ascent":500,"descent":-50,"path":"M487 375c7-10 9-23 5-36l-79-259c-3-12-11-23-22-31-11-8-22-12-35-12l-263 0c-15 0-29 5-43 15-13 10-23 23-28 37-5 13-5 25-1 37 0 0 0 3 1 7 1 5 1 8 1 11 0 2 0 4-1 6 0 3-1 5-1 6 1 2 2 4 3 6 1 2 2 4 4 6 2 3 4 5 5 7 5 7 9 16 13 26 4 10 7 19 9 26 0 2 0 5 0 9-1 4-1 6 0 8 0 2 2 5 4 8 3 3 5 5 5 7 4 6 8 15 12 26 4 11 7 19 7 26 1 1 0 4 0 9-1 4-1 7 0 8 1 2 3 5 6 8 4 4 6 6 6 7 4 5 8 13 13 24 4 11 7 20 7 28 1 1 0 4 0 7-1 3-1 6-1 7 0 2 1 4 3 6 1 1 3 4 5 6 2 3 3 5 5 6 1 2 3 5 4 9 2 3 3 7 5 10 1 3 2 6 4 10 2 4 4 7 6 9 2 3 4 5 7 7 3 2 7 3 11 3 3 0 8 0 13-1l0-1c7 2 12 2 14 2l218 0c14 0 25-5 32-16 8-10 10-23 6-37l-79-259c-7-22-13-37-20-43-7-7-19-10-37-10l-248 0c-5 0-9-2-11-5-2-3-2-7 0-12 4-13 18-20 41-20l264 0c5 0 10 2 16 5 5 3 8 6 10 11l85 282c2 5 2 10 2 17 7-3 13-7 17-13z m-304 0c-1-3-1-5 0-7 1-1 3-2 6-2l174 0c2 0 4 1 7 2 2 2 4 4 5 7l6 18c0 3 0 5-1 7-1 1-3 2-6 2l-173 0c-3 0-5-1-8-2-2-2-4-4-4-7z m-24-73c-1-3-1-5 0-7 2-2 3-2 6-2l174 0c2 0 5 0 7 2 3 2 4 4 5 7l6 18c1 2 0 5-1 6-1 2-3 3-5 3l-174 0c-3 0-5-1-7-3-3-1-4-4-5-6z"},"click":"function(gd) { \n        // is this being viewed in RStudio?\n        if (location.search == '?viewer_pane=1') {\n          alert('To learn about plotly for collaboration, visit:\\n https://cpsievert.github.io/plotly_book/plot-ly-for-collaboration.html');\n        } else {\n          window.open('https://cpsievert.github.io/plotly_book/plot-ly-for-collaboration.html', '_blank');\n        }\n      }"}],"cloud":false},"data":[{"x":["a","b","c"],"y":[1,2,3],"type":"bar","unsupported":"nonsense","marker":{"fillcolor":"rgba(31,119,180,1)","color":"rgba(31,119,180,1)","line":{"color":"transparent"}},"xaxis":"x","yaxis":"y","frame":null}],"highlight":{"on":"plotly_click","persistent":false,"dynamic":false,"selectize":false,"opacityDim":0.2,"selected":{"opacity":1}},"base_url":"https://plot.ly"},"evals":["config.modeBarButtonsToAdd.0.click"],"jsHooks":{"render":[{"code":"function(el, x) { var ctConfig = crosstalk.var('plotlyCrosstalkOpts').set({\"on\":\"plotly_click\",\"persistent\":false,\"dynamic\":false,\"selectize\":false,\"opacityDim\":0.2,\"selected\":{\"opacity\":1}}); }","data":null}]}}</script>

---
class: inverse, middle, center

# Three ways to widget<sup>1</sup>

`ggplotly()`: translates ggplot to widget

`as_widget()`: translates R lists to widget

`plot_ly()`: translate a custom R-specific grammar to widget

.footnote[
[1]: Actually, four, if you count `api_download_file()`
]

---
## Inspect the JSON behind any widget

```r
# In recent versions of RStudio -- gg %>% plotly_build() %>% View()
plotly_json(gg)
```

The `data`, `layout`, and `config` attributes are official plotly.js attributes covered in the [figure reference](https://plot.ly/r/reference/).

The other attributes are unique to the R package (don't worry about them).

---
## Modify any widget

`style()` modifies data attributes. `layout()` modifies the layout.

```r
gg2 <- gg %>%
  style(mode = "markers+lines", traces = 2) %>%
  layout(title = "A map of 2010 population", margin = list(t = 30))
```

---
# Note the modification!

```r
# The 'x' element stores the list converted to JSON
# plotly_json() just provides a more pleasant interface to gg$x
str(gg2$x$data[[2]])
```

```r
#> List of 14
#>  $ x         : num [1:238] -324546 -325004 -325571 -325589 -326462 ...
#>  $ y         : num [1:238] -110164 -119390 -130838 -131207 -149343 ...
#>  $ text      : chr "The state of Wyoming had <br /> 564358 people in 2010"
#>  $ type      : chr "scatter"
*#>  $ mode      : chr "markers+lines"
#>  $ line      :List of 3
#>   ..$ width: num 1.89
#>   ..$ color: chr "rgba(89,89,89,1)"
#>   ..$ dash : chr "solid"
#>  $ fill      : chr "toself"
#>  $ fillcolor : chr "rgba(19,43,67,1)"
#>  $ hoveron   : chr "fills"
#>  $ showlegend: logi FALSE
#>  $ xaxis     : chr "x"
#>  $ yaxis     : chr "y"
#>  $ hoverinfo : chr "text"
#>  $ frame     : chr NA
```

---
## Can also add data to any widget

There are a number of `add_*()` functions (e.g., `add_bars()`, `add_polygons()`, `add_trace()`).

```r
d <- gg$x$data[[52]]
add_polygons(gg, x = d$x, y = d$y, color = I("red"), inherit = FALSE)
```

---
## Resources for studying the figure reference

https://plot.ly/r/reference/

https://github.com/rreusser/plotly-doc-viewer

```r
# In recent versions of RStudio -- View(plotly:::Schema)
schema()
```

---
class: inverse, center
background-image: url(../your-turn.jpeg)
background-size: contain

## Your Turn

Overlay text on top of Wyoming using either a [scatter trace with text mode](https://plot.ly/r/reference/#scatter-mode) or an [annotation](https://plot.ly/r/reference/#layout-annotations)

**Tip**: Use `sf::st_centroid()` to find the center point of polygon(s).

**Bonus**: Can you label all the states?

.footnote[
Solution is [here](02-your-turn.R)
]

---
## Raster objects

* Raster objects are basically a matrix of color codes. These objects can be used to represent bitmap images.

```r
m <- matrix(hcl(0, 80, seq(50, 80, 10)), nrow = 4, ncol = 5)
(r <- as.raster(m))
#>      [,1]      [,2]      [,3]      [,4]      [,5]     
#> [1,] "#C54E6D" "#C54E6D" "#C54E6D" "#C54E6D" "#C54E6D"
#> [2,] "#E16A86" "#E16A86" "#E16A86" "#E16A86" "#E16A86"
#> [3,] "#FE86A1" "#FE86A1" "#FE86A1" "#FE86A1" "#FE86A1"
#> [4,] "#FFA2BC" "#FFA2BC" "#FFA2BC" "#FFA2BC" "#FFA2BC"
plot(r)
```

![](index_files/figure-html/unnamed-chunk-22-1.png)

---
## Embedding raster objects in plotly

```r
  plot_ly() %>%
  layout(images = list(
*   source = raster2uri(r), # converts a raster object to a data URI.
    xref = "x", yref = "y", x = 0, y = 0, sizex = 1, sizey = 1,
    sizing = "stretch", xanchor = "left", yanchor = "bottom"
  ))
```

---
## ggmap objects are bitmap images!

```r
library(ggmap)
basemap <- get_map(maptype = "satellite", zoom = 8)
p <- ggmap(basemap) +
  geom_polygon(aes(x = lon, y = lat, group = plotOrder),
    data = zips, colour = "black", fill = NA) +
  ggthemes::theme_map()
```

---
## Add zoom/pan/tooltips via `ggplotly()`

```r
ggplotly(p)
```

---
## A crossroads in ggplot2 mapping

* Going forward, `geom_sf()` will be the preferred way to map in **ggplot2**, but it is still under development.
* It will support _any map projection_, thanks to the magic of **sf**.

```r
library(maps)
library(sf)
world1 <- st_as_sf(map('world', plot = FALSE, fill = TRUE))
ggplot() + geom_sf(data = world1)
```

---
## Finding map projections

* <http://www.spatialreference.org/ref/> is invaluable for finding map projections.

---
## Mollweide projection of Canada

```r
# http://spatialreference.org/ref/sr-org/7/proj4/
canada <- subset(world1, ID == "Canada")
canada2 <- st_transform(canada,"+proj=moll +lon_0=0 +x_0=0 +y_0=0 +ellps=WGS84 +units=m +no_defs")
ggplot() + geom_sf(data = canada2)
```

---
## Now with `ggplotly()`

```r
ggplotly()
```

---
class: inverse, center
background-image: url(../your-turn.jpeg)
background-size: contain

## Your turn

(1) Peruse some of the examples on <https://plot.ly/r/maps/>. Which approach do you like best (`ggplotly()`, `plot_geo()`, or `plot_mapbox()`)? Let me know via Slack. Can you point out some advantages/disadvantages to each approach?

(2) See the last example on <https://plot.ly/r/lines-on-maps/> -- how does the plot know to render in 3D? Can you make a 2D version?

.footnote[
Not interested in maps? Peak through [tomorrow's slides](../day2/index.html). Tell me if you want to see something else!

Have a personal project (related to plotly) that you need help with? Ask me!
]