Data Visualization Lesson._pub.html

<hr/>

<h1>Data Viz Ain&#39;t Easy</h1>

<p>In almost every business meeting you&#39;ll attend, you will see a poorly designed chart, graph, or other visual representation of data. Most people simply lack the education required to emphasize a point.  </p>

<p>Bad visualizations can be:  </p>

<ul>
<li>Difficult or impossible to interpret</li>
<li>Filled with completely worthless information</li>
<li>Misleading (intentionally or unintentionally)</li>
<li>Redundant and boring</li>
<li>Inaccurate</li>
</ul>

<p>I have to give credit to <a href="http://junkcharts.typepad.com/j">Junk Charts</a> - it inspired a lot of this post.</p>

<hr/>

<h2>Let&#39;s take a look at some examples</h2>

<h3>Every Death in Shakespeare</h3>

<p><strong>Could you imagine a worse way to show this??</strong>  </p>

<p><img src="http://i.imgur.com/4kF0q5F.png" title="plot of chunk unnamed-chunk-1" alt="plot of chunk unnamed-chunk-1" style="display: block; margin: auto;" /></p>

<p><strong>Is this not insane!?!?!</strong>  </p>

<p>No one could ever glance at that and possibly want to read it. The only thing that would have made it worse would be if there had been a legend instead of data callouts. The author could easily have used a number of other tools to get the point across. I hate wordles but due to the fact that the article wasn&#39;t trying to show the exact proportions of type of deaths, a wordle easily illustrated the point.</p>

<p><img src="http://i.imgur.com/dlDh5H8.png" title="plot of chunk unnamed-chunk-2" alt="plot of chunk unnamed-chunk-2" style="display: block; margin: auto;" />
This example came from this <a href="http://junkcharts.typepad.com/junk_charts/2016/03/which-way-to-die-the-bard-asked-onelesspie.html">Junk Charts Article</a></p>

<hr/>

<p>To be clear, I&#39;m not even close to being perfect when it comes to visualizations in my blog. The sizes, shapes, font colors, etc. tend to get out of control and I don&#39;t want to take the time in R to tinker with all of the details. However, when it comes to displaying things professionally, it has to be spot on! So, we&#39;ll look at the theory and not worry too much about aesthetics (save that for a time when you&#39;re getting paid).</p>

<p>Let&#39;s load up some libraries and get started.</p>

<pre><code class="r">library(ggplot2)
library(dplyr)
library(tidyr)
library(lubridate)
library(scales)
</code></pre>

<h2>Decide on what you&#39;re trying to accomplish first.</h2>

<p>Ask yourself the following questions to help drive your decision:  </p>

<ul>
<li>Are you making a comparison?</li>
<li>Are you finding a relationship?</li>
<li>Are you showing a distribution?</li>
<li>Are you finding a trend over time?</li>
<li>Are you showing composition?</li>
</ul>

<p>Once you know which question you are asking, it will keep your mind focused on the outcome and will quickly narrow down your charting options.</p>

<h4>Rule of Thumb</h4>

<ul>
<li>*<em>Trend: *</em> Column, Line<br/></li>
<li>*<em>Comparison: *</em> Area, Bar, Bullet, Column, Line, Scatter<br/></li>
<li>*<em>Relationship: *</em> Line, Scatter<br/></li>
<li>*<em>Distribution: *</em> Bar, Boxplot, Column<br/></li>
<li>*<em>Composition: *</em> Donut, Pie, Stacked Bar, Stacked Column<br/></li>
</ul>

<p>Obviously, there are choices beyond these and you need to think through your choice wisely. </p>

<p>Side Note: I <strong><em>hate</em></strong> donut and pie charts! When used properly, they&#39;re terriffic! However, I&#39;m very used to gagging every time one appears on a projector screen due to how frequently they&#39;re used inappropriately.</p>

<p>For this project, I&#39;ll use some oil production data that I found while digging through <a href="http://data.world">http://data.world</a> (pretty great site). The data can be found <a href="http://www.eia.gov/dnav/pet/pet_crd_crpdn_adc_mbbl_m.htm">here</a>  </p>

<pre><code class="r">#Custom data preparation
#GitHub (linked to at bottom of this post)
source(&#39;data_preparation.R&#39;)
data = getData()
</code></pre>

<pre><code class="r">head(data)
</code></pre>

<pre><code>##   Location Month Year ThousandBarrel       Date
## 1  Alabama   Mar 2013            883 2013-03-01
## 2  Alabama   Apr 2013            844 2013-04-01
## 3  Alabama   May 2013            878 2013-05-01
## 4  Alabama   Feb 2013            809 2013-02-01
## 5  Alabama   Mar 1982           1687 1982-03-01
## 6  Alabama   Apr 1982           1567 1982-04-01
</code></pre>

<hr/>

<h2>Trend - Line Chart</h2>

<p><strong>Objective:</strong> See what the oil production in the US looked like from 1981 - 2016 by year. I want to illustrate the changes over the time period. This is a very high-level view and only shows us a decline followed by a ramp up at the end of the period.</p>

<p>I decided to use a line chart to show the trend over time. When using discrete data you should use a column chart to avoid any confusion that in between these years the data actually was simply linear. However, it paints a much clearer picture this way and is not misleading.</p>

<h3>Which of these views would you rather see?</h3>

<h4>Poor Version</h4>

<p>The x-axis is a disaster and the y-axis isn&#39;t formatted well. While it gets the point across, it&#39;s still almost worthless.</p>

<pre><code class="r">df = data %&gt;% 
  group_by(Year) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))

p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1)) 
p + geom_line(stat=&#39;identity&#39;) + 
  ggtitle(&#39;Oil Production Over Time&#39;) + 
  theme(plot.title = element_text(hjust = 0.5),plot.subtitle = element_text(hjust = 0.5)) + 
  xlab(&#39;&#39;) + ylab(&#39;&#39;)
</code></pre>

<p><img src="http://i.imgur.com/QjuY0mP.png" title="plot of chunk unnamed-chunk-4" alt="plot of chunk unnamed-chunk-4" style="display: block; margin: auto;" /></p>

<h4>Better Version</h4>

<p>The title gives us a much better understanding of what we&#39;re looking at. The chart is slightly wider and the axes are formatted to be legible.</p>

<pre><code class="r">p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1)) 
p + geom_line(stat=&#39;identity&#39;) + 
  ggtitle(&#39;Thousand Barrel Oil Production By Year in the U.S.&#39;) +
  theme(plot.title = element_text(hjust = 0.5),plot.subtitle = element_text(hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1)) + 
  scale_y_continuous(labels = comma)
</code></pre>

<p><img src="http://i.imgur.com/DmOdVOK.png" title="plot of chunk unnamed-chunk-5" alt="plot of chunk unnamed-chunk-5" style="display: block; margin: auto;" /></p>

<h2>Comparison - Line Chart</h2>

<p><strong>Objective</strong>: Identify which states affected the trend the most. Evaluate them simultaneously in order to paint the picture and compare them.  </p>

<p>From this visual you can see the top states are Alaska, California, Louisiana, Oklahoma, Texas and Wyoming. Texas seems to break the mold quite drastically and drove the spike which occurred after 2010.</p>

<h3>Which of these views would you rather see?</h3>

<h4>Poor Version</h4>

<p>There are far too many colors going on here. Everything at the bottom of the chart is relatively useless and takes our focus away from the big players. </p>

<pre><code class="r">df = data %&gt;%
  group_by(Location, Year) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))

df$Year = as.numeric(df$Year)

p = ggplot(df,aes(x=Year,y=ThousandBarrel,col=Location))
p + geom_line(stat=&#39;identity&#39;) + 
  ggtitle(paste(&#39;Oil Production By Year By State in the U.S.&#39;)) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1))
</code></pre>

<p><img src="http://i.imgur.com/LdhZsPF.png" alt="plot of chunk unnamed-chunk-6"/></p>

<h4>Better Version</h4>

<p>This focuses attention on the top producing states. It compares them to each other and shows the trend per state as well.</p>

<pre><code class="r">n=6 #Arbitrary at first, after trying a few, this made the most sense
topN = data %&gt;%
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  arrange(-ThousandBarrel) %&gt;%
  top_n(n)

df = data %&gt;%
  filter(Location %in% topN$Location) %&gt;%
  group_by(Year,Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))

df$Year = as.numeric(df$Year)
df$Location = as.factor(df$Location)

p = ggplot(df,aes(x=Year,y=ThousandBarrel,group=1))
p + geom_line(stat=&#39;identity&#39;) + 
  ggtitle(paste(&#39;Top&#39;,as.character(n),&#39;States - Oil Production By Year in the U.S.&#39;)) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1)) + 
  facet_wrap(~Location) + 
  scale_y_continuous(labels = comma) 
</code></pre>

<p><img src="http://i.imgur.com/Vkxymeb.png" alt="plot of chunk unnamed-chunk-7"/></p>

<h2>Relationship - Scatter Plot</h2>

<p><strong>Objective</strong>: See if Alaska and California data is correlated (This probably isn&#39;t important but it allows us to use the same data).</p>

<h3>Which of these views would you rather see?</h3>

<h4>Poor Version</h4>

<p>Lots of completely irrelevant data! Size of the point should have nothing to do with the year.  </p>

<pre><code class="r">statesList = c(&#39;Alaska&#39;,&#39;California&#39;)
df = data %&gt;%
  filter(Location %in% statesList) %&gt;%
  spread(Location,ThousandBarrel) %&gt;%
  select(Alaska,California,Month,Year)

p = ggplot(df,aes(x=Alaska,y=California,col=Month,size=Year))
p + geom_point() + 
  scale_y_continuous(labels = comma) +
  scale_x_continuous(labels = comma) +
  ggtitle(&#39;Oil Production - CA vs. AK&#39;) + 
  theme(plot.title = element_text(hjust = 0.5))
</code></pre>

<p><img src="http://i.imgur.com/ta0Uv6a.png" alt="plot of chunk unnamed-chunk-8"/></p>

<h4>Better Version</h4>

<p>The trend line is nice because it helps to visualize the relationship even more. While it can sometimes be misleading, it makes sense with our current data.  </p>

<pre><code class="r">df = data %&gt;%
  filter(Location %in% statesList) %&gt;%
  spread(Location,ThousandBarrel) %&gt;%
  select(Alaska,California,Year)

p = ggplot(df,aes(x=Alaska,y=California))
p + geom_point() + 
  scale_y_continuous(labels = comma) +
  scale_x_continuous(labels = comma) +
  ggtitle(&#39;Monthly Thousand Barrel Oil Production 1981-2016 CA vs. AK&#39;) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  geom_smooth(method=&#39;lm&#39;)
</code></pre>

<p><img src="http://i.imgur.com/Kxtvyrc.png" alt="plot of chunk unnamed-chunk-9"/></p>

<h2>Distribution - Boxplot</h2>

<p><strong>Objective</strong>: Examine the range of production by state and year over the time period to give us an idea of the variance.</p>

<h3>Which of these views would you rather see?</h3>

<h4>Poor Version</h4>

<pre><code class="r">df = data %&gt;%
  group_by(Year,Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))

p = ggplot(df,aes(x=Location,y=ThousandBarrel))
p + geom_boxplot() + 
  ggtitle(&#39;Distribution of Oil Production by State&#39;)
</code></pre>

<p><img src="http://i.imgur.com/QYyBcNn.png" alt="plot of chunk unnamed-chunk-10"/></p>

<h4>Better Version</h4>

<p>This gives a nice ranking to the plot while still showing their distributions. While it was semi-apparent in the line charts, the variance of Texas is huge compared to the others! We could take this a step further and separate out the big players from the smaller players.</p>

<pre><code class="r">p = ggplot(df,aes(x=reorder(Location,ThousandBarrel),y=ThousandBarrel))
p + geom_boxplot() + 
  scale_y_continuous(labels = comma) +
  ggtitle(&#39;Distribution of Annual Oil Production By State (1981 - 2016)&#39;) + 
  coord_flip()
</code></pre>

<p><img src="http://i.imgur.com/AunupK4.png" alt="plot of chunk unnamed-chunk-11"/></p>

<h2>Composition - Stacked Bar</h2>

<p><strong>Objective</strong>: Check out the composition of total production by state.</p>

<h3>Which of these views would you rather see?</h3>

<h4>Poor Version</h4>

<p>My favorite, the beautiful pie chart! There&#39;s nothing better than this&hellip;</p>

<pre><code class="r">df = data %&gt;%
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  mutate(ThousandBarrel = ThousandBarrel/sum(ThousandBarrel))

df$ThousandBarrel = round(100*df$ThousandBarrel,0)

library(plotrix)
pie(x=df$ThousandBarrel,labels=df$Location,explode=0.1,col=rainbow(nrow(df)),main=&#39;Percentage of Oil Production by State&#39;)
</code></pre>

<p><img src="http://i.imgur.com/vv1q2FX.png" alt="plot of chunk unnamed-chunk-12"/></p>

<h4>Better Version</h4>

<p>The 1980&#39;s and 2010&#39;s will be missing years in terms of a &ldquo;decade&rdquo; due to the data provided (and it&#39;s only 2017). While the percentage labels are slightly off center, it&#39;s certainly much better than the pie chart. It&#39;s not quite &ldquo;apples-to-apples&rdquo; for a comparison because I created different decades, but you get the idea.</p>

<p>I also created an &ldquo;Other&rdquo; category in order to simplify the output. When you are doing comparisons, it&#39;s typically a good idea to find a way to reduce the number of variables in the output while not removing data by dropping it completely.</p>

<pre><code class="r">data$Decade = &#39;1980s&#39;
data$Decade[data$Year &gt;= 1990] = &#39;1990s&#39;
data$Decade[data$Year &gt;= 2000] = &#39;2000s&#39;
data$Decade[data$Year &gt;= 2010] = &#39;2010s&#39;
data$Decade = as.factor(data$Decade)

top5 = data %&gt;%
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  arrange(-ThousandBarrel) %&gt;%
  top_n(5) %&gt;%
  select(Location)

top5List = top5$Location

data$State = &quot;Other&quot;

for(i in 1:length(top5List)){
  data$State[data$Location == top5List[i]] = top5List[i]
}

df = data %&gt;%
  group_by(Decade,State) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  mutate(ThousandBarrel = ThousandBarrel/sum(ThousandBarrel))

df$ThousandBarrel = round(df$ThousandBarrel,3)
df$text = paste(round(100*df$ThousandBarrel,0),&#39;%&#39;, sep=&#39;&#39;)

p = ggplot(df,aes(x=Decade,y=ThousandBarrel,col=reorder(State,ThousandBarrel),fill=reorder(State,ThousandBarrel)))
p + geom_bar(stat=&#39;identity&#39;) + 
  geom_text(aes(label=text),col=&#39;Black&#39;,size = 4, hjust = 0.5, vjust = 3, position = &quot;stack&quot;) + 
  scale_y_continuous(labels = percent) +
  ggtitle(&#39;Percentage of Top Oil Producing States by Decade&#39;) + 
  guides(fill=guide_legend(title=&#39;State&#39;),col=guide_legend(title=&#39;State&#39;)) + 
  theme(plot.title = element_text(hjust = 0.5))
</code></pre>

<p><img src="http://i.imgur.com/I7Cyf6Z.png" alt="plot of chunk unnamed-chunk-13"/></p>

<h3>Some other fun concepts are below!</h3>

<p>Some of them are nice, others are terrible! I won&#39;t comment on any of them, but I felt it was necessary to include some other ideas I toyed around with. </p>

<p>Have fun with your data visualizations. The charts I showed here are extremely simple. Being creative by using things other than R wind up making visuals people can remember. There are plenty of examples around, but they all tend to follow basic principles of design. There are <strong><em>A TON</em></strong> of good books out there on this topic. </p>

<p>Now it&#39;s your turn!</p>

<pre><code class="r">df = data %&gt;% 
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  arrange(-ThousandBarrel)
p = ggplot(df,aes(x=reorder(Location,ThousandBarrel),y=ThousandBarrel))
p + geom_bar(stat=&#39;identity&#39;) + 
  ggtitle(&#39;Oil Production 1981 - 2016 By Location&#39;) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  coord_flip()
</code></pre>

<p><img src="http://i.imgur.com/DwJ52D9.png" alt="plot of chunk unnamed-chunk-14"/></p>

<pre><code class="r">top10 = data %&gt;%
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) %&gt;%
  arrange(-ThousandBarrel) %&gt;%
  top_n(10)
</code></pre>

<pre><code>## Selecting by ThousandBarrel
</code></pre>

<pre><code class="r">print(top10)
</code></pre>

<pre><code>## # A tibble: 10 × 2
##       Location ThousandBarrel
##          &lt;chr&gt;          &lt;dbl&gt;
## 1        Texas       23447172
## 2       Alaska       15775279
## 3   California        9988225
## 4    Louisiana        4267246
## 5     Oklahoma        3701224
## 6      Wyoming        2894624
## 7       Kansas        1708873
## 8     Colorado        1288643
## 9         Utah         894657
## 10 Mississippi         861999
</code></pre>

<pre><code class="r">df = data %&gt;% 
  group_by(Location,Year) %&gt;%
  filter(Location %in% top10$Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel)) 
p = ggplot(df,aes(x=Year,y=ThousandBarrel,col=Location,fill=Location))
p + geom_bar(stat=&#39;identity&#39;) + 
  ggtitle(&#39;Oil Production - Top 10 States&#39;) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1))
</code></pre>

<p><img src="http://i.imgur.com/Y21hxkY.png" alt="plot of chunk unnamed-chunk-15"/></p>

<pre><code class="r">df = data %&gt;%
  filter(Year == 1990)%&gt;%
  group_by(Location) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))
df$Location = tolower(df$Location)

#Add States without data
States = data.frame(Location = tolower(as.character(state.name)))
missingStates = States$Location[!(States$Location %in% df$Location)]
appendData = data.frame(Location=missingStates,ThousandBarrel=0)
df = rbind(df,appendData)

states_map &lt;- map_data(&quot;state&quot;)

ggplot(df, aes(map_id = Location)) + 
    geom_map(aes(fill=ThousandBarrel), map = states_map) +
    expand_limits(x = states_map$long, y = states_map$lat)
</code></pre>

<p><img src="http://i.imgur.com/jT1XTmF.png" alt="plot of chunk unnamed-chunk-16"/></p>

<pre><code class="r">df = data %&gt;% 
  filter(Location == &#39;Texas&#39;) %&gt;%
  group_by(Year,Month) %&gt;%
  summarise(ThousandBarrel = sum(ThousandBarrel))

p = ggplot(df,aes(x=Month,y=ThousandBarrel))
p + geom_line(stat=&#39;identity&#39;,aes(group=Year,col=Year)) + 
  ggtitle(&#39;Oil Production By Year in the U.S.&#39;) + 
  theme(plot.title = element_text(hjust = 0.5)) + 
  theme(axis.text.x = element_text(angle = 90, hjust = 1))
</code></pre>

<p><img src="http://i.imgur.com/4OwEmjV.png" alt="plot of chunk unnamed-chunk-17"/></p>

<p>As always, the code used in this post is on my <a href="https://github.com/stoltzmaniac/Data-Visualization-Lesson">GitHub</a></p>