Introduction

The main purpose of this tutorial is to put together all 5 key functions from dplyr and use them to create summary tables and graphs in RMarkdown. The functions and their purposes are listed as follows:

We will continue to practice our skills using the dataset flights by loading the R package nycflights13.

Part 1: Summarizing the Data

Using the pipe %>% to chain functions from dplyr, modify the code below to create a dataset named flight.summary that contains the following ordered modifications:

  1. Starts with the raw dataset flights in the package nycflights13.

  2. Transform delay variables so that they are measured in minutes since midnight, and then create a metric to measure accuracy based on those transformed delay variables.

  3. Remove observations with missing accuracy.

  4. Group data based on combinations of “origin”, “dest”, and “carrier”.

  5. Summarize the data using the number of observations, the accuracy mean, the accuracy variance, and the mean distance for each combination of “origin”, “dest”, and “carrier”.

  6. Filters summarized data to remove all combinations of “origin”, “dest”, and “carrier” with less than or equal to 10 flights throughout 2013.

  7. Create a variable called “proportion”” that represents the proportion of flights in each combination out of the whole dataset.

flight.summary = 
  
  #1
  DATASET %>%
          
  #2
  mutate(
    dep_min       = (dep_time%/%NUMBER)*NUMBER+dep_time%%NUMBER,
    sched_dep_min = COMPLETE,
    arr_min       = COMPLETE,
    sched_arr_min = COMPLETE,
    dep_delay_min = COMPLETE,
    arr_delay_min = COMPLETE,
    accuracy      = ACCURACY_FORMULA
  ) %>%
  
  #3
  filter(CRITERIA) %>%
  
  #4
  group_by(VARIABLES) %>%

  #5
  summarize(
    count=FUNCTION1,
    mean.acc=FUNCTION2,
    var.acc=FUNCTION3,
    mean.dist=FUNCTION4,
    .groups="drop"
  ) %>%
  
  #6
  filter(CRITERIA) %>%
  
  #7
  mutate(proportion=FORMULA)
head(flight.summary,10)

Part 2: Building Charts From Summary Data

The purpose of creating this plot was to investigate if flight accuracy gets worse or better for longer flights. The chart above displays the approximated linear relationship between flight accuracy and flight distance for each the 3 NYC airports. This first image was created using a combination of geom_point() and geom_smooth(). Recreate this image using ggplot() and remember to modify the defaults of geom_smooth() to get linear trend lines without standard error regions.

ggplot(data=flight.summary) +
  geom_point(MORE)+
  geom_smooth(MORE)

The first thing we notice is that not many flights are over 3000 miles in distance. Design a new plot similar to the one above that ignores the scenarios where the distance exceeds 3000. This will prevent rare occurrences from effecting our linear trend comparison and present these relationships in a zoomed-in window where the majority of data exists.

ggplot(data=filter(flight.summary,CRITERIA)) +
  geom_point(MORE)+
  geom_smooth(MORE)

Based on your work, think of answers to the following 2 discussion questions:

  1. Which trend line was most affected by the removal of few occasions where the average distance exceeded 3000 miles?

  2. Each trend line is doing different things. What do you find from the three trend lines? (Hint: relationship between accuracy and distance for different airports)

Part 3: Build a Nice Table to Be Displayed in RMarkdown

The dataset we created, flight.summary, summarizes the raw data but still contains 370 observations. Below I have provided code, that produces a dataset called flight.summary2 that only contains the top 5 and bottom 5 combinations of “origin”, “dest”, and “carrier” based on mean accuracy. Closely examine this code and ensure that you understand what is happening here. Set eval=T to create flight.summary2.

flight.summary2 = 
  flight.summary %>%
  mutate(rank=min_rank(mean.acc)) %>%
  filter(min_rank(mean.acc)<=5 | min_rank(desc(mean.acc))<=5) %>%
  arrange(rank)

The kable() function in the knitr package allows for creating HTML tables. Furthermore, the kableExtra package allows you to add beauty to those internet-ready tables. Click here for a helpful introduction. Once you understand the function of kable() and the modifications you can apply, start an R code chunk to place the information required to produce an HTML table. Click here for additional guidance on the options required to turn a tibble into HTML code using kable() and then HTML code into an integrated webpage table.

flight.summary2 %>%
  kable() %>%
  kable_styling(bootstrap_options = c("striped", "hover"))

Conclusion

In this class you will be forced to analyze data you probably care very little about. This is the part in the class where you begin to act like you care. An important rule in this class is to “love thy data as thyself.” Consider the flights data from both the view of the consumer and the airline executive. Think of the different questions that may arise on both sides. A consumer, such as myself, might view the data as an opportunity to explain to American Airlines why they are embarassing all Americans. The airline executive might want to know how their carrier compares to other carriers on departure and arrival delays. Once you have the question you want to answer, the next step is using the data to answer your question.