Run vs. Pass Play Prediction:

class: center, middle, inverse, title-slide

# Run vs. Pass Play Prediction:
## Incorporating NFL Tracking Data
### Tej Seth
### Nicole Tucker
### November 6th, 2021

---

# Previous Play Type Prediction Models- NFLFastR

- Situational Factors from Play by Play Data
  - Quarter
  - Down
  - Yards to Go
  - Yardline Number
  - Half Seconds Remaining  
  - Score Differential
  - Shotgun Formation

- XGBoost Model

- Accuracy: 70.1 %

---
# The Data

NFL 2017 Tracking Data Week 1 Through Week 6

Corresponding NFL Play by Play, Games and Players Data

NFL FastR 2017 Play by Play Data

<div id="gnrkfacppv" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
  font-family: -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#gnrkfacppv .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#gnrkfacppv .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#gnrkfacppv .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#gnrkfacppv .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 4px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#gnrkfacppv .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#gnrkfacppv .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#gnrkfacppv .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#gnrkfacppv .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#gnrkfacppv .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#gnrkfacppv .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#gnrkfacppv .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#gnrkfacppv .gt_group_heading {
  padding: 8px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#gnrkfacppv .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#gnrkfacppv .gt_from_md > :first-child {
  margin-top: 0;
}

#gnrkfacppv .gt_from_md > :last-child {
  margin-bottom: 0;
}

#gnrkfacppv .gt_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#gnrkfacppv .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 12px;
}

#gnrkfacppv .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#gnrkfacppv .gt_first_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
}

#gnrkfacppv .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#gnrkfacppv .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#gnrkfacppv .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#gnrkfacppv .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#gnrkfacppv .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#gnrkfacppv .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding: 4px;
}

#gnrkfacppv .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#gnrkfacppv .gt_sourcenote {
  font-size: 90%;
  padding: 4px;
}

#gnrkfacppv .gt_left {
  text-align: left;
}

#gnrkfacppv .gt_center {
  text-align: center;
}

#gnrkfacppv .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#gnrkfacppv .gt_font_normal {
  font-weight: normal;
}

#gnrkfacppv .gt_font_bold {
  font-weight: bold;
}

#gnrkfacppv .gt_font_italic {
  font-style: italic;
}

#gnrkfacppv .gt_super {
  font-size: 65%;
}

#gnrkfacppv .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 65%;
}
</style>
<table class="gt_table">
  
  <thead class="gt_col_headings">
    <tr>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">nflId</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">x</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">y</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">dir</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">dis</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">s</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">event</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1">displayName</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">frame.id</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">playId</th>
      <th class="gt_col_heading gt_columns_bottom_border gt_right" rowspan="1" colspan="1">gameId</th>
    </tr>
  </thead>
  <tbody class="gt_table_body">
    <tr><td class="gt_row gt_right">2495340</td>
<td class="gt_row gt_right">41.56</td>
<td class="gt_row gt_right">16.54</td>
<td class="gt_row gt_right">78.90</td>
<td class="gt_row gt_right">0.41</td>
<td class="gt_row gt_right">3.91</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">Anthony Sherman</td>
<td class="gt_row gt_right">1</td>
<td class="gt_row gt_right">44</td>
<td class="gt_row gt_right">2017090700</td></tr>
    <tr><td class="gt_row gt_right">2495340</td>
<td class="gt_row gt_right">41.95</td>
<td class="gt_row gt_right">16.62</td>
<td class="gt_row gt_right">79.16</td>
<td class="gt_row gt_right">0.40</td>
<td class="gt_row gt_right">4.28</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">Anthony Sherman</td>
<td class="gt_row gt_right">2</td>
<td class="gt_row gt_right">44</td>
<td class="gt_row gt_right">2017090700</td></tr>
    <tr><td class="gt_row gt_right">2495340</td>
<td class="gt_row gt_right">42.40</td>
<td class="gt_row gt_right">16.73</td>
<td class="gt_row gt_right">79.46</td>
<td class="gt_row gt_right">0.47</td>
<td class="gt_row gt_right">4.66</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">Anthony Sherman</td>
<td class="gt_row gt_right">3</td>
<td class="gt_row gt_right">44</td>
<td class="gt_row gt_right">2017090700</td></tr>
    <tr><td class="gt_row gt_right">2495340</td>
<td class="gt_row gt_right">42.85</td>
<td class="gt_row gt_right">16.82</td>
<td class="gt_row gt_right">79.76</td>
<td class="gt_row gt_right">0.46</td>
<td class="gt_row gt_right">5.04</td>
<td class="gt_row gt_left">NA</td>
<td class="gt_row gt_left">Anthony Sherman</td>
<td class="gt_row gt_right">4</td>
<td class="gt_row gt_right">44</td>
<td class="gt_row gt_right">2017090700</td></tr>
    <tr><td class="gt_row gt_right">2495340</td>
<td class="gt_row gt_right">43.36</td>
<td class="gt_row gt_right">16.92</td>
<td class="gt_row gt_right">80.12</td>
<td class="gt_row gt_right">0.51</td>
<td class="gt_row gt_right">5.39</td>
<td class="gt_row gt_left">kickoff</td>
<td class="gt_row gt_left">Anthony Sherman</td>
<td class="gt_row gt_right">5</td>
<td class="gt_row gt_right">44</td>
<td class="gt_row gt_right">2017090700</td></tr>
  </tbody>
  
  
</table>
</div>

---
# The Data

---

# Created a Baseline Situational Factors Predictive Model for Comparison Purposes

- Predictors:

- Quarter
  - Down
  - Yards to Go
  - Yardline Number
  - Half Seconds Remaining  
  - Score Differential
  
--

- Accuracy: 63.27%
- Brier Score: 0.2085

---

# Defining Variables Created from Tracking Data

- Width of Formation + Standard Deviation
- Width of Offensive Line + Standard Deviation 
- Depth of Offensive Line

---
# Defining Variables Created from Tracking Data

- Width of Formation + Standard Deviation
- Width of Offensive Line + Standard Deviation 
- Depth of Offensive Line
- Deep Backfield RB- Indicator Variable
- QB Position (Under Center, Shotgun, Pistol)
- Personnel (Number of WRs, RBs, TEs)
- Fullback- Indicator Variable
- Maximum Wide Receiver Distance Off Line of Scrimmage
- Man in Motion- Indicator Variable
- Tight End Starting Near Ball- Indicator Variable

---
## A Larger Formation Width May Indicate a Pass Play

---
## A Larger Linemen Width Standard Deviation May Indicate a Run Play

---
## Using a Larger Sample Size to Model Expected Pass for Situational Factors

- Trained model on 2012-2017 NFLFastR Play by Play Situational Data
- Created a Situational Expected Pass Variable

---
## Comparing Model Types

---
## Using a Mixture of Tracking Data Variables and Situational Variables Produces the Most Accurate Model

- Situational Factors and Tracking Data Variables in a Random Forest Accuracy: 75.2 %
- Situational Factors and Tracking Data Variables in a Random Forest Brier Score: 0.167

---
##  The Predictability of Play Type Varies by Team

---
## Each Team's Pass Rate Over Expected for 2017- Accuracy

---

## Each Team's Pass Rate Over Expected for 2017- Brier Score

---
## Mixed Effects Model- Effect of Team

<img src="mixedeffects.PNG" width="65%" style="display: block; margin: auto;" />
---

## Model Strengths- Down and Distance

---

## Model Calibration- Field Position

---

## Model Calibration- Down and Distance Situation

---

## Interpreting the Model

---
## Expanding Our Model to Update Probability of Pass After Snap

Updates every 0.10 of a second up to 2.5 seconds after the snap

Variables Created:
- Average Offensive Linemen Distance from Line of Scrimmage
- Average Offensive Linemen Speed
- Average Receiver Distance from Line of Scrimmage
- Average Receiver Speed
- Probability of Pass Before Snap

---
## Average Prediction Accuracy Every 1/10 Second

---
## Single Play Probabilities

---

# Completed Work

- Created variables using tracking data
- Created a basic model using only play by play data
- Created models including tracking data variables
- Investigated the predictability of each team
- Looked at how our model does in specific subsets
- Expanded our model to update predictions within the first 2.5 seconds of the play

# Acknowledgements
A big thank you to our project advisors!
- Brian Macdonald
- Sam Ventura

And a big thank you to our program instructor, Ron Yurko, for all the help and support this summer!