Traffic Incident Management Performance Measures: Ranking Agencies on Roadway Clearance Time ()
1. Introduction
When crashes occur, they can result in added congestion for other roadway users as well as increased risk associated with secondary collisions. Previous studies suggest that the occurrence of secondary crashes is highly influenced by the duration of clearance time [1] [2] [3] [4] . Thus, a prime concern for emergency responders is to minimize the clearance time of any event on the roadway. The national-level Traffic Incident Management (TIM) program has three main objectives: 1) Reduce roadway clearance time, 2) reduce incident clearance time, and 3) reduce the number of secondary crashes [5] . As these all are measurable metrics, performance measures can be defined for these objectives which are expected to be met by emergency response teams. These performance measures vary across regions and specific values are often used for particular facilities. Performance evaluations based on performance measures for these objectives are essential for allocating resources and improving TIM responses. However, there is a lack of published literature on methods for performing these evaluations and comparing agencies for the purpose of providing additional training, resources, etc. to improve TIM programs and responses. To fill that research gap, this study proposes a framework for performance evaluation of emergency response teams based on observed roadway clearance times.
2. Literature Review
The Federal Highway Administration (FHWA) defines roadway clearance time (RCT) as the “Time between first recordable awareness of incident by a responsible agency and first confirmation that all lanes are available for traffic flow” and incident clearance time (ICT) as “the time between the first recordable awareness and the time at which the last responder has left the scene” [5] . The previous research related to incident clearance time consists of two main categories: prediction of clearance time and impact of influential factors on clearance time. The scope of this study matches the latter of these—important factors impacting clearance time. For ranking the agencies, this study utilizes variables that impact the clearance time to make adjustments (i.e., making the average adjusted clearance times fair for comparison across agencies).
Primarily, this study collects influential factors suggested by previous literatures. For instance, one study discussed the time-varying effect of influential factors on incident clearance time by using a hazard-based model and found 18 variables that impact the incident clearance time significantly [6] . For example, injury involved, fire involved, AADT, summer, disabled vehicle, collision type, single lane blocked, multiple lanes blocked, short response time, medium response time, long response time, debris, abandoned vehicle, heavy truck involvement, nighttime, weekends, and traffic control were found to be significant predictors for incident clearance times. In a similar study, the factors influencing the response and clearance time were investigated along with incident reporting by using a copula-based approach [7] . This study included incident characteristics, traffic characteristics, roadway characteristics, and environmental characteristics in their model. This study found that incidents involving multiple vehicles, tire service, morning and evening peak, and night-time incidents increased the roadway clearance time and, on the other hand, police reported cases, summer months, and weekends decreased the clearance time [7] . Moreover, the study observed that the location of a crash is related to clearance time and usually the longer distance from the central business district increases the total incident duration.
Another group of researchers predicted the clearance time by combining four statistical and four machine learning models which were used frequently in the previous studies [8] . The significant variables impacting the clearance time are estimated from information gained from the machine learning models. Among those variables, lane closure and incident type were significant for all machine learning (ML) and statistical models. The month of the year, High Occupancy Vehicle (HOV), and weather variables were significant in ML models only. However, the variables related to the number of injuries and response time were not found significant from the ML model. Although, other studies have also found all these variables to be important [6] . Another study predicted freeway incident clearance time using the gradient boosting decision tree method and found that the incident response time and traffic volume were the variables with the highest predictive power [9] .
While searching for relevant literature, it was found that the previous studies mainly focused on prediction accuracy whereas the focus of our current paper is on the interpretation and estimation of influencing factors that will help create agency ranks based on their performance measure metrics. The definition of performance measures can be described as “performance measures are standards that express the degree to which tasks are accomplished efficiently and effectively” [10] . In practice, performance measure goals for Traffic Incident Management (TIM) are mostly agency-specific and those agencies are police and public safety officials. Usually, both the transportation and public safety/police departments have the same set of performance measures: Reducing the response time and Reducing the clearance time.
The following section describes the performance appraisal systems documented in previous incident management reports. A study was developed by using a regression-based model combining the variables: road name, direction of travel, time of day and vehicle position [10] . The authors calculated the predicted dispatch time and fixed the benchmark with one standard deviation from the mean. Use of the variables—specific roads (e.g., 95th street) and direction of travel might be useful for dispatch time, but it also may not have a direct impact on roadway clearance time. Moreover, the position of the vehicles gave some results which are complicated to anticipate and utilize. For example, vehicle position: Lane 1 shows 6.5 minutes of addition in the dispatch time. Whereas, Lane 2 shows 28.5 minutes decrease in the clearance time. There should be reasonable explanations that lead to such a big difference in the results. Furthermore, using the one standard deviation as a benchmark might not give proper performance measure goals that should be followed by agencies. Another study suggests that the performance measure process is evaluated by incident or injury severity, roadway type, vehicle type, notification of incidents, etc. [11] . For further contextualization, it was suggested to use the weather condition, lighting condition, roadway condition, incident in the work zone, number of towing services, etc. These variables are important for performance measures, but the report does not provide how to use the combined effect of these variables. Some regions follow a fixed value for performance measures such as clearing all incidents within 90 or 120 minutes for a particular injury type [12] . But, using injury type alone to differentiate performance measures is not actually helpful in identifying agency performances, as each crash incident under an agency has multiple characteristics which need to be considered for performance evaluation. In another study, roadway clearance time was modeled with peak hour, response time, number of lanes blocked, and number of incident response team members [13] . Furthermore, the investigators modeled the total incident duration with incident type and roadway clearance time. But, both the roadway clearance time and total incident duration can be impacted by the same set of variables that are omitted in the model and might lead to endogeneity issues.
3. Objective
Evaluating the performances of agencies depending on clearance times is a multi-criteria-based decision-making process and should not be handled by a fixed threshold of clearance time. However, current ranking methods in traffic safety research account mainly for crash hotspots which is often a single and direct approach for ranking sites for potential improvements [14] . Developing a proper performance evaluation is crucial as it helps allocate resources that shape an agency's future performance. A relevant study showed how the performance evaluation helps improve agencies’ responses and found that the low performing agencies aspire to improve to be above average and the high-performance agencies want to improve performance relative to their own [15] . While searching for relevant literature, it was difficult to find any comprehensive ranking framework for transportation agencies to the best of our knowledge. In this regard, our study provides an extensive framework for ranking incident management agencies depending on their performance measures (i.e., Roadway clearance time).
4. Data Description
This study has used crash data from the state of Iowa, provided by Iowa Department of Transportation (DOT), which comprises 150,004 crashes from June 2018 to August 2021. This dataset contains both the incident clearance time and the roadway clearance time. Staying true to the objective, this study has developed the ranking method based on the roadway clearance time. In addition to that, all 99 counties in Iowa and their respective 373 accident response teams were considered in this study to rank according to their historical performance. The accident response teams are mainly police and public safety agencies who ensure the safe and rapid clearance of incidents [10] . Regarding variable selection for modeling the roadway clearance time (RCT), this study included variables which cannot be controlled by the response teams. For example, a response team cannot control the weather or environmental conditions during a crash incident. The dependent variable for this modeling purpose is RCT and all the independent variables in the dataset are mostly categorical variables. For all the sub-categories under each category, indicator variables were used. The categorical variables, their base category and counts of the base categories (Among 149,174 crashes) are listed in Table 1.
This study included multiple filtering steps before using the data in the final model. First the dataset was filtered with a 99.5 percentile value RCT which is 306 minutes (larger values removed). Some crashes had roadway clearance time = 0 minutes which might be due to some error and noise in the data or due to all vehicles involved driving off the road before reporting the collision. Those are also removed from the dataset. Additional filtering was done depending on the counts of each categorical variable. Apart from the property damage cost estimate, all the used variables in this model are categorical. The final dataset has 149,174 crashes with roadway clearance time. Figure 1 shows the histogram of RCT for these 149,174 crashes.
5. Methodology
The main objective of this study is to rank agencies to understand where additional training and resources can be used to improve the overall system. The whole ranking process is carried out in two steps:
1) Modeling: This gives the coefficient estimates of the variables;
Figure 1. Histogram of roadway clearance time (in minutes).
Table 1. Variable list and description.
2) Ranking: This ranks the agencies on mean adjusted roadway clearance time (RCT) where adjustments are made using the coefficient estimates from the regression model.
Figure 2 represents the flowchart of the methodology used in this study. As indicated, Tobit Regression is used for normalization. Details of the Tobit
regression method and consideration of survival models for the methodology are provided below.
In the dataset, the RCT ranges from 1 to 306 minutes. To determine the relationship of the predictor variables with RCT, a Tobit model is used. Tobit model is a regression method where the linear relationship between the dependent variable and independent variable is estimated given that the dependent variable is left censored, right censored, or both. Otherwise, it has similar properties as linear regression. In this study, the RCT cannot be negative so the model is left censored. Equation (1) shows the model specification:
(1)
where μi = the predicted value of the outcome (i.e., roadway clearance time), βi = an estimated coefficient, and xi = predictor variable i. Using this specification, the outcome (yi) is shown in Equation (2);
(2)
where εi = the error term which is approximately normally distributed.
For the Tobit model, it is typically estimated using maximum likelihood procedures. The log-likelihood utilizes the fact that the outcome is truncated at value YL. The log-likelihood function for the Tobit is provided in Equation (3) [16] .
(3)
where, I(yj) = 0 if yj < YL, I(yj) = 1 if yj ≥ YL, φ = the probability density function of the standard normal distribution, Φ = the cumulative density function of the standard normal distribution, and σ = the estimated latent standard deviation of the residuals. For this analysis, YL = 0 and the lowest reported time in the dataset is 1 minute. Thus, Equation (3) can be simplified to Equation (4).
(4)
The CDF of the normal distribution does not have a closed form, so Equation (4) cannot be easily expanded. However, implementing Equation (4) in maximum likelihood estimation is straightforward.
As with linear regression, here are four major assumptions related to Tobit regression models [17] . They are:
1) Linearity: the dependent variable has a linear relationship with the response variable;
2) Homoscedasticity: The distribution of the error terms (εi) has constant variance;
3) Independence: The error terms (εi) are independent of each other;
4) Normality: The error terms (εi) are normally distributed.
After estimating the coefficients for the variables, RCT of each crash is normalized using the regression coefficients. The normalization process is described below with an example for one categorical variable:
1) Let, β1 = the estimate for crash severity = fatal;
2) Base category for crash severity = property damage only (PDO);
3) Let, R1 is the actual roadway clearance time of a fatal crash;
4) If R1 is normalized to a PDO crash, then the normalized RCT is = max (R1 − β1, 0).
While performing the normalization process, 49 variable estimates were used. These 49 variables were selected based on feedback from experienced officials. This selected list incorporates the variables that are not controllable by agencies and at the same time might cause substantial increase or decrease in the RCT. The RCT of 149,174 crashes was normalized using these 49 variables. Then the average normalized RCT time of each agency is calculated and ranked from highest to lowest adjusted RCT.
While the regression method applied in this paper is a Tobit model, other models such as survival models could be applied to accomplish the same outcome (i.e., rankings with normalization). Survival models were explored for this paper – including lognormal, gamma, Weibull, generalized gamma, and Gompertz but the results using survival models did not change the findings and the statistical measures for comparison (AIC, BIC, and Chi-Squared tests) indicated that the Tobit model fit the data better than the survival models. Thus, the Tobit model was used for the regression model estimation and normalization. This is supported by a recent study where Tobit model has similar performance as complex machine learning models for predicting incident clearance time [18] .
It is worth noting that the choice of regression model can have a significant impact on the results of the analysis. In the case of survival models, the factors have a multiplicative effect on the outcome, whereas they have an additive effect when using the Tobit model. As a result, normalization using survival models would require slight adjustments to the normalization procedure, although these adjustments would be minor in nature.
Overall, the selection of the appropriate regression model is crucial in ensuring the accuracy of the findings. While other methods may be explored in future studies, the Tobit model proved to be the most effective approach for our particular dataset.
6. Results
6.1. Model Results
This section describes the results from the Tobit model and shows the ranks of different agencies. In the ranking process, 49 variables are used. Their estimates are shown in Table 2. The coefficients indicate the adjustments to be made in minutes. The adjusted RCT, like the Tobit, were left truncated at 0 minutes (i.e., the adjusted RCT could not have negative values). The right three columns in Table 2 shows the estimates, standard error and P-value, respectively for variables used in the ranking from Tobit regression model.
The estimates from the regression model indicate that if the crash is in urban areas (Urban/Rural in Table 2), on average, the RCT is 7.381 minutes lower than the rural areas. This seems relevant as there is a possibility that the incident response teams have greater resources in urban areas compared to the rural areas which helps to lower the RCT. The RCT time is found to be 0.89 minutes higher in the dark (Lighting in Table 2) compared to daylight which matches with the results of [19] who found that the RCT time is slightly higher at night. The estimates also indicate that if the major cause of the crash is related to oversized vehicles or separation of trailer units, it results in an increase in the RCT. Similar results are found in the published literature for heavy vehicles [7] . In Table 2, almost all the coefficients indicate an increase in the RCT and the corresponding small p-values exhibit the usefulness of these variables in the model. This information could be used by agencies to allocate more resources in the crash site if these characteristics are present.
If there is any irregular weather condition, it will influence the RCT. In Table 2, there are specific categories of different weather conditions coded as “Primary Weather”. Foggy weather conditions are associated with a 5.2-minute increase for RCT on average compared to clear weather conditions. The sleet and snowy condition show that there will be a slight decrease in the RCT. Also, the snowy and sleety weather conditions have larger p-values which is consistent with the result of [6] who found that snowy weather does not impact the RCT significantly. The blowing snow condition increases the RCT by around 3.3 minutes as it is usually associated with windy conditions and hence could impact the clearance work.
Table 2. Result from Tobit regression model for variables used in ranking method.
If the roadway does not have lighting, it is associated with increased RCT time. Thus, the result indicates that crashes at night on roadways where there is no lighting may require greater attention from the response teams. The variable surface condition shows different results than expected. It shows that there will be a slight decrease in the RCT for snowy, icy, or slushy surfaces when compared to dry surfaces than dry surfaces. However, these variables have higher p-values. Thus, the accurate interpretation of these variables is not possible from these model results.
From the regression results, crash severity level has the largest impact on the RCT of the variables tested. If the severity level is fatal, it is associated with a 108.8-minute increase in the RCT, which coincides with the result of [6] and [20] . Similarly, the RCT will be increased by 28.5, 8.7, and 4.2 minutes for major, minor and possible injuries, respectively.
The “Road Type” variable shows if the crash happens at a railroad crossing, it is associated with higher times for clearance. Similarly, 4-way intersection, T-intersection, Y-intersection, Five points or more-Intersection and L-intersection also increase the clearance time. It was found that the intersection crashes in general have higher clearance times. On-ramp, off-ramp, on-ramp merge area, off-ramp diverge area and mainline of interchange will cause a significant increase in the RCT [7] . These roadway locations should be studied further to reduce the clearance time during these conditions. If the crash happens in a work-zone area, it takes an additional 2.1 minutes for RCT compared to the non-work zone areas.
6.2. Model Implementation & Ranking
After obtaining the variable estimates from the developed model, the variables were used in adjustments for the ranking process. Estimates from the model are considered as weights and the observed RCT is normalized according to these weights. For example, assume a crash had a severity level of fatal injury and required 200 minutes of clearance time. According to Table 2 results, a fatal injury crash usually has 108.8 minutes higher RCT than property damage crashes. The crash with 200 minutes RCT will be normalized into a PDO crash by subtracting the 108.8 minutes. Thus, the normalized RCT will be = 200 − 108.8 = 91.2 minutes. This normalization process is estimating what would be the clearance time, if all the crashes were PDO crashes in ideal conditions. If the crash occurs in an urban area, the RCT time decreases by 7.3 minutes (normalized to equivalent rural area). So, if the above-mentioned fatal crash occurs in an urban area, the adjusted RCT will be = 91.2 + 7.3 = 98.5 minutes. In this way, the process incorporates all 49 variables mentioned in Table 2.
The adjustment process described above was then implemented using all 49 variables on all crashes. Each crash would then contain the original RCT along with an adjusted RCT that normalizes the RCT for equivalent comparisons across crashes. One method utilized for comparison is to average the adjusted RCT of each agency and county. The agencies with the highest average adjusted RCT can then be reviewed to determine if additional resources or training would be beneficial for improving responses. Additionally, agencies with low average adjusted RCTs could be recognized for their work and efforts. Finally, a dashboard was prepared to show the ranking result based on the normalized RCT. Figure 3 shows all the counties of Iowa. In this paper, the actual ranking is not presented in Figure 3 due to data sensitivity. A specific county can be selected from the agency map (Figure 3), and the corresponding figures in the dashboard would change as per the selection. To minimize the data sensitivity and maximize the anonymousness of the ranking result, each of the figures presented in this paper, is plotted from different counties in Iowa without mentioning the county names.
If a single county is selected in dashboard, it will show the ranking of all emergency response teams within that county (in Figure 4). The agency names in Figure 4 have been redacted and simply referred to as agency numbers.
Figure 5 shows the normalized RCT of each crash for an agency over the time period. As the plot shows the normalized RCT of each crash with severity level, crashes still having a higher RCT suggests a in depth look into the crash reports.
Figure 3. Showing all the counties in Iowa included in the ranking process.
Figure 4. Showing the average normalized RCT for all agencies in a county.
7. Conclusions
This study provides an overall framework on how emergency response teams can be ranked based on their corresponding performance measures which is crucial for resource allocation and better response. This framework utilizes regression models (i.e., Tobit regression) to adjust roadway clearance time (RCT)
Figure 5. Showing the normalized RCT (minutes) of each crash for an agency over time.
and ensure fair comparisons across agencies. The adjustments are made for variables that influence RCT but are beyond the control of the responding agencies. In summary, this study:
• Provides a comprehensive framework for ranking agencies;
• Ensures the fair comparison among agencies while ranking;
• Once the adjustments are made, agencies can be ranked based on average adjusted RCT;
• Low performing agencies can then be reviewed to determine what resources and training may be beneficial for improving performance. High performing agencies can be commended for their efforts. In this way, the framework can be used to optimize use of resources for reducing overall RCT.
8. Limitations and Future Research
This study has several constraints. Some of the variables used in this study are derived variable by Iowa DOT officials and might be subjective. The variable, indicating the incident happened in the work-zone, have high percentage of missing values which are considered as “not work-zone” in this study. In most cases, information for response time is unavailable. Developing a model with response time and even normal summary statistics of response time would provide greater information for ranking purposes. In addition, the data does not have proper traffic volume related information (e.g., traffic volumes immediately prior to and during the incident through clearance). However, this volume data might be useful for better interpretation of the model. A number of trained personnel in those agencies might provide important information as well. In the regression analysis, a Tobit regression model was used. In the future, additional models can be used to estimate the weights of the variables. Overall, the framework developed in this study is transferable and this method can be followed by other DOTs to evaluate their teams.
Acknowledgements
We would like to thank Zachary Hans for assistance with variable selection for ranking process which helped to improve our methodology. We would also like to thank the many employees at Iowa Department of Transportation who provided feedback throughout the project.