Evaluating and Improving the Effectiveness of Vehicle Inspection and Maintenance Programs: A Cost-Benefit Analysis Framework

Ying Li^*

Department of Environmental Health, College of Public Health, East Tennessee State University, Johnson City, USA.

DOI: 10.4236/jep.2017.812095                

Abstract

Motor vehicle inspection and maintenance (I/M) programs are designed to identify high-emitting vehicles and mitigate their impacts on air quality and climate. I/M programs have been traditionally ranked superior among various vehicle emission control measures by the results of cost-benefit analysis, based on the assumption that these programs will achieve the targeted emission reduction outcomes. However, the actual effects of I/M programs may be greatly uncertain and when this uncertainty is taken into account, these programs may become suboptimal. This study develops a new a cost-benefit analysis framework that links various program design consideration, such as program participation rate, identification rate and effective repair rate, to the public health benefits as well as costs of the programs. This framework helps decision makers to investigate minimum implementation requirements that at least ensure the benefits are greater than the costs of implementing the programs in order to improve the overall effectiveness of the I/M programs. To illustrate the applications of the framework, it was applied to a particulate matter oriented I/M program targeting all diesel-fueled vehicles in the city of Bangkok, Thailand, a large metropolitan area that has been suffering from severe ambient PM pollution mainly attributable to its wide use of diesel-fueled vehicles and motorcycles. It was found that the health benefits achieved from the program are sensitive to several key program design elements, including participation rate and problem vehicle identification rate, fraction of effective repairs and illegal operation rate. Other variables, such as the testing cut-points and vehicle population growth rate, only have modest effects on the overall emission reduction and consequent health benefits. Overall, the performance of multiple variables associated with I/M program design needs to be improved simultaneous in order to achieve the targeted benefits of the program.

Keywords

Inspection and Maintenance Programs, Cost-Benefit Analysis, Effectiveness, Program Design

1. Introduction

Motor vehicle inspection and maintenance (I/M) programs are designed to identify high-emitting vehicles and mitigate their impacts on air quality and climate [1] . In western developed countries such as the United States (US), these programs have been considered to be cost-effective and are required by the Clean Air Act Amendments of 1990 in regions with the most challenging air pollution problems [2] [3] . The main purpose of I/M programs is to encourage better maintenance for in-use vehicles and to assure the vehicle emission control systems are functioning properly through periodic inspections. The rationale for an I/M program is that the emission distribution among a vehicle population is highly skewed: A small portion of vehicles (estimated at 5% - 10%), sometimes called gross polluter vehicles, is responsible for a substantial fraction (variously estimated at 50% to 80%) of total vehicle emissions [4] [5] . Moreover, not only old vehicles can be gross polluters, but also vehicles of all model years may include some proportion of gross polluters [5] , due to the factor that vehicle emission levels are heavily dependent on maintenance. This problem can be even more pronounced in developing countries, where vehicles have a long lifetime and are often poorly maintained [6] . In this case, upgrading maintenance practices and replacing the worst engines should be considered first before moving on to better technology [7] . In addition, despite technological and regulatory advances, new vehicle standards are not sufficient to achieve pollution abatement goals if vehicles deteriorate rapidly, resulting in increasing emission rates [6] . Therefore, to control rapidly growing vehicle emissions, governments must not only affect the behavior of vehicle manufacturers and fuel suppliers, but also the actions of drivers in terms of how well they maintain their vehicles regardless of their vehicle ages [8] .

However, although simple in concept, the detailed design and implementation of I/M programs is challenging. For example, when emission control equipment malfunctions, vehicle performance may be unaffected, hence the driver has no private incentive to seek repairs, and demanding private expenditures of money and time by vehicle owners will create the usual tensions that lead many actors to try to evade the regulation in numerous ways [8] . The practices of I/M programs in the US have shown various barriers that may cause failure of these programs to generate the emission reduction originally anticipated by policy makers. For instance, motorists may have many opportunities to evade required repairs, such as testing vehicles numerous times until they happen to pass. By and large, launching an effective I/M program requires massive behavior change among the drivers of a region [8] .

Due to these challenges, the actual cost-effectiveness of I/M programs often remain uncertain. In theory, vehicular emission reductions available from I/M programs are mainly determined by the failure thresholds, or cut points, used to identify problem vehicles, but are also quite sensitive to a variety of factors, such as the actual percentage of problem vehicles identified, the percentage of problem vehicles waived from repairs or operating illegally, the emission reduction achieved by repairs, the durability of repairs, and so on [2] . In practice, these factors are often overlooked in designing and evaluating a regional I/M program.

This paper aims at developing a cost-benefit analysis framework for evaluating the effectiveness of vehicle inspection and maintenance programs, on the basis of the emission reduction assessment tool called “I/M Design” developed by [2] (referred to as EISINGER2005 hereafter), and the health benefit analysis tool developed in our previous study [9] . This combined analysis framework incorporates various factors that affect the level of emissions achieved by an I/M program, and links emission reduction with health benefits (avoided mortality and illnesses). The new framework is then applied to a hypothetical particulate matter (PM) oriented I/M program targeting all diesel-fueled vehicles in the city of Bangkok, Thailand, a megacity that has been suffering from severe adverse health effects attributable to PM for a few decades, to illustrate how the framework may help to evaluate the cost-effectiveness of I/M programs.

2. Methods and Data

2.1. Theoretical Framework

I/M programs are one of the essential policy tools to control emissions from in-service vehicles in severely polluted urban areas [2] . An I/M program has the potential to reduce emissions in a number of ways, such as better maintenance of vehicles by motorists as a result of the program, repairs made in anticipation of an I/M inspection (referred to as pre-test repairs) or as a result of failing the test, and early scrapping of vehicles that are not worth repairing [3] . Figure 1 illustrates the sources of emissions reductions resulting from an I/M program.

Based on this conceptual framework, EISINGER2005 developed a spreadsheet tool to evaluate the effectiveness of I/M programs in terms of levels of emission reductions achieved by those programs. The theoretical basis of this tool is that the amount of vehicle emission reductions resulting from an I/M program is a function of the following variables [2] :

・ Pre-I/M test repair work: potential vehicle repairs motivated by instituting I/M for the vehicles that would otherwise be identified as problems.

・ Post-I/M test repair work: emission reductions from repair work for the vehicles identified as problems by inspections. This variable itself is a function of the number (or the percentage) of the problem vehicles identified by I/M and the number (or the percentage) of identified problem vehicles being repaired effectively.

Figure 1. Sources of emission reductions from I/M programs (Partial information source: [3] ).

・ Vehicles scrapped: emission reductions will be generated from scrapping gross polluting vehicles that would otherwise continue to be used, and replacing them by low emission vehicles. This variable also includes high-pol- luting vehicles that are transferred outside the I/M region due to the implementation of the programs.

Based on this theoretical basis, this study modified the equations developed by EISINGER2005 and included six equations in the Spreadsheet used to estimate the benefits and costs of an I/M program as follows (all six equations Equations (1)-(6) were originally developed by EISINGER2005, and adopted in this study with minor modifications):

Equation (1) describes the percentage of all problem vehicles that are identified by an I/M program. This variable is a function of the program participation rate and problem vehicle identification rate:

$\text{ProbVeh}=\text{PartiRate}\times \text{IndenRate}$ (1)

where:

ProbVeh: Percent of all problem vehicles that are identified by an I/M program.

PartiRate: Percent of total required vehicles that participate in an I/M program.

IndenRate: Percent of problem vehicles inspected that fail the test.

Equation (2) describes the percentage of problem vehicles that are both identified by I/M and subsequently undergo repair work:

$\text{PercentRep}=\text{ProbVeh}\times \left[\left(\text{1}-\text{ScrapFrac}\right)\times \left(\text{1}-\text{VehWaive}\right)\times \left(\text{1}-\text{IllegalVeh}\right)\right]$ (2)

where:

PercentRep: Percent of all problem vehicles that are failed (identified) by I/M and subsequently repaired.

ProbVeh: Fraction of problem vehicles (vehicles emitting above certification standards) identified by the I/M program test (this is a function of false pass rates).

ScrapFrac: Percent of failed vehicles that are retired from the fleet within one year of failing their I/M test.

VehWaive: Fraction of problem vehicles (vehicles emitting above certification standards) identified by the I/M program test but allowed a waiver from needed repair work (a function of money spent on repairs).

IllegalVeh: Fraction of problem vehicles (vehicles emitting above certification standards) operating without obtaining the requisite repairs or certifications needed to pass or waive out of the I/M inspection process.

Equation (3) describes the percent of a vehicle’s total emissions that are reduced through repair work:

$\text{PercentRed}=\text{GoodRep}\times \text{ExEm}\times \text{DurRep}\times \text{EmisFrac}$ (3)

where:

PercentRed: Percent of total vehicle emission reductions achieved by repairs, for the vehicles failing I/M and getting repaired (does not include vehicles that fail I/M and are scrapped, waived, or illegally operating).

GoodRep: Fraction of repairs that are “good” (effective), as measured by percent of repaired vehicles that immediately pass a retest.

ExEm: Fraction of excess emissions (where “excess” means emissions above allowable levels, usually referred to as the “cutpoint”) reduced from identified problem vehicles that receive good repairs (I/M does not address all excess emissions, for example cold start emission problems).

DurRep: Durability of good repairs, as measured by percent of vehicles with good repairs that pass retests at 12 or 24 months.

EmisFrac: Fraction of total vehicle emissions represented by pre-repair excess emissions (this is a function of the “cutpoint” used to define the point at which a vehicle is allowed to pass I/M; emissions above the passing cutpoint are considered excess). In other words, emissions below I/M cutpoints are essentially acceptable, emissions above cutpoints are excess; this variable represents the percent of total vehicle emissions considered excess.

Equation (4) describes the benefits of repair work after I/M test:

$\text{BenefitsRep}=\text{PercentRep}\times \text{PercentRed}$ (4)

where:

BenefitsRep: The percentage of vehicles repaired (PercentRep), multiplied by the percentage reduction achieved per repair (PercentRed); units are in percent of total vehicle emissions reduced.

Equation (5) describes emission reduction benefits from vehicle retirements due to I/M test failures:

$\text{BenefitsScrap}=\text{ProbVeh}\times \text{ScrapFrac}\times \text{ScrapEmis}$ (5)

where:

BenefitsScrap: Percent reductions from all problem vehicles, due to vehicles that are scrapped (considering emissions from the replacement vehicles).

ScrapEmis: Percent of total vehicle emissions reduced, for each vehicle retired from the fleet, after accounting for replacement vehicle emissions.

Equation (6) describes total program benefits, in terms of the percent emission reduction in total vehicle emissions from the I/M program:

$\text{BenefitsTotal}=\text{BenefitsGrow}+\text{BenefitsRep}+\text{BenefitsScrap}$ (6)

where:

BenefitsGrow: Percent emission reductions achieved due to the changes in vehicle population growth as a result of I/M enforcement. It is calculated by using the projected total number of vehicles under the baseline and the I/M scenarios, as well as the fleet-average emission rates. This variable is not in the original “I/M Design” spreadsheet, but is developed by this study to reflect future I/M effectiveness as vehicle fleets change over time.

The unit for BenefitsTotal is percent of total vehicle emissions reduced, for the entire problem vehicle fleet.

This spreadsheet tool, developed by modifying the tool given in EISINGER2005, allows users to adjust the values of parameters in the model and obtain the resulting percentage of emission reductions in total vehicle emissions. Although the tool is developed in the US for the most common gasoline vehicle I/M programs, the fundamental ideas of vehicle I/M programs are universal and thus the theoretical modeling framework applies to I/M programs targeting other pollutants in other regions. However, the values of the variables must reflect the specific contexts and issues of concern with respect to the interested area and programs¹.

The present study used this spreadsheet tool as a cost-benefits framework that links I/M design considerations with health benefits associated with the programs in order to understand the impacts of some key issues regarding I/M design, such as compliance rates, testing cut-points and effectiveness of repairs, on the potential health benefits of the programs.

Our previous study estimated the potential health benefits associated with the proposed PM-oriented I/M programs targeting all diesel-fueled vehicles and motorcycles in the city of Bangkok, Thailand, a megacity in Asia that has been suffering from severe adverse health effects attributable to ambient PM for decades [9] . In that study, the health benefits as a function of different levels of PM₁₀ (all particulates with an aerodynamic diameter of less than or equal to 10 μm) emission reductions were analyzed and compared with the social costs of the I/M programs (these different levels of reductions were considered due to the significant uncertainty involved in the actual emission reduction benefits of I/M programs). It was found that a minimum of about 4% reduction of the total PM₁₀ emissions from motor vehicles is required in order for the total benefits to be greater than the total costs of implementing the programs. On the basis of that study, the main purpose of the current study is to examine how key variables affect PM₁₀ emission reductions available from the same I/M programs, and the desirable performance of these variables in order to achieve the 4% minimum emission reduction objective. The best available information about I/M experience in Thailand and elsewhere were used as the inputs to the models in spreadsheets.

2.2. A Framework to Estimate the Effectiveness of I/M Programs

Empirical evidence on the performance of I/M programs and on the important elements affecting I/M emission reductions is fairly limited. Evidence on the performance of PM-oriented I/M is even less available given that these programs are still relatively new. In Thailand, the data collected by a World Bank study [10] on their pilot motorcycle inspection and upgrade project in Bangkok are the most comprehensive dataset on I/M programs in the Bangkok Metropolitan Region (BMR). Very little information is available on the performance of I/M targeting diesel-fuel vehicles including buses and trucks. Given the limitation of data, in running the spreadsheet developed by EISINGER2005, the values of most variables in the tool are derived based on the best available information in the U.S. and some extrapolation is performed to the BMR. Definitions of the variables in “I/M Design” and their input values used in this study are presented in Table 1.

In addition to the input variables listed in Table 1, the spreadsheet also needs the following inputs related to the characteristics of the vehicle population studied: 1) Problem vehicles as percent of total vehicles: The values used in the spreadsheet were consistent with the assumptions made in our previous study― 10% of buses and heavy trucks, 17.5% of light trucks and 25% of motorcycles in the BMR are problem vehicles [9] . In the uncertainty analysis, the upper and lower bounds of this parameter were assumed to be 1.5 times and half of the mean estimate, respectively. In lack of empirical evidence to support the form of PDF of this parameter, the uniform distribution was selected based on the authors own judgment; 2) Problem vehicles as percent of total PM emissions: Studies usually suggest that the gross-polluting vehicle pool is responsible for a substantial fraction―ariously estimated at 50% to 80%―of total vehicle emissions [4] [5] . Based on this, this study assumes that 50% and 80% are the lower and upper limits, respectively, and the mean value of them, 65%, is the best estimate of total vehicular emissions are generated by problem vehicles. Also, the uniform distribution was selected in the uncertainty analysis based on the author’s own judgment; 3) Number of vehicles (under both the baseline and the I/M scenarios), average annual vehicle kilometers traveled (VKT) per vehicle and baseline fleet-average emission rates (in the unit of g/km-vehicle).

The testing cut-points for each type of vehicle need to be determined and input into the spreadsheet. Emission cut-points are established in I/M programs to

Table 1. Variables governing I/M program emission reductions (All variables in this table were developed and defined in EISINGER2005 unless noted).

¹The input values used in EISINGER2005 were based on light-duty vehicle hydrocarbon (HC) inspection data from an enhanced I/M program in southern California’s South Coast Air Basin. ²The original study examines the emission reduction benefits by an I/M program in a previous year using data on the actual number of vehicles inspected. It assumed that all vehicles subject to inspection participated in the program. ³Triangular distribution was selected for all the variables in Table 1 except for VehWaive. There is little empirical evidence to support the PDFs of the variables in Table 1. Therefore, the PDFs were selected based on the author’s own judgment. When there is more confidence in values near the central value than in values far away on either side, the triangular distribution was selected. In the case of the variable VehWaive, there is no reason to believe that some values between the lower and upper limits are larger than the others, and therefore, the uniform distribution was selected. Furthermore, the range was obtained based on the mean and the known theoretical limit of the variable, namely, 0% for the lower limit or 100% for the upper limit.

identify the worst polluters and minimize false failures [2] . In reality, vehicle emission rates usually span a wide spectrum. Conceptually, if an I/M targets a 25% reduction of the total emissions from motor vehicles, a cut-point equal to 75% of the current fleet-average emission rate will reduce the emission rates of all vehicles to 75% of the current level or lower, and thus ensure that the 25% emission reduction target is reached with confidence. However, given that the emission rates of the large portion of “good” vehicles (e.g. 90% of total vehicles) are usually much lower than the small portion of problems vehicles (e.g. 10% of total vehicles), it is not necessary, or probably not feasible either, to cut the emission rates of all vehicles to 75% of the current average level or lower in order to achieve the 25% reduction goal². More stringent cut-points may be able to fail more vehicles, in particular those with emission rates close to the failure

cut-points. However, more stringent cut-points are also likely to increase the social costs of I/M programs, and to suffer from problems such as technological infeasibility and motorist acceptance of the programs. While how to select and modify testing cut-points in I/M design to optimize the program effectiveness is beyond the scope of this study, this study uses the “ideal” cut-points discussed above, i.e. cut-points equal to 75% of the baseline fleet-average emission rates for each vehicle type in the BMR for the “best estimate” case, followed by an examination of the impacts of alternative cut-points on overall emission reduction levels. Just for comparison, the enhanced I/M in southern California’s South Coast Air Basin studied in EISINGER2005 used a rate of 86% of the baseline the fleet-average emission rate as the failure cut-point (the baseline rate was 1.25 g/mi and the cut-point was 1.08 g/mi). Table 2 summaries the fleet-average

Table 2. Fleet-average PM₁₀ emission rate in the BMR under the baseline and I/M Scenario (Year: 2008).

Notes: ¹The baseline emission rates were derived from data published in [9] using PM₁₀ emission rates in the year 2000 and the assumption that a 5% annual PM emission factor decrease rate for all types of vehicles in the BMR. ²The emission rates under the I/M scenario were calculated as baseline rate × 75%, assuming cut-points equal to 75% of the baseline fleet-average emission rates for each vehicle type.

PM₁₀ emission rates for various vehicle type under the baseline scenario and the I/M scenario.

EISINGER2005 considered that a small fraction (in the range of 0% - 7.5%) of the initial problem vehicles seek repairs in anticipation of I/M tests, and they were assumed to be “good” vehicles in inspection and pass the I/M test. Although the emission reductions resulting from this kind of “pre-test” repairs were taken into account in EISINGER2005, these reductions are only responsible for a small fraction of total emission reduction benefits achieved by I/M programs, approximately ranging from 0% - 2%. It is expected that the fraction of problem vehicles seeking emission repairs before I/M will be even smaller in a developing country than that in the US, given that people are generally less wealthy and less able to afford the costs of maintenance and repairs. For simplicity, this study did not consider the emission reductions resulting from pre-test repairs.

3. Results and Discussion

3.1. Estimating Emission Reduction Effectiveness of the PM-Related I/M Programs in the BMR

Using the “best estimate” values of the variables listed in Table 1 and the estimated 2008 vehicle population, emission rates and VKT [9] , the “I/M Design” spreadsheet was run for the year 2008. The results show that the PM-oriented I/M programs are expected to reduce total PM₁₀ emissions from motor vehicles in the BMR by 10.6%. Table 3 summarizes the findings.

Therefore, in the “best estimate” case, the proposed PM-oriented I/M programs in the BMR are expected to yield health benefits that exceed the social costs of the programs (the “threshold” for achieving this goal is a 4% overall PM emission reduction achieved by the programs, as found in [9] .

3.2. Examining the Roles of Key Design Elements on the Emission Reduction Benefits of I/M Programs

A series of sensitivity analyses were conducted to examine the sensitivity of the effectiveness of the I/M programs in terms of the percent of overall PM₁₀ emission reduction to the key design elements.

3.2.1. The Effects of Testing Cut-Points on Overall Emissions Benefits

The “best-estimate” in Section 3.1 is based on the assumption that failure cut-points are 75% of the baseline emission rates for each vehicle type. Since the cut-points determine the size of the initial problem vehicle pool (a more stringent testing cut-point is likely to result in more vehicles with “excess” emissions and, hence, subject to repair or replacement), changes to the cut-points will result in changes in the other two inputs: problem vehicles as percent of total vehicles and problem vehicles as percent of total emissions. It is difficult to estimate the magnitude of changes in these two variables as a result of the changes in failure cut-points without knowing the distribution of emission rates. Here it

Table 3. Estimated PM₁₀ emission reduction benefits of PM-related I/M programs in the bangkok metropolitan region.

was assumed that slight changes in cut-points do not change the values of the two variables (this may be true in the case that the majority of good vehicles have emission rates much lower than the cut-points, and the majority of problem vehicles have emission rates much higher than the cut-points), so in this case failure cut-points only affect the new emission rates of problem vehicles after repairs and retests. Based on this assumption, a cut-points sensitivity analysis was conducted. Figure 2 shows the results.

Figure 2 was generated based on the assumption that all the other input variables are independent of the failure cut-points, i.e. changing the cut-points while holding all other variables constant to examine the sensitivity of overall PM emission reductions to failure cut-points. Figure 2 indicates that the when cut-points decrease from 100% to 60% of the baseline emission rates, the percent of overall emission reduction increases from 9.8% to 11.1%. Therefore, cut-points modifications within a certain range (e.g. from 60% - 100% of the

Figure 2. Effects of testing cut-points on the percentage of overall emission reduction by I/M programs.

baseline emission rates in this case) only have modest effects on the overall emission reduction benefits (the percent of overall PM₁₀ emission reductions changes from approximately 10% to 11% as the result of changing the failure cut-points from 60% - 100% of the baseline emission rates), because most vehicles are considered to emit at levels well outside the range (either higher or lower the cut-points). This study considers that other I/M design elements may have more significant effects on the emission reduction benefits achieved by I/M programs. In general, important variables in I/M design that policy makers need to address include program participation rate, problem vehicle identification rate, effective emission repair rate and problem vehicle illegal operation rate. In the following section, the impacts of these variables on emission reduction were analyzed.

3.2.2. Key Variables Affecting I/M Effectiveness

1) Participation rate and problem vehicle identification rate associated with I/M programs. Participation rate (PartiRate) and problem vehicle identification rate (IdenRate) are two key elements to address in designing I/M programs. Participation rate represents the levels of program enforcement. A successful I/M program minimizes vehicle violations (vehicles required by an I/M program do not participate in the program). The “best estimate” case in Section 3.1 assumes the majority (90%) of vehicles in the BMR required by I/M will participate in the programs, i.e. they will undertake appropriate emission inspection (reasons discussed in Table 1). However, if a large fraction of vehicles subject to I/M tests escape from the inspection process, the emission reduction benefits of I/M are expected to decrease considerably. Problem vehicle identification rate represents the ability of I/M programs to identify vehicles that exceed the emission standards and thus need emission repairs.

Sensitivity tests of each of the two variables were conducted by changing the input value of one variable while holding all other inputs constant (presuming that all input variables are independent of each other). The goal is to examine the sensitivity of the percent of overall emission reduction to program participation rate (PartiRate) or problem vehicle identification rate (IdenRate). Figure 3 shows the results.

The square-marked and the triangle-marked lines represent the percent of overall PM emission reductions achieved by I/M programs as a function of program participation rate (PartiRate) and problem vehicle identification rate (IdenRate), respectively (as noted in the figure). Each of the two lines was generated by incrementing the value of an individual variable (PartiRate or IdenRate) by 10% at a time (starting from 0% and ending at 100%), while setting all other inputs to their best estimates. And the red solid line represents the minimum percentage of emission reduction required in order for the benefits of the programs to outweigh the costs (the value was 4% as found in [9] ).

Figure 3 indicates that both participation rate and problem vehicle identification rate are important determinants of overall PM emission reduction benefits achieved by I/M programs. For participation rate, when the value of this variable increases from 0% (lower bound) to 100% (upper bound), the percent of overall emission reductions from vehicles increases from 0.6% to 11.7%; for problem vehicle identification rate, the percent of overall emission reductions from vehicles increases from 0.6% to 20.6% when the variable’s value changes from the lowest to the highest. Comparing the effects of the two variables in Figure 3 shows that problem vehicle identification rate has a greater impacts on the overall emission reduction benefits than program participation rate, since for the same increment (e.g. 10%) in the two variables, the incremental emission reduction benefits resulting from the change in the problem vehicle identification rate

Figure 3. Effects of participation rate and problem vehicle identification rate on the percent of overall PM emission reduction by I/M programs.

are greater.

In order to achieve the goal of 4% PM emission reduction from motor vehicles, the participation rate is required to be greater than 30%, if all other inputs remain the same values as in the “best estimate” case. And the requirement for problem vehicle identification rate is 17% when setting the other variables in the spreadsheet to their “best estimate” values.

2) The impacts of the effectiveness of problem vehicle repairs. Repairing problem vehicles to meet emission standards is the major source of emission reduction available from I/M programs. Three variables in the I/M Design spreadsheet are related to the effectiveness of repairs: GoodRep―Percent of repair work initially effective; ExEm―Percent of excess emissions (emissions above allowable levels) from identified problem vehicles reduced by repairs that properly pass an I/M test immediately; and DurRep―Percent of good repairs that remain durable until the next I/M inspection. Figure 4 shows the impact of each individual variable on the levels of emission reduction achieved by I/M programs.

The three marked lines (named as GoodRep, ExEm and DurRep) were generated using the same approach as used to generate Figure 3: Each line is generated by incrementing the value of the individual variable it represents (GoodRep, ExEm or DurRep) by 10% at a time (starting from 0% and ending at 100%), while setting the other inputs to their “best estimate” values. The red solid line also represents the minimum PM emission reduction target of 4%.

Figure 4 illustrates that the increase in the values of any of the three variables related to problem vehicle repairs results in considerable improvement in emission reduction performance by the I/M programs, as these variables are key

Figure 4. Effects of problem vehicle repairs on the percent of overall PM emission reduction by I/M programs.

determinants of the effectiveness of emission repairs. Comparing the three marked lines in Figure 4 shows that while the same increment in any one of the three variables results in approximately similar incremental emission reduction benefits, the impact of initially effective repair rate (GoodRep) is slightly greater than the impacts of the other two variables. The 4% emission reduction target requires a minimum of 22% of repair work initially effective (GoodRep), or 25% of excess emissions from identified problem vehicles reduced by repairs (ExEm), or 26% of repairs that properly pass an I/M remain durable until the next I/M inspection (DurRep).

3) The impacts of illegal operation by problem vehicles. Illegal operation here refers specifically to failed vehicles that continue to run on roads without appropriate repairs or certificates of waiver (the variable IllegalVeh in Table 1). There are other types of illegal operation in I/M program implementation. For example, vehicles may run on roads without taking the inspection required by the programs. This latter type of illegal operation is considered in the program participation rate variable, so it is not taken into account here. Illegal operation by failed vehicles may considerably damage the performance of I/M programs, since these vehicles are identified as gross emitters. Using the same sensitivity test approach as in Figures 3-5 was generated, which shows the effects of failed vehicle illegal operation rate on the levels of overall emission reduction achieved by I/M programs.

Therefore, the increases in illegal operation rate by failed problem vehicles can substantially reduce the emission reduction benefits achieved by I/M programs. In order to achieve the goal of 4% PM₁₀ emission reduction from motor vehicles, the rate of failed problem vehicle illegal operation should not go over 75%, presuming that the performance of the other variables is at the level of the “best

Figure 5. Effects of problem vehicle illegal operation on overall emission reduction benefits.

estimate”. In reality, it may be unrealistic to achieve 0% illegal operation by failed problem vehicles. However, minimizing the problem vehicle illegal operation rate is an essential I/M design element to improve the effectiveness of I/M programs.

3.3. Improving the Emission Reduction Effectiveness of I/M Programs

For the PM-oriented I/M programs in this study, a 25% PM₁₀ emission reduction in the BMR was originally proposed as an upper bound target of the I/M programs based on the past experience of similar programs in the US [9] . Also as discussed earlier, the levels of PM₁₀ emission reductions actually achieved by the programs are significantly uncertain. The results in Figures 3-5 indicate that based on the assumptions made in Table 1, the improvement in the performance of any individual element is not sufficient to achieve the upper bound target of 25% overall emission reduction initially expected in proposing the programs to be adopted in the BMR.

For example, when one of the key variables discussed above reaches the upper bound, i.e. 100% (for IllegalVeh, the upper bound is 0%), while holding the other variables the same as in the “best estimate” case, the percent of overall PM₁₀ emission reduction is summarized in Table 3.

The results in Table 4 show in the case that only one variable in the spreadsheet increases while the values of the others remain the same as assumed in Table 1, even if the variables achieve complete success, the maximum level of PM emission reduction benefits is 20.6% (when IndenRate reaches 100%). Therefore, the performance of more variables needs to be improved simultaneously.

As two illustrations, when the values of PartiRate, IndenRate, IllegalVeh, GoodRep, ExEm and IllegalVeh were replaced by the lower levels found in the I/M program in southern California’s South Coast Air Basin [2] , the percent of PM emission reduction from motor vehicles increased to 19.1%; and when the values of all the key variables discussed above (PartiRate, IndenRate, IllegalVeh, GoodRep, ExEm, and DurRep) were set to the upper values found in California’s I/M, the percent of PM reduction increased to 30.9%. Table 5 summarizes the

Table 4. Emission reduction benefits in the case that one key variable reaches the upper bound.

Table 5. Improving the effectiveness of the I/M Programs in the BMR by increasing the values associated with key design elements.

*Note: Data in parentheses were not used in the calculation since they are smaller than the “best estimate” in this study.

replaced variable values used and the new emission reduction estimates.

3.4. Emission Reduction Benefits Due to the Change in Vehicle Population Growth

The “best estimate” in Table 3 was based on the assumption that the implementation of the new PM-related I/M programs causes 10% decrease in average annual vehicle growth rate in the BMR [9] . This assumption is associated with the variable BenefitsGrow (the percent of emission reductions achieved due to the changes in vehicle population growth as a result of I/M enforcement). Sensitivity test was conducted by changing the percentage decrease in annual vehicle growth rate from 10% to 0% (no change in the annual growth rate), 20% or 30%, while holding all other input unchanged. Figure 6 summarizes the results.

Past experience in rapidly developing metropolitan areas in Asia shows that the introduction of vehicle I/M programs may slightly slow down the fast growth of motor vehicles in these areas and it is expected that the percent decrease in average annual vehicle growth rate falls into the range of 0% - 30%. Figure 6 indicates that the change in the assumption about the percent decrease in annual vehicle growth rate has modest impact on the overall emission reduction benefits, when the change falls into the range of 0% - 30%.

3.5. Uncertainty Analysis Results

Contribution to variance is a measure of the fraction of the total uncertainty (variance) in the risk estimate that comes from the uncertainty in a particular parameter, when all parameters are allowed to vary simultaneously [12] . Research to reduce uncertainty then should focus limited resources on narrowing the uncertainty in the premise showing the greatest contribution to variance [12] . Table 6 summarizes the top 10 variables that contribute the most significantly to the uncertainty in the percent of overall PM₁₀ emission reductions. The analysis was conducted using Monte Carlo simulation based on the variable PDFs listed in Table 1, and were performed in the Oracle Crystal Ball software. The sample size was set as 5000.

Table 6 indicates that the problem vehicle identification rate of light-duty trucks showed the greatest contribution to variance. The contribution to variance of this premise is large probably both because the overall PM₁₀ emission reductions are sensitive to this variable (as found in Figure 3), and because the

Figure 6. Impact of the change in vehicle growth rate on the overall PM₁₀ emission reductions by the I/M programs.

Table 6. Variables showing the greatest contribution to variance.

uncertainty in this premise is large (falls into the range of 0% - 100%, see Table 1). Moreover, the problem vehicle identification rates of several other vehicle types (buses, city trucks and motorcycles) are also among the top 10 premises that contribute the greatest to the total uncertainty. Therefore, policy design considerations need to focus on increasing problem vehicle identification rate in order to narrow its uncertainty and improve its effectiveness.

3.6. Summary of Sensitivity Test Results

Table 7 summarizes the sensitivity test results. It indicates that the level of PM₁₀ emission reductions available from the I/M programs is the most sensitive to the variable “problem vehicle identification rate (IdenRate)”, since when increasing a variable from its lower limit to the upper limit while holding all the other variables constant, the greatest change happened with this variable (increased from 0.6% to 20.6%). Also, the uncertainty analysis demonstrates that the same variable “IdenRate” contributes the greatest to variance. Moreover, the variables associated with light-duty trucks play a relatively major role on the effectiveness of the I/M programs due to its role as the largest contribution to total PM₁₀ emissions from motor vehicles. These findings suggest that program effectiveness can be improved by narrowing the uncertainty in the problem vehicle identification rate and by identifying a greater percentage of problem vehicles. Also, attention should be directed toward the light-duty diesel vehicle fleet in introducing the programs.

Over all, the sensitivity analysis performed here indicates that, in order to increase the problem vehicle identification rate, a key point is to improve testing procedure to maximize the ability of the programs to detect vehicles that need emission repairs. Second, studies have suggested that using more stringent testing cut-points can increase the percent of problem vehicles that are discovered by the inspection [2] . Further research on how to maximize the problem vehicle identification rates associated with I/M programs is warranted.

4. Conclusions

Since air pollution control usually imposes substantial costs on a society, an understanding of the link between specific control policies and associated health benefits would provide valuable information to decision-makers. Despite the fact that the actual effects of some mitigating policies, such as the in-use vehicle inspection and maintenance (I/M) programs, may be greatly uncertain, this uncertainty issue has traditionally been ignored in evaluating the impacts of a policy on public health. An important uncertainty issue, namely, the uncertainty about the actual effects of pollution mitigating policies, has traditionally been ignored in assessing the public health benefits of control policies. By assuming full implementation of a policy measure, an evaluation may considerably overestimate the health benefits achieved by that policy, or simply shift the focal point of decision-making processes, if the emission savings are in fact considerably

Table 7. Sensitivity test of the percent emission reductions to the key design variables.

uncertain for decision makers or likely to be far less than anticipated. Consequently, despite that I/M programs have been traditionally ranked superior among various vehicle emission control measures by the results of Cost-Benefit Analysis (CBA) or Cost-Effectiveness Analysis (CEA), these programs may be suboptimal if the uncertainty issue mentioned above is taken into account. Given this, this study developed a new cost-benefit analysis framework for evaluating the effectiveness of I/M programs. This framework takes into account the effects of various program design considerations, such as program participation rate, identification rate and effective repair rate, on the health benefits of policy implementation, and examines what are the minimum implementation requirements that at least ensure the benefits are greater than the costs of implementing the programs.

Applying the framework to a PM-oriented I/M program targeting all diesel- fueled vehicles in the city of Bangkok, Thailand, it was found that the health benefits achieved from the program are sensitive to several key program design elements, including participation rate and problem vehicle identification rate, fraction of effective repairs and illegal operation rate. Other variables, such as the testing cut-points and vehicle population growth rate, only have modest effects on the overall emission reduction and consequent health benefits. Overall, the performance of multiple variables associated with I/M program design needs to be improved simultaneous in order to achieve the targeted benefits of the program.

The main limitation of using the new analysis framework to evaluate the effectiveness of the I/M programs is that presently there is very limited information on the performance of the important program design elements globally. The findings from running the “I/M Design” spreadsheet could be improved when more empirical data worldwide for the input variables are collected. Secondly, one source of emission reductions achieved by the I/M programs is improved maintenance of vehicles in anticipation of the required inspection process, but this kind of emission reduction relative to the baseline is not considered in estimating the emission reductions delivered by the programs. This portion of emission reductions can be large for diesel-fueled vehicles since pollution levels from these vehicles are heavily dependent on maintenance, perhaps resulting in an underestimate of the emission reduction benefits of the programs. Given the information gap, further research is warranted to examine the potential emission reductions due to improved maintenance by vehicle owners in anticipation of required I/M testing. Finally, further research needs to concentrate on the question that how to design and implement a PM-oriented I/M program in order to improve the performance of the key variables found in this study, e.g. the problem vehicle identification rate associated with an I/M program, so that more concrete and practical advice is provided to decision makers based on the theoretical conclusions of the present study.

List of Abbreviations

BMR: Bangkok Metropolitan Region

CBA: Cost-Benefit Analysis

CEA: Cost-Effectiveness Analysis

CO: Carbon Monoxide

I/M: Inspection and Maintenance

NOx: Nitrogen Oxides

HC: Hydrocarbon

PDF: Probability Density Function

PM: Particulate Matter

PM₁₀: Particulate Matter Having an Aerodynamic Diameter of Less than or Equal to 10 Micrometers

NOTES

¹Based on a conversation with the author of the paper, Dr. Douglas S. Eisinger.

²A hypothetical example is provided here: Assuming that a vehicle fleet has an average emission rate of 1.0 g/km. 10% of all vehicles are gross polluting and they are responsible for 50% of the total emissions. Based on this information, it can be derived that the average emission rates for good and problem vehicles are 0.56 g/km and 5.0 g/km, respectively. As long as the average emission rate for problem vehicles goes down to 2.46 g/km, the fleet-average rate will decrease to 0.75 g/km. Therefore, if all problem vehicles can be properly identified and fixed, a cut-point of 2.46 g/km will ensure the 25% emission reduction goal accomplished.

Conflicts of Interest

The authors declare no conflicts of interest.

References

[1]	Posada, F., Yang, Z. and Muncrief, R. (2015) Review of Current Practices and New Developments in Heavy-Duty Vehicle Inspection and Maintenance Programs. The International Council on Clean Transportation White Paper, Washington DC.
[2]	Eisinger, D.S. (2005) Evaluating Inspection and Maintenance Programs: A Policy-Making Framework. Journal of the Air & Waste Management Association, 55, 147-162. https://doi.org/10.1080/10473289.2005.10464609
[3]	National Research Council (2001) Evaluating Vehicle Emissions Inspection and Maintenance Programs. National Academies Press, New York.
[4]	Harrington, W. (1997) Fuel Economy and Motor Vehicle Emissions. Journal of Environmental Economics and Management, 33, 240-252. https://doi.org/10.1006/jeem.1997.0994
[5]	Beaton, S.P., Bishop, G.A., Zhang, Y., Ashbaugh, L.L., Lawson, D.R. and Stedmam, D.H. (1995) On-Road Vehicle Emissions: Regulations, Costs, and Benefits. Science, 268, 991-993. https://doi.org/10.1126/science.268.5213.991
[6]	Sterner, T. (2003) Policy Instruments for Environmental and Natural Resource Management. Resource for the Future Press, Washington DC.
[7]	World Bank (2001) Vehicular Air Pollution: Setting Priorities, South Asia Urban Air Quality Management Briefing Note No. 1. http://siteresources.worldbank.org/PAKISTANEXTN/Resources/UrbanAir/VehicularAirPollution.pdf
[8]	PA Government Services, Inc. (2004) Vehicle Inspection and Maintenance Programs: International Experience and Best Practices. Report, The US Agency for International Development Office of Energy and Information Technology, Washington DC.
[9]	Li, Y. and Crawford-Brown, D.J. (2011) Assessing the Co-Benefits of Greenhouse Gas Reduction: Health Benefits of Particulate Matter Related Inspection and Maintenance Programs in Bangkok, Thailand. Science of the Total Environment, 409, 1774-1785. https://doi.org/10.1016/j.scitotenv.2011.01.051
[10]	World Bank (2003) Thailand: Reducing Emissions from Motorcycles in Bangkok. Report 275/03, World Bank Energy Sector Management Assistance Programme.
[11]	USEPA, United States Environmental Protection Agency (2007) Cleaner Diesels: Low Cost Ways to Reduce Emissions from Construction Equipment. United States Environmental Protection Agency National Center for Environmental Innovation.
[12]	Crawford-Brown, D.J. (1999) Risk-Based Environmental Decisions: Methods and Culture. Kluwer Academic Publishers, Boston, MA. https://doi.org/10.1007/978-1-4615-5227-7