New report: FMCSA's safety measurement system is sound, implementation improvements needed

While the Federal Motor Carrier Safety Administration's (FMCSA) Safety Measurement System (SMS) used to identify commercial motor vehicle carriers at high risk for future crashes is conceptually sound, several features of its implementation need improvement, says a new congressionally mandated report from the National Academies of Sciences, Engineering, and Medicine. Over the next two years, FMCSA should develop a more statistically principled approach for the task, based on an item response theory (IRT) model—an approach that has been applied successfully in informing policy decisions in other areas such as hospital rankings. If the model is then demonstrated to perform well in identifying motor carriers that need interventions, FMCSA should use it to replace SMS.

Commercial motor vehicle (CMV) carriers are responsible for moving freight and passengers in a safe manner over the nation's highways. About 100,000 fatality- or injury-causing crashes involving large trucks and buses occur in the United States each year, the report says. Approximately 3.5 million CMV roadside inspections are conducted yearly by specially trained inspectors on approximately 900 potential violations of safety regulations that fall under six categories: unsafe driving, hours of service compliance, vehicle maintenance, controlled substances/alcohol use, hazardous materials compliance, and driver fitness.

FMCSA uses information that is collected mainly during these inspections to identify motor carriers that are operating unsafely and therefore are likely to be at higher risk for future crashes. Carriers found to have frequent violations are subject to interventions from FMCSA, which can include warning letters and investigations, the report says.

The report identified a number of data quality issues and recommended that FMCSA should continue to collaborate with states and other agencies to improve the collection of data on vehicle miles traveled and on crashes, data which are often missing and of unsatisfactory quality. Including vehicle miles traveled data by state and month will enable SMS to account for varied environments where carriers travel—for example, in icy winter weather in the North. In addition, there is information available in police narratives not represented in the data used that could be helpful in understanding the contributing factors in a crash.

In addition, the report says FMCSA should research ways of collecting data on carrier characteristics—including driver turnover rates, type of cargo, and method and level of compensation. For example, compensation levels are relevant because it is known that drivers who are better-compensated, and those not compensated based on miles traveled, have fewer crashes. This additional data collection would require greater collaboration between FMCSA and the states to standardize the effort and to protect carrier-specific information.

The committee that conducted the study and wrote the report was unable to recommend whether SMS percentile ranks should be made public because it would require a formal evaluation to understand the consequences of public knowledge of the information.