Quantifying the benefits of safety improvements is an essential step in developing road safety infrastructure programmes. We measure the value of improving safety on our roads by quantifying how many deaths and safety injuries can be avoided, maximising the return on safety for dollar spent.
Earlier this year, Professor Bhagwant Persaud from Canada visited New Zealand on sabbatical and was a keynote speaker at the 2020 Transportation Conference. Bhagwant is a leading road safety researcher at the cutting edge of road safety analyses, including the development of crash prediction models and before-and-after evaluations of safety treatments.
The timing of his visit was fortunate, with fellow statistician Graham Wood and I providing strategic advice on safety programme evaluations to one of our key clients. We uncovered several challenges with these evaluations, most notably dealing with regression-to-the-mean. Regression-to-the-mean is a major issue in safety evaluation studies due to bias-by-selection (the treatment of predominately high-risk sites). By not adjusting for regression-to-the-mean, the benefits of safety countermeasures and safety programmes are overstated.
Another challenge is sites that have multiple safety treatments. Evaluation studies often do not control for the extra benefits associated with additional treatments. The safety benefits are ascribed to the main treatment, thereby overestimating its benefits. For example, when realigning a highway, the curve radii maybe increased (main treatment) along with road widening, improved delineation, and installation of safety barrier. These other changes also contribute to the overall safety benefit. Sometimes the number of road users changes when a treatment is implemented. For example, the number of cyclists often increases when cycle lanes or paths are developed as people are attracted to the improved facilities. Adjusting for increase in use is difficult as the relationship between safety benefits and the number of users tends to be non-linear.
With Bhagwant in Christchurch we were able to meet up and discuss the best statistical methods for handling these challenges. Here are some learnings that Bhagwant discovered across evaluation studies:
Bhagwant suggests that many people new to this area of analysis, and even some veterans, don’t do their homework and fail to address these important considerations when planning evaluation studies. Unfortunately this has led to some poor outcomes, with benefits of some treatments being overstated.
To improve general practice in this area he recommends practitioners consult the Highway Safety Manual and the Guide to Developing Quality Crash Modification Factors (available online), both produced in the US, and Ezra Hauer’s 1997 book on before-and-after studies.
As billions of dollars are invested in road safety countermeasures, it is more important than ever that we do safety evaluation studies correctly. Are we investing in the right countermeasures? Will our programmes achieve the investment outcomes of a fifty percent reduction in deaths and serious injury globally by 2030? Given lives are on the line, and a lot of money too, it is time to invest in quality evaluation studies to give us confidence in our safety investments.