Posted on: June 4, 2020 at 10:51 AM    

Written by Dr Shane Turner, with input from Professor Bhagwant Persaud and Professor Graham Wood

Quantifying the benefits of safety improvements is an essential step in developing road safety infrastructure programmes. We measure the value of improving safety on our roads by quantifying how many deaths and safety injuries can be avoided, maximising the return on safety for dollar spent.

Earlier this year, Professor Bhagwant Persaud from Canada visited New Zealand on sabbatical and was a keynote speaker at the 2020 Transportation Conference. Bhagwant is a leading road safety researcher at the cutting edge of road safety analyses, including the development of crash prediction models and before-and-after evaluations of safety treatments.

The timing of his visit was fortunate, with fellow statistician Graham Wood and I providing strategic advice on safety programme evaluations to one of our key clients. We uncovered several challenges with these evaluations, most notably dealing with regression-to-the-mean. Regression-to-the-mean is a major issue in safety evaluation studies due to bias-by-selection (the treatment of predominately high-risk sites). By not adjusting for regression-to-the-mean, the benefits of safety countermeasures and safety programmes are overstated.

Another challenge is sites that have multiple safety treatments. Evaluation studies often do not control for the extra benefits associated with additional treatments. The safety benefits are ascribed to the main treatment, thereby overestimating its benefits. For example, when realigning a highway, the curve radii maybe increased (main treatment) along with road widening, improved delineation, and installation of safety barrier. These other changes also contribute to the overall safety benefit. Sometimes the number of road users changes when a treatment is implemented.  For example, the number of cyclists often increases when cycle lanes or paths are developed as people are attracted to the improved facilities. Adjusting for increase in use is difficult as the relationship between safety benefits and the number of users tends to be non-linear.

With Bhagwant in Christchurch we were able to meet up and discuss the best statistical methods for handling these challenges. Here are some learnings that Bhagwant discovered across evaluation studies:

  1. Regression-to-the-mean is not a trivial effect and cannot be overlooked – even when engineers swear there is no bias-by-selection.
  2. It is important to control for changes in traffic volume, especially since safety treatments can lead to increased traffic – for example more cyclists on cycle paths.
  3. Safety benefits can vary depending on the circumstances in which the treatment is applied. Treatments are typically more effective in locations where they are most warranted.
  4. Driver adaptation can result in counter-intuitive effects for some treatments at locations where behavioural changes can result – for example increasing curve radii and widening seal width can result in higher speeds (behaviour change) and more severe crashes.
  5. Some treatments, like red light cameras, have a general deterrence effect, resulting in safety benefits at untreated sites. Conversely, some treatments cause crash migration, resulting in an increase in crashes at untreated sites.
  6. Partially addressing these evaluation challenges is as good as not addressing any.

Bhagwant suggests that many people new to this area of analysis, and even some veterans, don’t do their homework and fail to address these important considerations when planning evaluation studies. Unfortunately this has led to some poor outcomes, with benefits of some treatments being overstated. 

To improve general practice in this area he recommends practitioners consult the Highway Safety Manual and the Guide to Developing Quality Crash Modification Factors (available online), both produced in the US, and Ezra Hauer’s 1997 book on before-and-after studies.

As billions of dollars are invested in road safety countermeasures, it is more important than ever that we do safety evaluation studies correctly. Are we investing in the right countermeasures? Will our programmes achieve the investment outcomes of a fifty percent reduction in deaths and serious injury globally by 2030? Given lives are on the line, and a lot of money too, it is time to invest in quality evaluation studies to give us confidence in our safety investments.

 

iStock 545576976 v2