Blackspots are sections of road that are considered high risk because a high number of crashes have occurred on them.
The number of deaths on Victoria’s roads declined from 776 in 1989 to 266 in 2019. Road safety strategies, such as mandatory seatbelts, breath testing, speed cameras and safety infrastructure upgrades at blackspots, have successfully reduced Victoria’s road toll.
However, data from the Transport Accident Commission (TAC) shows that reductions in the road toll have slowed in the last five years. It also shows that in the last 10 years, most road accidents that resulted in death or serious injury occurred on rural roads. These accidents primarily involved head on collisions or collisions with roadside objects, such as trees.
In 2013, the government announced the $1 billion Safe System Road Infrastructure Program (SSRIP) as part of its Road Safety Strategy 2013–2022. In 2016, it launched the 2016–2020 Towards Zero Strategy and Action Plan (the Towards Zero Strategy) to complement the SSRIP. This included a $340 million program of road safety initiatives.
Wide centreline treatment involves widening the distance between opposing traffic lanes.
Rumble strips are small raised bumps that vibrate when driven over. This alerts the driver that they are straying out of their lane.
TAC funded the Towards Zero Strategy with the proceeds from annual vehicle registrations, which road users pay. Most of this funding was set aside for VicRoads—now part of the Department of Transport (DoT)—to deliver 27 road safety projects. These projects, collectively referred to as the Top 20 Program, involve installing road safety infrastructure, such as flexible barriers, wide centrelines and rumble strips, to 20 of Victoria’s highest-risk rural roads.
The Towards Zero Strategy aims to reduce the annual number of road deaths to 200 or less by the end of 2020 and reduce the number of serious injuries by 15 per cent within the five-year period from 2016. Installing flexible barriers is a significant part of the strategy. The strategy states that flexible barriers can reduce fatalities and serious injuries from run off road and head on serious casualty crashes by up to 85 per cent.
Large infrastructure projects can be complex, expensive and risky. In previous audits we have found that detailed planning, diligent project management and comprehensive oversight are critical to their success.
For this audit our objective was to determine if Victoria’s regional road barriers program is delivering its intended safety outcomes. To do this, we focused on TAC and VicRoads’ roles in planning, installing, maintaining and monitoring flexible barriers installed under the Top 20 Program.
While flexible safety barriers save lives and reduce serious injuries on Victoria’s roads, they are not as cost-effective as VicRoads and TAC intended.
A crash reduction factor is the percentage that a certain type of crash will reduce by after a road safety treatment is installed.
We undertook a statistical analysis to assess the effectiveness of the road safety treatments that VicRoads installed through 18 of the 21 completed projects under the Top 20 Program. We found that these treatments, which include flexible barriers and other safety measures, have most likely reduced fatalities and serious injuries on the treated sections of road by 46.5 per cent. This result is below the individual crash reduction factors that VicRoads set for 17 of the 18 completed projects. If this result persists, then the projects will not achieve their expected benefits and the Top 20 Program will be less cost-effective than intended.
VicRoads did not sufficiently plan its flexible barrier installation projects under the Top 20 program. As a result, it is likely to spend at least 22 per cent more installing the barriers than it originally budgeted for. Additionally, it has failed to properly maintain and monitor the barriers it installed, which increases the risk that they will not perform as intended.
VicRoads has not kept adequate records about project start and completion dates, and in some cases, it is missing data. This will impede its ability to fully evaluate the Top 20 Program’s outcomes and hold itself accountable to the community for the effectiveness of its investment.
Planning and approval
The Road Safety Executive Group consists of executives from different government bodies that are involved in road safety, such as Victoria Police, the Department of Health and Human Services (DHHS) and the Department of Justice and Community Safety.
The Ministerial Council for Road Safety is made up of government ministers who have portfolios that are relevant to road safety.
It is best practice for agencies to prepare a business case to support investment in a major infrastructure project. While it was not required by the Department of Treasury and Finance (DTF), TAC and VicRoads did not prepare a business case to support the SSRIP, the Towards Zero Strategy or the decision to install flexible barriers and other safety infrastructure treatments, which was initially costed at $340 million, on the top 20 roads.
The Road Safety Executive Group and the Ministerial Council for Road Safety both approved the Towards Zero Strategy. The government also approved TAC to fund it. TAC and VicRoads presented these groups with modelling, costings and a copy of the draft strategy. However, without a business case, there is no consolidated document that shows how TAC and VicRoads analysed different options and justified why the recommended investment was the best value-for-money option. It also means that the executive group, ministerial council and government were not presented with a consolidated picture of the strategy’s costs, timelines and risks as well as evidence for the effectiveness of flexible barriers.
TAC and VicRoads undertook extensive modelling of road trauma data and held workshops with experts to develop the Towards Zero Strategy. TAC and VicRoads also modelled the effectiveness of a range of different treatment options. However, a consolidated business case would have helped TAC and VicRoads demonstrate that the Top 20 Program was the best use of funds at the time. A consolidated business case would have also provided enough information for the government to make a better-informed decision about investing in flexible barriers and understanding the investment’s value for money in comparison to other options.
The Towards Zero Strategy stated that VicRoads would make 20 of Victoria’s highest-risk rural roads safer. VicRoads used two criteria to select the highest risk roads:
- roads with a speed limit of at least 100 kilometres per hour
- roads with a daily traffic volume of 3 000 vehicles or higher.
A serious casualty crash is a road accident that results in a driver, passenger or pedestrian being seriously injured or killed.
After identifying the roads that met both of these criteria, VicRoads assessed how many serious casualty crashes occurred on them over five years. It then selected the 20 roads that had the highest rate of serious casualty crashes during this period.
VicRoads did not formally document its reasoning for its road selection approach. While VicRoads has data to support why it identified high speed roads as high risk, it does not have sufficient evidence to support the relationship between traffic volume and risk.
Project design and approval
In 2014, TAC and VicRoads signed the Safe System Road Infrastructure Program Funding Deed (the SSRIP Funding Deed). The Top 20 Program is partly funded by the SSRIP.
VicRoads is delivering the Top 20 Program through 27 projects under five investment plans. TAC approved these investment plans, which set financial limits for the project proposals that sit beneath them.
VicRoads used a 10 step process to develop these project proposals. It introduced this process to speed up project development and approval. As part of the process, VicRoads developed scope approval reports, which detail its planning for each proposed project. A joint committee of VicRoads and TAC staff were then meant to use these scope approval reports to endorse the project proposals. Figure A outlines this process.
SSRIP approval process
SSRIP funding program
$1 billion SSRIP funding for road safety initiatives
Outlines a program of projects
Endorsed by a joint committee of VicRoads and TAC staff
Final proposal approved by VicRoads
Contains specific details about Top 20 Program projects
VicRoads could not demonstrate that the joint committee received the scope approval reports for 15 of the 27 Top 20 Program project proposals. Consequently, there is no evidence that the joint committee was informed about key project details—such as the required length, type and cost of the flexible barriers—when it endorsed them.
The next 16 are an additional 16 regional road sections where VicRoads is planning to install flexible barriers and other road safety infrastructure.
A 95 per cent confidence interval is the range of values of which you can be 95 per cent confident that the true value lies within.
For the next 16 roads, VicRoads completed more detailed project design plans before approving the proposal and submitting it to the joint committee.
Evidence supporting flexible barriers
The Towards Zero Strategy states that flexible barriers have been shown to reduce run-off-road and head-on serious causality crashes by up to 85 per cent. However, VicRoads does not have strong evidence to support this statement.
VicRoads based this statement on research that it commissioned from the Monash University Accident Research Centre (MUARC). MUARC’s research states that on the 11 roads it tested, flexible barriers led to an overall 56 per cent reduction (with a 95 per cent confidence interval of 25 per cent to 74 per cent) in run-off-road and head-on serious casualty crashes.
Two of these roads had a reduction of 87 per cent and 83 per cent, which is the basis for VicRoads’ 85 per cent statement. Respectively, the width of the 95 per cent confidence interval for these roads was 1 to 98 per cent and 47 to 94 per cent. VicRoads should not have relied on these results. It could not be confident that the real result was near 85 per cent because for the one road with a result reported as above 85 per cent, the confidence interval is too wide to be useful.
Use of the study’s overall result, a 56 per cent reduction, in the Towards Zero Strategy would have been more statistically robust. The result of our statistical analysis is also more consistent with this lower figure.
Crash reduction factors and benefit-cost ratios
A benefit-cost ratio is the ratio between the benefit of a project (as a dollar amount) and the project’s cost.
Each of VicRoads’ project proposals state the crash reduction factor and benefit cost ratio for the proposed safety treatment. VicRoads used these measures to convey the expected benefits of its Top 20 Program projects to the joint committee. The joint committee endorsed the projects and VicRoads’ chief executive officer (CEO) approved them.
VicRoads used each project’s crash reduction factor to calculate its benefit cost ratio. Under the SSRIP Funding Deed, project approval required the project to have a benefit-cost ratio of 1.5 or higher. This means that the benefit of a project must be at least 1.5 times more than its cost. If the crash reduction factor is overstated, then the benefit-cost ratio will be, which can mislead decision-makers, such as the joint committee.
VicRoads did not consistently calculate crash reduction factors for its Top 20 Program projects.
VicRoads advised that the crash reduction factors for these projects, including continuous flexible barrier installation, were based on an 85 per cent reduction informed by the results for the two highest performing roads in the MUARC research. VicRoads adjusted the 85 per cent crash reduction factor depending on how much flexible barrier was included in a project. For example, if less of the road segment was treated with barrier, the crash reduction factor was relatively less than 85 per cent.
However, VicRoads could not provide its calculations to support its adjusted crash reduction factors. Additionally, three past evaluations that MUARC completed of the Safer Road Infrastructure Program (SRIP) 1, 2 and 3 programs found that VicRoads has a history of overestimating crash reduction factors by up to 32 percentage points. For these reasons, we could not determine if the benefit cost ratios that VicRoads presented to its CEO and the joint committee were reasonable. Consequently, the program may be less cost-effective than intended because its cost may not achieve the intended outcome.
VicRoads’ crash reduction factors are based on crash data for all vehicle types, of which passenger vehicles make up the majority. VicRoads’ Top 20 Program investment plans and project proposals contain no information about how effective flexible barriers are for different types of road users, such as motorcyclists and heavy vehicle drivers.
While the Towards Zero Strategy references two studies about the effectiveness of flexible barriers for motorcyclists, neither of these studies have enough data for VicRoads to rely on.
VicRoads did not have finalised stakeholder communication and engagement plans for any of the top 20 roads before it started construction. These would have given VicRoads a better opportunity to identify and consistently address stakeholder concerns before beginning works.
VicRoads needed to make two infrastructure changes after it commenced works because it had not identified stakeholder concerns. These changes included:
- constructing additional emergency crossover areas on the Calder Freeway
- removing newly installed flexible barriers on stretches of the Princes Highway East to ensure that motorists have a clear line of sight when turning onto the highway.
VicRoads improved the quality of its stakeholder engagement plans for the next 16 roads by including more information about key messages, staff responsibilities and its engagement approaches for each identified stakeholder.
Time and cost
As of April 2020, VicRoads had completed 21 of the 27 Top 20 Program projects and installed 92 per cent of the planned 3 458 kilometres of safety infrastructure, including flexible barriers, wide centrelines and rumble strips.
While VicRoads aimed to complete the Top 20 Program by June 2020, it has scheduled the eight remaining projects for delivery by February 2021, which is eight months later than planned.
When VicRoads’ project proposals were initially approved, TAC provided $450 million in funding through the SSRIP. In April 2020, VicRoads estimated that the final cost of the Top 20 program would be $550.2 million, with 12 of the 27 projects expected to exceed their budget. As a result, the program’s current cost is $99.9 million (or 22 per cent) more than its initial budget.
Road design standards exist to ensure that road infrastructure is consistent and safe. VicRoads has a comprehensive suite of quality assurance mechanisms and sign offs to ensure that contractors install flexible barriers in compliance with road design standards.
Repairs and maintenance
Flexible barriers need ongoing repairs and maintenance to ensure that they perform as expected. VicRoads estimates that unrepaired barriers can be up to 34 per cent less effective at reducing serious injuries and fatalities.
VicRoads inspects its road safety infrastructure to identify any hazards and damage. Its inspection processes vary significantly between its different regions. While VicRoads does not set requirements for what inspectors should assess, it does specify how often inspections should occur.
VicRoads has limited oversight of how maintenance contractors inspect flexible barriers. Only one region could demonstrate how it conducts inspections, which means that VicRoads cannot guarantee that inspections are comprehensive or effective across the state.
Repairs and maintenance works
When a barrier is damaged, VicRoads responds before repairing the affected section. These interim responses include actions such as assessing the damage or placing signs to warn traffic about the damaged section.
VicRoads has timeliness standards for responding to damaged barriers, but not for conducting repairs. In October 2018, VicRoads proposed timeliness standards for repairs to TAC, but neither TAC nor VicRoads adopted them.
Since 2015, VicRoads has improved the timeliness of its repairs and maintenance works. Despite a 163 per cent increase in barrier repair and maintenance jobs, its average repair time decreased from 94 days to 30 days between 2015 and 2019.
VicRoads does not keep records about how often it maintains flexible barriers. VicRoads advised that it performs routine maintenance on some stretches of flexible barriers, such as checking the tension of wire rope barriers, on an ad hoc basis. For others, VicRoads said it does not perform any maintenance.
Funding for repairs and maintenance
In addition to the funding TAC provides to VicRoads to install flexible barriers, it also provides funding to repair and maintain them. VicRoads also allocates some of its asset maintenance funding to its regions for general road maintenance, which includes maintaining barriers not funded by the TAC, managing roadside vegetation and fixing potholes. Regions allocate this funding according to their regional road priorities. This means that funding for maintaining these barriers competes with other types of maintenance.
VicRoads does not centrally record what condition stretches of flexible barrier are in. Consequently, it does not know which barriers require maintenance at specific points in time. This means that VicRoads’ regional road maintenance priorities are not informed by the condition of flexible barriers.
Monitoring and evaluation
Key performance indicators
VicRoads did not develop a benefits management plan at the start of the Top 20 Program. Benefits management plans are better practice because they clearly outline key performance indicators (KPIs) and targets, as well as measures to determine if a project is going to meet its original objective.
While VicRoads’ crash reduction factors could be viewed as project targets for the Top 20 Program, VicRoads did not formalise these.
VicRoads did not establish an evaluation framework when it was planning the Top 20 Program. Additionally, it could not give us accurate construction start and end dates for its Top 20 Program projects, which are needed to conduct a proper evaluation. If VicRoads had established a formal evaluation framework at the beginning of the program, then it could have used it to collect relevant data and hold itself more accountable for its performance.
VicRoads did not commission a contractor to develop an evaluation framework for the Top 20 Program until May 2017. Additionally, it did not start planning and contracting for a short and medium-term evaluation of the program until 2017. It advised that it is planning to start a long term evaluation in 2021, which will finish in 2026.
As VicRoads has not evaluated how effective the road safety treatments it installed have been at reducing road deaths and serious injuries, we completed our own evaluation.
We found that as at the end of January 2020, installing flexible barriers, wide centrelines and rumble strips has led to a 46.5 per cent reduction in run off road and head-on serious casualty crashes for 18 of the completed projects. Within a 95 per cent confidence interval, this reduction could be between 25.6 per cent and 65.6 per cent. We were not able to determine how much of this reduction can be attributed to flexible barriers alone because VicRoads did not have enough data on barrier location for us to perform this analysis.
Record keeping and data quality
VicRoads’ inadequate record keeping and data quality has limited its ability to manage projects, provide evidence for key decisions and evaluate if the Top 20 Program will achieve its expected benefits. We discuss a few examples of this below.
VicRoads used crash data to select the top 20 roads, develop investment plans and project proposals and monitor the effectiveness of its road safety treatments.
VicRoads sources crash data from Victoria Police but does not verify it against TAC’s records to check its accuracy. TAC’s data is more reliable because it is based on road injury claims that are supported by medical records and other evidence. Victoria Police only assess injuries at the crash site. While it is unlikely that it would have changed VicRoads’ final selection, by using Victoria Police’s data VicRoads missed six injuries in the data it used when it was selecting the 20 highest risk roads.
DoT, TAC and Victoria Police are now reviewing all crashes that occurred between 2006 and 2018 to ensure that their data is correct. They plan to complete this review in June 2020.
Key program dates
VicRoads has not accurately recorded the construction dates for its 21 completed Top 20 Program projects. This is because each of its regional offices manage their projects differently and there is no central record for this information. For this reason, our evaluation only included 18 of the completed projects where VicRoads could provide accurate construction start and end dates.
This lack of consolidated information will significantly impede VicRoads’ ability to monitor the Top 20 Program’s success and evaluate if it has achieved the intended safety benefits.
VicRoads’ asset register does not separately record where it has installed different types of barriers on its roads.
This hinders VicRoads’ ability to:
- plan to install flexible barriers in the future
- accurately evaluate the effectiveness of flexible barriers
- provide emergency services with the exact locations of gaps between sections of barrier for emergency vehicle access
- know which barriers are damaged or require maintenance at specific points in time
- schedule strategic maintenance and repairs.
In June 2019, VicRoads started identifying and mapping the locations of flexible barriers it has installed under the Top 20 Program with funding from TAC. It is due to complete this work after it finishes all of its Top 20 Program projects.
We recommend that the Department of Transport (VicRoads):
1. develops business cases for major investments funded by the Transport Accident Commission in line with the Department of Treasury and Finance’s better practice Investment Lifecycle and High Value High Risk Guidelines (see Section 2.2)
2. uses statistically robust methods that account for statistical variation in crash data to select future roads for treatment (see Section 2.2)
3. develops crash reduction factors that are supported by multiple peer reviewed evidence sources, and clearly indicates these in its project approval documentation. If multiple peer reviewed evidence sources are not available, then the Department of Transport (VicRoads) should clearly indicate this in its project documentation and state a conservative estimate (see Section 2.2)
4. ensures that its project proposals contain:
- a project’s key details, including expected completion dates, key milestones, detailed costings of individual treatments, key scoping information (such as expected kilometres of infrastructure) and expected benefits (such as crash reduction factors)
- peer-reviewed evidence sources
- risks and mitigation strategies
- stakeholder engagement plans (see Section 2.3)
5. maintains comprehensive records of all project approval documentation, including meeting minutes and supporting documentation put before the project approvers (see Section 2.3)
6. ensures that its asset management systems include key information about flexible barriers, including their location, installation date, state of repair and maintenance schedule (see Section 3.3)
7. develops standards for maintaining flexible barriers that detail how they should be conducted and how often (see Section 3.3)
8. introduces timeliness standards for repairing flexible barriers (see Section 3.3).
We recommend that the Transport Accident Commission:
9. requires VicRoads to develop a business case before it approves funding for major infrastructure projects (see Section 2.2)
10. develops a reporting framework with VicRoads to establish regular reporting on the Top 20 Program’s benefits and costs (see Section 4.2).
Responses to recommendations
We have consulted with DoT (VicRoads) and TAC and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments. We also provided a copy of this report to DTF.
DoT (VicRoads) and TAC accepted all of our recommendations and have both provided a detailed action plan to address them. The full responses are included in Appendix A.