Safety on Victoria's Roads—Regional Road Barriers

Tabled: 18 June 2020

2 Planning and approval

When public money is invested in a major project, a rigorous planning and approval framework is necessary. DTF provides guidance to all government entities for planning major projects through its Investment Lifecycle and High Value High Risk Guidelines.

We examined VicRoads’ planning and approval processes that secured TAC’s investment in flexible barriers. To do this, we assessed if:

  • TAC and VicRoads developed a detailed business case or equivalent before the project was approved
  • VicRoads followed an evidence-based process to select the 20 highest-risk road sections
  • TAC and VicRoads provided the government with strong evidence to support the effectiveness of flexible barriers, including their effectiveness for different types of road users.

2.1 Conclusion

VicRoads’ plans for the Top 20 Program did not contain a sufficient amount of detail for a major infrastructure project that would normally be consolidated in a business case. As a result, VicRoads and TAC cannot be sure that the investment in flexible barriers will achieve the intended benefits and that it was the best value-for-money option.

VicRoads’ approval documentation was inconsistent and lacked detail about its projects’ timelines, scope, risks, costs and how it selected the 20 high-risk roads. For some Top 20 Program projects, we saw no evidence that TAC and VicRoads had sufficient information about key project details before they approved funding.

While there is evidence that flexible barriers effectively reduce fatalities and serious injuries, VicRoads has likely overstated these benefits.

2.2 Business case

It is best practice for an agency to develop a business case when it is proposing to invest public funds in a major project.

A business case shows a decision-maker:

  • that the proposed investment is worthwhile and has merit over other projects competing for funding
  • that the proposed solution is the best value-for-money option
  • when the project will be delivered and how much it will cost.

A business case should also provide evidence that options, costs, risks and benefits were fully considered before the project was approved.

Installing continuous flexible barriers to sections of 20 high-risk rural roads is a major infrastructure project. While DTF did not require TAC and VicRoads to develop a business case for this program, it would have been better practice, given its cost and scale.

The impact of funding sources

A major government project’s funding source can determine if it requires a business case or not. It can also influence if other project planning steps must be completed before the project is approved.

Funded by the government budget

If a major project is expected to cost over $250 million and will be funded through the government budget (or is assessed as being high risk), then it must meet certain planning requirements before it is approved. DTF govern this process through the Investment Lifecycle and High Value High Risk Guidelines.

These guidelines require agencies that are seeking government funding for a project to develop a detailed business case for DTF and the government’s approval. Once approved, the project must also pass a series of rigorous assessments, known as gateway reviews, throughout its lifespan. These gateway reviews give DTF, as an independent reviewer, oversight of the project’s timing, budget and benefit outcomes.

Internally funded by a government agency

If a major project is internally funded by a government agency and is outside the government’s budget process, then it is not required to comply with the Investment Lifecycle and High Value High Risk Guidelines.

As TAC internally funded the Top 20 Program, VicRoads’ flexible barrier installation projects were not required to go through DTF’s gateway review process.

However, DTF’s framework is better practice, and TAC’s funding came from the Victorian community. For these reasons, we expected that the Top 20 Program’s planning and approval processes would have been subject to a similar level of scrutiny to other major infrastructure projects. It was not.

Developing the Towards Zero Strategy

While developing the Towards Zero Strategy, TAC and VicRoads:

  • modelled road trauma trends
  • held workshops with road safety experts to identify treatment options
  • engaged with government bodies, such as Victoria Police, the Department of Justice and Regulation (now the Department of Justice and Community Safety) and DHHS
  • modelled road trauma data to estimate the expected reductions in deaths and serious injuries for different road safety treatment options and investments.

Through this process, different road safety treatments were identified, costed and modelled to determine their expected benefit. These treatments form the Towards Zero Strategy, which went through the following approvals:

Date

Approval

July 2015

The Road Safety Executive Group approves the strategy, subject to the Ministerial Council for Road Safety’s approval.

August 2015

The Ministerial Council for Road Safety approves the strategy, subject to the government’s approval and funding.

December 2015

The government approves TAC to fund the strategy.

VicRoads and TAC did not prepare a consolidated business case for any of these approval stages. There was no approval stage where a decision maker formally considered and selected installing flexible barriers on 20 high risk rural roads over alternate options.

Without a business case, VicRoads and TAC were unable to provide us with evidence that decision-makers were presented with a consolidated package that included all of the following information before approving the investment:

  • a range of costed options and evidence for why installing continuous flexible barriers was the best solution
  • an analysis of different roads and justification for why the selected 20 were identified as the riskiest
  • detailed and accurate evidence to prove the effectiveness of flexible barriers for different types of road users
  • timelines and expected project milestone dates
  • risks and potential mitigation strategies.

A consolidated business case would have provided enough information for decision-makers to make an informed decision about the investment in flexible barriers in comparison to other options.

Selecting the top 20 roads

Road selection process

VicRoads’ data analysis found that most serious casualty crashes occur on high-speed rural roads due to vehicles crossing the centreline into oncoming traffic. VicRoads determined that installing flexible barriers would solve this problem and selected 20 high speed rural roads to treat under the Top 20 Program. The Towards Zero Strategy lists these 20 roads.

VicRoads advised that it jointly developed its road selection process with TAC, the then Department of Justice and Regulation, DHHS and Victoria Police.

VicRoads developed two criteria for selecting the top 20 roads:

  • high speed—roads with a speed limit of at least 100 kilometres per hour
  • high volume—roads with a daily traffic volume of at least 3 000 vehicles.

After applying these criteria, VicRoads selected undivided roads with the highest crash rates. As Figure 2A shows, this resulted in the selection of nine divided and 11 undivided roads.

Figure 2A
Top 20 roads selection process

Figure 2A Top 20 roads selection process

Note: VicRoads calculated the FSI per 100 km/h rate for undivided roads using five years of crash data from July 2010 to June 2015.
Source: VAGO.

VicRoads and TAC could not assure us that this approach was the most effective way to identify and prioritise roads for treatment. There were no existing external guidelines for VicRoads to refer to and it had no internal process to review or approve its selection approach.

VicRoads and TAC did not document how they decided on the high-speed and high-volume criteria. While VicRoads advised that it did consider roads that did not meet the speed and volume criteria, it could not demonstrate this or that it considered other road selection approaches.

Validity of VicRoads’ road selection approach

After examining VicRoads’ road selection process and high-volume criteria, we identified two key issues:

  • VicRoads used unverified crash data to calculate fatality and serious injury (FSI) crash rates.
  • VicRoads does not have clear evidence that links high traffic volume with high FSI risk.

Despite this, the selected roads are dangerous, and installing barriers will save lives. However, in a constrained funding environment, VicRoads must ensure that it selects the most dangerous roads.

Unverified crash data

VicRoads used unverified crash data throughout its road selection process. It did not verify the crash data it sourced from Victoria Police with TAC’s records to ensure accuracy. While it is unlikely that it would have changed VicRoads’ final selection, by using Victoria Police’s data, VicRoads missed six injuries when it was selecting the 20 highest risk roads.

While TAC’s data comes from road injury claims made by the public, Victoria Police’s data comes from an assessment of injuries at the crash site. Injuries assessed by Victoria Police may become more or less serious by the time a TAC claim is made. This means that TAC’s crash data is more reliable because claims occur after longer periods of time and are supported by medical records and other evidence.

While VicRoads identified this issue in 2016, it did not start reviewing its crash data until August 2019. DoT, TAC and Victoria Police are currently reviewing all crashes that occurred between 2006 to 2018 to ensure that their data is correct. This review will not be completed until June 2020.

High-volume criteria

VicRoads does not have clear evidence to show that roads with high volumes of traffic are riskier. To identify the most dangerous rural roads, VicRoads looked at how many run-off-road and head-on serious casualty crashes had occurred per 100 kilometres of road. Most serious casualty crashes occur on low-volume roads that carry fewer than 3 000 vehicles per day. However, the rate of crashes per 100 kilometres is lower on these roads because there are more kilometres of low-volume roads in Victoria than high-volume roads.

This means that while the FSI rate per 100 kilometres is lower on low-volume roads, more serious casualty crashes occur on them, which makes them more dangerous. This is shown in Figure 2B.

Figure 2B
Serious casualty rates per 100 kilometres compared to the actual number of serious casualties on low-volume to high-volume roads between 2011 and 2015

Figure 2B  Serious casualty rates per 100 kilometres compared to the actual number of serious casualties on low-volume to high-volume roads between 2011 and 2015

Note: This graph includes run-off-road and head-on crashes between 2011 and 2015 that resulted in serious casualties.
Source: VAGO, based on VicRoads’ data.

There are valid reasons for using FSI rates per 100 kilometres to identify high risk roads, including cost and visibility. In relation to cost, there are more kilometres of low-volume roads than high-volume roads. This means that VicRoads would have to spend more money to treat the low-volume roads, which may be impractical. However, VicRoads did not internally document or communicate its reasons to the public.

Evidence of benefits

Without a business case or equivalent plan, there is no consolidated record of the research that VicRoads and TAC relied on to support the Top 20 Program’s expected benefits.

None of VicRoads’ investment plans or project proposals provide evidence to support the claimed effectiveness of continuous flexible barriers. However, the Towards Zero Strategy references a MUARC study, which we discuss below.

Evidence used in the Towards Zero Strategy

The Towards Zero Strategy publicly states that flexible roadside and centreline barriers have been shown to reduce run-off-road and head-on serious casualty crashes by up to 85 per cent.

The Towards Zero Strategy attributes this claim to a 2009 peer-reviewed study conducted by MUARC titled Evaluation of the Effectiveness of Flexible Barriers Along Victorian Roads.

This paper, which VicRoads commissioned, analyses the effectiveness of flexible barriers in reducing run-off-road and head-on crashes on Victoria’s roads. In the study, MUARC tested 11 sites where VicRoads had installed flexible barriers and conducted before-and-after analyses.

The study found that installing flexible barriers resulted in a 56 per cent reduction in run-off-road and head-on serious casualty crashes (with a 95 per cent confidence interval of 25 to 74 per cent).

Even at the best possible outcome within the confidence interval, these results do not represent an 85 per cent reduction in run-off-road and head-on serious casualty crashes.

The study found that for two of the 11 sites it examined, reductions of 87 per cent and 83 per cent were statistically significant. This is the finding that underpins the claim in the Towards Zero Strategy. For one of these roads, the confidence interval was too wide for the results to be meaningful. The results showed that the researcher was 95 per cent confident that the true reduction lay somewhere between 1 and 98 per cent. For the other road, the confidence interval was 47 to 94 per cent.

Quoting the study’s overall result of a 56 per cent reduction, based on all 11 sites, would have provided a more realistic reflection of the effectiveness of the barriers, instead of reporting that barriers can reduce crashes by ‘up to 85 per cent’.

Crash reduction factors

VicRoads uses crash reduction factors to calculate the expected benefits of its road safety projects. Crash reduction factors are the amount (as a percentage) that new safety infrastructure will reduce a certain type of crash by.

VicRoads sets crash reduction factors to calculate benefit-cost ratios, which we discuss in the next section.

Figure 2C shows the crash reduction factors that VicRoads specified for its Top 20 Program investment plans.

Figure 2C
Crash reduction factors for Top 20 Program investment plans

Investment plan

Crash reduction factor

Type of run-off-road and head-on crash

Hume Highway (Investment Plan 5)

85 per cent

All casualty crashes(a)

Geelong to Bacchus Marsh Road (Investment Plan 9)

90 per cent

Serious casualty crashes

Goulburn Valley Highway (Investment Plan 10)

81 per cent

All casualty crashes

Divided roads (Investment Plan 14)

85 per cent

All casualty crashes

Undivided roads (Investment Plan 15)

40 to 47 per cent (for flexible middle barriers)

17 to 32 per cent (for flexible roadside barriers)

Serious casualty crashes

Note: (a)All casualty crashes are crashes that result in a death or any type of injury.
Source: VAGO, based on an analysis of VicRoads’ investment plans.

VicRoads could not give us evidence to show how it calculated the crash reduction factors for its investment plans. However, it told us that it based the crash reduction factors for Investment Plans 5 and 14 on the 85 per cent reduction in run-off-road and head-on serious casualty crashes sourced from the MUARC study. As stated earlier, this crash reduction factor is likely overstated because it does not reflect the most likely result presented in MUARC’s research.

It is also unclear why the crash reduction factor for Investment Plan 15 is significantly lower than the crash reduction factors in the previous four investment plans, which were for similar treatments.

VicRoads’ previous overestimations of crash reduction factors

As Figure 2D shows, VicRoads has a history of overestimating crash reduction factors. In its evaluations, MUARC commented on the difference between its findings and VicRoads’ crash reduction factors for the SRIP 1, 2 and 3 funding programs.

Figure 2D
MUARC’s evaluation of crash reduction factors

Program

Treatment

VicRoads crash reduction factor(a)

Evaluation date

MUARC evaluation finding (with 95 per cent confidence interval)

SRIP 1 (2004)

Run-off road treatments

54 per cent

May 2011

22 per cent (14–29 per cent)

SRIP 2 (2005)

Run-off road treatments

52 per cent

August 2011

26 per cent (13–37 per cent

SRIP 3  (2007)

Safety barriers without shoulder sealing

64 per cent

March 2016

27 per cent for casualty crashes (7–43 per cent)
35 per cent for serious casualty crashes (8–54 per cent)

Note: (a)VicRoads’ pre-project estimates.
Source: MUARC’s evaluations of the SRIP 1,2 and 3 programs.

In light of these overestimations, there is no evidence that VicRoads updated its approach to calculating crash reduction factors for the Top 20 Program. VicRoads calculated the crash reduction factors that we reviewed in the period between 2015 to 2017. VicRoads advised that it updated its approach to determining crash reduction factors in August 2019.

Impact on different vehicle types

The crash reduction factors that VicRoads used in its investment plans and project proposals do not differentiate between different types of vehicles, such as motorcycles, cars or trucks.

The Towards Zero Strategy references two studies about the effectiveness of flexible barriers for motorcyclists. Neither of these studies had enough data for VicRoads to rely on. We saw no evidence to support how effective flexible barriers are for other road users, such as heavy vehicle drivers.

Passenger vehicles make up most of the vehicles within the crash data that VicRoads uses to calculate crash reduction factors. This means that the crash reduction factors for other types of vehicles that are less common, such as motorcycles and heavy vehicles, could be under or overestimated.

This also means that the joint committee endorsed, and VicRoads approved, each project without understanding how flexible barriers impact different types of road users.

Benefit-cost ratios

Decision-makers use benefit-cost ratios to justify major infrastructure projects and prioritise projects that are expected to deliver higher benefits for lower costs.

Under the SSRIP Funding Deed, the joint committee could not endorse a project proposal with a benefit-cost ratio of less than 1.5. This means that a project’s expected benefit needed to be at least 1.5 times more than its cost.

Crash reduction factors are vital for calculating benefit-cost ratios. If a project’s crash reduction factor is overstated, then its benefit-cost ratio will also be overstated. As discussed earlier, VicRoads likely overstated the crash reduction factors for its Top 20 Program projects, and therefore likely overstated the benefit cost ratios in the corresponding investment plans and project proposals.

Approved projects below the 1.5 threshold

The joint committee endorsed, and VicRoads approved, four of the 27 project proposals with a benefit-cost ratio that did not meet the 1.5 threshold. VicRoads should not have approved these projects for funding under the SSRIP. While VicRoads’ and TAC’s CEOs jointly approved exemptions for three of these roads, they did not justify why. These three roads were:

  • Paynesville Highway, which had a benefit-cost ratio 0.4
  • Midland Highway, which had a benefit-cost ratio 1.1
  • South Gippsland Highway, which had a benefit-cost ratio 0.7.

2.3 Project design and approval

Governance over approvals

Under the SSRIP Funding Deed, TAC and VicRoads have different responsibilities for developing and approving investment plans and project proposals, which is shown in Figure 2E.

Figure 2E
VicRoads and TAC’s project approval process

Figure 2E VicRoads and TAC’s project approval process

Source: VAGO.

We asked to see the documentation that VicRoads presented to the joint committee before it endorsed the Top 20 Program projects. We also requested the joint committee’s meeting minutes. VicRoads could only give us full documentation for 12 of the 27 endorsed projects. VicRoads also advised that there is no central repository for the committee’s meeting minutes and other documents.

VicRoads advised that establishing Road Safety Victoria in 2019 has strengthened governance over the program. We did not assess this as part of our audit.

Project planning

As there was no business case or equivalent plan for the Top 20 Program, we assessed VicRoads’ five investment plans and 27 project proposals that made up the program.

VicRoads developed a 10-step process to guide the project design and endorsement processes for these project proposals:

Step 1

VicRoads analyses the problem.

Step 2

VicRoads’ regions and the SSRIP team hold a collaborative concept workshop.

Step 3

VicRoads’ regions begin project scoping and development.

Step 4

VicRoads’ regions present the project proposal to the SSRIP team.

Step 5

VicRoads refines the approval documentation.

Step 6

The joint committee endorses the project proposal.

Step 7

VicRoads begins design work and selects the procurement method.

Step 8

VicRoads announces the project and commences construction.

Step 9

VicRoads monitors construction.

Step 10

VicRoads completes the project.

VicRoads developed a scoping approval report or presentation for each project to outline the design activities it undertook between steps 1 and 5. VicRoads presented this report or presentation to the joint committee at Step 6 to inform its endorsement.

VicRoads’ project proposals lack the level of detail that decision-makers need to make an informed decision. While its scoping approval reports contain varying levels of information, VicRoads only provided these for 12 of the 27 Top 20 Program projects.

Due to the lack of documentation, there is no evidence that the joint committee were aware of the projects’ key details, such as their time frames, costs and risks, before it endorsed some of them.

Time frames and completion dates

Under the SSRIP Funding Deed, VicRoads was not required to outline the time frames or completion dates for its Top 20 Program projects in its investment plans or project proposals.

Of the 27 project proposals, only two had completion dates. However, these dates were not specific—they were financial year ranges. For another three projects, the scoping approval reports outlined their delivery schedules. The joint committee endorsed, and VicRoads approved, the remaining 22 projects with little or no insight into how long each project would take and what the key milestones would be.

Project design and costs

VicRoads did not begin detailed planning and full project design for the 27 projects until Step 7. As outlined earlier, this step occurred after VicRoads had approved the final project proposals. While its project proposals included total estimated costs, VicRoads calculated these before it finalised the detailed specifications for some projects, including:

  • the exact length of barrier required
  • the type and brand of barrier to install (and its cost per metre)
  • the amount of vegetation that needed to be removed
  • the environmental approvals required to commence works.
  • Figure 2F outlines the two project proposals that the joint committee endorsed, and VicRoads approved, that had limited details.

Figure 2F outlines the two project proposals that the joint committee endorsed, and VicRoads approved, that had limited details.

Figure 2F
Project proposals endorsed with limited detail

Project

Approved funding

Project details given

DK751: Midland Highway (Ballarat to Creswick)

$12 million

Project length: 12 kilometres. 

Project Description: ‘3x Roundabouts. Centre Barrier Treatment.

DK629: Geelong to Bacchus Marsh Road (Ballan Road to Woolpack Road)

$10 million

Project length: 12.25 kilometres. 

Project Description: ‘WRSB(a) Left, WRSB Centre, 2x Roundabout (Exford Rd & Nerowie Rd).’

Note: (a) WRSB stands for wire rope safety barrier.
Source: VicRoads.

These project proposals did not specify the specific length of barrier that needed to be installed. While VicRoads’ scope approval reports and presentations sometimes contained more information about a project’s design and cost, it was not able to provide these to us for 12 of the 27 projects.

Insufficient detail in project planning can lead to costs blowing out. The costs of 11 of the 27 projects increased from their proposals, with one requiring a 77 per cent funding increase from $20 million to $35.4 million.

Risks

We could only find evidence that VicRoads presented key environmental, planning and project risks and potential mitigation strategies for six of the 27 projects to the joint committee. This means that the joint committee may not have been aware of important considerations that could have delayed the other 21 projects or increased their cost.

Stakeholder engagement

We expected that VicRoads would have documented its stakeholder engagement approach for each of the Top 20 Program projects to ensure consistent messaging across its regions.

VicRoads engaged contractors to develop 12 communication and engagement plans from August 2018. As VicRoads did not have finalised engagement plans before it started installing flexible barriers, it had limited opportunities to identify and address any required changes due to stakeholder concerns before it approved the project proposals.

As a result, VicRoads made two post-implementation infrastructure changes after receiving stakeholder feedback:

  • It constructed additional emergency crossover areas on the Calder Freeway.
  • It removed a small new section of barrier on the Princes Highway East to ensure that motorists have a clear line of sight when turning onto the highway.

If VicRoads had engaged stakeholders at an earlier stage, it could have considered these requirements before it approved the projects, which would have saved time and money. Figure 2G outlines one example relating to concerns from the Country Fire Authority (CFA).

Figure 2G
CFA and Calder Freeway case study

Media outlets reported that CFA had concerns about the reduced distance between the edge of the traffic lane and the centre median barriers on the Calder Freeway near Elphinstone. VicRoads advised that it responded by constructing an additional emergency crossover area.

In March 2018, VicRoads and CFA released a joint media release that outlined a forward plan of how VicRoads would address local brigades’ remaining concerns. The forward plan included the following actions:

  • VicRoads to conduct detailed mapping of the barriers to identify locations of road access breaks, emergency crossovers and interchanges
  • VicRoads to attend all local emergency management planning meetings in regions where it is installing barriers under the Top 20 Program.

In August 2018, VicRoads also launched a new training facility at its Wendouree depot to train first responders on how to dismantle and remove barrier posts during an emergency.

Source: VAGO, based on VicRoads’ data and publicly available information.

Improvements

VicRoads is completing more advanced design plans for its next 16 project proposals before the joint committee endorses them. These project proposals also include a breakdown of key milestone dates. This means that VicRoads is approving these project proposals with a more accurate understanding of their costs and expected time frames.

VicRoads has also improved its stakeholder engagement plans for the next 16 projects. For example, VicRoads has developed a public engagement plan toolkit that includes better practice principles and guidance on what needs to be considered as part of public engagement planning. As a result, its engagement plans for the next 16 roads include greater detail about key messages and responsibilities as well as an engagement approach for each identified stakeholder. This will help VicRoads support consistent public engagement practices across different projects and mitigate potential project risks.

Back to Top