fbpx Follow up of Managing the Level Crossing Removal Program | Victorian Auditor-General's Office

Follow up of Managing the Level Crossing Removal Program

Tabled: 14 October 2020

Snapshot

Have the Department of Transport and the Major Transport Infrastructure Authority effectively addressed recommendations made in our 2017 audit Managing the Level Crossing Removal Program?

Why this audit is important

Our 2017 audit identified weaknesses in the Level Crossing Removal Project's (LXRP) design and delivery. 

In 2018, the Victorian Government expanded the LXRP from 50 to 75 sites, which increased the project’s total estimated cost from $8 billion to $14.8 billion. 

It is vital that agencies implement lessons learnt from the first stage of the project to improve the second stage’s delivery and value for money.

Who we examined

  • Department of Transport (DoT) 
  • Major Transport Infrastructure Authority (MTIA). 

What we examined

Whether agencies addressed our 2017 recommendations (six for DoT and four for MTIA). 

What we concluded

DoT and MTIA have fully addressed seven recommendations. DoT has partially addressed one and is still addressing a further two.

Unlike the process used to select the first 50 level crossing removal sites, DoT and MTIA used a transparent selection process for stage two (LXRP2). They fairly balanced the principles of safety, congestion and delivery efficiency. Consequently, LXRP2 is on track to meet the project’s overall aim of removing dangerous and congested level crossings.

DoT and MTIA have also improved how they measure the project’s benefits and reviewed their procurement approach to ensure it minimises costs. 

However, DoT and MTIA did not complete a full business case for LXRP2. As a result, the government did not receive advice about the project's expected economic benefit before it made its decision to fund LXRP2.

DoT is also yet to complete work on engineering standards and network integrity controls. This creates a risk that project delivery agencies and contractors may be unsure about the technical standards their projects need to meet to integrate with the transport network. 

Video presentation

Video transcript

Key facts

Key facts

What we found

We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A. 

Summary of progress and outcomes

The table on the next page summarises DoT’s and MTIA’s progress on each of our 2017 recommendations. Appendix C contains the full list of recommendations from our 2017 audit.

Of the 10 recommendations:

  • seven have been fully implemented
  • one has been partially addressed, with no further work planned
  • two are still in progress and further work is needed to ensure that the underlying issues are addressed.

There is a detailed assessment of each recommendation throughout the report.

Agencies' progress on addressing our 2017 recommendations

No. Issue Status and comments
Developing LXRP2
2 Site selection process   Addressed
MTIA used a comprehensive and transparent site prioritisation framework to select the LXRP2 sites.
1 Business case   Partially addressed
DoT and MTIA have completed some of the High Value High Risk (HVHR) framework’s requirements. However, they did not prepare a business case or cost-benefit analysis for LXRP2.
7 Options assessment   Addressed
Unlike the process for stage one of the Level Crossing Removal Project (LXRP1), which varied for different sites, MTIA is now using a consistent and transparent process to select grade separation options. 
Procurement and packaging
8 Evaluation of contract structure   Addressed
While there has not been a formal independent evaluation of deferred price contracting, MTIA and the Office of Projects Victoria (OPV) have reviewed its benefits.
10 Packaging approach   Addressed
While MTIA did not commission a formal evaluation of its packaging approach, it has incorporated lessons learnt into its packaging approach for LXRP2.
9 Benchmarking tool    Addressed
MTIA’s benchmarking tool is embedded in its process for awarding additional works packages. The tool is working as intended to achieve cost efficiencies.
Managing benefits
3 Key performance indicators   Addressed
While MTIA did not develop new key performance indicators (KPIs), it has improved the level of detail and usefulness of the data it uses to report against the existing KPIs.
4 Monitoring outcomes   Addressed
MTIA has developed a project-wide benefits framework that progressively monitors outcomes.
Network integrity and standards governance
6 Network integrity controls   In progress
DoT reviewed its network integrity controls in 2018 and is currently reviewing them again following machinery of government changes. It is too early to tell if these changes will be effective.
5 Network rail standards   In progress
DoT has developed some network rail standards and is on track to finalise several more. However, it needs to conduct further work to develop network requirements and fully embed the governance process it uses to manage changes to engineering standards. 

Note: Refer to Appendix C for the full recommendations.

Back to top

1. Audit context

Established in 2015, the LXRP is one of the government’s major transport infrastructure projects. It aims to reduce pressures on the transport network by easing road congestion and travel delays caused by level crossings. It also intends to improve safety by decreasing the chance of accidents involving trains and road users or pedestrians.

Our 2017 audit Managing the Level Crossing Removal Program examined the effectiveness of the LXRP and made 10 recommendations to improve it. 

1.1 Risks posed by level crossings

A level crossing is an intersection where a rail line crosses a road or path at the same level. 

Level crossings use boom gates to manage the flow of road and foot traffic across the rail line. When boom gates are down for extended periods of time, traffic can become congested. This can increase risk-taking behaviours by drivers and pedestrians.

Between 2005 and 2015, there were more than 149 level crossing collisions involving a train and a vehicle or pedestrian in metropolitan Melbourne. Of these, 38 resulted in fatalities and 22 in serious injuries. 

The alternative to a level crossing is constructing an overhead bridge or underground tunnel to separate the rail line from road and foot traffic.

1.2 The LXRP’s evolution

LXRP1: the first 50 sites

Following its election in 2014, the Victorian Government pursued its pre-election commitment to remove 50 of Melbourne’s most dangerous and congested level crossings by 2022. These 50 sites are listed in Appendix D. At the time, the project's estimated cost was $6.9 billion. 

In 2015, the government established the Level Crossing Removal Authority (LXRA), which was an administrative office within the then Department of Economic Development, Jobs, Transport and Resources (DEDJTR).

LXRA was responsible for delivering LXRP1 and achieving:

  • more reliable and efficient transport networks by addressing congestion and delays caused by level crossings
  • better connected, liveable and thriving communities by reducing delays and increasing the attractiveness of living and investing in areas surrounding removed crossings
  • safer communities by removing conflict points between trains and road users and pedestrians to reduce the number of crashes. 

LXRP2: a further 25 sites

In October 2018, the government announced that it would remove a further 25 level crossings through LXRP2. 

The official LXRP covers 75 sites. However, MTIA is removing a further three sites that need to be removed at the same time as one of the official project sites. Appendix D contains the full list of all LXRP sites and their progress to date.

Project budget

As at 30 June 2020, MTIA has spent $6.2 billion of its $14.8 billion total funding. MTIA is forecasting that it is on track to complete all 75 level crossing removals within budget.

To date, the Department of Treasury and Finance (DTF) has approved the release of $1.28 billion of the project's risk provision to address cost escalation pressures. The remaining risk provision is $879 million.  

Chapter 3 discusses how MTIA is using its procurement and packaging approach to minimise project costs.

1.3 Changes to government agencies since our 2017 audit

Machinery of government changes are changes to the administrative structure of government agencies. These can include transferring functions from one agency to another or abolishing entire departments.

Since our 2017 audit, machinery of government changes have altered the roles and responsibilities of the agencies delivering the LXRP.  

As Figure 1A shows, on 1 January 2019, the government abolished DEDJTR. It replaced it with DoT and the Department of Jobs, Precincts and Regions. DoT also absorbed the functions of:

  • VicRoads, which managed the state’s road networks
  • Transport for Victoria (TfV), which integrated the planning and coordination of the state’s transport system 
  • Public Transport Victoria (PTV), which coordinated the public transport network and managed its integrity.

At the same time, the government abolished LXRA and transferred responsibility for the LXRP to MTIA—a newly formed administrative office within DoT. 

MTIA is also responsible for delivering the government’s other major transport infrastructure projects, including the Metro Tunnel, West Gate Tunnel, North East Link, Regional Rail Revival and Melbourne Airport Rail.

FIGURE 1A: Machinery of government changes relevant to this audit

2017 department/agency Structure since 1 January 2019
DEDJTR DoT
TfV
PTV
VicRoads
LXRA MTIA
North East Link Authority
Western Distributor and West Gate Tunnel Authorities
Major Road Projects Authority
Melbourne Metro Rail Authority

Note: In 2013, VicRoads began planning works to remove 10 level crossing sites. The government transferred responsibility for this to LXRA in 2015.
Source: VAGO, based on information from DoT and MTIA.

Since the machinery of government changes:

  • DoT is responsible for delivering the six recommendations we made to DEDJTR
  • MTIA is responsible for delivering the four recommendations we made to LXRA.

Throughout this report we refer to DoT and MTIA, except for when we discuss former agencies’ specific actions. 

1.4 Project delivery

Alliance contracting is when a state agency works with a group or alliance of private sector parties to deliver a project.

MTIA is delivering the LXRP using an alliance contracting model. Rather than allocating individual contracts to remove each level crossing, MTIA groups multiple sites into packages and then contracts program alliances to deliver them. 

Program alliances

Following a competitive tender process, MTIA established four program alliances in 2017 to deliver different work packages:

  • North Eastern Program Alliance (NEPA)
  • North Western Program Alliance (NWPA)
  • Southern Program Alliance (SPA)
  • Western Program Alliance (WPA).

In 2018, NEPA split into two streams for road and rail:

  • NEPA Rail
  • NEPA Road.

This change was designed to maximise the expertise of the construction companies that are part of NEPA.

In 2019, NEPA Rail and NEPA Road changed their names to the South Eastern Program Alliance (SEPA) and the Metropolitan Roads Project Alliance (MRPA) respectively.

Legally, there are still four program alliances. However, in practical terms MTIA is working with five. Each program alliance has different program directors and leadership teams. 

Appendix D shows the full list of LXRP2 sites and Appendix F lists their program alliance allocation.

Procurement approach

At the time of our 2017 audit, LXRA planned to use a deferred price contracting structure to procure contractors to remove the remaining 32 level crossings.

Under this structure, program alliances would competitively bid for and complete an initial package of works of between one and four level crossing sites. If LXRA was satisfied with an alliance’s performance, it would then ask it to develop a formal proposal for delivering additional sites. This meant that only the initial works package would be subject to full price competition. 

In February 2019, MTIA introduced a new delivery strategy and allocated the remaining LXRP1 sites and 25 additional LXRP2 sites to the program alliances. 

While MTIA has now allocated all 75 sites, program alliances still need to provide a detailed works proposal and costings before they are awarded a construction contract for a package of sites. MTIA can reallocate sites to a different program alliance if it is not satisfied with an alliance’s works proposal or past performance.

We discuss MTIA’s approach to procurement and packaging in Chapter 3.

1.5 Options for removing level crossings 

As shown in Figure 1B, there are four options for separating the rail line from the road to remove a level crossing. These are known as grade separation options.

MTIA analyses high-level issues, benefits and constraints for each level crossing site before selecting an indicative preferred option.

FIGURE 1B: Grade separation options

Option Description Potential negative impacts Potential benefits
Rail over road
Rail over road
A rail bridge is built over the road.
The road remains at the existing level.
Train stations may need to be modified to suit the new rail level. It can improve pedestrian access and create opportunities to use the area beneath the rail line.
Rail under road
Rail under road
A rail tunnel is built beneath the existing road. 
The road remains at the existing level.
Nearby train stations may need to be modified or rebuilt to suit the new rail level. Additional pedestrian or cycling bridges can be built to further improve access across the lowered rail line.
Road over rail
Road over rail
The rail line remains at the existing level. 
A road bridge is constructed over the rail line.
Service roads and alternate access options need to be built.
Pedestrian access to train stations needs to be maintained.
Modifications to train stations are usually not needed because the rail level does not change. 
Road under rail
Road under rail
The rail line remains at its existing level. 
An underpass is built beneath the rail line for the road.
Service roads and alternate access options need to be built.
Pedestrian access to train stations needs to be maintained.
Modifications to train stations are usually not needed because the rail level does not change.

Source: VAGO, based on images and information from MTIA.

Selecting a replacement option

To determine which grade separation option to use for each level crossing site, MTIA considers:

  • road functionality and rail operational requirements
  • road and rail horizontal and vertical alignments
  • construction challenges
  • station locations and opportunities to improve the surrounding precinct
  • requirements for land acquisition.

MTIA can also choose a hybrid approach that combines elements of different options. 

1.6 Applying the HVHR framework 

The HVHR framework is a series of project assurance checks that DTF manages. These checks scrutinise major infrastructure projects to increase the likelihood that they will achieve their stated benefits on time and within budget. 

DTF uses an assessment tool to determine a project’s risk profile. It considers a project HVHR if it is:

  • high risk according to its risk assessment tool
  • medium risk according to its risk assessment tool and has a total estimated investment of between $100 million and $250 million
  • low risk according to its risk assessment tool, but has a total estimated investment of more than $250 million 
  • identified by the government as needing the level of scrutiny applied to HVHR investments.

According to DTF’s criteria, the LXRP is a HVHR project. While it is low risk, it requires a total estimated investment of more than $250 million.

Investment life cycle and HVHR guidelines and gateway reviews

Like other HVHR projects, the LXRP is subject to DTF’s investment life cycle (ILC) and HVHR guidelines. The guidelines outline the government’s process for investment decision-making and project delivery across three stages:

  • the business case
  • procurement
  • delivery.

HVHR projects need to complete ongoing quarterly progress reports for OPV. They also undergo external gateway reviews to identify risks to their budget or delivery. These reviews occur at the following six ’gates’:

OPV is as an administrative office within DTF that works with infrastructure delivery agencies to report on major projects’ performance, costs, timelines, scope and risks. 

  • gate 1: concept and feasibility
  • gate 2: business case
  • gate 3: readiness for market
  • gate 4: tender decision
  • gate 5: readiness for service
  • gate 6: benefits analysis.

The ILC and HVHR guidelines recommend that if a project involves multiple physical sites, then the responsible agency should package gateway reviews for efficiency. For the LXRP, works packages that meet the $250 million HVHR threshold undergo gateway reviews, rather than the whole project. 

Back to top

2. Developing LXRP2

Conclusion

Our 2017 audit found that DEDJTR did not analyse if the 50 level crossing sites that the government had committed to remove were the most dangerous and congested. We also found that DEDJTR and LXRA did not develop a business case to outline the project’s intended benefits before starting construction. 

Since then, LXRA developed and applied a transparent process to select sites for LXRP2. This new process has improved the project’s cost-effectiveness because it uses delivery efficiency as one of the criteria for site selection.

However, DEDJTR and LXRA did not develop a new business case for LXRP2. This means that the government did not have vital information about the project's expected economic benefit before it made its decision to invest. 

2.1 LXRA’s site selection process for LXRP2

In our 2017 audit, we found that DEDJTR did not assess the value of removing the first 50 level crossings because they were based on the government’s 2014 election commitment.

DEDJTR and LXRA did not provide detailed advice to the government on the project's expected benefits before construction began. This meant that the government could not be assured that the selected sites met the project’s stated objective to remove the most dangerous and congested level crossings. 

We recommended that LXRA improve its site selection process for any future level crossing removals. 

Status of 2017 recommendation

Recommendation 2: Site selection process
We recommend that DEDJTR, in conjunction with the LXRA, develop a transparent selection and prioritisation process for targeted removal of level crossings beyond current commitments made by government.
Status: Addressed
  DEDJTR accepted this recommendation in 2017. It noted that the government had not committed to removing any further level crossings. 
MTIA (on behalf of DoT) has developed and applied a comprehensive and transparent site prioritisation framework to select LXRP2 sites.

Progress and outcomes

From mid-2017 to late 2018, LXRA developed, refined and applied a transparent site selection process for LXRP2. Figure 2A shows a timeline of LXRA’s work on LXRP2 during this period. 

LXRA used three principles in its initial planning work as criteria to select LXRP2 sites:

  • safety
  • movement
  • place.

Following advice from DoT, MTIA refined this to include a fourth principle—delivery efficiency.

LXRA used these principles to select the most dangerous and congested sites for LXRP2. LXRP2 also minimises delivery time and cost by removing sites that are next to LXRP1 sites or other major projects. 

Unlike its process for LXRP1, LXRA provided DEDJTR and the government with a range of different project proposals before finalising the LXRP2 sites. These options varied in size from 21 to 45 sites and ranged from $4 billion to $16 billion in cost. 

MTIA’s advice to the government for LXRP2 was a significant improvement to LXRP1. This is because it provided the government with a comprehensive understanding of what it could achieve for different investment levels. 

FIGURE 2A: Timeline of planning and development work for LXRP2

FIGURE 2A: Timeline of planning and development work for LXRP2

Source: VAGO, based on information from MTIA and DoT.

Selecting LXRP2 sites 

LXRA developed a site prioritisation framework to respond to our recommendation. This framework outlines LXRA’s four principles for prioritising level crossing removals. Figure 2B describes these principles and the underpinning factors that LXRA used to assess each site. 

FIGURE 2B: Site selection principles and underpinning factors

FIGURE 2B: Site selection principles and underpinning factors

Source: VAGO, based on information from MTIA.

As Figure 2C shows, LXRA’s framework has a five-stage process to determine which sites to recommended for removal. 

FIGURE 2C: LXRA’s site selection process

FIGURE 2C: LXRA’s site selection process

Source: VAGO, based on information from MTIA. 

ALCAM is a nationally used tool for identifying potential risks to, or deficiencies at, level crossings. It is also used to prioritise sites for upgrades based on their incident history, near misses, collisions, fatalities and traffic data.

In stage one, LXRA analysed 276 level crossings using data from several sources, including the Australian Level Crossing Assessment Model (ALCAM) and VicRoads’ traffic data. 

In stage two, LXRA used the safety, movement and place principles to categorise each site as having a ‛very high need’ or a ‛high need’ for removal. For example, Figure 2D shows how LXRA defines high or very high need for the movement principle.

FIGURE 2D: How LXRA used its movement principle to prioritise level crossing removal sites

FIGURE 2D: How LXRA used its movement principle to prioritise level crossing removal sites

Source: VAGO, based on information from MTIA.

In stage three, LXRA identified and prioritised the sites that met the threshold for very high need and high need for more than one principle. 

As Figure 2E shows, LXRA then considered these sites using the delivery efficiency principle in stage four. 

FIGURE 2E: Stage 4 of LXRA’s site prioritisation framework

FIGURE 2E: Stage 4 of LXRA’s site prioritisation framework

Source: VAGO, based on information from MTIA.

LXRA used the following factors to assess if removing a level crossing would have a delivery efficiency benefit: 

LXRA considered …

to assess if …

future needs

the level crossing was likely to need removal in the medium to long-term (beyond 2026) from a movement, place or safety perspective.

corridor completion

removing two or fewer level crossings would separate an entire rail corridor section from road and foot traffic.

land use opportunity

removing the level crossing would lead to other land use and transport benefits, such as significant precinct or urban renewal.

network importance

removing the site would significantly improve the movement of people and goods across the transport network.

In stage five, LXRA produced a consolidated list of 25 sites to submit to the government for approval. Figure 2F shows that each site meets at least one of the selection principles or criteria for delivery efficiency, adjacency or future need.

FIGURE 2F: LXRP2 sites’ prioritisation ratings for principles, delivery efficiency, adjacency and future need

Level crossing site Safety Movement Place Delivery efficiency Adjacency Future need
Argyle Avenue, Chelsea     High  
Camms Road, Cranbourne      
Cardinia Road, Pakenham   High    
Chelsea Road, Chelsea     High  
Cramer Street, Preston        
Evans Road, Lyndhurst      
Fitzgerald Road, Ardeer Very high High        
Glen Huntly Road, Glen Huntly   High    
Greens Road, Dandenong South Very high          
McGregor Road, Pakenham   High    
Main Street, Pakenham High   High  
Mont Albert Road, Mont Albert          
Mt Derrimut Road, Deer Park Very high Very high High      
Munro Street, Coburg      
Murray Road, Preston   High    
Neerim Road, Glen Huntly   High    
Oakover Road, Preston        
Old Geelong Road, Hoppers Crossing High High High  
Racecourse Road, Pakenham   High    
Reynard Street, Coburg      
Robinsons Road, Deer Park   Very high        
Station Street/Gap Road, Sunbury   High    
Swanpool Avenue, Chelsea        
Union Road, Surrey Hills High Very high      
Webster Street, Dandenong Very high          

Source: VAGO, based on information from MTIA and ALCAM.

Balancing the site selection principles 

While the site selection principles were not formally weighted, LXRA prioritised sites that scored highly on the safety and movement principles. This means that LXRP2 is more closely aligned with the project’s overall objective of removing dangerous and congested level crossing sites than LXRP1. 

Safety

LXRA used data from ALCAM to consider the number of past traffic incidents and the risk of future incidents occurring at each site. 

By using the safety principle to prioritise sites, LXRA selected:

  • the three most dangerous level crossing sites in the state (Fitzgerald Road, Ardeer; Mount Derrimut Road and Robinsons Road, Deer Park). While ALCAM classifies these sites as ‛non-metropolitan’, LXRA selected them over metropolitan sites with higher movement or congestion ratings because the risk scores for these sites are significantly higher than those ranked fourth and below
  • 18 sites in ALCAM's top 100 riskiest sites in the metropolitan area 
  • two sites that were rated as very high against the safety principle (Greens Road and Webster Street in Dandenong). ALCAM ranked these sixth and eleventh in its list of the state’s most dangerous sites.
Movement 

The movement principle focuses on sites that have high levels of traffic congestion. Congestion can increase the risk of accidents and risk-taking behaviour at level crossings. 

LXRA used data from TfV, VicRoads and ALCAM to assess sites against the movement principle. 

LXRP2 sites have an above average level of daily trains and vehicle traffic. They also have a higher number of predicted annual collisions. In particular, LXRP2 sites have:

  • 160.4 trains per day on average, compared to the state average of 19.55
  • 10 400 vehicles per day on average, compared to the state average of 1 822
  • 64 per cent more predicted annual collisions on average.

A further 32 per cent of LXRP2 sites have a medium to high risk of collisions occurring. The level crossing with the highest level of traffic in the state (46 000 daily vehicles) is Bell Street, Preston. This site is part of LXRP1 and is adjacent to three LXRP2 sites.

Place 

The place principle considers a site’s impact on pedestrian and cycle paths. It also assesses how the level crossing affects access to important local community facilities, such as schools, medical facilities and local shops. While LXRA consistently applied the place principle in its site assessments, it did not select any of the LXRP2 sites based on this principle alone. 

LXRA identified five sites as having a high need according to the place principle. Of these five sites, four had a delivery efficiency need. The other site—Mount Derrimut Road, Deer Park—also had very high movement and safety needs. 

Delivery efficiency

In our 2017 audit, we highlighted the absence of delivery efficiency in the criteria for LXRP1. We suggested that LXRA should consider the potential cost and time savings of removing sites located close together. 

By adding delivery efficiency as one of its four site selection principles, LXRA took a more strategic and future focused approach for LXRP2. It selected LXRP2 sites that minimised construction costs and rail service disruptions.

In our 2017 audit we identified five sites that LXRA should have considered for removal because they were adjacent to other level crossing removal sites: 

  • three sites in Chelsea (Swanpool Avenue, Chelsea Road and Argyle Avenue)
  • two sites in Glen Huntly (Neerim Road and Glen Huntly Road).

LXRA included all of these sites in LXRP2. 

As Figure 2G shows, there is close association between LXRP1 and LXRP2 sites.

FIGURE 2G: LXRP1 and LXRP2 site locations

FIGURE 2G: LXRP1 and LXRP2 site locations

Source: VAGO, based on information from MTIA.

LXRA also considered delivery efficiency by:

  • selecting eight sites in LXRP2 based on delivery efficiency alone 
  • ensuring that 80 per cent of LXRP2 sites met the delivery efficiency principle. Of these sites, 56 per cent are adjacent to other LXRP2 sites.
  • selecting five sites that are located near current and future major transport projects, such as the Cranbourne Line Upgrade and Metro Tunnel Project. 

Figure 2H shows examples of how LXRA used the delivery efficiency principle to select sites.

FIGURE 2H: Examples of delivery efficiency outcomes for LXRP2 sites 

LXRP2 sites Delivery efficiency outcome
Munro Street and Reynard Street, Coburg Delivered as a single works package with the LXRP1 level crossings at Bell Street, Coburg and Moreland Road, Brunswick.
Murray Road, Cramer Street and Oakover Road, Preston Integrated as a single works package with the LXRP1 level crossing at Bell Street, Preston.
Argyle Street and Swanpool Avenue, Chelsea Packaged with LXRP1 sites Edithvale Road, Edithvale and Station Street, Bonbeach.
Old Geelong Road, Hoppers Crossing Packaged with the LXRP1 site at Werribee Street, Werribee.
Webster Street and Greens Road (Dandenong), Evans Road (Lyndhurst) and Camms Road (Cranbourne) These sites, which are the four remaining level crossings on the Cranbourne line, are packaged for removal in conjunction with the Cranbourne Line Duplication project.

Source: VAGO, based on information from MTIA.

As Figure 2A shows, LXRA developed several options before submitting its final 25 site option to the government. The key difference between the site options it developed in March 2018 and its final list was determined by the delivery efficiency principle. Some sites that LXRA had initially considered for removal narrowly missed inclusion in LXRP2 because they did not have a high delivery efficiency need. However, none of these sites had a very high safety need. 

2.2 Business case and advice to government

The primary purpose of a business case is to provide the government with enough information to make an informed investment decision. Preparing a business case is one of DTF’s key requirements for all HVHR infrastructure projects. 

In our 2017 audit, we found that DEDJTR did not complete a business case until the LXRP had been underway for two years. Consequently, DEDJTR did not follow DTF’s HVHR framework to provide the government with a range of project options.

We also found that DEDJTR did not follow the HVHR framework to update the business case to reflect significant project changes. While DEDJTR added two level crossings to the project after completing the business case, it did not update it. 

Status of 2017 recommendation

Recommendation 1: Business case
We recommend that that DEDJTR follow the HVHR guidelines in developing a business case as the basis for government's decisions, including timing of approval, presenting a range of project options and updating the business case with any significant changes.
Status: Partially addressed
  DEDJTR accepted this audit recommendation in 2017. It noted that it would continue to work with DTF for future investments to apply HVHR requirements.
While MTIA (on behalf of DoT) has not completed a business case or cost-benefit analysis for LXRP2, it has followed other ILC and HVHR guidelines throughout LXRP2.

Progress and outcomes

We found that DoT and MTIA only partially addressed our recommendation. DoT did not complete a new business case for LXRP2. Instead, DoT and MTIA approached LXRP2 as an extension of the original project. They considered a new business case unnecessary as the government had already made its decision to invest in an expanded project. 

While MTIA did not follow DTF’s ILC and HVHR guidelines for completing a business case, its advice to government for LXRP2 did include most of the information that HVHR projects require. This included a clear definition of the problem that the investment will address and estimated costs and delivery timelines. 

However, MTIA did not complete a cost-benefit analysis for the full 75-site project. Cost-benefit analysis is a valuable tool for government decision-making that quantifies the economic benefits of an investment decision. While the LXRP is on track to achieve its intended safety and congestion benefits at each individual site, a cost-benefit analysis would have enabled government to better understand the expected economic benefits of its almost $15 billion investment.

Advice to government for LXRP2 

A business case is the first step in DTF’s ILC and HVHR guidelines, which state that the agency delivering a project should:

  • establish the need that the project aims to address
  • define the project’s intended benefits
  • explore project options
  • estimate costs
  • outline the delivery process.

As part of the 2019–20 state budget process, LXRA made a funding submission for LXRP2. While LXRA did not complete a business case for LXRP2, most of the key information it provided to the government aligned with the requirements of the business case stage of DTF’s ILC and HVHR guidelines, including the project’s:

  • cost estimates and budget, including budget allocations, scope, milestones and estimates
  • site prioritisation framework, including LXRP2’s overall benefits and alignment, prioritisation process and selection options 
  • key considerations, including stakeholders, future needs, corridor completion, land use and network importance
  • delivery process, including formal responsibilities, agreements and risk mitigation strategies. 

Gateway review process

By not completing a business case, MTIA missed the first three gates in DTF’s gateway review process, which provide early and independent scrutiny of a project’s deliverability. Instead, MTIA established an internal review process that broadly covered gates 1 to 3. This process included a: 

  • detailed assessment of the project’s progress by internal MTIA staff and external consultants contracted by MTIA
  • review by MTIA’s executive team and DTF representatives, which focused on the budget estimate for each works package
  • final stage where the contracted program alliance presented detailed works package information and costings for MTIA’s executive review team and DTF representatives, to assess.

DTF’s gate 4 reviews are designed to confirm that a project’s procurement and tender evaluation stages are fulfilling its objectives, benefits plan and statutory requirements. 

MTIA has completed 11 gate 4 reviews that cover four works packages. The independent gateway reviewers engaged by DTF gave each package a ‘green’ project compliance rating. This means that the reviewer found no significant issues that threatened the cost, quality or timeliness of a work package’s delivery. 

2.3 Assessing options for removing level crossings

Our 2017 audit raised concerns that LXRA was not using a consistent approach to assess grade separation options for all 50 sites. 

As part of LXRP’s business case, which was finalised in 2017, LXRA developed a grade separation options assessment framework. However, prior to this, LXRA and VicRoads selected the first 20 sites without clear criteria. 

In its grade separation options assessment framework, LXRA included a multi criteria analysis tool to help it select the remaining 30 sites. However, we found that this framework did not weigh the selection criteria.

We also found that LXRA used a different approach for Frankston line sites, which considered the impact of removal options on adjacent properties. LXRA did not use this criterion to assess any grade separation options for LXRP2. 

Status of 2017 recommendation

Recommendation 7: Options assessment
We recommend that the LXRA apply options assessments transparently and consistently.
Status: Addressed
  LXRA accepted this recommendation in 2017. It also noted that it may add additional information and criteria to its grade separation options assessment framework to provide the best possible advice to government.
MTIA has:
  • revised the process it uses to select grade separation options
  • applied the process consistently and transparently for the 25 LXRP2 sites.

Progress and outcomes

Since our 2017 audit, MTIA has transparently and consistently applied its grade separation options assessment framework for LXRP2 and the remaining LXRP1 sites. The framework uses a three-stage approach to select a grade separation option for each site: 

  • Stage 1: MTIA conducts a preliminary assessment to identify an ‘indicative’ option to estimate a site’s removal costs.
  • Stage 2: MTIA conducts a detailed site investigation, design assessment and consults with the local community.
  • Stage 3: MTIA submits its preferred option to the Minister for Transport Infrastructure for final approval. 

In response to our 2017 audit, MTIA completed detailed site investigations to reassess grade separation options for the remaining LXRP1 sites. MTIA has not used the approach it took for LXRP1 sites on the Frankston line to select any other grade separation options. 

In late 2018, the government announced the 25 LXRP2 sites and specified an indicative grade separation option for each. 

MTIA advised us that over the course of the LXRP, it has developed its knowledge about what options are feasible for different sites. Consequently, it has changed fewer grade separation options after its Stage 1 assessments. This means that MTIA’s community consultations only focus on options that are achievable within engineering and budget constraints. 

Back to top

3. Procurement and packaging

Conclusion

Our 2017 audit found that LXRA’s procurement approach may not achieve value for money because not all sites would be subject to full price competition. While LXRA had set up mechanisms to minimise costs, these were untested. 

MTIA has since changed its procurement approach. While this has further reduced price competition, MTIA is using its benchmarking tool effectively to manage costs. It is also incentivising the program alliances to share lessons learnt to achieve cost savings across the whole project. 

3.1 MTIA’s procurement approach for LXRP2

At the time of our 2017 audit, LXRA had planned to apply full price competition to 25 per cent of the 32 remaining LXRP1 sites and procure the rest through a deferred pricing contract structure.

We found that this could pose a risk to achieving value for money because not all sites would be subject to full price competition.

We also found that it could encourage program alliances to engage in loss-leading behaviours, such as bidding low in initial tendering to secure an advantage when bidding for the remaining sites. 

Status of 2017 recommendation 

Recommendation 8: Procurement
We recommend that the LXRA commission an independent evaluation and report on whether the deferred pricing contract structure is cost-effective and has delivered its benefits.
Status: Addressed
  LXRA accepted this recommendation in 2017. It noted that LXRA would commission an independent evaluation of the LXRP’s deferred pricing contract structure at an appropriate time during its delivery.
MTIA reviewed and incorporated lessons learnt into its procurement strategy for LXRP2. Additionally, OPV has independently reviewed the LXRP’s contracting structure as part of its project assurance review process. Together, these actions have assured MTIA that its contracting approach is cost-effective.

Progress and outcomes

Following the government’s announcement in 2018 that it would remove a further 25 level crossings, MTIA developed a new procurement strategy to integrate these sites into the existing project. 

MTIA’s new procurement strategy allocated all of the remaining LXRP1 sites and the LXRP2 sites to one of the five existing program alliances. This approach differed from LXRA’s original plan to leave some sites unallocated to drive competition among program alliances for the remaining sites. Appendix F shows the program alliance packages for LXRP1 and LXRP2.

Since 2017, the infrastructure construction market in Victoria has continued to experience unprecedented demand. This has shifted market dynamics and has encouraged contractors to seek higher profits, more favourable risk allocation and be more selective in the projects they bid for. 

However, the program alliances had already agreed to the key costs of their LXRP works packages, including corporate overheads, profit percentages and labour on costs for staff. By providing the program alliances with a stable pipeline of works, MTIA has secured better resource allocation and planning because contractors have their resources locked in over the short to medium term.

MTIA’s procurement approach also mitigates the risk of delays due to competition with other projects over access to the same section of rail track because MTIA and DoT can plan works well in advance.

Project assurance review

Project assurance reviews provide timely independent advice on the progress of HVHR projects to both the government (as the investor) and the delivery department or agency.

In August 2019, OPV completed a project assurance review on the LXRP. It found that MTIA’s procurement model was a ‛good basis’ for delivering the 75 level crossing removals by 2025. 

OPV acknowledged that MTIA’s packaging approach is less competitive than a traditional tender process. However, OPV found that MTIA is successfully using several mechanisms to achieve efficiencies. These mechanisms include a cost benchmarking tool, independent price estimator and a performance reward system that incentivises program alliances to collaborate. We discuss these mechanisms further in Section 3.3. 

3.2 Packaging sites for removal

The LXRP’s 2017 business case considered two options for packaging the then 30 remaining level crossings sites:

  • a corridor approach, which grouped sites along the same rail corridors
  • a discipline-based approach, which grouped sites that required similar types of works (such as modifying stations, power, signalling, and rail track).

LXRA determined that the corridor approach effectively balanced time pressures and the need to manage service disruptions. As a result, it packaged most of the remaining LXRP1 sites using this approach.

In our 2017 audit, we found that LXRA did not consider which sites had the highest priority for removal from a safety or rail efficiency perspective in its packaging approach. 

We recommended that LXRA evaluate its packaging approach as the project progressed to ensure that it could incorporate lessons learnt into its future packaging decisions.

Status of 2017 recommendation 

Recommendation 10: Packaging
We recommend that the LXRA, in conjunction with TfV, evaluate its packaging approach and incorporate lessons learned into future crossing removals.
Status: Addressed
  MTIA accepted this recommendation in 2017. It noted that it will incorporate the lessons it learns from its LXRP1 packaging approach when planning future level crossing removals.
MTIA has not commissioned a formal evaluation of its packing approach. However, it has incorporated lessons learnt from LXRP1 into its packaging and procurement approach for LXRP2.

Progress and outcomes

MTIA has continued to take a broadly corridor-based approach to packaging sites. However, this approach is more flexible and considers the benefits of grouping sites according to construction technique and proximity to other work sites. This approach is allowing MTIA to balance cost, timeliness and the expertise of different program alliances.

In February 2019, the government approved MTIA’s new site allocation framework, which outlines how it allocates works packages to the five program alliances. The framework builds on LXRA’s initial corridor-based approach and includes the following new criteria for packaging sites:

  • construction technique, which involves allocating works based on the expected grade separation option
  • capability, which involves allocating works to program alliances based on their past performance
  • proximity, which involves allocating works based on whether a site is near another major transport infrastructure project site
  • capacity, which involves allocating works based on workforce skills or availability.

MTIA did not formally review its packaging approach when it developed its site allocation framework. However, MTIA did incorporate lessons learnt from LXRP1, such as packaging sites to minimise disruption for commuters and reducing costs by grouping sites that use similar construction techniques.

As discussed in Chapter 2, delivery efficiency was one of MTIA’s core principles for selecting LXRP2 sites. This principle required MTIA to consider the cost and time savings of selecting sites that were close to other LXRP sites or major projects. 

This is an important shift since our 2017 audit and shows that MTIA is proactively incorporating lessons learnt from earlier removals into LXRP2. By packaging sites that are close enough to be removed during the same construction period, MTIA can minimise disruptions to public transport users and local residents. 

3.3 Benchmarking and minimising costs

Our 2017 audit found that while using program alliances with partial price competition saves time, it removes competitive tension. To address this risk, LXRA developed a number of mechanisms to minimise costs, including a benchmarking tool.

At the time of our 2017 audit, LXRA had only applied the benchmarking tool to its procurement for NEPA. 

Status of 2017 recommendation 

Recommendation 9: Benchmarking
We recommend that the LXRA embed its benchmarking tool into the procurement process before using it to award additional works sites.
Status: Addressed
  MTIA accepted this recommendation in 2017. It noted that it had already embedded the benchmarking tool in its procurement approach for awarding additional works packages.
MTIA has:
  • embedded its benchmarking tool into its process for awarding additional works packages. The tool is working as intended to achieve cost efficiencies
  • successfully implemented other mechanisms to manage costs, including a Joint Coordination Committee (JCC) that allows program alliances to share knowledge and improve efficiency for future removals.

Progress and outcomes

In this context, a TOC is the estimated cost of designing, constructing and delivering a package of level crossing removals and any associated works, such as constructing new stations.

Since our 2017 audit, MTIA has fully embedded its benchmarking tool into its process for awarding additional works packages. The tool is working to ensure that packaged works are cost-efficient by requiring program alliances to submit a target outturn cost (TOC) that is broadly in line with the LXRP benchmarking tool. MTIA has also successfully implemented other mechanisms to manage costs, including the JCC. 

Awarding additional works packages

MTIA has allocated all of the remaining sites to program alliances. However, program alliances still need to develop a detailed project proposal and TOC before they are formally awarded an additional works package.

MTIA has developed a framework to assess value for money when awarding additional works packages. This framework includes:

  • developing a benchmark cost estimate using its benchmarking tool
  • engaging an independent estimator to examine the costs proposed by a program alliance
  • assessing a program alliance’s performance using performance monitoring reports
  • assessing non‐price elements, including a program alliance’s proposed approach to construction, delivery, continuous improvement, risk management and stakeholder management. 
Benchmarking tool

MTIA’s benchmarking tool is a database that includes cost information from previous level crossing removals, the Regional Rail Link Project and VicRoads. MTIA adds new data over time, which ensures that the tool can provide realistic construction cost estimates.

MTIA uses the benchmarking tool to develop a benchmark cost estimate for each additional works package. MTIA provides program alliances with a high-level price to assist them when developing their cost estimate.

MTIA incentivises program alliances to submit a TOC that is less than the benchmark by increasing their performance award if they are successful. 
In this way, MTIA’s benchmarking tool encourages program alliances to compete against MTIA when developing cost estimates for additional works packages, instead of competing against each other.

Independent estimator

MTIA also appoints an independent estimator to examine TOCs submitted by program alliances. 

The independent estimator uses their own benchmarking data to assess if a program alliance’s TOC is realistic and based on accurate assumptions. 

Using an independent estimator provides assurance that both MTIA and program alliances are using robust and comprehensive price information. 

Site reallocations

To date, MTIA has reallocated two additional works packages to a different program alliance due to cost and performance issues: 

MTIA reallocated

Because ...

High Street, Reservoir from NEPA to NWPA

  • NEPA could not develop a competitive TOC using a suitable design solution.

Werribee Street, Werribee from MRPA to WPA

  • WPA had a strong capability to deliver MTIA’s preferred grade separation option.
  • WPA would achieve delivery efficiency and minimise disruptions for commuters because it could remove the Werribee Street level crossing at the same time as the Old Geelong Road level crossing in Hoppers Crossing.

These reallocations demonstrate that MTIA’s benchmarking tool is working as intended. They also indicate that MTIA is taking a proactive approach to monitoring value for money.

Sharing lessons learnt

While there is now less direct competition between program alliances, MTIA has successfully incentivised them to share lessons learnt and re-use ideas to achieve cost savings. 

Typically, contractors are in competition with each other and are keen to protect their intellectual property because it gives them a competitive advantage when bidding for work.

By tentatively allocating all of the remaining sites, MTIA has encouraged program alliances to work together and share information to help achieve cost efficiencies across the whole project. 

The JCC is one mechanism that program alliances can use to discuss their work and share ideas. The JCC includes 13 subcommittees, which are organised by discipline or topic, such as sustainability. The JCC has an online document sharing site that all program alliances can access to share information. 

MTIA modelled this approach on a similar framework used for the Regional Rail Link project. It has further strengthened this approach by offering program alliances incentives to collaborate.

In 2018, OPV found that the JCC was a key mechanism for encouraging a culture of sharing and continuous improvement across the LXRP.

At the time of our 2017 audit, LXRA’s performance reward regime included two indicators that focused on improvement—one for continuous improvement and another for innovation. In 2018, LXRP replaced these two indicators with:

  • ‛adding value’, which combines the continuous improvement and innovation indicators and measures if program alliances share learnings about better practice 
  • ‛adoption’, which rewards a program alliance when it uses a solution from another alliance, such as the same station design.

These indicators encourage program alliances to share ideas, rather than rewarding them for developing unique design solutions. By doing this, the LXRP has the potential to achieve cost savings through economies of scale because program alliances can adopt similar designs that require the same standard components and materials. 

Back to top

4. Managing benefits

Conclusion

In 2017 we found that LXRA was using unclear KPIs to measure the benefits of individual level crossing removals. We also found that LXRA was not progressively monitoring whether the project was achieving its intended benefits.

MTIA has improved the amount and quality of data it uses to measure KPIs for individual sites. It has also introduced a process to annually monitor the project’s overall progress. As a result, MTIA now has a comprehensive understanding of how the project is progressing and whether it is on track to achieve its intended benefits.

4.1 KPIs for level crossing removal sites

Benefits management is a core part of effective project delivery. It involves setting defined objectives and KPIs at the start of a project and monitoring if these are achieved. 

Under the LXRP benefits management plan, MTIA uses 14 KPIs to measure if each individual site is achieving its intended benefits. However, our 2017 audit found that these KPIs did not set clearly defined benchmarks for improvement. 

Many of the KPIs were binary measures. This means that simply removing a level crossing would be considered an improvement. For example:

  • 100 per cent of sites with road-based public transport will have improved punctuality (KPI 1.3b)
  • 100 per cent of sites will have improved access to local activity centres and major services (KPI 2.3b).

In 2017, we recommended that MTIA’s benefit reports include a discussion of the results and the extent of any improvements, rather than simply stating if a KPI has been met. 

Status of 2017 recommendation 

Recommendation 3: KPIs
We recommend that DEDJTR develop comprehensive KPIs and targets to meaningfully measure achievement of intended benefits.
Status: Addressed
  DEDJTR accepted this recommendation in 2017. It noted that it would implement it by enhancing the analysis and commentary in its site benefits reports under the LXRP benefits management plan.
MTIA has since improved the amount and quality of information it uses to report against the KPIs. Consequently, MTIA’s site benefits reports now provide a more comprehensive qualitative assessment of whether level crossing removals are achieving their intended benefits or not.

Progress and outcomes

MTIA advised us that following our 2017 audit, it considered if it could develop benchmarks for each KPI. However, it determined that it was not feasible to fully model data for every KPI at every site to create a specific benchmark. Instead, it decided to focus on providing more detailed information and commentary.

At the time of our 2017 audit, LXRA had four draft site benefits reports, but these included limited data. 

Since our 2017 audit, MTIA has increased the level of detail and analysis in its site benefits reports. 

As a result, the reports now provide more comprehensive information about the extent to which an individual level crossing removal has achieved its intended benefits.

At June 2020, MTIA has:

  • completed 10 individual site benefits reports
  • 19 reports in draft 
  • a further five sites where drafting has not commenced. MTIA advised us that this is because traffic patterns have not returned to normal yet. Drivers avoid level crossings sites during construction, and it takes some time for them to return once the area reopens.

The 10 completed reports include the four reports that were in draft at the time of our 2017 audit. Each individual site report is approximately 35 pages and provides contextual information and an assessment of the extent to which each KPI has been met. For example, instead of just stating that removing a crossing as improved travel time for vehicles (KPI 1.1a), the reports specify the minutes saved.

This qualitative and contextual material is more meaningful than a binary report that simply states if the KPIs have been met. 

Appendix E lists the 15 KPIs that MTIA set for the LXRP benefits management plan.

KPI results

Of the 10 sites with completed benefits reports, MTIA has:

  • achieved 70 per cent (105) of the KPIs
  • not met 21 per cent (31) of the KPIs
  • not finalised a further 9 per cent (14) of the KPIs.

Figure 4A shows these results.

FIGURE 4A: Percentage of KPIs achieved for the first 10 level crossings removed

FIGURE 4A: Percentage of KPIs achieved for the first 10 level crossings removed

Source: VAGO, based on information from MTIA. 

The completed site benefit reports show improved travel times and an increased level of traffic flow. For example: 

  • the number of vehicles travelling through the 10 sites has increased by 24 per cent on average (KPI 1.1b—reducing congestion and improving traffic flow)
  • the average improvement in travel time for vehicles at the 10 sites is one minute during the morning and evening peak periods (KPI 1.1a)
  • Blackburn Road, Blackburn, had the greatest improvement for vehicle throughput with an increase of 60 per cent for northbound peak hour traffic in the morning.

Of the unmet KPIs, 16 are due to a lack of pre-removal data for sites that VicRoads was initially responsible for. As the LXRP progresses, there will be fewer sites without suitable baseline data, and the quality of MTIA’s KPI reporting will therefore improve.

KPI 3.1a measures if 100 per cent of treated sites have zero crashes and near miss incidents involving trains as a result of the level crossing removal and no negative safety outcomes as a result of the removal works.

Other unmet KPIs are also due to data gaps. For example, MTIA cannot yet conclude against KPI 3.1a because it requires three years’ worth of data to assess if a level crossing removal has had any negative safety outcomes.

4.2 How MTIA monitors the project’s outcomes

In our 2017 audit, we found that there was insufficient data to make an informed judgement on how well the project was progressing towards achieving its intended outcomes.

At the time, LXRA advised us that when it completes all of the removals, it intends to combine the individual site benefit reports into a holistic project report. However, we found that waiting until the project is complete would mean that LXRA would have limited insight into how the LXRP is progressing towards realising its benefits.

Status of 2017 recommendation 

Recommendation 4: Monitoring project outcomes
We recommend that DEDJTR, in conjunction with the LXRA, progressively monitor the progress of achievement of LXRP outcomes to facilitate timely insight into how the program is progressing towards benefits realisation.
Status: Addressed
  DEDJTR accepted this recommendation in 2017. It noted that it was already implementing it because it was also a requirement under the HVHR framework.
MTIA (on behalf of DoT) has developed a project-wide benefits framework that progressively monitors its outcomes. According to the framework’s annual reports, the LXRP is on track to achieve slightly more benefits than the original business case anticipated.

Progress and outcomes

Following our 2017 audit, MTIA developed the LXRP Benefits Framework. This framework is designed to increase the transparency of the LXRP’s overall progress, outcomes, benefits and performance.

Project benefits reports

The LXRP Benefits Framework requires MTIA to annually report on the LXRP’s overall progress. Complementing the site benefits reports, these project benefits reports provide a progressive and timely assessment of whether the project is achieving its overall objectives.

The project benefits reports also aim to assess the wider economic, social and environmental benefits that the project has created or enabled, which are broader than the site specific KPIs. 

MTIA measures and reports on the project benefits across four domains, as outlined in the 2017 business case:

  • Communities are more mobile and connected.
  • Communities are more productive.
  • Communities are more vibrant and safer.
  • Communities are more sustainable.

MTIA measures the benefits of each domain at two levels:

  • the project level, which aggregates information about all sites
  • the corridor level, which describes the combined benefits of removing level crossings along a specific rail corridor. For example, the Cranbourne to Pakenham rail corridor.

MTIA has completed two project benefit reports—one at 30 June 2018 and one at 30 June 2019. 

The 2019 report includes data analysis and economic modelling undertaken by an external consultant. Based on data from 26 removed sites, the 2019 report found that the project is achieving:

  • 7.6 per cent higher travel time savings for road and train users at removed level crossing sites than forecast
  • 17.5 per cent higher reliability savings than forecast.

This means that travel time benefits across the whole transport network are 3.5 per cent higher than the original business plan forecast.

While the annual project benefits reports provide a useful point-in-time assessment, the results are uneven. This is because the individual sites do not contribute equally to the project’s overall benefits. The removal of some sites will have a greater impact on the network than others. 

For example, while the 2019 report found that overall travel time benefits are 3.5 per cent higher than the original business plan, this is a decrease compared to the benefits reported at 30 June 2018. At that time, LXRA had data for nine sites and observed that travel time benefits were 17 per cent higher than forecast in the business case.

The project benefits reports are not publicly available. However, MTIA does include some information from case studies on its website. Given that the business case for the first 50 sites was not completed until LXRA had started construction, providing public updates on the LXRP’s benefits would improve transparency around the project. 

Gate 5 and 6 reviews 

Gate 5 reviews assess if an asset or service is ready for delivery. Gate 6 reviews examine if a project has delivered its benefits as defined in its business case.

In addition to MTIA’s annual project and individual site benefits reports, DTF’s gateway reviews are providing independent assurance that the LXRP is achieving its intended benefits.

As discussed in Chapter 2, all HVHR projects must go through DTF’s gateway review process. In December 2018, MTIA received a joint gate 5 and 6 report for 10 LXRP1 sites. 

MTIA advised us that DTF combined the gate 5 and 6 reviews for some packages. This was because some packages included sites that had already been operational for two years by the time the review started, which limited the benefits of a standalone gate 5 review.

The 10 sites that DTF reviewed in 2018 were part of packages 1, 2 and 4, which VicRoads started and LXRA completed. LXRP packages are listed in Appendix F. 

The independent gateway reviewer engaged by DTF assessed the 10 sites’ overall delivery confidence as ‛good’. It also noted that LXRA had successfully managed a high volume of work, a challenging timeline and the current ‛heated’ construction market. 

While the gate 5 and 6 report was positive, it did not cover any sites that were fully delivered by LXRA/MTIA through the program alliance model. This means that MTIA is yet to fully compare the benefits of its contracting approaches. 

MTIA advised us that it has not set the next gate 5 and 6 review date with DTF. It estimates that this will take place in early 2021. This review will assess the benefits of MTIA’s new procurement approach compared to the earlier approach. 

It is important that DTF and MTIA continue to complete timely gate reviews so they can identify any issues that could impact the LXRP achieving its benefits as early as possible.

OPV's major projects performance reports

In addition to internal reports and gate reviews, MTIA and DoT provide information to OPV for inclusion in its quarterly major projects' performance reports. These reports to government highlight emerging risks and issues in major infrastructure and information technology projects, which includes the LXRP.  

In its June 2020 report, OPV noted that LXRP is experiencing cost pressures related to the escalating cost of labour and materials. However, OPV gave the project an overall 'amber' risk rating. This means that it considers that MTIA is managing the identified risks and no government action is needed at this time.  

As discussed in Chapter 3, MTIA is using several mechanisms to minimise costs.

Back to top

5. Network integrity and standards governance

Conclusion

Our 2017 audit found significant weaknesses in PTV’s network integrity controls and the process it used to manage changes to engineering standards. 

While DoT has introduced new systems to govern standards and assure network integrity, 2019 machinery of government changes have made further work necessary. Until this work is complete, there is a risk that project delivery agencies and contractors may be unsure about the standards their projects need to meet.

DoT is also yet to develop system-level network requirements, which means it lacks a holistic view of the public transport network’s technical needs. DoT does not have a documented plan to complete this work in a timely way.

This chapter provides discusses:
  • Network integrity controls
  • Network standards

5.1 Network integrity controls

Network integrity refers to how functionally effective, reliable, maintainable, secure, safe and environmentally compatible a public transport network is.

Strong network integrity controls support a transport system to work as a coherent whole. They also ensure that individual infrastructure projects fit seamlessly into the network. 

In our 2017 audit, we found that PTV did not have adequate resources to ensure the transport network’s integrity. We also noted that while PTV had established a new network integrity governance framework in November 2017, it was too early to tell if it would be effective.

Status of 2017 recommendation 

Recommendation 6: Network integrity
We recommend that DEDJTR, in conjunction with PTV, monitor the effectiveness of PTV’s controls to improve its network integrity function.
Status: In progress
  DEDJTR accepted this recommendation in 2017. It noted that TfV and PTV had completed a review and implemented a revised network assurance governance framework.
  • significantly improved its network integrity controls and capability in 2018 and 2019
  • introduced a new project and network governance framework following machinery of government changes.
As this new framework is still at an early stage, DoT will need to monitor its effectiveness.

Progress and outcomes

During 2018 and 2019, PTV and DoT progressively monitored their network integrity controls. They also made changes to improve how the controls ensure that major infrastructure projects integrate with the public transport network. 

While DoT introduced a new network and project governance framework in March 2020, work on this recommendation is still in progress:

Since 2017 DoT has …

However …

secured funding to fulfill its network integrity role.

DoT has not confirmed resourcing for this beyond 2020–21.

introduced a new network and project governance framework.

It is too early to tell if this framework will be effective.

established an assurance process to manage how major projects integrate with the network.

DoT will need to refine this assurance process to align with the new network and project governance framework. There is also a risk that this may confuse delivery agencies and contractors.

Assurance process for infrastructure projects

In 2019, DoT introduced a network integrity assurance management procedure. This procedure outlines the assurance activities that DoT and infrastructure delivery agencies need to complete throughout a project’s life cycle to ensure it will integrate with the overall public transport network.

Under the assurance procedure, DoT conducts reviews throughout a project’s life cycle to confirm that the delivery agency is managing network risks. DoT also assesses how effectively the project will integrate with the existing public transport network. 

MTIA is following the assurance procedure for LXRP works packages and providing DoT with the evidence it needs to complete its assurance reports.

Network and project governance framework

In January 2019, machinery of government changes transferred the responsibility for the road network from VicRoads to DoT. In March 2020, DoT introduced a network and project governance framework to integrate road and rail network integrity controls.

While infrastructure delivery agencies’ project assurance responsibilities have stayed the same, the framework has replaced two key governance bodies with new committees.

DoT has established a change management process to ensure that infrastructure delivery agencies and contractors understand the new framework. DoT is updating its key policies, including its network integrity assurance management procedure, to align with the new framework. 

5.2 Network standards

As part of its network integrity and assurance role, DoT is responsible for setting network technical standards (NTS). These define the specific engineering and technical requirements that transport infrastructure and assets need to meet to ensure compatibility and consistency across the entire transport network. In addition to NTS, rail operators also develop their own engineering standards that relate to the parts of the network they are responsible for.

In 2017, we found that PTV did not have NTS. Instead, Victoria’s rail network relied on the Victorian Rail Industry Operators' Group standards, which PTV did not oversee. 

This meant that PTV did not have control over new standards or changes to existing standards. Consequently, it could not manage the risk that ad hoc changes could affect the network’s integrity. 

We also found a risk that changes to engineering standards could have time and cost impacts on in-progress projects that had been approved using earlier standards. The agencies delivering these projects would need to alter works or seek waivers from the new standards.

Status of 2017 recommendation 

Recommendation 5: Network standards
We recommend that DEDJTR, in conjunction with PTV, develop contemporary network rail standards, so that agencies delivering rail projects have an understanding of network requirements and what is required to assure projects meet engineering, network integration and safety requirements.
Status: In progress
  DEDJTR accepted this recommendation in 2017 and noted that it was already being implemented. At the time, TfV was developing new rail standards with input from PTV, Metro Trains Melbourne (Metro Trains), LXRA and the Melbourne Metro Rail Authority.
DoT has since developed a set of NTS and a standards governance framework. However, it still has work to do to further develop system-wide technical standards and embed standards governance.

Progress and outcomes

Since 2017, PTV and DoT have endorsed 13 NTS, which cover all of the rail network’s major technical components. These include track, traction power, public transport precincts and operational control management systems. DoT and PTV are developing a further five NTS.

In addition, DoT has introduced a standards governance framework to oversee changes to NTS. However, DoT still has work to do to address the risks raised in our 2017 audit:

DoT has not …

This creates a risk that …

updated its standards governance framework to align with its new project and network governance framework.

delivery agencies and contractors may be confused about their responsibilities as key governance committees have been replaced or merged.

developed high-level network requirements.

infrastructure projects may not align with the transport network’s future technical needs because DoT has not set a clear vision.

New standards governance framework

Before construction begins on an infrastructure project, the delivery agency outlines the project’s agreed outcomes. It also defines the minimum system requirements that contractors need to meet to ensure that the project seamlessly integrates with the transport network.

NTS and engineering standards change over time. If changes have significant safety impacts, then contractors may need to change their plans to meet the new requirements. 

As part of its franchise agreement with rail operator Metro Trains, PTV established a standards governance group in early 2018. This group ensures that Metro Train’s engineering standards align with the network’s overall technical standards. 

In 2019, DoT built on this governance group and introduced an expanded standards governance framework. The new standards governance framework:

Axle counters detect if a section of rail track is clear or occupied by recognising the presence of train axles. Axle counters are an important part of rail safety.

  • manages the development and maintenance of NTS
  • manages the relationship between NTS, operator-developed engineering standards and related user policies and guides
  • includes several DoT-run governance committees that:
    • endorse new or revised standards
    • ensure that any changes align with network requirements
    • determine if in-progress projects need to adopt a new or changed standard
  • clearly defines the roles and responsibilities for DoT, project delivery agencies, rail operators and contractors.

MTIA advised us that while DoT's new standards governance framework is a significant improvement since our 2017 audit, some changes are still causing confusion for delivery agencies and contractors. 

One example of ongoing confusion is Metro Train’s recent engineering directive on axle counters. 

FIGURE 5A: Case study: Metro Trains’ directive on axle counters

In December 2019, Metro Trains issued a chief engineer’s directive on rail condition requirements for axle counters. This was a stopgap measure before the new engineering standard was endorsed. 

In January 2020, MTIA alerted DoT that Metro Trains had issued the directive without consulting them, which is usually required if an operator wants to change a standard. 

LXRP contractors were confused about the status of the directive, and MTIA was concerned that there would be cost implications if in-progress removals needed to follow it. 

In May 2020, DoT established a working group on axle counters to sit outside the standards governance structure. The working group aims to respond to MTIA’s concerns about the directive and consider a network wide risk assessment. 

DoT has recently updated its standards governance framework to address the gap that allowed the axle counter issue to arise. It now requires Metro Trains and DoT to consult with agencies on urgent engineering directives before releasing them.

This example shows that unclear changes are still occurring. However, DoT is continuing to refine its standards governance process as needed.

Source: VAGO, based on information from DoT and MTIA.

With the introduction of DoT's network and project governance framework in March 2020, DoT has changed the names and responsibilities of the committees involved in standards governance. To avoid confusion, DoT needs to update its standards governance framework to reflect these changes. 

Network requirements

While DoT has developed NTS, it advised us that it is still developing higher level network requirements, including a Victorian Rail Plan that will provide the agreed architecture for the network's configuration and service levels.  Until that work is completed, DoT lacks a consolidated understanding of the public transport network’s technical needs to guide project development and future system-wide improvements. 

Back to top

Appendix A. Submissions and comments

We have consulted with DoT and MTIA and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments. 

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Responses were received as follows:
Response provided by the Secretary, DoT

DoT Response letter page 1

DoT Response letter page 2

Back to top

Appendix B. Acronyms, abbreviations and glossary

Acronyms  
ALCAM Australian Level Crossing Assessment Model
DEDJTR Department of Economic Development, Jobs, Transport and Resources
DoT Department of Transport
DTF Department of Treasury and Finance
HVHR High Value High Risk
ILC investment life cycle
JCC Joint Coordination Committee
KPI key performance indicator
LXRA Level Crossing Removal Authority
LXRP Level Crossing Removal Project
LXRP1 Level Crossing Removal Project—Stage 1
LXRP2 Level Crossing Removal Project—Stage 2
MTIA Major Transport Infrastructure Authority
MRPA Metropolitan Roads Project Alliance
NEPA North Eastern Program Alliance
NTS network technical standards
NWPA North Western Program Alliance
OPV Office of Projects Victoria
PTV Public Transport Victoria
SEPA South Eastern Program Alliance
SPA Southern Program Alliance
TfV Transport for Victoria
TOC target outturn cost
VAGO Victorian Auditor-General’s Office
WPA Western Program Alliance

Abbreviations

 
Metro Trains Metro Trains Melbourne

Back to top

Appendix C. Scope of this audit

Who we audited What we assessed What the audit cost
DoT and MTIA We assessed if DoT and MTIA have effectively addressed the recommendations from our 2017 audit Managing the Level Crossings Removal Program. The cost of this audit was $225 000. 

Our methods

For this follow-up audit we assessed whether agencies: 

  • made accurate attestations about implementing our 2017 recommendations 
  • have taken timely action to address our recommendations
  • have plans to address incomplete recommendations 
  • are monitoring and reviewing the impact of their actions to implement our recommendations
  • made transparent and appropriate decisions in cases where they decided not to implement recommendations
  • have addressed the performance issues we identified, and made recommendations about, in our 2017 audit. 

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. We also provided a copy of the report to the Department of Premier and Cabinet.

Unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

FIGURE C1: Our 2017 audit recommendations

Recommendation number Description
1 DEDJTR follow HVHR guidelines in developing a business case as the basis for government's investment decisions, including timing of approval, presenting a range of project options and updating the business case with any significant changes.
2 DEDJTR, in conjunction with MTIA, develop a transparent selection and prioritisation process for targeted removal of level crossings beyond current commitments made by government.
3 DEDJTR develop comprehensive KPIs and targets to meaningfully measure achievement of intended benefits.
4 DEDJTR, in conjunction with MTIA, progressively monitor the progress of achievement of LXRP outcomes to facilitate timely insight into how the program is progressing towards benefits realisation.
5 DEDJTR, in conjunction with PTV, develop contemporary network rail standards, so that agencies delivering rail projects have an understanding of network requirements and what is required to assure projects meet engineering, network integration and safety requirements.
6 DEDJTR, in conjunction with PTV, monitor the effectiveness of PTV's controls to improve its network integrity function.
7 MTIA apply options assessments transparently and consistently.
8 MTIA commission an independent evaluation and report on whether the deferred pricing contract structure is cost-effective and has delivered its intended benefits.
9 MTIA embed its benchmarking tool into the procurement process before using it to award additional works sites.
10 MTIA, in conjunction with TfV, evaluate its packaging approach and incorporate lessons learned into future level crossing removals.

Note: See Section 1.3 for changes to responsible agencies.
Source: VAGO.

Back to top

Appendix D. LXRP sites

FIGURE D1: The first 50 level crossing removal sites (LXRP1)

Level crossing location Current status (July 2020)
Abbotts Road, Dandenong South Removed
Aviation Road, Laverton Removed
Balcombe Road, Mentone Removed
Bell Street, Coburg Construction—contract awarded
Bell Street, Preston Removed
Blackburn Road, Blackburn Removed
Buckley Street, Essendon Removed
Burke Road, Glen Iris Removed
Camp Road, Campbellfield Removed
Centre Road, Bentleigh Removed
Centre Road, Clayton Removed
Chandler Road, Noble Park Removed
Charman Road, Cheltenham Construction—contract awarded
Cherry Street, Werribee Construction—contract awarded
Clayton Road, Clayton Removed
Clyde Road, Berwick Construction—contract awarded
Corrigan Road, Noble Park Removed
Edithvale Road, Edithvale Construction—contract awarded
Eel Race Road, Carrum Removed
Ferguson Street, Williamstown In planning
Furlong Road, St Albans Removed
Glenroy Road, Glenroy Construction—pending contract
Grange Road, Alphington Removed
Grange Road, Carnegie Removed
Hallam Road, Hallam Construction—pending contract
Heatherdale Road, Mitcham Removed
Heatherton Road, Noble Park Removed
High Street, Reservoir Removed
Koornang Road, Carnegie Removed
Kororoit Creek Road, Williamstown North Removed
Lochiel Avenue, Edithvale* Construction—contract awarded
Lower Plenty Road, Rosanna Removed
Main Road, St Albans Removed
Manchester Road, Mooroolbark Construction—contract awarded
Maroondah Highway, Lilydale Construction—contract awarded
Mascot Avenue, Bonbeach* Removed
McKinnon Road, McKinnon Removed
Melton Highway, Sydenham Removed
Moreland Road, Brunswick Construction—contract awarded
Murrumbeena Road, Murrumbeena Removed
Mountain Highway, Bayswater Removed
North Road, Ormond Removed
Park Road, Cheltenham* Removed
Poath Road, Hughesdale Removed
Seaford Road, Seaford Removed
Scoresby Road, Bayswater Removed
Skye/Overton Road, Frankston Removed
Station Street/Bondi Road, Bonbeach Construction—contract awarded
Station Street, Carrum Removed
South Gippsland Highway, Dandenong Construction—contract awarded
Thompsons Road, Lyndhurst Removed
Toorak Road, Kooyong Removed
Werribee Street, Werribee Construction—contract awarded

Note: *Level crossing not included in original list. MTIA added these later for delivery efficiency with LXRP1 sites.
Source: VAGO, based on information from MTIA.

FIGURE D2: LXRP2 level crossing removal sites

Level crossing location Current status (July 2020)
Argyle Avenue, Chelsea Construction—contract awarded
Camms Road, Cranbourne In planning
Cardinia Road, Pakenham Construction—contract awarded
Chelsea Road, Chelsea Construction—contract awarded
Cramer Street, Preston Construction—pending contract
Evans Road, Lyndhurst Construction—contract awarded
Fitzgerald Road, Ardeer In planning
Glen Huntly Road, Glen Huntly In planning
Greens Road, Dandenong South Construction—contract awarded
McGregor Road, Pakenham In planning
Main Street, Pakenham In planning
Mont Albert Road, Mont Albert In planning
Mt Derrimut Road, Deer Park In planning
Munro Street, Coburg Construction—contract awarded
Murray Road, Preston Construction—pending contract
Neerim Road, Glen Huntly In planning
Oakover Road, Preston Construction—contract awarded
Old Geelong Road, Hoppers Crossing Construction—contract awarded
Racecourse Road, Pakenham In planning
Reynard Street, Coburg Construction—contract awarded
Robinsons Road, Deer Park In planning
Station Street/Gap Road, Sunbury In planning
Swanpool Avenue, Chelsea Construction—contract awarded
Union Road, Surrey Hills In planning
Webster Street, Dandenong In planning

Source: VAGO, based on information from MTIA.

Back to top

Appendix E. LXRP benefits management plan

FIGURE E1: Site assessment framework—LXRA benefits and KPIs

Benefits KPI Benefits management plan section Target 
1. Improved productivity from more reliable and efficient transport networks
Network efficiency
 
 
1 1.1a Travel time 100% of sites will have an improvement in travel time following removal of level crossing
1.1b Throughput 100% of sites will have increased throughput of vehicles following removal of level crossings
100% of sites will have increased throughput of cyclists or pedestrians following removal of level crossings
Reliability of travel times on the road and rail network 2 1.2a Travel time standard deviation 100% of sites with boom gate closures of more than 25% of the AM peak will have an improvement to the reliability of travel time following removal of level crossings
1.2b Train punctuality 100% of sites will have an elimination of passenger weighted minutes as a result of signal faults at the level crossing following removal of level crossings
Public transport improvements 3 3.1a Percentage of line grade separated Percentage of line grade separated as a result of the LXRP (% based on each site)
1.3b Public transport punctuality 100% of sites with road-based public transport will have improved punctuality of road-based public transport (for example, an increase in the number of services that are on time or not as late) following removal of level crossings
Economic productivity 4 1.4a Access to labour markets 100% of national employment clusters will have improved access to labour markets following the removal of level crossings
2. Better connected, liveable and thriving communities  
Frequency and severity of accident 1 2.1a Community satisfaction At least 60% of survey respondents at each site are satisfied with the changes as a result of the level crossing removal
Infill land developments around rail corridors 2 2.2a Integrated development opportunities All sites identified with integrated development opportunities at the project proposal stage will achieve an increase in residential units and/or let‐able retail/business floor space
Access to jobs, education and services 3 2.3a Access for employment 100% of sites will improve access to jobs, education, and services
2.3b Access for local activity centres 100% of sites will have improved access to local activity centres and major services
Public transport intermodal connectivity 4 2.4a Drop-off time and distance 100% of sites have reduced distance and/or travel time between collection and drop-off points
3. Safer communities
Frequency and severity of accidents 1 3.1a Number of incidents 100% of sites have zero crashes and near miss incidents involving trains as a result of the level crossing removal and no negative safety outcomes as a result of the level crossing removal works
Exposure to risk 2 3.2a Improved ALCAM 100% of sites have an ALCAM risk score of zero

Source: VAGO, based on information from MTIA.

Back to top

Appendix F. LXRP works packages

FIGURE F1: LXRP1 works packages

Works/program alliance package Level crossing removal site
Package 1 Burke Road, Glen Iris
Centre Road, Bentleigh
McKinnon Road, McKinnon
North Road, Ormond
Package 2 Blackburn Road, Blackburn
Furlong Road, St Albans
Heatherdale Road, Mitcham
Main Road, St Albans
Package 3 Chandler Road, Noble Park
Clayton Road, Clayton
Centre Road, Clayton
Corrigan Road, Noble Park
Grange Road, Carnegie
Heatherton Road, Noble Park
Koornang Road, Carnegie
Murrumbeena Road, Murrumbeena
Poath Road, Hughesdale
Package 4 Mountain Highway, Bayswater
Scoresby Road, Bayswater
Individually packaged sites Melton Highway, Sydenham
Thompsons Road, Lyndhurst
North Eastern Program Alliance (NEPA) Bell Street, Preston
Clyde Road, Berwick**
High Street, Reservoir
Grange Road, Alphington
Hallam Road, Hallam*
Lower Plenty Road, Rosanna
Manchester Road, Mooroolbark*
Maroondah Highway, Lilydale*
South Gippsland Highway, Dandenong**
Toorak Road, Kooyong*
North Western Program Alliance (NWPA) Bell Street, Coburg
Buckley Street, Essendon
Camp Road, Campbellfield
Glenroy Road, Glenroy
Moreland Road, Brunswick
Skye/Overton Road, Frankston
Southern Program Alliance (SPA) Balcombe Road, Mentone
Charman Road, Cheltenham
Eel Race Road, Carrum
Edithvale Road, Edithvale
Mascot Avenue, Bonbeach
Park Road, Cheltenham
Seaford Road, Seaford
Station Street/Bondi Road, Bonbeach
Station Street, Carrum
Western Program Alliance (WPA) Abbotts Road, Dandenong South
Aviation Road, Laverton
Cherry Street, Werribee
Ferguson Street, Williamstown
Kororoit Creek Road, Williamstown
Werribee Street, Werribee

Note: Packages 1 to 4 preceded the program alliance framework.
Note: *sites allocated to NEPA rail.
Note: **sites allocated to NEPA road.
Source: VAGO, based on information from MTIA.

FIGURE F2: LXRP2 program alliances

Program alliance Level crossing removal site
Southern Program Alliance (SPA) Argyle Avenue, Chelsea
Chelsea Road, Chelsea
Swanpool Avenue, Chelsea
Glen Huntly Road, Glen Huntly
Neerim Road, Glen Huntly
South Eastern Program Alliance
(previously NEPA rail)
Mont Albert Road, Mont Albert
Union Road, Surrey Hills
Metropolitan Roads Project Alliance
(previously NEPA road)
Camms Road, Cranbourne
Cardinia Road Pakenham
Evans Road, Lyndhurst
Fitzgerald Road, Ardeer
Robinsons Road, Deer Park
North Western Program Alliance (NWPA) Cramer Street, Preston
McGregor Road, Pakenham
Main Street, Pakenham
Munro Street, Coburg
Murray Street, Preston
Oakover Road, Preston
Racecourse Road, Pakenham
Reynard Street, Coburg
Western Program Alliance (WPA) Greens Road, Dandenong South*
Mount Derrimut Road, Deer Park
Old Geelong Road, Hoppers Crossing
Construction—contract awarded
Station Street/Gap Road, Sunbury**
Webster Street, Dandenong

Note: *Greens Road is now allocated to the Cranbourne Line Duplication's program alliance.
Note: **Station Street is now allocated to the Rail Infrastructure Alliance program alliance.
Source: VAGO, based on information from MTIA.

Back to top