Managing the Level Crossing Removal Program

Tabled: 14 December 2017

Audit overview

Melbourne is facing unprecedented population growth in the coming decades, which will result in an increase in travel demands. Plan Melbourne 2014 stated that, by 2050, Melbourne's road and rail network will need to accommodate an additional 10.7 million daily trips, on top of the 14.2 million daily trips recorded in 2014.

The Level Crossing Removal Program (LXRP) is one of the Victorian Government's key transport infrastructure projects. Part of an overarching policy to improve Melbourne's transport network, its stated aim is to remove 50 of the most dangerous and congested level crossings.

Following its election in 2014, the government upheld its pre-election commitment to remove 50 crossings by 2022. The May 2015 Budget included an additional commitment to remove 20 crossings by 2018. Since the LXRP's announcement, the government has identified an additional two level crossings for removal, bringing the total to 52.

Established in 2015, the Level Crossing Removal Authority (LXRA) took over responsibility for managing the program from VicRoads. At September 2017:

  • 10 level crossings had been removed
  • 16 were in design and construction stages
  • 11 had a contract awarded
  • nine were in the tender stage
  • six were in early planning stages.

As the LXRP is only two years into an eight-year program, our assessment at this time serves to identify risks, lessons and opportunities for improvement for future works.

We have examined whether the LXRP is cost-effective in terms of whether it has improved, or is expected to improve, the safety and efficiency of the state's road and rail network.

Our audit focused on the role of the Department of Economic Development, Jobs, Transport and Resources (DEDJTR), LXRA, Public Transport Victoria (PTV), and Transport for Victoria (TfV) in providing oversight and strategic focus for the LXRP.

Conclusion

Contrary to publicly stated objectives, not all of the 50 level crossings selected for removal are the most dangerous and congested. In this sense, the LXRP is not fully effective when compared to the stated objective.

The delivery of the program is ahead of schedule, and LXRA expects to surpass its target of removing 20 crossings by 2018. However, this pace presents risks to achieving value for money. These risks are compounded by an inadequate and delayed business case, and poor indicators to measure program benefits.

While LXRA is managing some aspects of the program well, including rail occupations and monitoring integrated development opportunities (IDO), opportunities remain to improve the LXRP—in particular, the rigour applied to and transparency of site selection and prioritisation, site packaging and procurement, benefits management, and integration of various rail projects.

Addressing these concerns will provide greater assurance that the program will both improve the safety and efficiency of the state's road and rail network, and maximise value for money.

Network integrity refers to a functionally effective, reliable, maintainable, secure, safe and environmentally compatible public transport network.

Gaps in rail network integrity have had significant consequences for the integration of the LXRP and other concurrent rail projects, including the Metro Tunnel Project (MTP) and the Cranbourne Pakenham Line Upgrade (CPLU).

Now that TfV is responsible for setting the state's strategic transport direction, it needs to address the gaps in rail network planning and requirements, so that future network requirements appropriately inform planning for major rail transport projects. TfV's development of transport planning frameworks has the potential to improve rail network planning.

Findings

Program business case

The LXRP business case was finalised in April 2017, almost two years after the program had commenced.

Weaknesses in the business case undermine its purpose and its value as a basis for the government's decision to commit to the investment. This situation needs to be remedied in future advice to government about investment decisions for crossing removals, if these decisions are to be based on a sound understanding of the costs, benefits and options of the investment.

Site selection and prioritisation

The LXRP business case is not consistent with the stated objective of the LXRP—to remove 50 of the most dangerous and congested level crossings—in that it omits the word 'most'.

This important difference arises in part because DEDJTR did not assess the merits of the 50 level crossing sites identified for removal, which were part of an election commitment in 2014.

As a result, the April 2017 approved business case does not include any analysis or rationale for why the 50 level crossings were selected as higher priority—or demonstrate that they were more dangerous and congested—than other level crossings across Melbourne. The 50 selected level crossing sites include a number of sites that have not been identified as dangerous and congested.

This means that the business case was necessarily limited in its function to fully inform the incoming government about available options and whether the selected sites provided the best approach for it to achieve its desired policy objectives.

Options assessment for grade separation

LXRA and VicRoads completed their options assessments for grade separations—separating the level of rail and road—of 20 crossings before the business case was completed.

LXRA's options assessments for the remaining 30 crossings—developed as part of the business case—differed in approach and in the level of detail to the first 20. The options assessment framework for these 30 crossing sites included a multi-criteria analysis tool to guide the selection of grade separation types. This framework aims to ensure that the chosen options are cost‐effective and defensible, and the process is comprehensive, transparent and consistent.

The options assessment framework does not weight criteria to indicate the relative importance of each criterion. While this allows for flexible decision-making, it also means options assessments can be inconsistent and that the selection of the preferred option lacks transparency.

LXRA refers all preferred grade separation options, including supporting options analysis, to the Minister for Public Transport (the Minister) for approval. The Minister has discretion to approve the option or, where it is likely to be contentious with key stakeholders, refer it to government for endorsement. LXRA's identification of contentious sites is subjective, with no clearly documented rationale, and offers limited transparency around how decisions were made.

In advising government on options for the Frankston line sites, LXRA used different assessment criteria than it used for the other sites. LXRA advised that it did this to accommodate additional information gathered about the sites through more detailed analysis and community consultation. However, the existing option assessment framework is designed to incorporate additional information as site investigation proceeds.

There are important differences between the two sets of criteria. In particular, the criteria used for the Southern package of crossing removals focus on aspects like impact on adjacent properties whereas the original criteria focus on project alignment and outcomes. While the modified criteria produced similar results to the original assessments in this instance, the decision to use different criteria indicates that LXRA is not consistently or transparently applying the options assessment framework.

Program cost

The cumulative cost of the program has increased by more than 38 per cent—based on the initial estimate of $5–6 billion in 2015—to $8.3 billion at July 2017.

DEDJTR did not follow the High Value High Risk (HVHR) guidelines to update the business case to reflect ongoing changes to program cost estimates arising from scope changes, including the addition of two more sites. The identification of future network requirements also did not result in updates to the program costs in the business case. The change in cost, with no assumed benefit increase, would result in a reduced benefit–cost ratio (BCR).

The business case was set up on the assumption of like-for-like replacement of existing infrastructure, but this was not realistic. For example, as the program progresses, works at level crossing removal sites along the Frankston and Cranbourne/Pakenham lines—such as required traction power upgrades and High Capacity Metropolitan Train (HCMT) requirements—are being identified. These requirements were known at the time of the business case development.

The business case provides for future proofing works—for example increasing station platform length, extending the width of the rail corridor, or building wider road bridges—for 16 crossings. LXRA has made provision of $148 million for this work, which is less than 2 per cent of the $7.6 billion identified in the business case.

Procurement and delivery

The 52 level crossing removal sites are divided into packages for delivery—eight packages contain between two and nine level crossings and two packages have one crossing each. There are six crossings that LXRA has not yet allocated to a package.

The LXRP is being delivered using a mix of contract types:

  • Competitive alliance contracts—two or more shortlisted parties develop competing project costs (26 sites).
  • Partial price competitive alliance contracts—shortlisted parties develop pricing for some elements of the total project cost. The sole successful party then prices the remaining elements alongside the owner of the project (24 sites).
  • Design and construct contracts—the client develops a limited design and invites potential suppliers to tender on the basis of completing and constructing the design. This contract typically allocates construction and design risk to the suppliers (two sites).
Partial price competition

While LXRA and VicRoads procured the initial 20 sites through competitive price competition, LXRA will only apply full price competition to a quarter of the remaining 32 sites, procuring the rest through partial price competition. In one instance, for the North Eastern package, LXRA procured the proponent for all four level crossings in the package using partial price competition. The main consideration for this decision was to fast track procurement so that LXRA could meet the government's committed time frame for the LXRP and the duplication of the Hurstbridge line.

While partial price competition can streamline the tendering process—as LXRA does not need to compare two fully priced proposals—it also removes an element of competitive tension. Victorian policy permits the use of partial price competition in some circumstances—for example where the community needs works to commence immediately. However, LXRA's rationale for selecting this method for its North Eastern Package—to fast-track the package to meet a government commitment—is not in itself a sufficient justification.

Contract structure

Under its contract structure for the North Eastern, North Western, Western and Southern packages, parties bid competitively for the initial sites. If LXRA is satisfied with the alliance's performance on those sites, it asks the alliance to develop a formal proposal for the additional sites in the package. This means the additional sites will not be subject to full price competition.

LXRA advises that there are benefits to this structure. For example, according to LXRA, the structure allows for stable engagement with the private sector and locks in key resources prior to significant construction activity across Australia in 2019–22. However there are some inherent risks in the procurement process, primarily due to reduced price and design competition. These risks are compounded in the North Eastern package, where LXRA did not use full price competition to procure a proponent for the initial sites.

Managing procurement risks

To manage its procurement risks, LXRA is using benchmarking and the commercial frameworks of its alliance agreements.

LXRA's benchmarking framework includes a tool for comparing price and productivity efficiencies across the LXRP alliances. The tool allows LXRA to monitor whether it is achieving value for money in delivery of the program. To date, LXRA has only applied the benchmarking tool to the North Eastern package.

When awarding initial works packages, LXRA locks in overhead and profit for all remaining sites in that package. This introduces an element of competition into awarding the remaining sites. However, there is a risk that parties can bid low on overheads and profit with the expectation of increased direct job costs on later sites. To date, overhead and profit as a percentage of direct job costs has been lower for packages using the deferred pricing model than for the initial four alliances that did not use this model. This reinforces the need for LXRA to continue to develop its benchmarking tool to assess proposals effectively and identify potential under- or overpricing. It is important that LXRA continues to improve its framework and embed it into future procurements.

LXRA is also using a risk and reward regime for each alliance, which compares actual and target performance in both cost and non-cost areas. The non-cost elements of LXRA's risk and reward regimes are in line with national guidance and include clear methods for calculating performance. However, LXRA has capped the amount that its contractors pay in the event of a cost overrun. This means there is a risk that if the cost overrun for a package exceeds the cap, the state will effectively bear all additional project risk.

Packaging and sequencing

The LXRP business case considered two options for packaging the remaining 30 level crossing removals:

  • a corridor approach—in which crossing sites along rail corridors are packaged together
  • a discipline-based approach—where similar work types are packaged together (such as stations, power, signalling and rail track).

LXRA's evaluation of these two options emphasised that time, management of disruptions, and risk management were important considerations. LXRA ultimately determined that the corridor approach better matched the LXRP's cost and timeliness procurement objectives.

LXRA did not analyse which of the crossings had the highest priority for removal from a safety or rail efficiency perspective, to inform the sequencing of crossing removals. Of the 50 level crossings, LXRA has prioritised those that had greater preparatory work completed and that had a less complex design solution. This has brought forward some crossing removals, and LXRA expects to exceed the government target of 20 crossing removals by 2018.

The remaining crossings are more complex and more challenging to complete within the allocated time frame and budget. However, it is too early to tell the impact of the current packaging and sequencing regime on the overall program.

LXRA is using 10 packages across the whole program. However, it did not package the following crossings using the corridor approach:

  • Abbotts Road, Dandenong South—part of the Western package instead of the Caulfield to Dandenong (CTD) package
  • Skye Road, Frankston—part of the North Western package instead of Southern package.

The level crossing at Skye Road was initially part of the Southern package. However, following a decision to accelerate the removal of this level crossing, LXRA moved Skye Road into the North Western package.

LXRA estimated that this cost an additional $11 million, due to a reduction in planned program efficiencies. While $11 million is not material in the context of the whole LXRP program, accelerating its removal by moving responsibility to the North Western Alliance helps LXRA to meet the government's committed target of removing 20 level crossing by 2018.

Rail occupation management

Removing level crossings involves the temporary closure of the rail line and its occupation to enable construction works. LXRA and alliance members have processes in place to manage rail occupations appropriately and have managed them well. However, more than half of the planned occupations are yet to occur.

Planning for major occupations commences at least six months before the proposed occupation because contractors usually work 24 hours a day, seven days a week during the occupation.

As at July 2017, only three of the 48 completed major occupations were late. Two of these delays were less than one hour, and the third was one day late. This is a good result and evidence of sound occupation planning and project management during the occupations.

Intended benefits

As completion of the LXRP is not due until 2022, there is insufficient data at present to make an informed judgement on how well the program is achieving intended outcomes.

DEDJTR has developed a benefits management framework in accordance with Department of Treasury and Finance (DTF) requirements. This includes development of an investment logic map, benefit map and benefit management plan. However, current monitoring and reporting of outcomes is not meaningful due to the lack of comprehensive key performance indicators (KPI) and targets.

Key performance indicators and targets

KPI targets specified in the benefit management plan only require performance improvement without defining what constitutes the improvement. Examples of such targets are, '100 per cent of sites with road based public transport will have improved punctuality' or '100 per cent of sites will improve access to jobs, education, and services'. LXRA advised that this is deliberate because factors other than the crossing removal can affect outcomes. Given this, the KPIs themselves are unlikely to be suitable measures.

At March 2017, LXRA had prepared four individual draft benefit reports for the removed level crossings. These reports do not provide meaningful information about the extent of improvement and whether the progress results are in line with expectations.

Value capture

LXRA has developed a comprehensive plan aimed at maximising value capture opportunities through the LXRP. For the LXRP, value capture is limited to IDOs—surplus railway land from a grade separation that can be sold to the private sector to develop. IDOs can include broader social benefits, such as social housing, community services and open spaces.

LXRA developed an IDO strategy in December 2015, providing guidance on identifying IDOs and their potential benefits and how these benefits can be realised.

LXRA's ongoing monitoring of identified IDO projects is timely and comprehensive. Monthly IDO dashboard reports provide a comprehensive summary of the status of each level crossing site against project milestones.

Network integrity

One of PTV's key functions, since September 2016, is to ensure the integrity of the public transport network, to make sure it is functionally effective, reliable, maintainable, secure, safe and environmentally compatible. As part of this role, PTV is responsible for establishing:

  • network technical requirements—high-level technical requirements for a transport corridor or geographic area
  • network technical standards—derived from network technical requirements to inform and direct the development of system requirements and specifications, and accredited rail operators (ARO) engineering standards.

PTV does not have adequate resources to be effective in its network integrity role. This has had undesirable cost and scope consequences for the integration of the LXRP and other concurrent rail projects into the rail network. PTV, with support from TfV—which is now responsible for strategic future transport planning—will need to ensure that actions to address this shortcoming are effective.

Application of standards to the LXRP

LXRA sets project system requirements that contractors delivering the LXRP must comply with. These requirements translate PTV and Metro Trains Melbourne (MTM) requirements, as well as other standards and specifications. However, during the translation process, there is scope for variations which can, in turn, have cost implications for projects. For example, approval to not comply or partially comply may reduce project costs and time lines. MTM manages the approval processes for changes to MTM engineering standards and their interpretation using a standard waiver or design practice note.

However, as MTM has limited insight into network-level requirements and planned developments, there is a risk that these decisions can impact other or future transport projects. Furthermore, as MTM is also a member of all of the LXRP alliances, a perceived conflict of interest exists—MTM is both part of the project development team applying for a standard waiver and part of the organisation making decisions on variation requests.

Controls for network integrity risks

PTV has established a number of arrangements to improve oversight of and accountability over its network integrity function. These include a network integrity governance framework and a formalised standards governance process in the new agreement with MTM, beginning 30 November 2017.

PTV's additional controls to improve its network integrity function are happening in parallel with the delivery of a number of important transport projects. It is important for PTV and TfV to closely monitor the effectiveness of these new arrangements, particularly for programs such as the LXRP which is well underway.

Recommendations

We recommend that the Department of Economic Development, Jobs, Transport and Resources:

1. follow High Value High Risk guidelines in developing a business case as the basis for government's investment decisions, including timing of approval, presenting a range of project options and updating the business case with any significant changes (see Section 2.2)

2. in conjunction with the Level Crossing Removal Authority, develop a transparent selection and prioritisation process for targeted removal of level crossings beyond current commitments made by government (see Sections 2.2 and 2.5)

3. develop comprehensive key performance indicators and targets to meaningfully measure achievement of intended benefits (see Section 4.2)

4. in conjunction with the Level Crossing Removal Authority, progressively monitor the progress of achievement of Level Crossing Removal Program outcomes to facilitate timely insight into how the program is progressing towards benefits realisation (see Section 4.2)

5. in conjunction with Public Transport Victoria, develop contemporary network rail standards, so that agencies delivering rail projects have an understanding of network requirements and what is required to assure projects meet engineering, network integration and safety requirements (see Section 5.2)

6. in conjunction with Public Transport Victoria, monitor the effectiveness of Public Transport Victoria's controls to improve its network integrity function (see Section 5.2).

We recommend that the Level Crossing Removal Authority:

7. apply options assessments transparently and consistently (see Section 2.3)

8. commission an independent evaluation and report on whether the deferred pricing contract structure is cost-effective and has delivered its intended benefits (see Section 3.3)

9. embed its benchmarking tool into the procurement process before using it to award additional works sites (see Section 3.4)

10. in conjunction with Transport for Victoria, evaluate its packaging approach and incorporate lessons learned into future level crossing removals (see Section 3.5).

Responses to recommendations

We have consulted with DEDJTR, PTV, VicRoads and VicTrack, and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments. We also provided a copy of the report to the Department of Premier and Cabinet (DPC).

The following is a summary of those responses. The full responses are included in Appendix A.

DEDJTR accepted the recommendations, noting its intention to improve the delivery of the balance of the LXRP, and provided an action plan to address them.

PTV accepted the recommendations and provided an action plan on how it will implement them.

Although there were no recommendations directed toward VicRoads and VicTrack, both agencies provided a response. VicTrack advised it will continue its collaboration with LXRA and other agencies as required.

Back to Top