fbpx Assessing Benefits from the Regional Rail Link Project | Victorian Auditor-General's Office

Assessing Benefits from the Regional Rail Link Project

Tabled: 10 May 2018

Overview

The Regional Rail Link (RRL) project was, at the time of its approval and delivery, one of the largest and most expensive rail projects ever built in Victoria, with an estimated final cost of $3.65 billion. Since opening, the project has untangled regional and metropolitan train lines by providing a dedicated high-speed corridor for V/Line trains to access the inner areas of Melbourne.

In this audit, we assessed whether the RRL project is realising expected benefits. To do this, we examined whether:

  • expected benefits and achievement measures from the project were clearly defined
  • the project has achieved, or is on track to achieve, expected outcomes and benefits.

We made thirteen recommendations.

Back to top

Transmittal letter

Ordered to be published

VICTORIAN GOVERNMENT PRINTER May 2018

PP No 393, Session 2014–18

The Hon. Bruce Atkinson MLC
President
Legislative Council
Parliament House
Melbourne
 
The Hon Colin Brooks MP
Speaker
Legislative Assembly
Parliament House
Melbourne
 

Dear Presiding Officers

Under the provisions of section 16AB of the Audit Act 1994, I transmit my report Assessing Benefits from the Regional Rail Link Project.

Yours faithfully

Signature of the Auditor-General.png

Andrew Greaves 
Auditor-General

10 May 2018

Back to top

Acronyms

DEDJTR Department of Economic Development, Jobs, Transport and Resources
DOT former Department of Transport
DPC Department of Premier and Cabinet
DTF Department of Treasury and Finance
HVHR High Value High Risk
IA Infrastructure Australia
KPI Key performance indicator
MTM Metro Trains Melbourne
PTV Public Transport Victoria
RRL Regional Rail Link
RRLA former Regional Rail Link Authority
TfV Transport for Victoria
VAGO Victorian Auditor-General's Office

Back to top

Audit overview

The Regional Rail Link (RRL) project was, at the time of its approval and delivery, one of the largest and most expensive rail projects ever built in Victoria, with an estimated final cost of $3.65 billion.

Since its official opening in mid-2015, the RRL has transformed public transport journeys in key growth corridors to the west of Melbourne.

The RRL has untangled regional and metropolitan train lines by providing a dedicated high-speed corridor for V/Line trains to access the inner-urban areas of Melbourne. The new capacity and reduction in conflicting train movements has also provided the metropolitan system with room to grow.

The scope, scale and complexity of the RRL project made it an exceptional challenge for the public sector. The challenges included pressure to quickly deliver the project, due to a broader Commonwealth aim to use the project as a key stimulatory initiative to offset any impact of the global financial crisis on the Australian economy.

Taxpayers and parliaments expect transparency about whether such major infrastructure projects are worthwhile and have had a beneficial effect.

In this audit, we assessed whether the RRL project is realising its expected benefits. To achieve this, we examined whether expected benefits and measures of achievement were clearly defined for the project and also whether the RRL has achieved, or is on track to achieve, its expected outcomes and benefits.

The agencies involved in this audit and their roles are:

  • the Department of Economic Development, Jobs, Transport and Resources (DEDJTR)—successor to the former Department of Transport (DOT) and the former Regional Rail Link Authority (RRLA), which led the planning and delivery of the RRL project
  • the Department of Premier and Cabinet (DPC)—which provided a whole‑of‑Victorian‑government policy and leadership function for the project and liaised with the Australian Government
  • the Department of Treasury and Finance (DTF)—which provided financial oversight and project scrutiny through its gateway review process and High Value High Risk (HVHR) function
  • Public Transport Victoria (PTV)—which was the key coordinator and planner for public transport at the time of the RRL
  • VicTrack—the owner of Victoria's public rail assets and operator of the railway telecommunications and signalling equipment
  • V/Line—the operator of the rail services on the RRL.

Conclusion

The RRL project is now fully operational and has delivered the expected high‑level outputs articulated by the Victorian and Commonwealth governments when they funded the project in 2009.

The outcomes achieved are less clear and poor benefit management practices by DOT, made it very challenging, if not impossible, to measure today whether the project has delivered all its expected benefits, and thus the level of value for money achieved.

The project has not yet fully realised some specific benefits articulated in the Victorian Government's 2013 booklet Regional Rail Link Benefits for Victorians, particularly the creation of capacity for an additional 10 metropolitan services in the two-hour morning peak period at the RRL's opening, with only five new metropolitan services delivered to date. When these remaining five metropolitan services are running, DEDJTR will have demonstrated that the RRL project has delivered the desired extra capacity.

A meaningful evaluation of the performance of a public sector project requires careful planning and assigning of adequate resources for the evaluation. Many of the benefit management issues we observed are due to DTF and DOT not fully applying key public sector investment processes, such as the Investment Lifecycle Guidelines and the gateway review process.

DOT did not develop a benefit management plan. It had no objective baseline from which to assess whether the project has achieved expected benefits.

The lack of a methodical evaluation culture in the Victorian public sector means there is no systematic and objective collection of lessons learned from past projects, to better inform the planning and execution of future projects. We have observed this problem across various portfolios and noted the comparatively low number of post implementation Gate 6 'Benefits Realisation' reviews done as part of DTF's gateway review process.

Poor benefit management practices during the RRL project meant that DOT did not establish responsibility for monitoring or evaluating benefits at the start of the project. DEDJTR and DTF have not attempted to assess the value or achievement of any of the project's realised or potential benefits since the project began.

Putting aside the benefit management and measurement problems we identified during this audit, it is clear that the RRL project has delivered a number of lasting benefits for the rail network, for commuters and for the wider community.

Findings

Planning for the RRL project

We saw no evidence of a formal business case for the RRL project. The government's funding submission to Infrastructure Australia (IA) defined high‑level project objectives and expected outputs but did not fully expand these into attributable, measurable project benefits.

Key inputs for the decision-making process, such as economic appraisals, patronage forecasts and a procurement strategy, were either developed separately from the broader project submission, or were artefacts from the 2008 Victorian Transport Plan.

Due to the ongoing restructure and reorganisation of many agencies in the public transport sector since the planning of the RRL, we experienced difficulty finding and receiving evidence from agencies and former employees of DOT and RRLA. Transport for Victoria (TfV) needs to improve its knowledge management and record keeping practices to resolve this problem.

The movement of skilled and experienced staff to other public and private sector roles can result in the state losing valuable organisational knowledge. It is important to assign ongoing evaluation responsibilities for projects to entities that will exist well beyond the project's completion.

This would reduce the risk that agencies focused on delivery will neglect or avoid the final stages of benefit management and post-completion evaluations for major investments.

Expected benefits from the RRL project

DOT tried to define some expected benefits that the RRL project would deliver, but it is very difficult to assess whether they have been achieved for several reasons:

  • DOT did not develop a benefit management plan or equivalent for the project. The government designated the RRL project as one of the first projects to be scrutinised by the then new HVHR process after the decision to fund the project had been made. The HVHR process requires a project to have a benefit plan and a full business case before a funding decision is made. However, DOT did not go back and develop these key documents.
  • The baselines for the time frames, location and geographic boundaries of the benefits were not clear. This meant that even if the actual benefits could be reasonably inferred, it is difficult to measure the extent to which the RRL project has achieved them.
  • There were not always direct links or clear attributions of specific outcomes to the intended benefits of the RRL project.
  • It is difficult to attribute wider economic benefits directly to the RRL project. Other economic activity in the area and natural population growth might have occurred even if the RRL project did not proceed.

Another key issue was the lack of rigour around the definitions of project outputs, project outcomes, and project benefits.

This is a common problem across the public sector. The delivery of a project and its tangible components are often identified incorrectly as benefits, while the beneficial effects of the project are not identified or measured.

Despite the deficiencies we identified in the RRL project's benefit management framework, we observed that the project has delivered many practical and tangible benefits:

  • People living in western fringe suburbs who previously only had limited bus services or no public transport options at all now have access to reliable public transport.
  • Regional train bottlenecks have been relieved by giving V/Line trains a high‑speed dedicated route into Southern Cross Station.
  • Train paths have been made available on metropolitan tracks on the Werribee rail corridor, as well as from Sunshine to Southern Cross, that were previously used by V/Line trains, allowing more frequent metropolitan services.
  • The new stations at Wyndham Vale and Tarneit, and the rebuilt stations at Sunshine, West Footscray and Footscray are compliant with the Disability Standards for Accessible Public Transport.
  • The capacity for future track expansion has been allowed for, from West Werribee to DeerPark, by providing an adequate rail corridor reservation, station platform expansion areas, and wide bridge abutments.
  • The future electrification of the whole RRL route has been provided for, which allows the option to convert the RRL to a metropolitan line if population and demand requires a higher frequency and higher capacity service.

A number of intangible benefits are also apparent:

  • Project delivery capability has improved in the public sector from key individuals participating in such a large and complex project.
  • The rail and signalling industry's technical skills in Victoria have developed in response to the complexity of he project.
  • A 'success breeds success' phenomenon has given decision-makers more confidence to pursue complex and risky rail projects, such as the elevated rail sections between Caulfield and Dandenong, and the Melbourne Metro Tunnel.

Post-project completion challenges

The RRL faced a number of operational challenges soon after its completion:

  • New track and stations on the RRL route were ready but not enough rolling stock (that is, trains) was available to deliver the proposed timetable.
  • The VLocity trains experienced wheel wear issues, mainly due to the tight curve of the Dynon flyover track, which resulted in this rolling stock being withdrawn for urgent unplanned maintenance.
  • Higher than expected growth in passenger numbers on the new RRL services showed that the transport forecasting models used to inform the early planning processes did not adequately predict demand.

V/Line has resolved these operational issues for rail and rolling stock by carefully managing lubrication on tightly curved track as well as regularly checking the wheel wear of VLocity trains.

A key issue for V/Line, as operator of the services on this route, will be to manage overcrowding and service frequency in the future.

DEDJTR, through its TfV agency, will also need to assess whether there are adequate bus connections at the RRL stations to cater for the anticipated growth in commuter numbers from surrounding housing estates and new developments.

There are signs that some sections of the RRL are reaching a point of saturation for converging regional rail services. Realistically, the capacity to increase services in response to demand growth will be limited by track and stabling space, signalling system capacity, platform availability at Southern Cross Station and the RRL's design limit of 18 trains per hour.

There are engineering options to augment the RRL and electrify some current RRL services, such as those to and from Melton, which would provide options to increase capacity. However due to the RRL's already high use of available track capacity, these options will require the government to expand the track segment between Deer Park West and Sunshine, and remove the remaining three level crossings on this part of the RRL route.

Recommendations

We recommend that the Department of Premier and Cabinet and the Department of Treasury and Finance:

1. confirm in their advice to government on major capital projects (including High Value High Risk projects) that:

  • proposals should not be recommended for funding unless the business case includes an outline benefits plan as well as a service plan for how desired services will be delivered or enabled by the capital investment
  • proposals and business cases should identify benefits that are robust, measurable, attributable and soundly articulated, irrespective of the source of funding.

We recommend that the Department of Treasury and Finance:

2. ensure that projects subject to the High Value High Risk process:

  • conform with applicable investment life cycle guidance and undergo the required gateway review process steps, unless explicitly exempted by a government directive
  • have sufficient allocated funding to conduct post-completion assessments and a senior responsible officer within the agency to ensure these assessments occur after project delivery

3. track the progress of agency senior responsible officer implementation of gateway review recommendations relating to major capital projects (including High Value High Risk projects) and, if required, reallocate these to a new senior responsible officer or agency for action if significant organisational change has occurred in the period between gateway stages

4. use the Office of Projects Victoria, working with the gateway and High Value High Risk functions, to analyse and publicise lessons learnt from previous major projects to promote project success in the wider public sector, with an initial focus on defining problems and developing options in business cases, developing service plans for how desired services will be delivered or enabled by the capital investment, and preparing benefit management frameworks

5. conduct a diagnostic analysis of reasons for comparatively low adoption of Gate 6 'Benefits Evaluation' gateway reviews across the public sector

6. develop guidance for major infrastructure projects on the optimal timing and number of post-delivery reviews that agencies should conduct throughout a project's useful life to assess any emerging or changing benefits.

We recommend that the Department of Economic Development, Jobs, Transport and Resources:

7. improve how it specifies and measures benefits arising from major capital projects to ensure alignment with better practice guidance on benefit management from the Department of Treasury and Finance

8. identify any major projects underway that have not yet developed a robust benefit management framework and require these projects to rectify this deficiency

9. nominate an enduring point of accountability for project post-completion assessments that can withstand major organisational restructures and realignments and provide continuity of staff, corporate knowledge and other corporate resources

10. develop a knowledge management strategy and record-keeping action plan that will focus on:

  • unifying the disparate public records that have been inherited by Transport for Victoria from other agencies to allow for their efficient usage and retrieval, in conformance with best practice guidance issued by the Public Record Office Victoria
  • ensure continuity of operational and leadership responsibilities for projects and initiatives after major organisational and staffing changes.

We recommend that the Department of Economic Development, Jobs, Transport and Resources, with support from the Department of Treasury and Finance's Gateway Unit:

11. identify and review any outstanding gateway review recommendations directed to projects managed by Transport for Victoria, including those transferred in from other agencies, and, if necessary, allocate these recommendations to new senior responsible officers for action.

We recommend that the Department of Economic Development, Jobs, Transport and Resources, in conjunction with public transport operators:

12. ensure that future asset or network improvements are explicitly linked to a service delivery outcome and that other project dependencies—such as staff and rolling stock—are well aligned to any expected service enhancements or project benefits.

We recommend that the Department of Economic Development, Jobs, Transport and Resources, in conjunction with V/Line:

13. assess growth in patronage and define future service requirements with a particular focus on the capacity challenges that are emerging along the RRL route and at Southern Cross Station.

Responses to recommendations

We have consulted with DEDJTR, DPC, DTF, PTV, VicTrack, and V/Line during the audit.

We considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments.

The following is a summary of those responses. The full responses are included in Appendix A.

DEDJTR accepted all the recommendations directed to it, to help it contribute to improving how the government manages major projects. It also welcomed the acknowledgement that the RRL is providing many long-term benefits, enabling further capacity to be added to the public transport network.

DPC accepted its collective recommendation with DTF and recognised the important role it plays in guiding and supporting improvements to the public transport network to ensure government's investment decisions realise benefits.

DTF accepted five recommendations and partly accepted two recommendations relating to the Gateway Unit. DTF acknowledged that a Gate 6 'Benefits Realisation' review has not been done for the project and committed to work with DEDJTR to schedule a review in 2018.

PTV recognised the importance of the RRL and committed to working with DEDJTR to ensure transport projects deliver benefits, by setting standards for the condition of public transport infrastructure, ensuring new projects are compatible with existing assets and actively managing assets by undertaking audits.

VicTrack noted there were no recommendations for it in the report and made no further comment.

V/Line noted that the RRL has delivered lasting benefits for the regional rail network and community, with increased patronage across the network. It committed to support DEDJTR with its regional rail operator expertise in regard to emerging capacity issues on the RRL and at Southern Cross Station.

Back to top

1 Audit context

Originally budgeted at $4.32 billion, with an expected final cost of $3.65 billion, the RRL project was, at the time of approval and delivery, one of the largest and most expensive rail projects ever to be built in Victoria.

The RRL was designed to improve the capacity and reliability of rail services by:

  • providing new rail track for Geelong-corridor trains across the west of Melbourne
  • untangling western and north-western regional lines from metropolitan lines.

1.1 Victoria's metropolitan and regional rail systems

Since 1999, private operators, working under franchise agreements with the state, have run and maintained Melbourne's metropolitan rail system. The government appointed the current operator, Metro Trains Melbourne (MTM), for an initial eight‑year term in 2009. In September 2017, the government announced that it had awarded MTM a new seven-year contract, which commenced on 30 November 2017.

V/Line, a state-owned company, operates Victoria's regional rail network. V/Line is Australia's largest regional public transport operator.

Victoria's metropolitan and regional train network is shown in Figure 1A.

Figure 1A
Victoria’s train network

Victoria’s train network

Source: PTV, April 2018.

Patronage growth on the rail networks

Both the metropolitan and regional networks have experienced significant growth in patronage and scheduled services over the last decade. Between 2006–07 and 2015–16, patronage on the regional train network grew from 8.9 million to 16.3 million, an increase of 83 per cent, as shown in Figure 1B. Figure 1C shows that the annual number of scheduled services grew by 28 per cent, from 68 454 to 87 725.

Figure 1B
Regional annual patronage

Figure 1C
Regional annual number of services

Regional annual patronage

Source: VAGO from PTV and V/Line data.

Regional annual number of services

Source: VAGO from PTV and V/Line data.

As shown in Figure 1D, over the same period, patronage on the metropolitan network grew from 167.4 million to 223.7 million, an increase of 34 per cent. Figure 1E shows that the annual number of scheduled services grew 22 per cent, from 628 249 to 763 383.

Figure 1D
Metropolitan annual patronage

Figure 1E
Metropolitan annual number of services

Metropolitan annual patronage

Source: VAGO from PTV data.

Metropolitan annual number of services

Source: VAGO from PTV data.

1.2 The Regional Rail Link project

Aim of the RRL

Victoria's regional and metropolitan services terminate in Melbourne's central business district. Prior to the opening of the RRL, all regional lines used the metropolitan network for part of their journey.

Congestion on the shared parts of the network is one of the key factors affecting V/Line's performance, with rapidly rising patronage requiring more services likely to exacerbate this issue in the future.

The main aim of the RRL project was to fully separate V/Line services on the Geelong and Ballarat corridors from metropolitan lines all the way into the inner city. The project also separated V/Line services on the Bendigo corridor from Sunshine to Southern Cross Station. However, V/Line services continue to share metropolitan tracks from Sunbury to Sunshine.

When the project was funded in 2009, advice to government noted that the project was the first new major rail line to be built in Victoria in 80 years. It consisted of 90 kilometres of new track, two new stations, three rebuilt stations and the removal of two existing level crossings, as well as major bridge works and other infrastructure works along the route—see Figure 1F.

Figure 1F
The RRL route

1F.gif

Source: DEDJTR.

By separating regional services from the metropolitan network on three of V/Line's busiest corridors, the RRL was expected to relieve rail congestion, increase rail capacity and improve punctuality on both the regional and metropolitan networks.

History of the RRL

The RRL has a long planning history, with the first potential route proposed in 1993. A report on the proposed Werribee public transport corridor examined the feasibility of constructing a rail line between Deer Park and Werribee, and reinforced the need for staged development of the corridor in line with urban development and resulting increases in patronage.

In May 2006, the state government released Meeting Our Transport Challenges, a long-term transport plan which promised more than $6 billion to public transport over a 25-year period. The plan identified the construction of a third railway track between Sunshine and West Footscray as a key project for improving capacity on the metropolitan rail network.

The concept that evolved into the RRL began as a recommendation in the 2008 report Investing In Transport (also known as the Eddington report). The state government commissioned this report to investigate transport solutions to connect Melbourne's eastern and western suburbs.

The Eddington report proposed a new rail link from Werribee to Sunshine to end conflicting demands for track space between regional and metropolitan services on the Geelong (via Werribee) corridor. The report proposed that building the link would improve the reliability of services on the Geelong, Ballarat and Bendigo regional corridors, increase capacity on the Werribee metropolitan line, and provide access to rail transportation for residents of growth areas in the western suburbs.

In December 2008, in response to the Eddington report, the government released the Victorian Transport Plan. This was a 30-year integrated transport plan that replaced Meeting Our Transport Challenges and included responses to the individual recommendations of the Eddington report. In its response to the recommended rail connection from Werribee to Sunshine, the government committed to building the RRL and declared its intention to seek a funding contribution from the Commonwealth, through the newly established Building Australia Fund.

The Building Australia Fund came into effect on 1 June 2009 through the Nation-building Funds Act 2008. The Commonwealth initially allocated $12.6 billion to the fund for transport, communications, energy and water infrastructure. The fund was part of a broader strategy to accelerate investment in infrastructure to offset the potential negative effects of the global financial crisis on the Australian economy.

In early 2009, Victoria's DOT prepared a submission for IA, seeking Commonwealth funding for two public transport projects, one being the RRL. The submission sought $40 million in development funding for the RRL project, to determine scoping options and to develop a robust cost estimate.

In response to the submission, the Commonwealth's May 2009 Budget committed to a contribution to fund the entire RRL project, up to a cap of $3.225 billion.

In June 2009, the state confirmed that it would contribute $1.1 billion and would proceed with the RRL project at an initial total estimated cost of $4.32 billion. This total amount was publicly announced in October 2009 in a Victorian Budget information paper.

Delivery and governance of the RRL project

In June 2009, DOT established a special-purpose division to manage the delivery of the RRL. In August 2010, this division became RRLA, an administrative office in the same department.

The RRLA Board was established in August 2010, accountable for the successful delivery of the project. The chairman of the board was the head of RRLA who reported to the Secretary of DOT.

Project cost outcomes

The RRL opened in June 2015 and, once all final costs are settled, it is expected to have cost $3.65 billion, which is $667 million less than initially budgeted.

The total final cost has not yet been confirmed because land acquisition costs are not expected to be finalised until December 2018.

Figure 1G shows how the initial budget of $4.32 billion was revised throughout the project's life cycle.

Figure 1G
Revision to project budget

Year

Budget

Amount ($m)

Detail of revision

2009

Initial budget

4 317

 

2011

Revised budget

5 568

Increase in budget following completion of further planning work.

2013

Revised budget

4 807

Reduction due to scope certainty, construction efficiencies and innovation, and removal of contingency provisions.

2014

Revised budget

4 102

Further cost reduction based on delivery progress, while also adding $202 million of level crossing works at Main Road, St Albans, to the scope of the project.

2015

Revised budget

3 650

Further revision based on a reduction in risk contingency provisioning of $200 million, reduction in expenditure forecasts of $50 million, and the removal of $202 million of level crossing works at Main Road, St Albans from the scope of the project.

Source: VAGO.

Project time savings

Construction of the RRL project was completed in October 2014, eight months earlier than RRLA's target of June 2015.

RRLA attributed the cost and time savings to:

  • the use of a competitive alliance procurement model for some of the construction works
  • minimal industrial issues and contractual claims reducing the use of contingency funds
  • delivery of works ahead of schedule.

Project procurement approach

In November 2009, DOT commissioned a strategic procurement plan for the RRL, to identify preferred packaging and procurement models.

The plan concluded that, based on the conditions and risk profiles of the various work packages, a range of procurement models should be used. DOT, and later RRLA, procured and delivered the RRL project through six work packages using several different models.

One work package was procured and delivered through a franchisee works model. In this model, the state signs a project agreement with a franchise operator to deliver infrastructure works on behalf of the state. This package involved works at Southern Cross Station and its environs, which were undertaken by MTM.

Two work packages were procured and delivered through a design and construct contract, where the state undertakes limited design works, and then invites potential suppliers to complete and construct the design. This contract typically allocates construction and design risk to suppliers. The two RRL contracts in this category were for 'greenfield' environments, where there were no existing rail operations or other infrastructure. These were for the Deer Park to West Werribee corridor's new stations and railway works, as well as the West Werribee Junction where the RRL joined the existing Geelong line.

Three other work packages (City to Maribyrnong River, Footscray to Deer Park, and Rail Systems) were procured and delivered by using competitive alliances.

A project alliance contract is where the state collaborates with one or more private sector parties to share risks and responsibilities during the construction phase of a project. Alliancing is typically used to deliver larger, more complex and higher-risk infrastructure projects. The state's functions and capability complement the private sector's skills and expertise, allowing them to better manage risk, deliver the project on time, within budget and at the required quality.

A competitive alliance is where two or more short-listed private sector proponents, consisting of one or more private sector parties, provide the state with a concept proposal and an indicative price. The competitive selection process allows proponents to differentiate themselves by demonstrating their capabilities, capacity and commercial attractiveness. From the state's perspective, a competitive selection process can enhance the project's value for money as well as achieving other benefits of a project alliance, such as reducing disputes, encouraging collaboration on the design and actively sharing risks.

RRLA's selection of the alliance model was based on innovation, risk management, an ability to work with the accredited rail operator, time, flexibility, market appetite and stakeholder management.

RRLA used a competitive alliance model for three 'brownfield' work packages in the RRL corridor, the first time this model has been used in Victoria. Brownfield works involve sites with existing infrastructure where the land continues to be used during construction. These types of works tend to be more risky and complicated because of the potential conflicts that can arise from using existing infrastructure while works are underway.

1.3 Why this audit is important

In previous audits of major capital projects, we have identified weaknesses in agencies' identification, measurement and monitoring of project benefits, both before and after completion.

An increasing number of Victorians rely on train services to access work, education, entertainment, medical services and other daily activities. Given the high rate of growth in rail patronage, it is important to assess whether the RRL project has achieved its intended outcomes of creating enough capacity for metropolitan and regional services to reliably meet demand.

1.4 What this audit examined and how

In this audit, we assessed whether the RRL is realising the expected benefits.

We examined whether:

  • the business case or equivalent documentation for the RRL project clearly defined the expected benefits and measures of achievement
  • the RRL project has achieved, or is on track to achieve, its expected outcomes and benefits.

The audited agencies were: DEDJTR (successor agency for DOT and RRLA), DPC, DTF, PTV, VicTrack, and V/Line.

We conducted our audit in accordance with section 15 of the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. The cost of this audit was $525 000.

1.5 Report structure

The structure of this report is as follows:

  • Part 2 examines whether agencies identified and documented benefits when the RRL project was planned
  • Part 3 examines the extent and nature of the outcomes and benefits that the RRL project has realised.

Appendix B describes a typical better practice model for assessing benefits and the benefit management process, derived from local and international practices.

Back to top

2 Expected benefits of the RRL project

In the Victorian public sector, a business case is used to describe the anticipated benefits that a major capital project is expected to deliver and how the agency responsible for the project will measure them.

A sound business case or equivalent documentation should clearly identify:

  • the outcomes and benefits the project is expected to deliver
  • how or why the benefits are important to the government and the community
  • the performance indicators that will be used to measure whether the project has delivered the expected benefits
  • any key dependencies critical to the delivery of benefits.

The responsible agency also needs to confirm arrangements and resourcing for monitoring and evaluating the achievement of anticipated project benefits after the project is completed.

Where the scope changes during a project's life cycle, agencies need to document the impact on project outcomes and benefits, and ensure decision-makers understand these implications.

2.1 Conclusion

DOT did not develop a formal, comprehensive business case for the RRL project.

DOT and DTF did not fully apply key processes in the public sector investment framework, such as the Investment Lifecycle Guidelines and the gateway review process, which both assist agencies to monitor and evaluate project benefits.

DOT did not develop a benefit management plan so DEDJTR, its successor agency, is unable to show an objective baseline from which to assess whether the project has achieved any benefits.

Some expected project benefits were included in the submission to IA and in documents prepared after the project had received funding. The IA submission adequately defined the project objectives and expected outputs but did not fully develop these into attributable, measurable benefits.

2.2 Development of a business case

Funding submission to Infrastructure Australia

A funding submission to IA, prepared by DOT in 2009, was the key input in Victoria's request for Commonwealth funding for the RRL project.

The submission sought development funding for the RRL and for stage one of the Melbourne Metro Rail Tunnel. These projects were part of a seven-stage Rail Capacity Upgrade program that aimed to double passenger capacity on the Victorian rail network over a decade. The submission provided a detailed concept brief to support its bid for a suite of future rail projects.

The IA submission focused on:

  • the existing problems that the Rail Capacity Upgrade program was intended to address
  • the preferred solutions for both projects
  • a high-level summary of the expected outcomes and benefits from both projects.

However, the submission lacked the key elements that a well-constructed and researched business case could be reasonably expected to include, such as:

  • a finalised route and solution option
  • detailed corridor-specific modelling of trip demand and how commuters might be split across the different modes of transport
  • a finalised scope of works, with costings underpinned by detailed design and site investigations
  • a benefit management plan for the overall investment, including measures to help assess whether the project has delivered the expected benefits.

In June 2009, DTF identified a need to develop a business case for the RRL project. However, in early 2010, with the support of DTF, DOT decided that it would not go back and produce a business case because the Victorian and Commonwealth governments had already committed project funding.

This meant that DOT did not do some of the benefit management activities that an agency would typically undertake at this stage of a project's development, including:

  • finalising key performance indicators (KPI) to measure the delivery of expected benefits
  • allocating responsibility for benefit management.

Development work after funding was approved

In late 2009 and throughout 2010, after the RRL project was approved, DOT and RRLA carried out further development work to supplement the IA submission. During this period, they developed a number of key documents, including:

  • a strategic procurement plan, finalised by DOT in June 2010, which divided the project into different work packages and recommended preferred procurement methods for each package
  • project-wide scope and technical requirements, finalised by RRLA in December 2010, for the design and construction of the project
  • a 'concept of operations' document, finalised by DOT in June 2011, which specified the functional capability that the RRL would need to meet service requirements, clarified the project's scope and identified future-proofing requirements that should be incorporated into the project.

This work, completed after funding approval, identified that the initial project cost estimate—$4.32 billion—was approximately 30 per cent less than what would be needed. This revision to the forecast cost was based on several factors:

  • The initial cost estimate was a pre-feasibility estimate made before the route had been finalised. This meant that DOT had not finalised some scope options, such as the exact route the track would take between Deer Park and West Werribee.
  • DOT and the RRLA subsequently identified items that needed to be included in the scope, such as signalling works between Southern Cross and Sunshine stations. Due to obsolescence and poor condition, this infrastructure had to be replaced before new RRL signalling could be installed.
  • The project's land acquisition costs had increased, because the route had not been finalised at the time of the initial submission.

In March 2011, the state government decided to proceed with the project at a revised cost of $5.57 billion. Under the terms of the agreement between the two governments, the Commonwealth contribution was capped at $3.225 billion, and Victoria was responsible for meeting any cost increases beyond that.

Wyndham-Vale.PNG

A new station was constructed at Wyndham Vale as part of the RRL project. Photograph courtesy of DEDJTR.

Defining the problem, objectives, outcomes and benefits

Due to the absence of a comprehensive business case for the RRL project, we assessed the IA submission and supporting documentation against a better practice model for identifying and measuring benefits—see Appendix B for details.

We used this model to assess the project's delivery of expected benefits in the absence of baseline measures. We compared guidance from different jurisdictions to develop criteria for a qualitative assessment of project objectives, outcomes and benefits, as well as benefit management processes.

This model helped us to determine whether DOT and RRLA had clearly defined the problem that the project was intended to address, as well as the objectives and expected outcomes and benefits.

Defining the problem

The IA submission provided a clear rationale for the seven-stage Rail Capacity Upgrade program by articulating the cause and effect of the problems facing the metropolitan and regional rail networks, which the RRL project was intended to address.

The submission discussed the increase in patronage, and existing operational and infrastructure constraints that prevented the required increases in capacity, the impact these constraints had on existing services, and the need for government to address this problem.

Defining the objectives, outcomes and benefits

The IA submission contained an overall project objective and 10 supporting objectives.

All of the expected project outcomes in the IA submission clearly stated the capabilities, changes or results they expected the RRL to deliver. It was clear how achieving these outcomes would allow the project to meet its objectives.

Supporting documentation identified both financial and non-financial benefits that the project was expected to deliver. These were consistent with the benefits typically expected from a major public transport project.

Documentation also defined the short-term and long-term impacts that would result from the construction and operation of services on the RRL.

2.3 Monitoring and evaluating benefits

Throughout the RRL project, the agencies responsible for planning and delivery have not monitored or evaluated the project, and are unable to demonstrate that the project has achieved expected benefits.

Investment Lifecycle Guidelines

DTF has published policies and guidance on investment decision-making for public sector agencies since 1996. These include Investment Evaluation Policy and Guidelines (1996) and Business Case Guidelines (2003).

In July 2008, DTF issued the Investment Lifecycle Guidelines that described the need to define and validate the identified business need (the problem) and the benefits that the solution is expected to deliver. These guidelines expect agencies to develop:

  • an investment logic map, which defines the problem the investment will overcome as well as the benefits to be derived
  • an investment concept brief
  • a benefit management plan, incorporating KPIs to measure expected benefits.

Although DTF expects agencies to comply with the guidelines, DOT did not develop the required documents for the RRL project. Similarly, there was no evidence that DOT considered putting in place a post-implementation review process, which is also required by the Investment Lifecycle Guidelines.

Gateway review process

In March 2003, DTF introduced the gateway review process, shown in Figure 2A. The process includes reviews at six critical decision points in a project's life cycle to ensure that investments are well made and that the project meets business needs and government's strategic objectives, and achieves value for money.

Gateway reviews are a component of DTF's investment life cycle framework. When DTF introduced them, it intended agencies to apply them to high- or medium-risk projects across a range of procurements, including construction, property and information and communications technology.

As the gateway review process existed prior to the RRL project, DOT and RRLA should have used all six reviews for the RRL project. While the agencies reviewed the RRL project at five of the key decision points, by early 2017, when we commenced this audit, they had not scheduled the final review—Gate 6 'Benefits Evaluation'.

The first gateway review, completed in 2010, identified that the RRL budget and time line had been set based on the IA submission, and that the project would benefit from having a single clear document that stated the project's key objectives and expected outcomes. However, DOT did not complete this document.

At the Gate 4 review, completed in April 2014, DOT and RRLA advised the review team that they were developing a benefit realisation plan. However, in October 2017, PTV advised that this plan had not been completed.

The Gate 5 review in October 2014 recommended that DOT and RRLA:

  • review and document the history of the actions, documents, scope, definition and approvals that constitute the rationale for the RRL project since funding was obtained and define it as the business case
  • ensure benefits identified in the business case are captured and a benefit realisation plan is completed in accordance with DTF guidelines
  • ensure that in the handover process to PTV, the RRL team documents the risks arising from the commencement of passenger services in April 2015 on the Deer Park to West Werribee corridor.

The audited agencies could not provide documentation to confirm whether these recommendations had been actioned.

High Value High Risk framework

In December 2010, the government established the HVHR process within DTF to enhance the existing investment life cycle framework.

The HVHR process aims to increase certainty about the deliverability of infrastructure projects, including their intended benefits, and whether they can meet planned costs and time lines.

HVHR projects usually cost more than $100 million, and the Treasurer and DTF scrutinise them closely throughout the project life cycle. All HVHR projects must complete the gateway review process.

Figure 2A outlines the five stages of the HVHR framework:

  1. concept and feasibility
  2. planning and validation
  3. tendering
  4. implementation
  5. benefit delivery.

Figure 2A
HVHR project assurance framework and the budget process

2A.PNG

Source: DTF.

Because the RRL project commenced before DTF introduced the HVHR framework, DOT did not have to complete stages one and two. As a result of not completing these stages, DOT did not develop the required full business case, investment logic map and benefit management plan.

These documents would have been very worthwhile for DOT and RRLA, particularly as an objective tool for measuring whether the RRL project was achieving the expected benefits after rail services began.

RRL governance and staffing

The RRL Board was established in August 2010, to oversee the project's scope, cost, budget, program, procurement, stakeholder management, communications and risk management. The board's terms of reference do not mention monitoring and evaluating the project's expected benefits.

The RRL Board had its final meeting in September 2014, and the RRLA officially disbanded as an entity in January 2016. RRLA staff began moving to new roles from 2013. A number of RRLA staff obtained new positions in either the roads office or rail office at Moving Victoria. In May 2015, these two offices were renamed as the Level Crossing Removal Authority and the Melbourne Metro Rail Authority respectively.

Victoria has a history of establishing administrative offices like RRLA and the Level Crossing Removal Authority for specific time-bound projects. At the conclusion of the project, staff often move on to the next major project or to other roles in the private or public sector.

This movement of skilled and experienced staff to other public and private sector roles can result in the state losing valuable organisational knowledge, so it is important that time-bound authorities assign ongoing responsibility for projects to entities that will exist well beyond the project's completion. This would also reduce the risk that the final stages of benefit management and evaluation for major investments are neglected or avoided.

In other recent audits, such as Managing the Level Crossing Removal Program (December 2017), we found that agencies are improving their management of project benefits. However, agencies still need a more complete and effective benefit management process for major capital investments.

Transport for Victoria—residual benefits realisation role

TfV was established in 2017 and has overarching responsibility for the planning, coordination and integration of activities and agencies within the transport portfolio.

TfV has taken on responsibility for many functions previously held by parts of DEDJTR and PTV, and has also inherited many of the residual 'project sponsor' roles previously held within other departments and agencies. This project sponsor role includes responsibility for a project's realisation of expected benefits.

The transfer of these legacy responsibilities has not always been smooth, particularly due to multiple restructures within DEDJTR and other transport agencies.

Some of the areas where we observed internal challenges within the new TfV structure included:

  • records management
  • knowledge transfer and knowledge management
  • technical subject-matter expertise
  • transport modelling and data analysis
  • knowledge gaps created when staff move into new roles.

During this audit TfV advised us that it had strengthened its governance practices to ensure continued responsibility for post-implementation monitoring of benefits and outcomes, in the event of machinery-of-government changes or organisational restructures.

To achieve this, TfV has established a new governance body that reports to the Head of TfV. This body recently endorsed a new approach to project oversight and governance known as the Investment Lifecycle Framework.

The new framework requires reviews to be conducted at three key points in a project life cycle, to ensure that documents like business cases, investment logic maps, and benefit plans are available at the appropriate project stage.

2.4 Analysing scope changes for impact on benefits

The RRLA delivered the RRL through a complex suite of interrelated work packages. For a large and complex project such as the RRL, some degree of scope change is expected and typical.

Defined project scope

Together, RRLA's December 2010 project-wide scope and technical requirements documents constituted the approved scope for the RRL project.

These documents specified the scope of the six work packages, including provision for future infrastructure enhancements that the project would deliver to meet longer-term service requirements.

Costs and process to change project scope

The RRLA implemented a change management procedure to make sure that changes to the project's scope were appropriately initiated, identified, evaluated, approved and documented. The procedure specified the process steps, and the documentation and approvals required for changes initiated by both the project owner and contractors.

The RRLA Board had responsibility for the successful delivery of the project including direct oversight and accountability for the project scope. The board was responsible for approving:

  • scope changes where the value of the change was over $10 million
  • changes that resulted in a cost increase over $10 million
  • any use of contingency funds greater than $10 million.

As an HVHR project the Treasurer was also required to approve any major contract variations.

During construction of the RRL, there were 284 changes to the approved scope, and two level crossing removals on Anderson Road, Sunshine. These changes resulted in approved cost increases totalling $454.5 million, though this includes some scope changes that reduced costs by $4.1 million.

We examined the overall RRL project change register and then reviewed a selection of scope changes across all work packages, over the duration of the project. The register showed that an appropriately authorised person approved these changes. Reports to the RRLA Board included relevant details on the scope changes. Although the sample of reports we reviewed did not discuss the impact of scope changes on project benefits, we determined these changes did not have a material impact on the project's overall outcomes.

Cost increases resulting from the scope changes could be met by the existing budget, resulting in no increase in the initial project budget. The approved cost increases accounted for 12.45 per cent of the final project cost of $3.65 billion.

The average cost of each change was $1.6 million, with the largest being $24.7 million for service relocation costs arising from the removal of one level crossing.

None of the individual scope changes were material relative to the total project cost, and they had no negative impact on the realisation of the project's expected benefits.

Some of the scope changes included early activation of future infrastructure enhancements outlined in the project-wide scope and technical requirements documents. These additions have had a positive impact on the outcomes delivered by the RRL.

One example was a 'turn back' facility constructed on the Geelong side of Wyndham Vale Station which enables train services to turn around and return to Southern Cross Station. This facility improves the frequency of services for the new stations closer to the city, as well as reducing passenger loading on Geelong trains. This facility cost $9.4 million and was funded from an allocation in the existing budget for unplanned activities or risks necessary to deliver project outcomes.

Back to top

3 Outcomes and benefits of the RRL project

The RRL project was expected to provide a number of transport, economic, community, and environmental benefits including:

  • improving reliability and carrying capacity for regional and metropolitan train services
  • reducing road congestion by providing extra train services so fewer people choose to drive, and by removing more level crossings
  • providing new train services for Melbourne's western growth areas
  • boosting investment in Melbourne's western suburbs by improving links with the central business district.

In this part of the report, we examine the intended benefits that the RRL project achieved as well as some other benefits that we observed during the audit.

3.1 Conclusion

The RRL project has delivered a range of benefits for the rail network, commuters and the wider community by providing new train services for Melbourne's western growth areas. The RRL has also improved carrying capacity for regional and metropolitan train services which use, or connect with, the RRL.

Due to poor benefit management practices by DOT for the RRL project, it is very challenging, and perhaps impossible, to measure the expected benefits of the project. It is therefore unclear what level of value for money has been achieved.

In particular, DOT did not establish responsibility for monitoring or evaluating the expected benefits at the start of the RRL project. Since the opening of the RRL, DEDJTR and DTF have not attempted to assess the value or achievement of any realised or potential benefits delivered by the RRL.

Some expected benefits have not yet been fully realised—particularly the government's 2013 commitment to introduce capacity for an additional 10 metropolitan services in the two-hour morning peak period, with only five new services operating so far.

3.2 Measurement challenges

Key issues in measuring actual benefits

Although the IA submission identified some high-level benefits that the RRL was expected to deliver, it is not possible to form a definitive view of the extent to which the project has achieved these benefits, for several reasons:

  • There were no key indicators or targets set for the expected benefits.
  • The baselines for time frames, locations and geographic boundaries were not clear. This means that even if actual benefits could be reasonably estimated, it is very difficult to compare them to what existed before the RRL.
  • Agencies have not directly linked or attributed specific project outcomes to a publicly stated project benefit.
  • It is difficult to attribute wider economic benefits directly to the RRL. Other economic activity in the area serviced by the RRL, including natural population growth, might have occurred even if the RRL did not proceed.

Public communication about the benefits of the RRL

In 2013, the Victorian Government published a booklet titled Regional Rail Link Benefits for Victorians, in hard copy and electronic formats. The booklet provided a high-level overview of the project, the problems it was trying to address, and how the project would solve them.

Advice from the Chief Executive Officer of the RRLA to the Minister for Public Transport in August 2012 stated that the booklet would highlight a range of benefits, and was supported by commissioned research by two economic consultancy firms.

The booklet divided the benefits into four key categories:

  • transport benefits
  • economic benefits
  • community benefits
  • environmental benefits.

The benefits outlined in the booklet largely reflect the high-level benefits outlined in the IA submission, and, like the submission, the booklet did not set clear measures to assess the achievement of benefits.

Figure 3A summarises the key items contained in the booklet along with our analysis of their achievement and commentary on how they were presented in the booklet.

Figure 3A
Analysis of expected benefits stated in Regional Rail Link Benefits for Victorians

Expected benefit

Measure

Achieved

VAGO commentary

Transport

Expanding the rail network

90 km of new track

Yes

This is an output, not a benefit.

Removing major bottlenecks

Untangling regional and metropolitan trains

Yes

This is an output, not a benefit.

To be a benefit, it would need to describe the consequence of the untangling—for example, time savings, greater train throughput.

Improving reliability

No measure provided

Unable to assess

 

Increasing number of peak trains

23 extra metro trains
10 extra regional trains

Yes—in terms of theoretical capacity

This is an output, not a benefit.

To be a benefit, it would need to quantify the extra passengers carried or time savings.

Economic

Growing the economy

During construction, add $1 billion annually

Yes—based on direct project expenditure

Mainly an activity measure.

To be a benefit, the expenditure would need to be new money that otherwise would not have been spent in Victoria and that can be solely attributed to the RRL project.

Providing a major jobs boost

Employment for approximately
5 600 people

Partly—based on a commissioned economic advisory report

Only a benefit if the jobs would otherwise not have been created in Victoria and can be solely attributed to the RRL project.

Can only be a benefit for the period that the new job exists.

Reducing road congestion

45 000 cars off the road during peak periods, resulting in $300 million savings

No measurement has been conducted to quantify the number of motorists who switched to the RRL

Congestion reduction is a benefit.

However, in this case, it can only be a benefit if it avoids causing greater congestion than already existed due to the absence of a rail option before the RRL was built.

To validate this figure, agencies would need to identify current RRL patrons who were previously living in the catchment area and have stopped driving.

Community

Boosting investment in the west

Labour productivity will grow and boost gross state product by $1.6 billion

Impossible to measure as there is no geographic attribution that links to gross state product

This estimate was derived from a commissioned economic advisory report into wider economic benefits. The research was not scoped to identify a direct benefit.

Improving regional connections

More reliable and more frequent connections between Melbourne and regional centres

No measures of achievement specified—unable to assess

 

Creating modern transport hubs

Two new stations
One rebuilt station
One upgraded station

Yes
Yes
Yes

This is an output, not a benefit.
This is an output, not a benefit.
This is an output, not a benefit.

Removing level crossings

Two level crossings at Anderson Road

Yes

This is an output, not a benefit.

Ensuring no new level crossings

13 rail-road grade separations

Yes

This is an output, not a benefit.

Introducing new services

Connecting Wyndham Vale and Tarneit to services for the first time

Yes

This is an output, not a benefit.

 

Support sustainable growth in Melbourne's west and easier access to jobs, education, recreation and services

No measures of achievement specified—unable to assess

 

Environmental

Reducing greenhouse gas emissions

Extra passenger capacity will save 14 000 tonnes of greenhouse gases

No measurement has been conducted to quantify number of motorists who switched to the RRL

Greenhouse gas reduction is a benefit only if it is linked to the current RRL patrons who were previously driving and have now stopped using their vehicles.

Improving cycling connections

Will improve cycling connections

Allow for further cycling upgrades

Yes—bicycle paths and other infrastructure were considered in the design of the RRL and delivered along some parts of the corridor and at stations

This is an output, not a benefit.

Source: VAGO based on Regional Rail Link Benefits for Victorians.

Our analysis in Figure 3A shows that the majority of the 'benefits' presented in Regional Rail Link Benefits for Victorians were actually project outputs or activity measures.

Though these output and activity measures are worthwhile individually, they miss the next step which would link them to a benefit. A benefit measure should set a pre-project baseline and then specify a measurable, beneficial outcome for users of the RRL or the wider community that the project will deliver.

More focus on comprehensively describing expected benefits, in line with available guidance, would likely have resolved this measurement problem.

Self-evident outcomes and benefits from the RRL

Regardless of the measurement and attribution issues we have identified, there are some benefits from the RRL that are self-evident.

These include:

  • access to reliable public transport for people on Melbourne's western fringe
  • service improvements for regional and metropolitan passengers
  • better access to the public transport system for mobility-challenged people, enabled by facilities that meet the Disability Standards for Accessible Public Transport at the two new stations and three rebuilt stations
  • improved safety and amenity for the wider community as a result of new and revitalised railway stations
  • reduced impact of rail noise along the corridor through the development of noise-reduction standards and the installation of noise mitigation infrastructure, such as noise walls and double glazing.
New stations on the western fringe of Melbourne

The construction of new stations at Wyndham Vale and Tarneit provides access to rail services for residents in Melbourne's western growth corridor who previously only had limited bus services or no public transport options at all.

IA considers access to transport to be a critical social equity issue, particularly in outer suburbs.

These new stations facilitate quicker access to central Melbourne, reducing travel time to the city and enabling easier access to employment and education.

In April 2016—almost one year after the new stations opened—Wyndham City Council surveyed residents using the Tarneit and Wyndham Vale stations to understand their travel methods and preferences before and after the introduction of the RRL. The survey findings reported a positive attitude to the RRL, with 30 per cent of respondents saying they had increased their train travel but still felt that services could be improved.

The new Tarneit station

The new Tarneit station features lifts to comply with the Disability Standards for Accessible Public Transport. Photograph courtesy of DEDJTR.

Service improvements

The RRL project has improved train services in Melbourne's west in several ways:

  • It has freed up train paths—the capacity to run a service between two points on the network over a given time period—on metropolitan tracks on the Werribee rail corridor, as well as on tracks from Sunshine to Southern Cross that were previously used by V/Line trains. This allows for future frequency increases.
  • It has built in capacity for future track expansion from West Werribee to DeerPark by providing adequate rail corridor reservations, station platform expansion areas, and wide bridge abutments to accommodate future additional track configurations.
  • The potential electrification along the whole route has been allowed for, if demand continues to increase, enabling larger trains and more frequent services.

V/Line now operates 41 per cent of its services through the shared network and 59 per cent of its services on non‑metropolitan track. Before the RRL, all V/Line's broad-gauge services into metropolitan Melbourne shared track with suburban electric services.

Disability compliance and enhanced access

The RRL project delivered a range of benefits for passengers with mobility impairments by providing facilities that are compliant with the Disability Standards for Accessible Public Transport at the two new stations—Wyndham Vale and Tarneit—and the three rebuilt stations—Sunshine, West Footscray and Footscray.

These facilities include easy-incline ramps, lifts to platforms, level-access bus stops, tactile platform markings and signs that include braille.

These features allow passengers with mobility impairments to access public transport on an equal basis and to more easily connect with the wider community, workplaces, education opportunities, medical services, and leisure and cultural activities.

As the population ages and average life spans increase, the number of people with mobility impairment will grow.

Further, a mobility challenge is not only an issue for people with permanent disabilities—it may also affect other people on a temporary basis, for example, due to an injury, pregnancy or needing to transport young children.

This further broadens the potential long-term benefit that these facilities can provide for the wider community.

Station safety and amenity

The reconfiguration and refurbishment of Sunshine, West Footscray and Footscray stations has improved their amenity, with better access to facilities, improved connections to other public transport modes, and increased safety through better station access and modern surveillance technology.

The revitalisation of Sunshine Station included a new pedestrian overpass to provide a safe, open and well-lit connection to platforms and across the rail corridor. This was a distinct improvement from the previous subway connection which was poorly maintained, badly lit and known as a crime hotspot.

The upgrade also included reconfiguration of the bus interchange and a link to the retail district.

Noise reductions

The RRL provided intangible social benefits through noise-mitigation works, described in the case study in Figure 3B. An increase in the number of metropolitan and regional rail services using this corridor has increased the value of this benefit.

Figure 3B
Case study: Noise reduction and mitigations along the RRL route

When RRLA was planning the RRL project, there were no policies or guidance about mitigating noise and vibrations caused by passenger trains. Accordingly, RRLA did not cost such provisions during its planning.

Following the release of the government's Passenger Rail Infrastructure Noise Policy in April 2013, RRLA developed a noise management plan for the section of the RRL between Southern Cross Station and Deer Park. The plan identified locations along the corridor where the noise from the RRL would exceed specified thresholds. Most of the affected locations were in inner‑suburban areas between North Melbourne and Footscray, an area with significant existing rail operations.

RRLA engaged with the community through information sessions and the project website to help it select suitable noise-mitigation measures along the route. The preferred option was the installation of noise walls. However, community consultations identified that noise walls did not provide sufficient noise reductions for the upper levels of multistorey buildings.

As a result, RRLA offered to double-glaze windows in multistorey buildings directly exposed to the RRL tracks.

The final noise-mitigation works delivered by the RRL included:

  • noise walls across both existing and new sections of rail track
  • double-glazing of windows in inner-suburban areas
  • lowering Wyndham Vale station nine metres below the surrounding area to decrease noise impacts on the community.

Source: VAGO.

rail-cutting.PNG

A rail cutting designed to reduce train noise on the approach into Wyndham Vale Station. Photograph courtesy of DEDJTR.

Further unexpected benefits

To capture insights and lessons from the project, RRLA published a 16-volume Insights booklet series for RRLA staff, which provided reflections on RRLA's approach to the development, planning, procurement, delivery and commissioning of the RRL project.

The RRLA found that the project has resulted in some unexpected benefits—most for the Victorian public, but some for the transport industry.

During this audit, several senior public sector executives supported this view, noting that the project:

  • helped develop the public sector's capability in project delivery by giving staff experience on a large and complex project
  • built the depth and technical skills of Victoria's rail and signalling industry
  • resulted in a 'success breeds success' phenomenon which has given decision-makers more confidence to pursue complex and risky rail projects, such as the Level Crossing Removal Program and the Melbourne Metro Tunnel.

In its Insights series RRLA identified numerous benefits of its approach, and lessons for the future across 14 domains, including engineering and design, planning and environment, project development and procurement, communications and stakeholder relations, project controls, contract management and construction.

Ultimately, RRLA concluded that the project made a significant contribution to the body of knowledge for future projects.

The RRL project won an industry award for Infrastructure Project of the Year at the National Infrastructure Awards 2014 and it won the Premier's Sustainability Award in 2014.

3.3 Post-project challenges

Challenges for realising outcomes and benefits

While the project has delivered a range of infrastructure and service outcomes and benefits, there were also some post-implementation issues affecting physical infrastructure and the timetabling of services.

Our 2017 audit report V/Line Passenger Services found that on-time performance for V/Line trains was significantly lower in 2015–16 and from July 2016 to March 2017 than the period before the RRL opened.

This was due to significant and unexpected growth in patronage as well as wheel wear problems on the VLocity trains that emerged soon after the RRL track opened.

Rapid passenger growth has required trains to stay at platforms for longer, to enable all passengers to board, which has affected punctuality. Since the opening of the RRL, patronage on the Geelong line in the south-western corridor—V/Line's busiest line—has increased by 80 per cent.

VLocity train wheel wear and availability of rolling stock

VLocity train wheels experienced more wear than expected in the first few months of services on the RRL, which required V/Line to replace the wheels urgently and more frequently than planned. This meant that more VLocity trains were out of service than expected.

Excessive wheel wear mainly occurred on the Dynon-North Melbourne flyover, where the RRL track crosses over the top of existing metropolitan tracks. An independent report found that the increased wheel wear was due to tight curves on sections of the new RRL track, high friction between wheels and rails due to the absence of lubrication, and poor wear resistance due to the metals used to make the wheels and rails.

This operational issue has since been resolved through careful management of lubrication on tightly curved track as well as regular checking of VLocity wheels using advanced laser gauges. We observed these inspection procedures during a visit to the V/Line maintenance facility in West Melbourne.

Other actions taken to resolve this issue include:

  • regular replacement of worn rail on high-traffic curves
  • a permanent speed restriction of 30 kilometres per hour on the Dynon‑North Melbourne flyover
  • increased maintenance funding for V/Line's rolling stock.

The second issue that impeded timely improvements to capacity was a delay in the delivery of additional rolling stock. While the rail infrastructure was completed in 2014, additional rolling stock was progressively delivered through to 2016, which meant that not all capacity improvements could be immediately realised.

Meeting demand for services

Both the metropolitan and regional rail networks experienced significant growth in patronage over the four-year period prior to the finalisation of the IA submission. Annual growth averaged 10 per cent on the metropolitan network and 11.6 per cent on the regional network between 2004–05 and 2007–08. Both networks were expected to see strong continuing growth until 2021. The IA submission estimated average annual patronage growth of 7.9 per cent on the metropolitan network and 15.4 per cent on the regional network until 2021.

By separating V/Line and metropolitan services, the RRL was meant to provide additional train paths on both networks to enable extra services and accommodate sustained patronage growth.

As DOT had not finalised the route alignment and solution option in the IA submission, it did not include the number of train paths the RRL would provide on completion.

The further development work that RRLA undertook after the RRL had been funded informed the number of additional train paths. The RRL was expected to enable an additional 33 services in the peak two-hour travel period across both networks. This comprised capacity for an additional 10 regional services and 23 metropolitan services. Of these, six regional and 10 metropolitan services were meant to be operational when the RRL opened, with the remainder to be delivered after the purchase of more rolling stock.

Figure 3C shows the number of services run in the peak two-hour morning travel period between 7.00 am and 8.59 am:

  • at the time of the IA submission in 2009
  • prior to the opening of the RRL in 2012
  • after the opening of the RRL in 2015
  • after the latest timetable changes in late 2017.

Figure 3C
Analysis of two-hour peak services run pre- and post-RRL opening

Line

Pre-RRL 2009

Pre-RRL 2012

RRL opening 2015

RRL mature 2017

Increase since 2012

Increase since 2009

Metropolitan

Craigieburn

15

16

16

17

1

2

Sunbury(a)

15

17

17

18

1

3

Upfield

6

7

7

7

0

1

Werribee

8

12

13

14

2

6

Laverton/Altona

0

5

5

5

0

5

Williamstown

6

5

6

6

1

0

Total

50

62

64

67

5

17

Regional

Ballarat

3

3

4

4

1

1

Bacchus Marsh

4

5

6

6

1

2

Bendigo

5

5

4

5

0

0

Geelong

8

9

10

10

1

2

Wyndham Vale

0

0

3

3

3

3

Total

20

22

27

28

6

8

Combined metropolitan and regional total

70

84

91

95

11

25

(a) Prior to the completion of electrification in 2012, services to Sunbury were operated by V/Line. For the purpose of this analysis, the 2009 services to Sunbury that were operated by V/Line have been included in the metropolitan services.

Source: DEDJTR.

Figure 3C shows that at the opening of the RRL in 2015, there were an additional seven services in the peak two-hour morning travel period, compared to 2012, comprising two metropolitan services and five regional services.

By 2017, the RRL had added 11 services, made up of an additional five metropolitan services and six regional services. This does not match the 2013 commitment to introduce 10 additional metropolitan services in the two-hour morning peak period.

The 2013 commitment also included an option to further increase the number of services that the RRL could deliver, by proposing 'additional long term capacity with the procurement of additional rolling stock'.

This new capacity could increase the potential number of services in the two‑hour morning peak period by four regional trains and 18 metropolitan trains. However, no services making use of this new potential capacity have been run.

The current 11-service increase in the two-hour morning peak since 2012 means that the RRL has not reached its expected capacity.

Future capacity and expansion challenges

As the RRL moves towards its maximum designed capacity of 18 regional trains per hour, it is reasonable to expect that a number of future challenges may require the state to further invest in RRL infrastructure:

  • Further expansion of urban growth corridors to the north and west of the current RRL tracks could double or triple the potential catchment for the new stations and may place heavy demand on the already-crowded services. This will likely require agencies to:
    • procure higher capacity rolling stock
    • improve bus routes and interchanges
    • increase station waiting areas and other passenger facilities
    • expand bicycle facilities and car parks.
  • The proposed addition of more stations along the RRL route to meet future demand would slow down trains on the Geelong corridor.
  • Growing demand from greater Geelong and Melbourne's urban fringe areas may require infrastructure enhancements to allow longer-distance express trains to bypass stations on the RRL. Express trains consume more rail path capacity than trains stopping all stations and can limit, or even reduce, service capacity unless they are allocated dedicated express tracks or station bypass tracks.
  • A future electrification of services to Melton would require the RRL track segment between Deer Park West and Sunshine to be amplified to four tracks, to allow for the segregation of existing diesel and new electric services. This would also require the removal of three level crossings on that track section and likely require a complex rail-over-rail flyover to be built at Sunshine to manage the various merging lines.
  • A potential future train service to Melbourne Airport through the Sunshine corridor may also place pressure on Sunshine Station as a key interchange where regional and airport services meet.

Two new platforms constructed at Southern Cross Station

The approach to two new platforms constructed at Southern Cross Station as part of the RRL project. Photograph courtesy of DEDJTR.

Ultimately, any increase in regional services will be limited by platform availability at Southern Cross Station and the designed maximum capacity of the RRL tracks between Sunshine and central Melbourne stations. A future electrification of Melton services would free up paths and platform space for longer-distance services on the Ballarat corridor.

Back to top

Appendix A. Audit Act 1994 section 16—submissions and comments

We have consulted with DEDJTR, DPC, DTF, PTV, VicTrack, and V/Line, and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments.

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Responses were received as follows:

RESPONSE provided by the Secretary, DEDJTR

RESPONSE provided by the Secretary, DEDJTR Page 1

 

RESPONSE provided by the Secretary, DEDJTR Page 2

 

RESPONSE provided by the Secretary, DEDJTR Page 3

RESPONSE provided by the Secretary, DPC

RESPONSE provided by the Secretary, DPC Page 1

 

RESPONSE provided by the Secretary, DPC Page 2

RESPONSE provided by the Secretary, DTF

RESPONSE provided by the Secretary, DTF Page 1

 

RESPONSE provided by the Secretary, DTF Page 2

 

RESPONSE provided by the Secretary, DTF Page 3

RESPONSE provided by the Chief Executive Officer, PTV

RESPONSE provided by the Chief Executive Officer, PTV

RESPONSE provided by the Chief Executive, VicTrack

RESPONSE provided by the Chief Executive, VicTrack

RESPONSE provided by the Chief Executive Officer, V/Line

RESPONSE provided by the Chief Executive Officer, V/Line page 1

 

RESPONSE provided by the Chief Executive Officer, V/Line page 2

 

Back to top

Appendix B. Benefit management—a typical better practice model

To determine whether a project is realising its expected benefits the following need to be examined:

  • the project objectives, outcomes, benefits and measures that were identified and refined throughout the project
  • the quality and consistency of the benefit management process undertaken, including whether there was an outcomes and benefits realisation focus throughout the life of the project.

Defining project objectives, outcomes, benefits and measures

The project objectives, outcomes, benefits and measures should be aligned. There should be an explanation and a logical link between how the outcomes and benefits contribute to the achievement of the project objectives.

Figure B1 lists some of the key considerations when defining individual objectives, outcomes, benefits and measures.

Figure B1
Defining project objectives, outcomes, benefits and KPI measures

Objectives

Outcomes

Benefits

KPI measures

Are project objectives clearly specified?

What is an outcome?

  • A project outcome is a specific capability created, change made or result sought from the investment.
  • Project outcomes should be clearly specified and linked to the project objectives. If project outcomes are achieved then project objectives have been met.
  • Project outcomes should be measurable.

What is a benefit?

  • A benefit is a measurable improvement resulting from one of the specified project outcomes.
  • Expected benefits should be clearly specified and linked to specific project outcomes.

     

What is a KPI?

  • A KPI is a measure which demonstrates that an expected benefit has been, or is likely to be, delivered.
  • KPIs should be directly attributable to the investment.

Measuring the quality of benefits

Does each expected benefit display the following four attributes?

  • Is there a beneficiary (for example society, group or individual)?
  • Is there a gain?
  • Is the benefit attributable?
  • Is the benefit discernible (i.e. noticeable)?

Measuring the quality of KPIs

  • Is the KPI evidence-based?
  • Is the KPI relevant? Does the indicator have a logical and consistent relationship to the objectives, outcomes and benefit?
  • Is the KPI appropriate? Does the indicator gives sufficient information to assess the extent to which the project has achieved a pre‑determined target, goal or outcome? The indicator should reference:
    • the trend in performance over time
    • the performance relative to the performance of similar agencies
    • the performance relative to pre‑determined benchmarks.

Does each expected benefit demonstrate value?

  • Is the benefit measurable and evidence-based? It is critical that KPI baselines and target measures are established.
  • Is the benefit meaningful? Is there a direct relationship between the achievement of KPIs and the achievement of the benefits?
  • Is the benefit aligned to broader agency or government strategic objectives?
  • Is the KPI providing fair representation? The information provided must be measurable, represent consistently and without bias what it purports to indicate, and be reliable and auditable.

Source: VAGO.

When examining the benefits, also examine whether:

  • both financial and non-financial benefits are identified and have measurable KPIs
  • both benefits and disadvantages are identified and measured
  • the benefits are kept to a sensible number with priority being given to the benefits with the greatest potential value.

Assessing the quality and consistency of a benefit management process

The following four key phases of a benefit management process should be undertaken throughout the life of the project:

  1. Understand
  2. Plan
  3. Manage and report
  4. Evaluate

For each of these phases examine whether the key activities were undertaken and the expected outputs produced. These are listed in Figure B2.

Figure B2
Key activities and outputs at each stage of the benefit management process

No.

Stage of the investment life cycle

Benefit management process

Activities to be undertaken

Expected/typical outputs

1

Preliminary business case/investment proposal

Understand—identify and define the expected benefits. This includes any possible KPIs and targets that will make it possible to understand the potential value of the project.

  • Specify the project objectives and outcomes.
  • Identify the expected benefits and specify evidence-based KPIs to provide evidence that benefits are being delivered.
  • Specify baseline measures for the KPI.
  • Identify the key interdependencies critical to benefit delivery. This is where the realisation of an expected benefit is dependent on a task or action outside the project.
  • Outline the importance of the benefits to government. This should include a discussion on why the benefits reflect government or organisational policies, objectives or priorities.
  • Identify the risks that may affect benefit realisation and include these in the benefit realisation plan and risk register.
  • Draft benefit map.
  • Draft benefit realisation plan.

2

Full business case

Plan—establish the expected level of benefits and test the achievability and measurability of the benefit. Describe the approach that will be used to manage benefits and how benefits will be monitored and realised.

  • Identify both the financial and non-financial benefits, and complete a cost-benefit analysis for the financial benefits.
  • Specify KPIs for both the financial and non≠-financial benefits. Ensure that the KPIs are fully formed and understood, and that baseline measures and measurement time frames specified have been specified.
  • Identify the risks to benefit realisation, develop risk mitigation strategies and include these in the benefits realisation register or risk register.
  • Clearly allocate responsibility for benefit management.
  • Identify any project disadvantages.
  • Updated benefit map.
  • Updated benefit realisation plan.
  • Benefit realisation register.

3

Ready for service

Manage and report—put in place processes to ensure benefits continue to be realised after the project transitions into business as usual. Measure how successfully the program outcomes are being achieved.

  • Establish a benefits monitoring and reporting process.
  • Create a clear understanding of the roles and responsibilities for benefits realisation beyond project implementation.
  • Measure the realisable benefits.
  • Review and update benefit deliverables by:
    • validating that project outcomes and benefits are still valid and achievable
    • capturing any new emerging benefits and considering their impacts
    • deleting benefits that are no longer relevant or achievable and considering the impacts of their deletion
    • monitoring project disadvantages.
  • Updated benefit realisation plan.
  • Updated benefit realisation register or a benefit realisation report.

4

Post implementation

Evaluate—assess whether the project achieved its planned objectives.

  • Determine whether the investment satisfactorily addressed the problem or need it set out to resolve.
  • Determine to what degree the expected benefits are being realised, and how valuable they are.
  • Identify and measure any unanticipated benefits and disadvantages where applicable.
  • Set realistic performance targets for continuous improvement for the project.
  • Capture and communicate any lessons learnt from the benefit management and project management activities.
  • Updated benefit realisation register.
  • Benefit realisation report.
  • Post-implementation performance reviews.
  • Lessons learnt document.

Source: VAGO.

Back to top