Managing the Performance of Rail Franchisees

Tabled: 7 December 2016

Audit overview

Public transport is an essential service for many Victorians, and an effective transport system is critical to the state's economic development. Public Transport Victoria (PTV) is responsible for managing the contracts with the operators that run the metropolitan passenger train and tram networks, known as franchise agreements. Agreements must be well designed, and effective contract management policies, processes and systems must be in place to ensure these agreements deliver value for money to the state of Victoria.

In this audit, we looked at how PTV manages its franchise agreements and its assets, to improve the performance of the train and tram networks. We also assessed how PTV and the Department of Economic Development, Jobs, Transport & Resources have prepared for future franchise agreements.

Conclusion

Train and tram services have become more reliable and punctual under the current franchise agreements, and customer satisfaction has improved. However, weaknesses in the design and implementation of the performance regimes prevent PTV from maximising value from the current agreements.

It is positive to see that PTV has gained a good understanding of the strengths and weaknesses of the current agreements and is well prepared to negotiate better outcomes in future agreements. The challenge for PTV is to successfully negotiate and properly implement the new agreements and improve its management of contracts.

Our key area of concern has been PTV's lack of focus on managing its train and tram system assets, which it has only recently begun to address. As a result, PTV has limited assurance that these critical assets are being optimally maintained and used by operators to maximise the efficiency and effectiveness of train and tram services.

Findings

Managing contracts and performance

Measuring performance

PTV measures the performance of its train and tram operators, or franchisees, in several ways. It measures and publicly reports on the reliability and punctuality of train and tram services, and surveys customer satisfaction with these services.

One of the challenges for PTV has been that the current arrangements do not enable it to periodically review and reset reliability and punctuality performance thresholds. These thresholds set minimum standards for performance and outline consequences if performance falls below the threshold.

The franchise agreements also include operational and customer experience performance regimes that are linked to financial incentives and penalties.

Until recently, PTV has relied heavily on manual systems and on data reported by train operators to assess the performance of trains. In 2015, it introduced an automatic train position monitoring system to improve the way it keeps track of franchisees' performance.

PTV's automated system to monitor tram performance, introduced in 1987, uses old technology with significant limitations that reduce its effectiveness. PTV's own checking of the reliability of both systems is limited. It largely relies on the franchisees to assure the ongoing integrity of the monitoring systems and the data they produce. This means PTV has only limited assurance about the reliability of the underlying performance data that it reports publicly and uses to determine bonus and penalty payments for franchisees.

The customer experience performance regime under the current franchise agreements was poorly designed and costly to administer, making it ineffective. PTV decided that it was not delivering value for money, and largely abandoned it in 2012. Since then, PTV has relied on limited subjective data to assess the way franchisees fulfil their customer experience obligations and to determine the payment of associated bonuses and penalties.

Reporting performance

PTV's public reports identify reliability and punctuality performance thresholds for franchisees. However, they do not clearly explain that these thresholds actually represent underperformance levels instead of performance targets. PTV needs to ensure that its public reporting is easy to understand and clearly reflects franchisee performance.

Managing performance

While there have been improvement in train and tram reliability and punctuality, we found gaps in how PTV manages its franchise agreements:

  • The agreements are not reviewed periodically, to track benefits, evaluate how objectives are being achieved and identify risks, issues and opportunities to improve the way the agreements are managed.
  • There are no recorded knowledge management processes and inadequate recording of important information about franchisees' performance.

The current agreements give franchisees an exclusive right to negotiate a seven‑year extension of their agreement with the state, as long as they meet certain performance benchmarks. In December 2015, PTV assessed that both franchisees met the required benchmarks. However, PTV could not provide evidence to demonstrate the rationale for some of the benchmarks and how they were assessed.

PTV recognises the need for a more comprehensive contract management framework supported by effective systems and adequate resources. It has started to address these issues.

Managing assets

We found significant weaknesses in how PTV oversees the maintenance and renewal of assets leased to its franchisees under the current agreements. In particular, PTV does not have adequate medium- to long-term asset strategies for its train and tram network assets.

PTV also does not know enough about the condition of the assets used across both the tram and train networks. The lack of meaningful information about asset condition reduces PTV's capacity to appropriately plan, budget and prioritise maintenance and renewal work.

It is positive that PTV and the Department of Economic Development, Jobs, Transport & Resources have begun to address these weaknesses.

Extending the current franchise agreements

PTV has prepared for future franchise agreements by thoroughly examining the strengths and weaknesses of its current agreements. To help it negotiate with the current franchisees, PTV has developed an in-depth understanding of operating costs and has identified opportunities to improve the franchisee's financial reporting. This puts PTV in a good position to negotiate important improvements with franchisees.

Recommendations

We recommend that Public Transport Victoria:

  1. finalise and introduce a contract management framework with appropriate processes and systems to better:
  • manage knowledge and improve record keeping
  • define roles and responsibilities and decision-making processes
  • manage contract risks
  • track benefits and evaluate performance throughout the life of the contract to identify problems, opportunities and weaknesses

(see Section 2.2).

  1. improve how it monitors and manages the performance of train and tram franchisees by:
  • periodically auditing franchisees' performance data and related systems (see Section 2.3.1)
  • planning and implementing monitoring systems for the train and tram networks that provide comprehensive real-time journey information (see Section 2.3.1)
  • preparing performance benchmarks for future franchise agreements that drive improvement and are periodically reviewed and reset where necessary (see Section 2.3.2)
  • clearly explaining performance thresholds and how they relate to incentives and penalties in public reporting on train and tram performance (see Section 2.3.2)
  • developing an effective way to monitor customer experience performance, using lessons from the current agreements (see Section 2.4)
  1. before the next franchise agreement starts, work collaboratively with Victorian Rail Track (VicTrack) and the Department of Economic Development, Jobs, Transport & Resources to prepare and introduce a comprehensive framework for managing assets that is consistent with the Department of Treasury & Finance's Asset Management Accountability Framework and includes:
  • clearly defined roles and responsibilities for all agencies and franchisees, including responsibility for operational control and management system assets
  • systems and practices to capture and analyse comprehensive data about the condition of train and tram assets, including a baseline condition survey when the new agreements start and periodic reviews from then on
  • clearly defined and ranked standards for asset management
  • strategies for managing each groups of assets

(see Section 3.2)

  1. evaluate the performance-based right to a contract extension under the current franchise agreement, to understand the benefits and weaknesses of this approach for future agreements (see Section 4.2.1)
  2. improve its systems and capability to collect and analyse comprehensive information on franchisee costs throughout the life of franchise agreements (see Section 4.3.1).

In June 2016, the government announced a new agency—Transport for Victoria (TFV)—that will plan, coordinate and operate Victoria's transport system and associated agencies. We acknowledge that responsibilities for coordinating and implementing the above recommendations may be affected by this change.

Responses to recommendations

We have consulted with the Department of Premier & Cabinet, the Department of Economic Development, Jobs & Transport, Public Transport Victoria, and Victorian Rail Track and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions and comments.

The following is a summary of those responses. The full responses are included in Appendix A.

Public Transport Victoria accepted all of the recommendations in the report and provided a detailed action plan on how it plans to address these recommendations. The Department of Economic Development, Jobs & Transport responded with a commitment to support Public Transport Victoria in implementing the recommendations.

Back to Top