At a glance
The Melbourne Metropolitan Bus Franchise (MMBF) is intended to improve services, reduce costs and progress reform of the bus industry. This Part of the report examines the MMBF's progress in achieving these outcomes.
Public Transport Victoria (PTV) cannot yet reliably demonstrate that MMBF is achieving full value for money for the state due to shortcomings with performance data. Unless PTV promptly addresses these deficiencies, MMBF's full benefits are unlikely to be achieved.
- In 2013–14, the cost of MMBF services was almost $33 million less than the previous year under the old arrangements.
- PTV's failure to resolve longstanding data issues has meant that the performance regime could not be effectively implemented by April 2015 as intended.
- PTV has begun work to reach agreement with the operator on a proposal to address these data issues, but it has yet to be satisfactorily resolved.
- Any further delays to planning or project approval for future bus service procurements risks compromising wider bus industry reform.
- That Public Transport Victoria rectifies barriers to implementing the MMBF performance regime, promptly addresses offer commitment time line slippages, and documents and assesses the rationale for all decisions to waive the withholding of payments.
- That Public Transport Victoria and the Department of Economic Development, Jobs, Transport and Resources advise government on all key risks, options and required actions for reforming the metropolitan bus contracts expiring in 2018.
The state used an open tender procurement method to establish the new Melbourne Metropolitan Bus Franchise (MMBF) because it expected that this would result in a better service to customers at a reduced cost to the state.
It also sought to use the MMBF to test the savings achievable through increased competition and the potential benefits of an enhanced performance-based contract as a precursor to extending these reforms to the rest of Melbourne's metropolitan bus industry.
The state expected the MMBF would achieve value if it:
- reduced costs for the state
- delivered improved services for more customers
- progressed reform of the bus industry.
This Part of the report examines whether the MMBF is on track to achieve improved value for money for the state.
Although the MMBF has been operating for over 18 months, it cannot yet be determined with any certainty whether it is achieving full value for money for the state.
The MMBF has proven less expensive than previous arrangements by reducing the cost to the state in 2013–14 by almost $33 million. However, as it also sought to improve services for more customers and progress reform of the bus industry, this metric alone is not sufficient for demonstrating value for money.
Public Transport Victoria (PTV) failed to establish reliable performance data by April 2015 when the performance regime was due to be fully implemented. This means that the performance regime could not be effectively applied as originally intended to manage the contractor's performance. While full implementation of the regime has since been delayed pending approval of the Greenfields timetable refresh, PTV needs to urgently address these issues as they are compromising its capacity to reliably manage the contractor's performance, including achievement of the state's reform agenda.
PTV's rationale for waiving the withholding of some operator payments for delays in delivering required service improvements is also at odds with the MMBF's reform goals, and is a missed opportunity to enforce the principles of the contract and leverage better value for the state.
Substantive action is still required to establish the preferred procurement method for the remaining metropolitan bus contracts that expire in 2018. Consequently, there is a significant risk that the state will miss the opportunity to significantly progress reform of the bus industry and achieve better value for money.
2.3 Reduced costs for the state
The actual cost of MMBF services is less than under the previous procurement arrangements, and is in accordance with the tender price and advice provided to the state by PTV following tender evaluation.
As shown in Figure 2A, MMBF expenditure for 2013–14 was almost $33 million, or 18 per cent, less than the amount PTV estimated it would have spent if the previous contracts had continued.
2013–14 actual MMBF expenditure compared to tender estimate and previous contracts
Source: Victorian Auditor-General's Office based on PTV financial reports.
The MMBF cost savings have been consistent with PTV's estimates, and are expected to total approximately $380 million over the 10-year life of the agreement. These savings are attributable to efficiencies achieved by merging five contracts into one, as well as the new operator's proposed innovations and willingness to accept a greater share of the risk for poor service delivery.
However, MMBF cost savings alone are not sufficient for assessing its achievement of value for money. As noted in Part 1 of this report, the state determined in 2011 that MMBF will have achieved value for money if it:
- reduced costs for the state
- delivered improved services for more customers
- progressed reform of the bus industry.
2.4 Improved services for more passengers
A key goal of the state's reform agenda in establishing the MMBF incentive-based contract was to provide the state with the means to continuously improve and grow bus services through clearly defined performance standards and monitoring arrangements.
In this context, the MMBF contract established key obligations for the operator to:
- continuously improve bus punctuality, reliability, safety and customer services—including the provision of information to passengers
- increase bus passenger numbers
- identify opportunities to improve existing bus services—including improving connections between buses and other public transport modes.
The MMBF contract contains an incentive and penalty regime to support the achievement of these obligations. Effective monitoring by PTV of the operator's performance, including enforcing penalties when necessary, is therefore critical for assuring these obligations are met, and for realising the contract's potential to deliver better services.
Although the performance mechanisms are soundly based, they are impacted by longstanding data reliability issues, and delays in achieving full implementation. These issues risk compromising achievement of value for money from the contract, and PTV's capacity to effectively manage the contractor.
PTV failed to resolve longstanding data reliability issues by April 2015, when they were originally due to be fully implemented under the contract. It also failed to reach a timely agreement with the operator on the standard for determining incentive payments for improvements in bus patronage—in effect delaying the operation of this regime by 12 months.
PTV advised towards the end of this audit that implementation of the performance regime and related Greenfields timetable has since been delayed, and that it has begun work to reach agreement with the operator on a proposal to address these data issues.
It will be critical for PTV to effectively resolve these issues as they are compromising achievement of the state's reform agenda—including PTV's capacity to reliably assess and manage the contractor's performance.
2.4.1 Performance regime
The MMBF performance regime comprises the Patronage Incentive Regime (PIR), Operational Performance Regime (OPR) and Reliability Regime (RR), and uses financial incentives and penalties attached to service targets.
The regime is intended to encourage the operator to improve performance and increase passenger numbers. The performance regime supports achievement of the state's reform agenda by:
- providing financial incentives for improved performance
- encouraging innovative ways to improve services
- shifting some of the financial risk for poor service to the operator.
However, PTV failed to resolve longstanding data reliability issues by April 2015, which meant that the OPR was not able to be implemented as planned. It also failed to reach a timely agreement with the operator on the standard for determining incentive payments for improvements in bus patronage—in effect delaying the operation of this regime by 12 months.
PTV recently advised that full implementation of the OPR has since been delayed as the state has yet to approve the Greenfields timetable refresh proposed by the contractor.
As a result, a key aspect of the MMBF agreement remains inoperative, which compromises the achievement of the reform objectives—including value for money.
Patronage Incentive Regime
The PIR is intended to provide the operator with a financial incentive to introduce service improvements and other innovations that lead to increased passenger numbers above a benchmark agreed with PTV. Conversely, poor services leading to a decrease in passengers results in a financial penalty.
The MMBF contract and PIR measure patronage by myki touch‑ons. However, due to concerns with the reliability of myki data, and protracted negotiations with the operator over setting the PIR benchmark, it was not operational until February 2015—more than 12 months after it was due to be implemented under the contract. This issue is examined further in Part 4 of this report.
PTV resolved the myki data issue in July 2014. However, agreement with the operator was not reached until January 2015. The failure to reach a timely agreement on the passenger benchmark is unsatisfactory. While it is clear that passenger numbers have been increasing, the absence of an agreed benchmark meant PTV could not be certain that the operator was meeting its expectations during this period. Similarly, the operator did not necessarily know if more work was needed to further increase passenger numbers.
Operational Performance Regime
An MMBF service is considered punctual if it is observed at the specified service monitoring location, and had arrived there no more than 4:59 minutes late and departed no more than 59 seconds early.
PTV set an interim OPR punctuality benchmark of 70 per cent for all services at the start of MMBF operations, based on its estimate of the previous operators' performance. It proposes to set the permanent benchmark at 90 per cent when the new Greenfields timetable commences—at which point the OPR's financial penalties will start to be applied. The OPR penalises the operator by up to $2 million annually if it fails to meet its punctuality benchmark. It also provides for PTV to terminate the contract if punctuality drops below 75 per cent for a year without due cause. However, reliable bus punctuality data is not yet available—impeding the operation of the OPR.
The contract initially required the Greenfields timetable to be implemented by April 2015, but this has been delayed. As noted earlier, PTV is currently working with the contractor to resolve reliability issues with the data.
Reliability is the number of service kilometres scheduled compared to the number of service kilometres actually travelled, expressed as a percentage. The RR came into force at the commencement of the contract.
If reliability falls below 95 per cent for a quarter, or below 99 per cent for three quarters within a 24-month period, PTV can immediately terminate the contract. Reliability below 98 per cent in a quarter, or below 99 per cent for three quarters in 24 months, triggers a noncompliance event, compelling the operator to commit to remedial action.
However, data concerning bus reliability has been problematic.
Absence of reliable performance data
The effective operation of the performance regime depends on the availability of accurate and reliable data.
The MMBF agreement's OPR and RR were designed to use data gathered by PTV's bus tracking system (BTS). While PTV knew there were issues with the reliability of the BTS in June 2011 when it was developing the OPR and RR, it assumed that these issues would be overcome by March 2012, prior to the OPR and RR becoming operational. However, this did not occur and PTV's progress in addressing this issue has been unsatisfactory.
Our 2014 audit Coordinating Public Transport noted that PTV advised that it expected the new BTS to be fully operational by July 2014, however, this has yet to occur.
The BTS data issues stem from the number of services being monitored. The OPR and RR require 100 per cent of MMBF services to be monitored by the BTS. However, PTV now recognises that no technology solution is capable of 100 per cent functionality as there will always be periods, however brief, where hardware breaks down or software fails.
PTV has improved the monitoring of services throughout the MMBF contract from 89 per cent in 2014, to the current level of 93 per cent. While this is encouraging, the unmonitored services affect the bus reliability calculation required under the contract as these services are recorded as missed service kilometres, even if the service ran as scheduled. Punctuality is similarly affected as a bus is considered not punctual if it is not observed at a contract monitoring point, even if this is due to a BTS malfunction.
PTV documents supplied in March 2015 acknowledge that while this improvement in coverage better supports operational and service planning activities, it is not sufficient for assessing the punctuality of services as required by the contract.
PTV has proposed an alternative method to calculate service reliability and punctuality to the contractor, but this had not yet been implemented at the time of audit. PTV recognises that overcoming this issue is critical because it will otherwise not be able to implement the financial incentives and penalties that are due to begin after the Greenfields timetable refresh. PTV could not demonstrate that it was on track to address this issue by April 2015 when the Greenfields timetable refresh was initially due to occur.
Furthermore, we found that PTV's proposed alternative method for calculating reliability and punctuality is currently deficient as it relies heavily on self-reporting by the operator for determining reliability. PTV did not have a plan or procedures in place to audit and verify the accuracy of this information.
Urgent action is required by PTV to address this issue as it risks compromising the integrity of the performance regime and of related incentive payments.
PTV is also aware of a reliability issue with its myki touch-on data. At times myki machines can temporarily stop functioning—preventing bus passengers from touching on. While the system flags when a myki machine stops operating and for how long, PTV has no practical way of assessing the number of passenger boardings that are not recorded. However, PTV believes the completeness and accuracy of touch-on data is not materially affected.
2.4.2 Offer commitments
The MMBF requires the operator to deliver 21 service improvement projects—described as offer commitments—by time lines the operator committed to in its tender response. If the operator fails to deliver a project on time PTV can withhold payments until a satisfactory remedy has been implemented or remedial plans have been agreed to.
PTV has not exercised its contractual powers in relation to the operator's failure to deliver some projects as committed. This risks diminishing the effectiveness of the offer commitments performance incentive mechanism.
Projects not completed by due date
Four offer commitment projects have not been completed by the time frames originally agreed by the operator. Three projects had their due dates extended, as the operator was not able to complete these projects on time. Had PTV not chosen to extend the deadlines, it would have been able to withhold more than $200 000 of contract payments and require the operator to remedy the underperformance.
Additionally, PTV did not take any action for the operator's failure to deliver on its bus refurbishment offer commitment. At the end of the first contract year, PTV could have withheld up to $180 000 for buses that had not been refurbished.
PTV advised that it has only agreed to extend some contract time lines for initiatives offered by the contractor and now formalised in the contract, but which have no impact on core services to customers. PTV also wanted to build a business relationship with the operator.
While PTV has the discretion to withhold payments in response to these circumstances, it has never done so. This approach does not provide appropriate incentives for the operator to comply with its contractual obligations, and risks compromising achievement of the state's reform goals.
Delivery of key improvement projects
Three key offer commitment projects require the operator to reduce fare evasion, develop and implement a new bus timetable known as Greenfields, and increase customer satisfaction. As these projects directly relate to the reform agenda, they are important for assessing whether the state is getting better value for money from the MMBF.
The fare evasion project requires the operator to reduce the fare evasion rate on its services to 5 per cent by 1 August 2015 and then to 2.3 per cent by 30 June 2021. By May 2014, the fare evasion rate had risen to 13.1 per cent from 10.1 per cent in October 2013. In October 2014 the rate was 8 per cent indicating that reaching the target rate will be challenging for the operator.
While it is evident that the operator has taken a range of remedial actions, failure to achieve this result would entitle PTV to formally notify the operator that its noncompliance with the MMBF risks termination of its contract, or withholding of contract payments until a satisfactory remedy has been implemented.
In August 2014, the operator raised concerns that the methodology of the survey that PTV uses to calculate the level of fare evasion was not representative of the areas it operated. Although PTV subsequently reviewed and made some minor adjustments to the survey, it is unclear if these have been accepted by the operator.
A total rebuild of the current bus timetable, known as the Greenfields timetable, is another key offer commitment. By having the operator develop and propose a rebuilt timetable, PTV aims to improve the efficiency of these bus services and increase patronage at no additional cost to the state. Implementation of the Greenfields timetable was scheduled for April 2015. However, this has now been delayed, as the state has not yet given its approval to proceed.
The initial step toward Greenfields implementation was the July 2014 timetable refresh. The operator implemented a suite of service changes, including route changes, route cancellations and timetable changes. The timetable refresh was not as extensive as the full Greenfields implementation will be, but was still a much more significant change than bus patrons are accustomed to. These changes met with passenger resistance and criticism of the consultation processes—following passenger complaints and highly critical media coverage over the cancellation of one bus route, the then Minister for Public Transport intervened to reinstate this service.
The operator subsequently acknowledged that improved communication to passengers about the changes needed to occur, such as providing more information at bus stops, more media releases and earlier availability of printed timetables. The operator submitted its draft business case for the Greenfields timetable on time. However, as this was before the full impact of the July 2014 refresh was known, it did not take into account the concerns raised by customers. PTV advised that the draft was subsequently finalised after considering passenger feedback on the July 2014 service changes and proposed Greenfields timetable.
PTV further advised during the audit that, while it is not possible to achieve 100 per cent acceptance by the public of proposed service changes, it was confident the operator would implement the Greenfields timetable in April 2015 as originally intended by the contract. However, the state has since delayed its implementation. Failure to implement it successfully, and learn from past experiences, will have negative consequences in terms of service improvement.
The operator is required to improve its overall customer satisfaction rating to 80 per cent or more by 31 December 2016. If this result is not achieved, the operator will be required to implement remedial plans to address the factors negatively impacting on the rating.
Although PTV reviews the results of quarterly customer satisfaction surveys and discusses the results with the operator, it does not complement this work with specified interim targets to more rigorously track the operator's progress towards meeting the target.
2.4.3 Service specifications
Service specifications are performance standards set by the MMBF agreement that include:
- general obligations covering the quality of service provision
- ensuring staff are properly trained and experienced, and that drivers are appropriately authorised
- maintaining sufficient numbers of buses to meet operational requirements
- ensuring bus depots and other bus-related infrastructure meets all legislative requirements.
PTV has not defined the qualitative elements of the general obligations. For example, the operator is required to perform its obligations in a 'timely and expeditious way' and in a 'proper, competent, courteous, safe and reliable manner' but these standards are not defined. Nor does PTV undertake any form of monitoring of the operator's compliance with the service specifications, preferring to address issues as they arise.
As such, PTV is not as well placed to identify and respond to any noncompliance or to challenge the operator where PTV believes its standards have not been met.
2.5 Progressive reform of the bus industry
The MMBF was intended to form the basis for further reform of the Victorian bus industry. In this regard, the next round of procurements will be for the remaining 70 per cent of metropolitan services that are currently provided through 27 private contracts that expire in July 2018.
PTV and the Department of Economic Development, Jobs, Transport and Resources' (DEDJTR) slow progress in determining the preferred procurement option and tight project time frame for establishing follow-on contracts risks the state not achieving further bus industry reform and improving value for money.
2.5.1 Procurement risks
Establishing new contracts to replace the 27 that expire in July 2018 will be a complex and time-consuming project. The number and severity of risks facing this project will depend on how much further the state elects to pursue its reform agenda.
Absence of clear end-of-term rights
Throughout the 1990s, successive reports by the then Industry Commission, the Victorian Commission of Audit and VAGO all noted that the practice of establishing contracts through negotiation with incumbent operators, without exposing them to competition, minimises the state's ability to gain best value for money. Moreover, the Victorian Government Purchasing Board's guidelines are based on the presumption that all major procurements will be open for tender to ensure the market is fully tested, and that all qualified providers can compete for government work.
The 27 private contracts do not contain any clear end-of-term rights for the state or an explicit right to offer the services through tender. However, the bus operators claim their financial investment in establishing the services gives them proprietary rights over their routes, and deny the state has the right to offer their routes to tender. The state does not share this view, but has not yet tested its position.
Consequently, pursuing an open tender could lead to the current operators taking legal action against the state, thereby potentially delaying any tender process until a settlement is reached or a legal ruling is obtained.
Negotiating contractual reforms
If the state elects to establish new contracts through negotiation with incumbent operators, it is unlikely to be able to pursue contractual reforms as extensive as those sought during the MMBF's procurement.
When contracts were negotiated in 2008 the incumbent operators were resistant to major reforms, including open book access to their financial data, establishing end‑of‑term rights and asset rights. Our 2009 audit of this process, Melbourne's New Bus Contracts, found that only modest cost reductions were achieved, and that on a like-for-like basis, the costs of the contracts negotiated in 2008 were very close to the costs of those they replaced.
2.5.2 Timeliness of planning
As the state faces risks no matter which procurement option is chosen, early advice to government is critical to ensure that all risks are appropriately assessed during the decision-making process.
Our 2009 audit recommended the former Department of Transport provide early advice to government on the strategic options and constraints for future metropolitan bus contracts. While the department and PTV implemented this recommendation for MMBF, it has not been addressed in relation to the next tranche of bus contracts.
The project to establish the MMBF required multiple approaches to government over 14 months before the decision to proceed with the open tender was reached. A further 22 months elapsed before the MMBF commenced operating.
Achieving further bus industry reform through new contracts will be much more complex, high risk and time intensive, and consequently will be likely to require more than a three-year lead time. PTV and DEDJTR established a joint working group in July 2014 to commence preliminary planning for the next round of bus contact procurements. The working group commissioned initial research on bus structures in other jurisdictions and potential options for reform. An interdepartmental steering committee comprising PTV, DEDJTR, the Department of Treasury and Finance and the Department of Premier and Cabinet was also formed in early 2015 to inform the development of a related procurement strategy for the government's consideration by August 2015.
However, progress on this initiative has been slow, meaning the remaining time for planning and implementing this project is ambitious and not commensurate with the significant scale and complexity of the task.
This situation risks limiting the state's options in 2018—if delays result in insufficient time to pursue an open tender, negotiating with incumbent operators may be left as the only way to proceed. Such an outcome will likely compromise any opportunity for the state to achieve improved value for money from these bus contracts.
That Public Transport Victoria:
- promptly rectifies all barriers to implementing the Melbourne Metropolitan Bus Franchise performance regime, including data reliability issues
- documents and assesses the rationale for all decisions to waive the withholding of payments for non-performance
- closely monitors the delivery of key offer commitments and proactively addresses any slippage from contractual time lines
- systematically audits and verifies the reliability of performance data provided by the operator underpinning incentive payments.
That Public Transport Victoria and the Department of Economic Development, Jobs, Transport and Resources:
- advise government on all key risks, options and required actions for reforming the metropolitan bus contracts expiring in 2018.