Service Victoria—Digital Delivery of Government Services

Tabled: 18 March 2021

Snapshot

Has digital delivery of government services improved customer experiences and reduced costs? 

Why this audit is important

As Victoria's population grows, so do customer interactions with public services. 

Digitising services can help to reduce demand pressure, save money, and improve customer satisfaction. 

In 2015, the government established Service Victoria (SV) to seize this opportunity and deliver cross-government reform. 

The impact of COVID-19 and the need to make services available digitally further highlight the importance of this change.

Who we examined

The Department of Premier and Cabinet (DPC), including its administrative office SV.

What we examined

We examined: 

  • the implementation of SV and DPC’s oversight of it. 
  • the benefits SV is delivering.

What we concluded

While SV has improved customer experience through the services it delivers, it has not reduced transaction costs as intended. 

SV has delivered a repeatable and scalable digital platform and technology solution. SV has shown the benefit to government from this platform, such as through quickly rolling out COVID-19-related transactions including venue check-ins using QR code scans.  

However, SV has not delivered the wide range of transactions envisaged at its outset. With the exception of recent COVID 19 related transactions; the types and low volumes of transactions that SV delivers mean it has not realised its objective of reducing the costs of existing government transactions and improving compliance with regulation.

SV's implementation was hampered by a poor business case and inadequate stakeholder and risk management, exacerbated by the lack of a strong mandate. DPC and SV missed several opportunities to address these issues. 

 


This has meant that SV has fallen well short of its ambition to achieve whole-of-government digital transaction reform in its first five years.

What we recommended

We made five recommendations to DPC. Four recommendations relate to improving its reporting and guidance on transactional services and one relates to introducing a stakeholder engagement strategy.

We also made five recommendations to SV. Four recommendations relate to improving SV’s transaction baselines and benefits reporting and one relates to data collection to enable SV to understand agency feedback on its performance.

Video presentation

Video transcript

Key facts

Service Victoria key facts

What we found and recommend

We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A. 

Transaction reform 

Most people want to be able to complete transactions, such as pay bills, apply for services or update their details, anywhere and at any time. 

The Department of Premier and Cabinet (DPC) estimated that the Victorian Government conducted over 55 million transactions with customers in 2015, with only 1 per cent being fully digital. Most transactions were face-to-face, or via phone or mail. 

Recognising that such transactions are costly to government and often not preferred by customers, DPC and the former Department of State Development, Business and Innovation commissioned external consultants to examine the case for reform.

DPC stated in its 2015 Improving the efficiency and effectiveness of Victorian Government transactional services full business case (the Reform Program business case) that it cost government $461 million per year to deliver transaction services across six departments and seven major agencies. It predicted that without changes, by 2026, this would increase to $713 million per year. 

DPC, in its Reform Program business case, recommended: 

  • creating a new service unit to establish digital capability
  • optimising a subset of transactions
  • developing capabilities for transaction delivery that can be used across Victorian Government agencies. 

Optimising transactions means making them more efficient by making them consistent, reducing unnecessary steps and delivering them digitally.

The Victorian Government allocated $15 million for planning this new service unit—now known as Service Victoria (SV)—in the 2015–16 State Budget and a further $81 million for its development in the 2016–17 State Budget. As at 30 June 2020, SV has cost the Victorian Government $156.9 million.  

The implementation of SV was complex. It involved multiple interrelated projects, information and communications technology (ICT) and whole-of-government collaboration. 

The first three years

The Reform Program business case sets out three delivery 'horizons'. DPC anticipated that in the first three years, or Horizon 1, SV would deliver the scope shown in Figure A.

Figure A: Delivery of Horizon 1 scope (2015–18)

Horizon 1 scope Delivered? VAGO commentary

Establish a digital platform that allows customers to transact or get information via a website or mobile device

SV has achieved this. It has a website and mobile app that allows customers to transact with government using a consistent design.

Build common capabilities (for example, payment gateways or identity verification functions that any department or agency can use)

SV's capability includes:

  • payment gateways, digital licences, identity verification and analytics
  • integration with agency legacy systems
  • digital customer support, such as live webchat and a virtual assistant.

Optimise a subset of government transactions

Partial

While SV optimised a subset of transactions, these were still in a testing phase at the end of Horizon 1. 

The Reform Program business case anticipated that by the end of its first three years, SV would be delivering at least 14 different transactional services. SV did not achieve this. 

At the end of Horizon 1, SV had delivered transactions for six services (all in a testing phase only) and 12 services in its fourth year. It is still delivering a lower-than-expected volume of transactions.  

Source: VAGO.

The Reform Program business case

The primary purpose of a business case is to provide the government with enough information to make an informed investment decision. DPC's Reform Program business case clearly showed a need for change but lacked sufficient detail. 

Narrow range of options

DPC's Reform Program business case referenced alternative solutions interstate and overseas, but it did not explore these as project options. 

High-value transactions include transactions that involve manual processes, are costly to government, or could deliver significant savings when delivered on a large scale.

DPC explored four project options. All options were similar and included high-value transactions. The options only differed on the number of services that SV would deliver and whether SV would also offer a complaints portal and customer service strategy. DPC limited its cost–benefit analysis to these options. 

DPC did not adequately explore alternatives such as outsourcing to the market or delivering lower-risk transactions to first establish SV's capability.

Estimation of benefits

DPC outlined in its Reform Program business case that SV would drive benefits through transaction reform. Its benefits included: 

  • improving customer satisfaction and reducing the time it takes for customers to transact with government
  • saving the Victorian Government $61 million per year from the improved productivity of government departments and agencies
  • increasing the effectiveness of government policy and regulations by making it simple and easy for people to transact with government. 

However, achieving these benefits relied on SV:

  • optimising a set of existing high volume and high cost to government transactions such as VicRoads licensing services, Victoria Police applications, and the then Land Victoria property services
  • being able to deliver and measure improvements in compliance, even though this was not well scoped in the Reform Program business case.

For example, DPC predicted that SV would deliver increased revenue because more people would obtain the required licence or pay their bills if simple and fast digital options were available. DPC suggested that the new agency would measure this by the reduction in unpaid fines. 

However, at the time DPC developed the Reform Program business case, the government was already considering an alternative fines ICT solution. Now known as Fines Victoria, this began operation in December 2017. SV did not onboard fines transactions, nor did it have a way to measure whether people's use of SV resulted in fewer unpaid fines. As such, this was unlikely to be a real or attainable benefit.

Reliance on assumptions and estimates

There were inherent problems with some assumptions that DPC used to build its business case and estimate the value of SV's benefits to government. DPC's Reform Program business case assumed that:

  • the identified stakeholders would onboard with SV without a requirement for them to do so
  • SV would be able to achieve the desired shift from in-person, post, and phone transactions to digital interactions.

In addition, because not all departments and agencies had reliable transaction data, DPC relied on estimates of the cost, volume and type of transactions undertaken.

Lack of key details 

DPC's Reform Program business case did not contain the full details and documentation recommended by the Department of Treasury and Finance’s (DTF) business case guidelines, such as a detailed project plan, procurement strategy or stakeholder management plan, funding model, governance framework or a detailed risk management strategy.

Further, DPC did not include in its business case a comparative assessment of what it would cost agencies to implement their own digital solutions. Nor did it include several items that it claimed would be explored as the program progressed, such as whether SV would have a physical presence or baselines for customer satisfaction.

The government considered the business case in 2015. While it recognised that the business case lacked key details, it noted that there was policy merit in the initiative and approved SV for inclusion in the 2015–16 State Budget with:

  • an initial commitment of $15 million for planning—including the development of a detailed Program Implementation Plan 
  • further funding (up to $122 million), which the government would hold in contingency, with release subject to DPC meeting project milestones determined in the Program Implementation Plan.
Failure to revise benefits and costs

The HVHR assurance process subjects identified programs to more rigorous scrutiny and approval processes, including staged reviews. 

This includes Gateway and Program Assurance Reviews, where an independent external reviewer provides advice about a project’s progress and likelihood of successful delivery.

As a complex ICT project that required the collaboration of many agencies, SV was subject to the high value high risk (HVHR) assurance process. However, despite Gateway reviews recommending that SV update the business case with revised benefits and costs, neither DPC nor SV made these revisions. 

When SV developed Program Implementation Plans, these did not detail and quantify the program benefits or address changes to underlying assumptions even after significant changes and government policy decisions had occurred. 

This reduced:

  • transparency in the program
  • the ability for government to track SV's progress against its original vision, which would have assisted government to:
    • decide whether SV remained a viable option
    • take action to improve project benefits (such as assess whether the program should be mandatory).

SV’s intended transactional benefits

The Reform Program business case predicted that SV would deliver a net financial benefit of $61 million per year. While SV would achieve almost $8 million per year in benefits through savings in payments and streamlining physical service centres, most of SV's financial value was though providing cheaper and more efficient transactions. 

In 2019–20, SV reported that it achieved $6 million in transactional benefits. This is a significant shortfall from the Reform Program business case’s annual transaction benefit estimate of $53 million. The shortfall is because SV did not deliver the volumes or types of transactions that DPC outlined in its Reform Program business case. 

This transactional benefit also does not reflect the true transactional savings to government. Rather, it reflects the savings to the agency from transferring the transactions to SV. For most transactions, it does not capture the ongoing costs of the transactions such as agency staff time or other indirect costs. 

In 2015, DPC did not have formal agreements with any of its key stakeholders to deliver their transactions. While it had engaged with these stakeholders, it had not 'locked in' the transaction delivery via a contract or formal agreement. SV’s success relies on agencies using it. One of DPC's risk mitigation strategies was that it would not implement SV without a mandated vision. This did not occur. The government chose not to mandate the use of SV and it remains optional for agencies to use.

The subsequent failure to onboard two bundles of transactions that the business case had anticipated has significantly impacted SV’s benefits realisation. These are:

  • land registration and title services, with $23.4 million in projected benefits per year
  • the full scope of VicRoads transactions, with $17.6 million in projected benefits per year.

Land registration and title service transactions

DPC's Reform Program business case included transactions managed by the then Land Victoria. This was a tenuous assumption because at the time of the Reform Program business case, the then Land Victoria had already begun to deliver some digital transactions via a national digital property platform and had recently transferred to a different department.

In May 2017, the Victorian Government announced a scoping study of land titles registry functions. In 2018, it announced that it would commercialise the land titles and registry functions of Land Use Victoria. 

Annual financial benefit is the financial value that SV asserts that it has delivered to government from such things as reduced transaction costs, avoiding costs in investing in multiple platforms and other efficiencies. 

SV removed this transaction type and related benefits from its annual financial benefit in July 2017.

VicRoads transactions

DPC expected SV to deliver all licence and registration transactions for VicRoads. However:

  • SV does not deliver any VicRoads licence transactions
  • at December 2020, SV was only delivering 3 per cent of VicRoads’ online 12-month vehicle registration renewals and 53 per cent of registration checks. 

Over the last five years, VicRoads has developed its own digital capacity and now delivers several digital transactions, including online registration checks and renewals, licence address changes, appointment bookings, notifications of vehicle transfers and an option to have a central VicRoads account. 

Several factors have contributed to VicRoads' decision not to onboard all transactions with SV. Some of these factors included the development of its own transactions and technology, its uncertainty about SV’s ability to deliver at scale, and the future of the VicRoads registration and licensing division.

Duplicated systems

In its Reform Program business case, DPC highlighted that SV would drive reform, consistency, and the delivery of common capabilities across government, including payments and identity management systems. 

DPC advised that over the medium to longer term, the government could decommission high-cost legacy systems and save money. However, agencies such as VicRoads, Ambulance Victoria and Working with Children Check Victoria still run their own digital transactions. This has meant that the Victorian Government has missed the opportunity to realise benefits from one centralised platform. Until departments and agencies decommission legacy systems, the government will continue to pay for multiple, fragmented IT systems and their associated costs.

Overall transaction volume

SV is only delivering a small fraction of its intended transaction volume. DPC estimated that by the end of Horizon 1 (June 2018), SV would be delivering approximately 11 million transactions per year. During the 2019–20 financial year (the end of Horizon 2), SV delivered 814 282 transactions. While this is an increase on previous years, it is a significant shortfall from the business case estimates and makes up only 1.3 per cent of all Victorian Government transactions. 

These low transaction numbers are due to: 

  • the types of transactions that SV has onboarded
  • the low percentage of digital transactions that agencies have directed to SV for some transactions.
COVID-19 related transactions

A QR code is a type of barcode that can be scanned by mobile phone cameras. The QR poster is the printout of that barcode that businesses can use. 

Between July to December 2020, SV introduced five transactions related to COVID-19, including border permits and exemptions, QR check-ins and poster generations, and regional travel vouchers.

SV's ability to quickly stand up these transactions demonstrates the ability of SV to build and utilise its technology. For example, SV set up online applications for regional travel vouchers in 2.5 days.

The introduction of the COVID-19 transactions has increased SV’s transaction volume to 2.57 million transactions in six months. However:

  • QR scans are largely automated and are low value on a cost-per-transaction basis
  • SV counts each individual QR poster generation, which may not be a true indication of the use and value of the service
  • these transactions are likely temporary.
SV’s predicted volume

SV states that it is on track to deliver 6.7 million transactions (worth $20 million per year) by the end of its first 10 years, in 2025. It has not included COVID-19 transactions as a long-term benefit. 

SV’s predicted volume is still lower than Horizon 1's initial scope of 11 million transactions by 2018 (worth $53 million). SV’s ability to deliver its projected volume is dependent on it onboarding new services and delivering an increased proportion of the transaction volume for the services it currently delivers. SV has a pipeline of transactions that it is developing, but it is too early to tell whether it will deliver the scale of transactions anticipated.

Stakeholder engagement

As it is not mandatory for agencies to use SV, it is essential to SV's success that it can engage and earn the trust of government agencies. This includes understanding agencies' needs and retaining their commitment. 

Barriers to onboarding agencies

During the development of the Reform Program business case, various stakeholders supported the proposed SV. However, by the end of its first three years of operation, SV had still not onboarded many of the forecast transactions. Factors leading to this include: 

SV-related factors such as …

External factors such as …

lack of clarity on its scope and when it would be ready to onboard transactions

agencies' reluctance to onboard new services without first testing SV

technical solutions not being ready when agencies wanted, which meant that agencies continued with their own digital projects 

uncertainty about SV's ongoing funding, which affected stakeholders’ trust in its longevity

failure to secure agreements with stakeholders to lock them in at an early stage, despite early recommendations in Gateway and Program Assurance reviews 

government policy decisions for key transactions such as land titles and registration

DPC outlined in its Reform Program business case that SV would benefit from being close to central government. As such, SV was established within DPC, which was ultimately responsible for implementing SV and ensuring its success. However, we saw limited evidence of DPC encouraging agencies to use SV prior to 2020, when it advocated for SV to have a role in the reform of the fines system. 

Stakeholder feedback

Almost all of the agencies that use SV stated that SV's capabilities have grown. However, SV does not have a mechanism to collect formal feedback from agencies, such as an annual survey. Rather, it meets with agencies during, prior to and after it has onboarded transactions. This is a missed opportunity for SV to identify issues from a stakeholder’s perspective and track its stakeholder engagement performance over time. 

Recommendations about stakeholder engagement

We recommend that:   Response
Department of Premier and Cabinet  1. develops and implements a government agency stakeholder engagement strategy with the aim of increasing agency participation and uptake of Service Victoria's services and reusing existing technology (see Section 3.5). Accepted
Service Victoria 2. introduces an annual survey for agencies that use Service Victoria to track its performance over time and address any areas for improvement (see Section 3.5). Accepted

SV’s cost per transaction

Due to its low transaction volume, SV is costly on a per-transaction basis. In 2019–20, it cost SV $46.54 to deliver each digital transaction. As shown in Figure B, this was more expensive than face-to-face transactions. 

Figure B: SV’s cost per transaction (2019–20) compared to other channels

Figure B: SV’s cost per transaction (2019–20) compared to other channels

Note: * DPC outlined these estimates in the Reform Program business case following an assessment of the cost of Victorian Government transactions. We have adjusted these for inflation. SV currently only delivers a digital option. 
Source: VAGO, from SV data and the Reform Program business case.

The Victorian Government's annual state budget spend is documented in an annual Appropriation Bill. This grants the government permission to spend public money on the items in the budget. 

Direct costs are expenses that directly go into producing the good or service (such as such as technology and licensing fees) whereas indirect costs apply to more than one business activity (such as rent and general staffing costs

SV's cost per transaction is based on SV’s budget appropriation for 2019–20 divided by the number of transactions it delivered that year. As SV delivers more transactions, this cost will decrease as shown in Figure C. However, there are limitations to this calculation. It does not reveal SV's direct or indirect costs, nor does it reflect the complexity of the transaction. Further, if SV delivers one high volume, low complexity transaction (such as QR scans), this will skew the transaction cost despite the transaction requiring limited customer interaction. 

Figure C: Impact of COVID-19 transactions on SV's cost per transaction 

Between 1 July 2020 and 31 December 2020 SV delivered … Inclusion of these transactions using SV's method results in a cost per transaction of …
0.60 million transactions from its existing services $34.62
Plus 0.53 million new COVID-19 related transactions (excluding QR scans and posters)  $18.39
Plus a further 1.44 million QR scan and poster generation transactions $8.09

Source: VAGO from SV data.

SV's annual financial benefit

In 2015, DPC estimated that SV would deliver an annual financial benefit to the Victorian Government of $61 million per year. In January 2018, SV revised its benefit realisation framework and calculation of its annual financial benefit. It: 

  • revised down its target to $30 million per year to reflect the removal of Land Victoria transactions
  • added several benefits not explored in the Reform Program business case such as avoided costs, re-use benefits, intellectual property, payment savings and time returned to customer.

SV reported that it achieved this annual benefit for both 2018–19 ($75 million) and 2019–20 ($45 million). However, SV's actual benefit to government is likely to be less than SV reported. Across the two years, SV has claimed: 

  • $50 million in benefits from reusing its technology to deliver transactions
  • $52 million in avoiding costs to agencies through avoiding a more expensive market solution and procurement costs. Most of this benefit was from Solar Homes transactions (SV claimed $48 million for these transactions). 

As Figure D shows, there are limitations with these benefits. 

Figure D: Added SV benefits and their limitations

Benefit Basis for benefit calculation Limitations

Avoided costs

The amount that government agencies save by using SV. 

These savings may occur because SV: 

  • is cheaper than a market solution
  • has no cost or low cost to agencies
  • means that agencies do not have to undertake lengthy tender processes.

This is a valid benefit to government. However, the quantum that SV has claimed to date is based on several questionable assumptions and estimates. 

SV estimates the value of this benefit by using assumptions and available data. It is difficult to determine the accuracy of this estimate as in most cases, the agency does not need to get market quotes. 

In some instances, these costs are theoretical because:

  • there is no evidence that the government would have approved funding for a market solution
  • the agency may have decided to use an alternative option such as a less expensive digital solution, an internal solution or reducing its scale of change.

Re-use benefits

The value provided by SV’s reusable capabilities, such as cloud-based infrastructure, common capabilities (for example, digital licences), centralised payment gateways and digital support, which make the costs of onboarding additional transactions low. 

The reason that SV can deliver a cheaper alternative and avoid agency costs is because it can re-use its capability.

Including this benefit is double counting. For example, SV has claimed the avoided cost of building digital capability for the Solar Homes transactions as well as the savings from reusing its technology and processes to provide a solution. 

SV was set up to build common capabilities and its ability to deliver more services at less cost is reflected in the overall cost of transaction. The benefit is achieved when the capability is actually re-used.

Time returned to customer

The time customers save by transacting faster.

While this is a valid benefit to customers and is in line with the objectives of the Reform Program business case, it is not a clear and direct financial benefit to government and should be counted separately to SV's annual benefit.

Further, this benefit is driven by an efficient, digitised process. It does not need to be SV that delivers it. As such, there is potential that this benefit could be achieved by other new digital systems that are factored into avoided costs. 

Payments

Savings from the introduction of PayPal.

The ability to realise significant benefits from PayPal has a high degree of uncertainty as it depends on agencies offering and customers using PayPal. It may also be limited in its life span due to changing payment technologies.

Intellectual property

Reusing intellectual property across government, such as SV templates and research.

The calculation is based on assumptions about the value of the intellectual property and is not supported by clear evidence or a consistent underlying methodology.

Source: VAGO.

Recommendation about SV's annual financial benefit

We recommend that:   Response
Service Victoria 3. revises its annual benefit measure to ensure that it accurately reflects direct savings for government and does not include double counting of benefits, particularly re-use benefits or benefits to stakeholders other than government (see Section 3.3). Accepted

Reporting and oversight

DPC is the lead agency for the Victorian Government's Information Technology Strategy 2016–2020. A key priority of this strategy is ensuring that Victorians can interact digitally with government services in a way that is useful, easy, and always available. 

Key to this strategy was the need for DPC to drive increased digital uptake by Victorian departments and agencies. DPC's Reform Program business case outlined that SV would define the strategy, standards and policies that must be adhered to by departments and agencies in the delivery of transactional services. 

This model has changed over time. SV sets customer service standards and identity verification standards, but only for the services that it delivers. It does not define whole-of-government transactional strategy or provide a consultancy service for agencies.

DPC has published a series of best-practice guides that aim to make digitisation easier for agencies to understand and implement. It also suggests that agencies assess how they are performing by measuring:

  • user (customer) satisfaction 
  • time to completion
  • transaction completion rate
  • cost per transaction
  • digital take-up, which refers to how many customers are using the service digitally compared to in-person, via phone or mail.

DPC does not know how many transactions exist across government, what they cost and whether they are digital as agencies do not report to DPC or publicly on these measures. While DPC commissioned work to estimate this in 2015, neither SV nor DPC has updated this data. This reduces the ability of government departments and agencies to:

  • make data-driven decisions about how to improve their services
  • compare data across multiple government services
  • be open and transparent to the public about the performance of their services.

While SV reports on these measurements internally, it does not publicly report on its cost to deliver transactions or volume of services. SV's only public performance measure is customer satisfaction, which DPC includes in its annual report and the Victorian Government Budget Paper No. 3 (BP3) reporting. While customer satisfaction is an important aspect of SV's service delivery, it is also important whether Victorians use SV and whether SV is reducing the costs to government to process transactions. 

Recommendations about reporting and oversight

We recommend that:   Response
Department of Premier and Cabinet  4. introduces a Victorian Government Budget Paper No. 3 measure and target for Service Victoria's cost of transactions, volume of transactions and the number of services Service Victoria delivers (see Section 3.4) Accepted
5. establishes a requirement for departments and agencies delivering digital services to report to the Department of Premier and Cabinet on the key performance indicators outlined in its Measure how content performs—digital guide including:
  • user (customer) satisfaction 
  • time to completion
  • transaction completion rate
  • the percentage of service transactions that departments and agencies deliver digitally (see Section 2.5)
Accepted
6. develops a dashboard to report publicly on the performance of agencies delivering digital services against the key performance indicators outlined in its Measure how content performs—digital guide (see Section 2.5). Accepted

SV's customer satisfaction results

SV achieved its customer satisfaction target of 95 per cent in 2018–19 and 2019–20.  

Customer Satisfaction (CSAT) is a metric that measures how satisfied a customer is based on a specific scale and formula. It is separate from the term 'customer satisfaction', which can be used in its common form to mean how satisfied customers are with a service. 

Net Promotor Score measures how likely a customer would recommend a service and Customer Effort Score measures how easy it is to use the service. 

However, it is difficult to compare SV across like services as there is no standard customer satisfaction measure used for public sector agencies in Victoria and other states. Each department or agency chooses its own metric. This can vary from CSAT (a customer satisfaction methodology) to Net Promoter Score, Customer Effort Score, or their own customer satisfaction metric. 

SV has used its own approach to measure customer satisfaction. This approach has been consistent and relies on statistically reliable data. However, it only captures feedback from customers who have finalised their transaction with SV. This means it potentially excludes customers who did not complete their transaction due to dissatisfaction with the service. 

SV’s customer satisfaction approach …

The commonly used CSAT metric …

Uses a five-point scale for customer feedback as follows:

  • Great experience! (score of 5)
  • Good experience (score of 4)
  • Ok, I guess (score of 3)
  • Needs work (score of 2)
  • I'm not happy (score of 1)

Uses a five-point scale similar to SV 

Uses a mix of neutral and positive scores (3s, 4s and 5s) to calculate its result

Uses an average of only positive scores (4s and 5s) to calculate its result

Has a high target (95 per cent)

Is usually a lower target (75–80 per cent) 

Resulted in 96 per cent customer satisfaction for 2019–20

If used by SV, would result in 82 per cent customer satisfaction for 2019–20

Recommendation about customer satisfaction

We recommend that:   Response
Department of Premier and Cabinet  7. reviews its current digital standards to include a clear key performance indicator for customer satisfaction together with the preferred customer satisfaction measurement approach agencies should use (see Section 2.5). Accepted

Compliance with government regulations and policies

The Reform Program business case highlighted that the transaction reform would increase peoples' compliance with Victorian Government regulations and policies due to more consistent, accessible, and simplified processes. 

SV cannot show that it has achieved this benefit. SV's measures do not directly relate to SV's actions or specific policies and regulations, as shown in Figure E.

Figure E: Performance measurement and tracking

Reform Program business case measures used to assess achievement of this benefit Commentary

The transaction completion rate (the percentage of people who complete a transaction with SV)

There is no measurement of the number of digital transactions across the Victorian Government. We estimate current transaction numbers based on 2015 data.

SV is not the sole provider of all services and the measure does not reflect increased digital compliance or indeed more people completing core compliance transactions, such as licences. 

Increased use of government services in the longer term

SV has a target of 2 per cent increase on its services every year. This does not consider the growth rate of all government transactions or the growth in Victoria’s population. 

Increased revenue from greater compliance due to a reduction in unpaid fines

SV does not deliver many transactions that are subject to regulation and fines. Out of 17 end-to-end transactions SV delivered by 30 June 2020, it had five transactions where a customer may be fined for non compliance (such as a failure to hold a fishing licence or drive with a valid car registration).

SV also does not deliver fines transactions and has no way of tracking revenue or a reduction in unpaid fines. SV revised its measures in 2018 and no longer tracks or reports on this measure. 

Source: VAGO.

Recommendation about government regulations and policies

We recommend that:   Response
Service Victoria 8. reviews its benefits reporting and the inclusion of the compliance with government policy and regulations benefit given the challenges in the attribution and measurement of this (see Section 3.3). Accepted

Baselines used by SV

Most transactions are not new to agencies. Agencies should have data on how much a transaction costs to deliver together with other key metrics such as completion rate, customer satisfaction and length of transaction. These are referred to as transaction baselines

SV measures several key performance indicators (KPI) relative to the transaction baseline of the agency that uses SV. These measures include:

  • reduced cost of transactions
  • completion rate
  • improvement in customer satisfaction
  • reduced effort to undertake a transaction.

Agencies do not always have reliable data on their transactions. To address this, SV undertook its own research and used this to set baselines for measuring its performance against customer satisfaction, transaction cost, time to completion and completion rate. However, SV did not do this for all transactions. For the transactions that it did obtain baselines for, SV's approach to establish these baselines was unreliable due to low sample sizes. 

Recommendations about baselines

We recommend that:   Response
Service Victoria 9. reviews its current baselines for customer satisfaction, time returned to customer, completion rate and transaction cost to ensure that they are statistically reliable and relate to the benefits they measure (see Section 2.5) Accepted
10. develops processes to ensure that future transactions have reliable baselines (see Section 2.5), including:
  • sample sizes that are a statistically reliable sample
  • processes for calculating baselines to ensure that these are consistent across transactions.
Accepted

Back to top

1. Audit context

Population growth and changing consumer expectations are placing increasing pressure on the cost, timeliness and quality of government services. Digitising transactions can help to address this. 

The Victorian Government launched SV in 2015 to transform the way that government agencies deliver transactions to citizens, improve customer satisfaction, and reduce costs.

1.1 Victoria’s need for digital services

The demand to access services online is growing rapidly, as is people’s expectations of the range of services they should be able to access anytime and anywhere. 

In 2015, Victorians completed around 55 million state government transactions by mail, phone, online or face-to-face. Considering the state’s population growth since, and assuming government transactions have grown in proportion, we estimate that Victorians completed over 62 million transactions in 2020. Transactions include: 

  • making a payment, such as vehicle registration or topping up a myki (public transport) card 
  • applying for a document, licence, or government program, such as a fishing licence, birth certificate, or the Solar Homes rebate 
  • renewing or changing personal information, such as updating personal details for a driver’s licence or a Working with Children Check (WWCC). 

While some Victorian Government transactions are fully available online, others require customers to fill in paper applications or visit an office. It can also be difficult for customers to find information about services because the Victorian Government has hundreds of phone hotlines and websites. This is inefficient and costly to both consumers and the government.

The 2014 Improving the efficiency and effectiveness of Victorian Government transactional services Preliminary business case (the Preliminary business case) and the Reform Program business case, which were commissioned by DPC and the former Department of State Development, Business and Innovation, highlighted the need for the Victorian Government to reform the way it delivers transactions.

These business cases found that the government had not embraced modern technology and was lagging behind other jurisdictions. Figure 1A illustrates the state of digital service delivery in Victoria in 2015. 

FIGURE 1A: The state of Victorian Government digital service delivery in 2015

FIGURE 1A: The state of Victorian Government digital service delivery in 2015

Source: VAGO, from the Preliminary business case and the Reform Program business case. 

The Reform Program business case also highlighted that, in 2013, Victoria’s customer satisfaction and ease of completing transaction rates were low compared to other jurisdictions.

1.2 The Victorian Transactions Reform Program

In 2015, DPC recommended that the government establish a new service unit, which is now known as SV. This aimed to address three primary transaction delivery challenges, as Figure 1B shows. 

FIGURE 1B: The Victorian Government’s three primary transaction delivery challenges 

Challenge Impact
Ease and speed of transactions Not meeting the public’s expectations, which causes increased red-tape costs, reduced productivity and reduced public satisfaction (including non compliance)
Complex, duplicated and fragmented transactions Low digital uptake, which results in increased costs to the government
Increasing demand Delivery models will not meet growth in transaction volumes, which could result in service failure and non compliance by the public

Source: VAGO, from the Preliminary business case and the Reform Program business case.

Establishing SV

The vision DPC expressed in its Reform Program business case—to establish digital transactions across Victorian Government agencies—was consistent with the Victorian Government ICT Strategy 2014–15. This strategy had objectives to implement new digital and mobile channels for Victorians, standardise systems, improve productivity and increase the government’s capability to innovate. 

In its Reform Program business case, DPC predicted that SV would deliver three key benefits: 

  • improved customer satisfaction and productivity (reducing the amount of time it takes for customers to transact with the government) 
  • increased compliance with regulation (increased effectiveness of government policy and regulation) 
  • increased government productivity (cost savings). 

DPC highlighted in its Reform Program business case that to achieve these benefits, the government must:

  • transform its approach to transactional service delivery
  • make it simpler, faster, and more convenient for people to access government services. 

As Figure 1C shows, the Reform Program business case outlined that SV would be delivered in three key stages, or ‛horizons’. 

Common capability refers to projects such as proof of identity, customer record keeping and other core capabilities that can be used for all transactions. 

Optimising transactions means making them more efficient by improving consistency, reducing unnecessary steps and delivering the transaction digitally.

FIGURE 1C: The three horizons

FIGURE 1C: The three horizons

Source: VAGO.

1.3 Timeline

FIGURE 1D: Timeline of key dates

FIGURE 1D: Timeline of key dates

Source: VAGO.

1.4 Service Victoria 

SV's project management approach

The Victorian Transactions Reform Program (Reform Program) was complex. It involved establishing a new unit that would provide a whole-of-government approach to transactional service delivery, including the development and implementation of a technology solution. 

Agile project management is an iterative approach to delivering a project throughout its lifecycle. It uses feedback loops to continually review and improve its product. 

SV aimed to address past failures with government ICT projects, such as delays and budget blowouts, by blending an Agile and waterfall project management approach.  

The waterfall method is a more traditional project management method. Projects that follow this approach outline their specifications before they start and deliver them at predetermined stages. This approach often does not involve the end user until late in the project. Changing requirements at that late stage can be expensive and time consuming. This can contribute to projects not being delivered on time, within budget or at all.

SV therefore took an Agile and iterative approach to progressively define its program and technology requirements and build capability. It then used a waterfall model of defined program phases designed to provide checkpoints and governance at key stages. Figure 1E shows the components of these methodologies.

FIGURE 1E: Waterfall and Agile project management methodologies

FIGURE 1E: Waterfall and Agile project management methodologies

Source: VAGO, from publicly available information.

In addition to these methodologies, SV also used the lean start-up methodology in developing its product, with the core intent of maximising customer value while minimising costs. It involves challenging processes and the underlying thinking to determine whether it can be more efficient.    

The government may establish an administrative office that is linked to a department. 

The relevant department is responsible for the general conduct and effective, efficient and economical management of the administrative office's functions.

In some cases (such as with SV) legislation gives specific powers and responsibilities to the administrative office. 

SV today

On 1 July 2018, SV became an administrative office of DPC. The Service Victoria Act 2018 outlines SV’s functions. This Act:

  • enables SV to deliver government services
  • provides a regulatory framework for SV to provide identity verification functions
  • allows agencies to transfer their functions to SV. 

Customers can transact with SV via its website or a mobile app. Figure 1F shows what SV’s website and app look like.

FIGURE 1F: SV’s platforms

FIGURE 1F: SV’s platforms

Source: SV.

Customers have the choice to transact as a guest or create an account if they want SV to store their information for future use. The SV app also allows customers to store digital licences. For now, SV has only released digital fishing licences.

SV provides three types of services: 

  • end-to-end transactions, where a customer can complete the entire transaction via the SV website or app
  • links to other services, where a customer clicks a link on SV’s website and is redirected to the relevant agency’s website
  • white-label transactions—these are capabilities that SV has built for an agency but does not host on its website. For example, the identity verification component of a WorkSafe Victoria licence. 

We discuss the transactions that SV offers in Chapter 2. 

1.5 DPC’s role and responsibilities

DPC is responsible for developing and leading whole-of-Victorian-Government policies, including the Information Technology Strategy 2016–2020. This strategy’s key deliverables include providing better digital transactions and implementing SV. 

SV was initially approved as a project within DPC so it could be established quickly, rely on existing supporting policies and systems and be close to the centre of government and decision-making. DPC was responsible for the successful delivery of SV.

Program and project governance

The Senior Responsible Officer led SV's operation, management, and implementation. They were responsible for recommending program level changes or changes that represent a material departure from the business case to DPC's Secretary and the then Special Minister of State. 

SV had over 220 deliverables for the program. Many of these were interrelated. Rather than having individual project boards to oversee each key project, SV used an Agile approach to delivery and conducted frequent cycles of build–test–learn phases. It then used one project control board to oversee the delivery of the outcomes. 

Audit and Risk Management Committees

DPC's Audit and Risk Management Committee's (ARMC) purpose is to provide independent assurance and advice to DPC's Secretary on the effectiveness of the department's finance management, performance, compliance and risk management functions. 

In 2015, the DPC ARMC formed the SV subcommittee to assess and make recommendations on SV's governance and monitor risk. This subcommittee was disbanded in 2016 and replaced by a Program Control Board (PCB). Appendix I outlines the role of these committees and period of time they were operational. 

Current governance arrangements

In 2018, SV became an administrative office of DPC and gained powers and responsibilities under the Service Victoria Act 2018. SV's Chief Executive Officer reports directly to the Minister for Government Services for functions exercised under the Service Victoria Act 2018 and reports to DPC's Secretary on the general conduct of SV. This included the establishment of an internal audit function to provide assurance to SV, the DPC Secretary and the ARMC about SV’s operations. SV must follow DPC’s standards, policies, and governance frameworks.

SV's Board of Management (BoM) is responsible for overseeing SV’s operations and functions. It establishes SV’s strategic direction and provides assurance over SV's benefits realisation. The BoM is made up of SV’s Chief Executive Officer, Chief Information Officer, three executive directors and two directors. 

Figure 1G outlines SV’s governance arrangements.

FIGURE 1G: Governance arrangements 

FIGURE 1G: Governance arrangements

Note: SV was accountable to the former Special Minister of State between May 2015 and March 2020, after which time the Minister for Government Services replaced the former Special Minister of State.
Source: VAGO, based on legislation and SV’s governance policy.

Back to top

2. Benefits of SV

Conclusion

SV has delivered good customer satisfaction results for the limited services it offers. However, it has realised only a small proportion of the benefits expected in the Reform Program business case. 

In 2019–20, SV achieved transactional benefits of $6 million—well below the $53 million per year that the Reform Program business case predicted. It has also not achieved measurable improvement of people following government regulations and policy, as SV is only conducting about 1 per cent of government transactions. 

2.1 Realisation of the Reform Program’s expected outcomes

Now in its fifth year of operations (or Horizon 3 of the program), SV has delivered key components of the Horizon 1 scope. However, because of the limited range and type of transactions moved onto its online platform, SV has not achieved many of the expected outcomes.

Program scope and five-year vision

In its Reform Program business case, DPC established the scope for SV’s first three years (or Horizon 1) of transaction reform and set its longer-term vision as shown in Figure 2A.

FIGURE 2A: Program scope and SV's progress 

Timeframe DPC's expectations of SV Achieved? Commentary

Horizon 1
(years 1–3)

Establish a digital platform

SV has a website and mobile app with consistent design elements, allowing customers to transact with government using either platform. 

Build common capabilities that any department or agency can use

SV's capability includes:

  • payment gateway, digital licences, identity verification and analytics
  • integration with agency legacy systems
  • digital customer support, such as live webchat and a virtual assistant.
Optimise a subset of government transactions

Partially, but not those originally intended

While SV has optimised a subset of government transactions, it has delivered a different mix of services than outlined in the Reform Program business case, many of which are lower in volume and complexity.

Horizon 2
(years 4–5)

Extend its capability to the remaining Victorian Government transactions resulting in a potential annual benefit of $120 to $150 million a year

SV is not yet delivering this scale of transactions. 

SV is delivering a similar number of transactions to that anticipated in Horizon 1. 

While SV delivers transactions across different departments and agencies, it does not deliver transactions for all departments or any transactions from local councils or water authorities.

Horizon 3
(years 5 and beyond)

Provide a platform for Victorians to access a range of services including transactions from local councils and water authorities

In progress

Note: SV was allocated three years of funding in the 2020–21 State Budget to continue its operations and partner with local governments to assist with streamlining business licence processing.
Source: VAGO.

SV has also only fully achieved two out of 10 specific outcomes outlined in the Reform Program business case and expected to be delivered in the first three years. 

SV's ability to achieve these outcomes was impeded by:

  • changes in the program scope such as changed transaction mix and the decision not to proceed with a whole-of-government complaints system
  • government decisions, such as not to proceed with a storefront or physical presence
  • poor stakeholder engagement and agency decisions not to use SV resulting in fewer transactions onboarded than expected.

Figure 2B summarises the status of these outcomes. 

FIGURE 2B: Horizon 1 Reform Program outcomes and status

Expected outcome Achieved?
A consistent digital experience for the customer regardless of the transaction type and department. Government leveraging cloud and mobile technologies to improve customers’ experience, access and cost
Multipurpose ‛tokens’ (evidence of the transaction undertaken that can be used by multiple departments and agencies and can be stored digitally on phones and the web)
A new government unit that facilitates service delivery and sets standards with senior sponsorship and metrics fostering good service, reduced costs and increased pace and agility Partially
Standardised data verification and customer record sharing, leading to reduced form filling and fewer face to face interactions required to prove identity and apply for services Partially
Efficient and effective development and maintenance of legislation and policy impacting transactions Partially
Consistent use of low-cost, highly reliable payment methods across all Victorian Government Partially
A significant increase in the number of transactions completed with little or no human interaction
Foundations laid for an optimised physical footprint (for example, retail outlets and contact centres) enabled by a greater digital uptake
Reduced complaints, which are captured and analysed from a whole-of-government perspective
Lower operating expenditure and total costs of ownership, primarily through migration of transactions to lower cost channels

Note: Appendix D outlines SV’s achievement of Horizon 1 outcomes in further detail. 
Source: VAGO analysis of the Reform Program business case and SV data.

2.2 Delivering the intended transaction volume and mix

NPV is the value of future cashflows over the life of an investment. It takes into account all revenue and expenses together with the timing of each. 

A positive NPV results in profit, while a negative NPV results in a loss. 

In its Reform Program business case, DPC anticipated SV would deliver a net present value (NPV) of $97 million over 10 years and that in Horizon 1, SV would:

  • establish both an online and physical (storefront) presence
  • deliver the top 41 per cent of all government transactions by cost—approximately 11 million transactions each year.

SV did not achieve this transaction target, and so is not on track to deliver the anticipated NPV.

Customer accessibility: online versus storefront

DPC’s Reform Program business case outlined that SV would to be accessible both online and via storefront locations. DPC flagged that establishing an in-person presence would require further exploration and development of a separate business case. The government did not approve this later business case. As a result, SV adjusted its focus to digital-only transactions and has not achieved the added benefits that a physical presence would bring. 

Transaction scope and mix

By the end of Horizon 1, SV had delivered only one of the 14 transactions originally within scope of the Reform Program business case—VicRoads end-to-end registration renewals, although this was not exclusive. By 2019–20 it had added WWCC transactions.

DPC's estimates of SV benefits relied on SV onboarding particular transaction bundles, many of which were high volume and costly for the government to deliver via existing channels. Two of the most important bundles were for Land Victoria and VicRoads. However, SV was not able to fully onboard either of these:

SV was not able to onboard …

Because …

Impacting SV's delivery of financial benefits, as …

Land Victoria’s property registration transactions, which was estimated to save $23.4 million per year. 

  • The national Digital Property Exchange Reform was already underway and these transactions moved to the national platform.
  • In 2016, the government created Land Use Victoria under the Department of Environment, Land, Water and Planning. In December 2017 this agency completed a scoping study to commercialise the land titles and registry functions.  

The 2018 Program Assurance Review (Gate 5 equivalent review) estimated that removing land registry and titles transactions would result in a negative NPV of $44.2 million over 10 years. 

On this basis, establishing SV would not have been financially viable. 

Most of VicRoads’ registration and licensing transactions, which was estimated to save $17.6 million per year.

SV was not ready to release its beta version (a version of the technology which is still undergoing testing) until October 2017. As such, VicRoads continued developing its own digital capabilities including the establishment of a myVicRoads portal. It has released several digital transactions including online registration checks and renewals, licence address changes, appointment bookings, notifications of vehicle transfers and an option to have a central VicRoads account.

The timing of VicRoads' decisions about its own registration and licensing functions, along with SV's technical capabilities at the outset impacted what SV could deliver.

The Victorian Government is conducting a scoping study into reform options for VicRoads registration and licensing. These decisions may further impact the use of SV.

DPC's Reform Program business case estimated that delivering 80 per cent of registration transactions and 60 to 70 per cent of licensing transactions digitally would deliver annual benefits of $6.1 million and $8.4 million respectively.

As at December 2020, SV was not processing any VicRoads licence transactions. It delivered 3 per cent of VicRoads’ registration renewals and 53 per cent of registration checks. 
 

There are several reasons why SV did not deliver the original transaction mix, including that it was not mandatory for agencies to use SV and many decided to delay, defer or not onboard at all with SV. We discuss this further in Chapter 3. A detailed examination of transaction delivery is also shown in Appendix E.

Revising the transaction mix

SV is now aiming to deliver a positive NPV of $25.17 million over 10 years. This relies on it adapting the transaction mix and delivering additional benefits not listed in the original Reform Program business case. We discuss these new benefits further in Section 3.3.

To prove its capability, SV revised the transactions that it would onboard in 2018. These changes were listed in its Program Implementation Plans.

The types of transactions SV onboards influence the ultimate volume that it can deliver. For example, fewer people will register an 'absence from residence' with Victoria Police than would apply for a criminal history check. 

Figure 2C shows the number of services that SV has delivered over time and how this compares to estimates in the Reform Program business case. 

FIGURE 2C: SV's transaction volume (business case estimates and after changes to the transaction mix) 

  Reform Program business case estimate(a) 2017–18 actual(b) 2018–19 actual 2019–20 actual
Number of separate end to end services available via the Service Victoria website (such as fishing licences, registration renewals) 14 6 12 17
Number of digital transactions processed by SV 11 million 53 311 406 595 814 282
Number of Victorian Government transactions processed via all channels(c) 55 million
(SV would deliver 21%)
60.3 million
(SV delivered 0.09%)
61.5 million
(SV delivered 0.7%)
62.5 million
(SV delivered 1.3%)
 

Note: (a) The figures are based on Reform Program business case estimates prepared in 2015 with the transactions to be delivered at the end of Horizon 1 (i.e. by mid-2018).
(b) 2017–18 was SV's beta (testing) phase.
(c) DPC conducted an exercise to estimate transaction volume in 2015. The figures between 2017 and 2020 are VAGO estimates based on the impact of population growth. 
Source: VAGO analysis of SV data.

Other impacts on SV's transaction capacity and timing

SV's governing legislation, the Service Victoria Act 2018, did not come into force until 1 July 2018, almost two years after Cabinet gave in principle approval for the legislative model to transfer customer service functions to SV. The delay in passing this legislation contributed to delays in SV delivering transactions. 

Delivered SV transactions

SV offers a range of end-to-end transactions and links on its website to other external services. As Figure 2D shows, as at June 2020, SV had 17 end-to-end transactions available on its website and was the primary provider of seven of these. 

FIGURE 2D: Transactions offered by SV 2019–20

Category End-to-end transaction Transactions delivered in 2019–20 Other options for completing the transaction
Total delivered by SV Proportion of all transactions (%)
Crime and the law Victoria Police absence from residence 2 022 99 Physical form to lodge (by post or in person
Victoria Police Partysafe (register your party online) 6 552 99
Housing and property Apply for the Solar Homes rebate 62 544 99 None
Outdoor and recreation Buy a fishing licence 170 320 75 In person (at a retailer), or by requesting a form to lodge with the Victorian Fisheries Authority  
Buy a miner’s right 4 779 60 Via an agent
Find a kangaroo harvester 1 419 99 None
Personal Ambulance Victoria new membership 1 775 1 Digitally via Ambulance Victoria's website, by phone or in person
Ambulance Victoria update membership 1 316 1
Ambulance Victoria membership renewals 2 621 Less than 1
Transport and driving  VicRoads check registration 346 739 40
(increased in July 2020)*
Digitally via Department of Transport’s website
VicRoads renew registration 85 815 1
(increased in July 2020)*
Digitally via Department of Transport’s website, by phone or in person 
Boat registration renewals 572 Less than 1
Work and volunteering WWCC new application 373 Less than 1 Via WWCC website with identity verification at Australia Post
WWCC renewal 8 372 10
(increased in September 2020)*
Via WWCC website but only with a MyCheck account
WWCC update details 31 002 33
WWCC change from volunteer to employee  2 350 5
(increased in September 2020)*
WWCC teacher's notification 4 626 99 None (but a small number may approach WWCC Victoria direct)
Total SV customer- facing transactions 17 Services 733 197    
Non-customer facing transactions Transaction volume delivered in 2019-20
Kangaroo tag scans 30 121
Solar Homes identity verification scans 50 964
Total transactions delivered 2019–20 814 282

Note: The three Ambulance Victoria transactions are not currently available on SV’s website as Ambulance Victoria is updating its technical systems.
Note: We list the channel mix for services that are only available via SV as 99 per cent. This allows for a small volume of transactions that customers may complete outside of the agency options provided. 
Note: The percentage of transactions delivered by SV are based on SV data that forecasts the 2019–20 total agency volume.
Note: * these agencies have agreed to 'ramp up' or transfer 100 per cent of their digital transactions to SV.
Source: VAGO analysis of SV data.

Between 1 July and 31 December 2020, SV delivered five customer-facing transactions in response to the COVID-19 pandemic. It generated QR code posters, QR check-ins, border permits and exemptions, and the regional travel voucher scheme. 

SV's quick establishment of these transactions shows its ability to build on and utilise its technology. For example, SV established online applications for regional travel vouchers in 2.5 days.

This has resulted in significant growth in the volume of transactions delivered, with 2.57 million transactions delivered in six months. However, the longevity of services delivered in response to COVID-19 and their financial value is not yet clear. 

SV does not have a clear definition of a how it counts a service or a transaction. For example, SV counts each generation of a QR poster as a transaction. A business may print multiple posters to display at their premises; however, this should count as one transaction as only one QR code is generated. SV also includes non-customer-facing transactions (such as kangaroo tag scans and verification of solar installation scans) in its transaction count but does not count transactions for other services such as 'white label' transactions.

White-label transactions

Agencies can also use SV to deliver part of a transaction or service. These transactions are hosted on an agency website but use SV technology (such as online identity verification) for part of its transaction. SV calls these ‘white-label transactions’. As at 31 December 2020, SV delivered white-label services for:

  • Residential Tenancies Bond Authority
  • the Department of Environment, Land, Water and Planning (for its Melbourne Strategic Assessment program)
  • WorkSafe Victoria.

SV does not count the number of transactions delivered for white-label services in its transaction volume.

Factors affecting channel mix

Channel mix reflects the percentage of customers who transact face-to-face, by phone, mail or online. SV aims is to get more customers transacting online rather than using higher-cost methods. 

SV needs more customers to transact digitally for it to realise its expected cost benefits. Several factors can affect the decision of a customer to transact digitally, such as:

  • the complexity of the transaction
  • alternatives offered by agencies (such as their own online services as well as face to-face, phone or mail)
  • customer preferences.

If SV was the sole provider of all transactions for the 17 established end-to-end services it offered in 2019-20, it would have processed approximately 14 million transactions per year. SV processed about 6 per cent of this number in 2019–20. 

Agency onboarding and referrals

When onboarding with SV, an agency that already has a digital channel may decide to automatically redirect a percentage of its customers to SV’s portal. 


The rate at which agencies have elected to redirect (or 'ramp up') has varied:

  • Some agencies, such as Solar Victoria and the Department of Jobs, Precincts and Regions, have redirected 100 per cent of digital transactions to SV.
  • Others, such as WWCC Victoria, have progressively increased their redirection rate. 
  • VicRoads has been slow to ramp up. It was not until June 2020 that VicRoads agreed to:
    • increase its rate of referral of transactions delivered by SV (renew and check registration)
    • put the SV referral button on relevant VicRoads webpages. This is a button with SV branding that directs a customer to the transaction on SV's website.

In cases where SV is not the sole provider of the digital transaction (such as VicRoads, Ambulance Victoria and WWCC transactions), agencies run their own ICT systems, websites, customer service centres and phone lines. This means they are not realising savings through rationalisation of technology and system efficiencies. We discuss this further in Chapter 3.

2.3 SV transaction cost

Non-digital transactions are more expensive to provide than digital transactions. In 2014, DPC engaged a consultant to estimate how much transactions were costing the Victorian Government and what percentage of people were completing transactions digitally. 

Around the same time, Deloitte Access Economics looked at the economic benefits of digitising customer transaction services for federal and state government departments. While this work was not specific to Victoria, it suggested that the savings from moving transactions online could be even more pronounced than the Reform Program business case estimated, with digital transactions dropping to 40 cents per transaction. We reflect the business case estimates in Figure 2E. 

FIGURE 2E: Benchmark costs per transaction and channel mix (2015)

Channel Reform Program business case estimate
Transaction mix Cost per transaction estimates
Face-to-face 32% $21.52
Mail 24% $30.22
Phone 12% $13.21
Online 32% $6.84

Source: Reform Program business case. 

Marginal cost is the change in the total cost that arises when one additional unit is produced. 

Variable costs are a corporate expense that increase or decrease depending on the production output. 

SV calculates transaction costs by dividing its total running cost by the volume of transactions delivered. The marginal cost of onboarding new services is low. As SV increases its volume, its per unit transaction cost will decrease, assuming that its variable costs do not scale up with volume. 

However, there are limitations to this calculation. It does not reflect SV's direct or indirect operational costs, nor does it reflect the complexity of the transaction. Further, if SV delivers one simple, high-volume transaction (such as QR scans), this will skew the transaction cost despite the transaction requiring very limited customer interaction.

As shown in Figure 2F, in 2019–20 SV was expensive compared to similar services. 

FIGURE 2F: SV's cost per transaction compared to other services

FIGURE 2F: SV's cost per transaction compared to other services

Note: Service NSW’s costs are estimated based on the cost to deliver Service NSW and its transaction volume. It includes transactions conducted in person at a service centre. At year five of Service NSW (the same stage of development as SV), the cost per transaction was $26.
Note: The average cost of digital transactions in Victoria is from the Reform Program business case.
Source: SV data, Reform Program business case and publicly available information.

Between 1 July 2020 and 31 December 2020, SV introduced COVID-19 transactions. Including these transactions significantly changes the cost per transaction amount as shown in Figure 2G. 

FIGURE 2G: Impact of COVID-19 transactions on SV's cost per transaction

Between 1 July 2020 and 31 December 2020 SV delivered … Inclusion of these transactions using SV's method results in a cost per transaction of …
0.60 million transactions from its existing services $34.62
Plus 0.53 million new COVID-19-related transactions (excluding QR scans and posters only)  $18.39
Plus 1.44 million QR scan and poster generation transactions $8.09

Source: VAGO, based on SV data.

2.4 Customer benefits

DPC's Reform Program business case highlighted that digital reform would make it quicker and easier for customers to undertake transactions and improve customer satisfaction. 

SV has consistently achieved high customer satisfaction results for the transactions it delivers. 

Measuring customer satisfaction

Customer feedback provides important insights into SV’s products and services and helps it identify any problems.

DPC’s Measure how content performs—digital guide describes what better practice for digital services involves. It states that:

  • at a minimum, a service should invite customers to give feedback once they reach the page that confirms a successful transaction
  • a more robust approach is to capture data on people who abandon a form part way through the process. 

The Victorian Government does not have a prescribed approach for measuring customer satisfaction. This means that agencies can choose their preferred methodology to calculate it.

The BP3 outlines the government's priorities for the delivery of services, its performance targets and whether it is achieving these. 

Customer satisfaction is SV‘s only BP3 measure, and it has set itself a high target of 95 per cent satisfaction. SV reported that it achieved its target for both 2018–19 (97 per cent) and 2019–20 (96 per cent). Between 1 July and 31 December 2020, SV reported that it achieved 96.7 per cent satisfaction.

SV does not ask all customers for their feedback but uses a sample to determine customer satisfaction with its service. When an SV customer completes a transaction, they may be offered an option to provide feedback on a five-point scale from ‛great experience‘ to ‛I'm not happy‘. SV then calculates customer satisfaction by using the median responses of the 3, 4 and 5 ratings.

Figure 2H compares SV’s customer satisfaction rate to that of other digital transactions and services.

FIGURE 2H: Customer satisfaction scores for a sample of digital services and transactions

FIGURE 2H: Customer satisfaction scores for a sample of digital services and transactions

Note: It is unclear how Service NSW and the South Australian Government Services channels measured customer satisfaction levels.
Note: The Australian Taxation Office calculates its customer satisfaction rate using ‘satisfied’ and ‘very satisfied’ responses. Digital Marketplace asks respondents to score the average level of difficulty from ‘easy’ (100), ‘OK’ (50) and ‘difficult’ (0) and produces an average. 
Source: 2019–20 SV data, myGov performance dashboard (accessed August 2020), Service NSW’s Annual report 2018–19 and the 2019 South Australian Customer Satisfaction Measurement Survey.

Since its establishment, SV has used the same metric and methodology for measuring customer satisfaction. It uses this approach so it can compare its results with other similar jurisdictions. SV counts people who gave a score of 3 (OK, I guess) as satisfied. 

SV's calculation approach can impact the overall customer satisfaction ratings:

If SV had used ratings of …

Then its 2019–20 customer satisfaction score would be …

Instead of …

3, 4 and 5 to calculate the average value instead of a median value

95 per cent

96 per cent

4 and 5 to calculate the average value (the common CSAT approach) 

82 per cent

While using the CSAT approach would result in a lower customer satisfaction score, this measure is usually also associated with a lower target (75 to 80 per cent).

Other methods of gauging customer satisfaction

Sentiment analysis is a technique that is used to interpret subjective text and written commentary. For example, it looks at the words used and classifies the statement or review as positive, negative or neutral. 

In addition to its externally reported customer satisfaction metric, SV also: 

  • internally monitors scores of only 4s and 5s, which it calls its ‛happiness score’
  • analyses written feedback from customers using sentiment analysis. 

SV's sentiment analysis suggests that the most commonly used word for customers who leave written feedback is ‛easy’. 

SV also reviews and monitors:

  • error messages received by customers
  • anonymised screen recordings of real customer visits to understand customer behaviour and how customers move their cursors across the screen
  • the number of customer-initiated interactions, including live webchat, email, or online feedback forms
  • all rating scores of 1. These are: 
    • opened as cases and assigned to a customer experience officer to review and resolve if possible
    • reported to the Chief Customer Officer.
  • customer feedback and suggestions for continuous improvement shared by staff via Yammer, SV's internal chat system. This provides visibility of complaints across the organisation and allows staff to suggest solutions. 

The following case study (Figure 2I) illustrates the impact that proactively reviewing issues can have.

FIGURE 2I: Case study—VicRoads customer satisfaction with registration renewals

In 2018, SV received several negative comments on its registration renewal transaction.

SV reviewed this customer feedback, including all free-text comments. It found that the most common customer complaint was the credit card transaction fee, which is a VicRoads requirement.

Some of the comments SV received were:

  • ‛I have no choice but to pay by credit card and then you charge a fee. How about PayPal or direct deposit[?]’
  • ‛CARD FEE ARE YOU SERIOUS?’
  • ‛$5 charge just to pay using a Debit card! I’m paying $800 just for rego! This is criminal! Disgusted! I have to pay to pay? Haha, absolute joke.'

To address this issue, SV added other payment options, including BPAY. 

Data shows that customer satisfaction scores relating to BPAY improved following this change. There has been no change in SV’s overall customer satisfaction score for this service, which has remained high (96 per cent).

Source: VAGO analysis from SV data.

Gaps in assessing customer satisfaction

SV’s customer satisfaction assessment does not survey customers who end a transaction before completing it. This means that SV is likely excluding some dissatisfied customers from its survey and does not receive their reason for not completing the transaction. 

This is supported by the feedback received on SV's app, available via Google Play and the Apple App Store. 

Between January 2019 and December 2020, 87 people had reviewed SV’s app with a weighted average of 2.7 stars. 

As shown in the examples in Figure 2J, most reviews relate to the app’s lack of services or the information it can store. SV does not respond to these reviews.

FIGURE 2J: Customer reviews of SV's app in Google Play and the Apple App Store

‘Doesn't Store Anything You Already Have. You essentially have to renew or pay your rego or ambulance cover or working with children check just to get access. This app isn't for easy legitimate storage of your existing permits and licenses. Waste of time at the moment, hopefully they … do it right.’


‘Very easy to QR check in. Remembers my details. Nice one, gov peeps!’


‘After finding a free way to set up an account … and jumping through about 4 verification codes, I finally got in to find a nice interface with a few options (that I admittedly have no immediate use for). I’d really like to add my driver’s licence to this, and then have it show in my Apple wallet so my locked phone can display it. This would be so AWESOME, and I hope that’s where this app is going soon.’


‘Useless app because all the services aren’t inside it. You need to transition everything from myki, VicRoads, and others so there is no need to go to other sites. For now it’s useless except the Fishing Licence. Fingerprint login doesn’t work yet. Beta version.’

Source: VAGO, from Google Play and Apple App reviews.

Customers can also provide feedback via SV's website. There is a general ‛Give feedback’ form at the bottom of its home page where people can get in touch, send a compliment, or make a complaint. 

During 2019–20, 10 647 customers used the website form to contact SV. Of these, 12 per cent of people clicked the 'make a complaint' option. A review of a sample of this data shows that customers also left comments indicating their dissatisfaction in the ‘get in touch’ category. If these customers did not complete their transactions, this feedback would not be included in SV’s overall customer satisfaction score.

2.5 SV's performance against agency baselines

In its Reform Program business case, DPC anticipated that it would be able to obtain performance baselines for existing Victorian Government transactions. However, at the time of the business case, not all agencies and departments had clear baseline data. Performance reporting against digital transactions across the Victorian Government is still not strong. 

Performance reporting at a whole-of-government level

DPC's best practice guidelines highlight the importance of measuring performance against benchmarks.

Despite this, the Victorian Government does not have a platform for agencies to report on their performance against its digital standards. In particular:

There is no …

Which makes it difficult to …

Overarching standard across the Victorian Government on customer satisfaction or the time it takes to complete a transaction.

Assess any improvement in customer satisfaction or transaction efficiency.

Ability to track agencies’ performance against DPC’s digital standards.

Assess consistency in digital service delivery across agencies.

Central reporting of the number, type, or cost of transactions that the Victorian Government completes each year or the percentage that people complete digitally.

Identify areas of inefficiency, track progress and drive improvement.

These limitations hinder the Victorian Government’s initiatives to drive digital reform because agencies cannot compare their performance against clear and reliable benchmarks.

SV's actions to address gaps in agency data

Between 2017 and 2019, SV conducted some research to address this gap, but not for all transactions and often not with reliable sample sizes. A low sample size increases the risk that the true result may vary from the reported figure.

This lack of reliable data is problematic as several of SV's KPIs and measures compare its performance against an agency baseline. These include:

  • time returned to customer
  • improvement to customer satisfaction by transaction
  • completion rate
  • reduced transaction cost.

To address this lack of data, SV has updated its operating agreement template to ensure that agencies either provide baseline data or give SV access to their transaction data when onboarding future transactions.

Figure 2K shows SV's reported achievements against these KPIs together with limitations we found with the methodology. 

FIGURE 2K: SV's KPIs, achievement and limitations

SV KPI and method of calculation SV report on achievement of KPI (SV data 2019–20) VAGO commentary

Improvement in customer satisfaction (by transaction)

Compares SV customer satisfaction scores against the previous year (or a baseline if the service has been with SV for less than 12 months). 

Measured by the percentage change relative to the previous year (no specific target).

KPI not achieved.

Overall customer satisfaction dropped by 1 per cent against the previous year.

Of the transactions with baselines, SV states that it:

  • improved satisfaction for seven transactions 
  • decreased satisfaction for two transactions
  • had one new transaction with no result due to insufficient volume.

This KPI has value in ensuring that SV is continually attempting to improve its services, including making transactions simpler. However, year on year improvements may be difficult to continually achieve over time.

Of the 17 transactions SV delivers, only 10 have baselines. 

Seven of the customer satisfaction baselines used were developed with small sample sizes. 

For example, VicRoads renew registration transaction baselines were developed through a low sample size (23). While SV states the baseline is 88 per cent, using such a small sample means that actual customer satisfaction may vary from 70.5 to 96.7 per cent. 

Our analysis suggests that based on the sample sizes SV used, it can only be confident that customer satisfaction improved for three transactions rather than the seven that it has stated.  

Time returned to customer

Compares the time it took a customer to complete a transaction at the original agency (baseline) compared with completing the transaction with SV. 

Measured using weighted average of the time returned to customer. SV’s internal target is 25 per cent improvement.

KPI achieved.

SV reported a 26 per cent improvement in time returned to customer.  

Of the 17 transactions SV currently delivers, it reported that it is quicker for customers to use SV in seven cases.

SV does not have baselines for all transactions. For the baselines it does have, six were developed using research that had low sample sizes. 

This limits reliability of performance reported. For example, SV calculated the fishing licence baseline with a sample of 12 transactions. As a result, based on a 95 per cent confidence interval, the amount of time it takes to complete this transaction could vary by up to six minutes from the estimate.

Some agencies also kept their own digital offering (for example, VicRoads registration renewals). However, SV does not compare its performance against the equivalent digital transaction. Instead, it uses the weighted average of all channels (in person, via phone or digitally). This makes it difficult to compare like-for-like services. 

Improvement in completion rate

Calculates first-time resolutions (transaction completion rate) by comparing the number of transactions customers start with those they complete. 

Where available, measured against the agency baseline. 

Goal to improve on the baseline but no specific target.

KPI achieved.

SV reports a 3 per cent improvement in the number of people who completed transactions first time with SV.

Of the 12 transactions SV delivered in 2018–19, SV reported that it improved the completion rate in 2019–20 for four transactions. 

SV only has completion rate baselines for the agency transaction for six of its 17 transactions. 

Reduced cost per transaction

Estimated cost if the volume of transactions conducted with SV were instead conducted via the previous channel. 

SV aims to achieve a 30 per cent reduction in transaction costs.

KPI achieved.

SV reports a 42 per cent reduction in costs per transaction.

SV reports that it is cheaper for all of the 11 transactions that it has baselines for. 

This measure does not capture the true cost of the transaction. Instead, it captures the amount that would have been spent if the agency had delivered the transaction. 

It does not include the direct costs (such as technology and licensing fees) or indirect costs (such as staffing costs) of SV delivering the transaction as SV assumes that the marginal cost is $0. 

Further, agencies, such as VicRoads, Ambulance Victoria and WWCC Victoria still run their own systems and digital transactions. This has meant that the real cost is increased due to the need to run multiple systems. 

SV has cost baselines for 11 of its 17 transactions. However, these baselines do not include all costs of delivering the transactions, such as direct and indirect costs. 

Note: We summarise SV’s performance against all of its KPIs in Appendix F. 
Source: VAGO.

Failure to implement a cost attribution methodology

VAGO's 2015 Delivering Services to Citizens and Consumers via Devices of Personal Choice: Phase 1—Interim Report highlighted that at that time, the public service did not have a suitable method to determine the costs of delivering service transactions that included both direct and indirect costs. It recommended that DPC develop a method for departments to use. DPC allocated this task to SV. 

SV developed a cost attribution method in its 2015 Program Implementation Plan that did attempt to include indirect costs. However, it is not using this method to calculate the cost of transactions, nor is there evidence that other Victorian Government agencies use this method. 

2.6 Public compliance with regulation and policy

DPC highlighted that people may be more likely to comply with government policies and regulations if the process is quick and easy. For example, as customers can now purchase and renew digital fishing licences via a mobile device, it is easier for people to obtain them, meaning more may do so.

However, SV cannot show that it has improved people's compliance with regulation and policy. This is difficult to measure. Further, neither the business case nor SV tracks compliance of specific policies or whether digital reform impacts this directly. 

Figure 2L looks at DPC’s measurements outlined in the Reform Program business case together with SV’s changes and the relevant limitations. 

FIGURE 2L: Measurements used to assess whether SV has improved compliance with regulations and polices

Reform Program business case measure Limitations

Transaction completion rate (the percentage of people who follow an SV transaction through to completion)

  • There is no measurement of the number of digital transactions across the Victorian Government.
  • Completion rate does not reflect increased compliance with regulation or indeed more people completing transactions digitally. 

Increased use of government services in the longer term

  • SV set a target for a 2 per cent increase on its services every year. This does not consider:
    • the growth rate of all government transactions due to population growth and increased demand
    • individual transactions that may drive compliance (as opposed to services that are not mandatory for citizens —for example, obtaining a licence as opposed to paying for Ambulance Victoria membership). 

Increased revenue from greater compliance due to a reduction in unpaid fines

  • At the time DPC developed the Reform Program business case, the government was already considering an alternative fines ICT solution. Now known as Fines Victoria, this began operation in December 2017. SV did not onboard fines transactions, as such, this was unlikely to be a real or attainable benefit.
  • SV does not deliver many transactions that are subject to regulation and fines. Out of 17 end-to-end transactions SV delivered by 30 June 2020, it had five transactions where a customer may be fined for non-compliance (such as a failure to hold a fishing licence or drive with a valid car registration).
  • SV also has no way of tracking revenue or a reduction in unpaid fines. 
  • SV removed this measure in 2018 and does not track or report on it. 

Source: VAGO.

2.7 SV’s future projected benefits

Since its establishment, SV has increased its service offering each year, achieving a growth rate of 100 per cent of the total volume of transactions delivered as compared to the 2018–19 financial year. However, this is still only a fraction of the Victorian Government’s total transactions, which we estimate to be approximate 63 million transactions per year.

SV’s transaction growth between June 2018 and January 2020 was low. However, since then two key agencies have increased their transaction redirection rates to 100 per cent: VicRoads in June 2020 and WWCC Victoria in September 2020 (for its WWCC renewals and teacher notification transactions). Further, the introduction of COVID-19 services is predicted to significantly increase transaction volume in the short-term.

As Figure 2M shows, SV’s transaction delivery is expected to become more economical due to new services and increased volume in 2020–21 and in subsequent years.

FIGURE 2M: SV’s cost and volume to 2019–20 (actual) and 2020–21 to 2024–25 (predicted)

FIGURE 2M: SV’s cost and volume to 2019–20 (actual) and 2020–21 to 2024–25 (predicted)

Source: SV.

SV estimates that by 2024, it will cost $5.21 to deliver each transaction. This prediction is based on SV onboarding more transactions, attracting more agencies and increasing the ramp up rate of its current transactions. 

Back to top

3. Implementation and oversight of SV

Conclusion

DPC and SV have delivered a technology solution that is repeatable and reusable. However, DPC's implementation of SV did not focus on the delivery of financial benefits. This lack of focus, along with poor reporting and stakeholder management by DPC and SV, has resulted in lower-than-expected delivery of benefits.

The Reform Program business case was inadequate for a project of this complexity. It lacked key details such as governance arrangements, a detailed procurement strategy or project plans. The business case also included several risky assumptions, such as that certain agencies would use SV to deliver key transactions. DPC did not fully explore these risks to inform the investment decision. Further, when key changes to the program occurred, DPC did not amend the business case. This reduced the transparency of benefits delivery and the information available to key decision-makers. 

3.1 The business case for SV

A business case gives decision-makers information to help them to decide whether to invest in a project or not. It should provide decision-makers with the confidence that:

  • the strategic justification for the investment is valid
  • they are selecting the right investment option
  • the agency can deliver the investment as planned.

The strategic justification for SV

As discussed in Chapter 1, DPC's Reform Program business case clearly outlined the need to improve transaction delivery in Victoria. This aligned with the government’s Victorian Government ICT Strategy 2014–15.

DPC, through the Reform Program business case, looked at options to improve transaction delivery. Together with key stakeholders, it developed an investment logic map that identified seven strategic interventions to address the problem. 

As shown in Figure 3A, DPC then grouped these interventions into four strategic options before assessing what the preferred options should be. 

FIGURE 3A: The development of the preferred strategic option

FIGURE 3A: The development of the preferred strategic option

Source: VAGO.


DPC recommended Option 4 following a high-level initial assessment of the benefit, costs, and risks of each option. It determined that this option would maximise benefits from the digital initiatives that agencies already had in progress, build future capabilities, drive cost-efficiencies, and improve customer satisfaction.

However, the business case did not provide: 

  • detailed costs of alternative solutions, such as the government using and building on existing platforms or outsourcing the technology or transactions to a market provider
  • an overall risk ranking for each strategic option
  • an overall cost figure. Instead, it ranked costs for each strategic option as low, medium, or high without a clear rationale for this assessment. 

The Gate 1 review, conducted in 2014, also noted that there were conflicting views on the relative priority between two key drivers of the program: efficiency and improved service delivery. Clarity around these drivers is important as they informed SV's focus and establishment. 

Once DPC identified a preferred strategic option, it explored a range of project options. These reflect what the investment would look like in practice and at a more granular level.

Reform Program project options

There are often many ways in which government can implement a project. However, as Figure 3B shows, DPC, through the Reform Program business case, only considered a narrow range of options. Its project options:

  • all involved establishing a new service unit and optimising a set of transactions that were high volume or high cost
  • differed from each other only in relation to the volume of services and whether the new service unit would have a service/contact centre or complaints function
  • did not cost the use of an off-the-shelf or market solution, although the business case did note that the new service unit should explore myGov as an option for the identity verification project
  • did not include a comparative assessment of what it would cost agencies to implement their own digital solutions
  • did not consider whether it should have tested its technology solution with smaller lower-risk transactions during Horizon 1. Instead, each focused on the high-risk approach of optimising bundles of transactions that represented the government’s top 41 per cent of transactions by value. 

FIGURE 3B: Project options presented in the Reform Program business case 

To deliver the reform, DPC outlined a series of projects to establish the foundations and deliver the means to optimise transactions. Its Reform Program business case listed 14 separate projects in five categories: 

  • foundational projects (establishing the new service unit and customer experience projects)
  • common capability projects, such as policy and legislation, identity management, record keeping, payments and complaints management 
  • channel-specific projects, such as the web and service centre strategies
  • transaction optimisation projects
  • technology project.
Project Option  Transaction bundles in scope for optimisation  Other projects  Technology solution

Option 1 a

  • VicRoads licensing
  • VicRoads registration
  • Land Victoria registration and title searches

All projects except Service Centre Strategy, Contact Centre Strategy and Complaints Management

Full technology solution but with only three bundles of transactions (a reduced volume) 

Option 1 b 

All projects

Option 2 a 

  • VicRoads licensing
  • VicRoads registration
  • Land Victoria property services
  • Department of Justice (Births Deaths and Marriages)
  • Department of Justice (WWCC)
  • Victoria Police applications

All projects except Service Centre Strategy, Contact Centre Strategy and Complaints Management

Full technology solution with all transactions in scope 

Option 2 b 

All projects

Note: The Department of Justice was subject to machinery of government changes in 2015 and 2019 and is now known as the Department of Justice and Community Safety
Source: VAGO, from the Reform Program business case.

Selection of SV as the recommended solution

DPC's Reform Program business case included analysis of the benefits, costs, and risks of each of the four project options and recommended that Option 2 b would deliver the greatest financial and socio-economic benefits. 

The expected advantages and benefits of Option 2 b—establishment of a service unit now known as SV—included the: 

  • highest ongoing annual benefit ($61 million) and 10-year NPV ($97 million)
  • greatest impact on customer satisfaction, including that it would take customers the least effort to undertake a transaction and that it would reduce red tape
  • power to drive change across the Victorian Government and facilitate collaboration and cooperation across departments and agencies
  • capacity to improve the Victorian Government’s image and reputation around customer service and transaction delivery.

This recommendation heavily relied on DPC's calculation of the financial costs and benefits of reform. We found limitations with the analysis undertaken. 

Financial benefits of reform

To understand the cost and benefits of transaction reform, the Victorian Government needed to clearly understand the volume, type, and cost of transactions undertaken. However, this data was not readily available at the time (2014–15) because:

  • departments and agencies had inadequate performance monitoring and data collection for digital initiatives
  • it was difficult for departments and agencies to consistently map and estimate the types, volumes, and costs of their service transactions
  • there was no suitable cost attribution method that could be used across the public sector to determine the cost components of delivering digital transactions.
Development of the transactions catalogue

Physical footprint is the cost of having physical government service centres that people can visit in person.

To address the data limitations, DPC commissioned an external consultant to estimate how many transactions people complete with the Victorian Government each year, their cost, the channel used and their physical footprint. This was known as the transactions catalogue.

DPC used information from six departments and seven agencies including VicRoads, the State Revenue Office, Victoria Police and Public Transport Victoria. However, few departments and agencies were able to supply complete datasets or the required cost data. As a result, the consultant extrapolated transaction costs and volumes based on assumptions. 

DPC relied on these figures to estimate the base case position (the situation if the government did not invest in change) and the transaction benefits the investment would bring.  

Limitations with calculation of benefits

DPC used the transactions catalogue to help quantify the benefits of reform. It stated in the Reform Program business case that these figures were verified with stakeholders. However, there remained the risk that these costs and benefits were incorrect or would change with time. Despite this, neither DPC nor SV revisited the calculations. 

The Gate 2 review in 2015 recommended that DPC gain agreement from key stakeholders about:

  • the accuracy of benefit calculation
  • who was responsible for delivering benefits. 

The Gate 2 review team also recommended that DPC articulate a timeline for realisation of benefits. This was not clear in the Reform Program business case. 

Stakeholders expressed concern that the Reform Program business case was double counting benefits against agency initiatives that were already in progress or planned. While DPC did not include benefits from funded and implemented projects, it did include benefits from projects that were at an early stage of development. It also did not clearly cost these benefits. 

This increased the risk that DPC had overestimated the benefits of the program.  

DPC stated in its Reform Program business case that some of its benefit targets were ‛bold’. However, it believed they were achievable. The Reform Program involved multiple stakeholders, interdependent projects, and new technology solutions. Variations in any of these factors, changes in policy or consumer behaviour may impact the delivery of costs and benefits. 

A real options analysis allows experts to consider multiple alternative future scenarios and their impact on a project’s costs and benefits when developing a business case. This is particularly useful when there is uncertainty as to what will happen in the future due to, for example, the changing pace of technology.

Where investments involve uncertainty, DTF recommends that the investor incorporate flexibility into the investment option and undertake a detailed real options analysis. DPC did not do this. It instead completed a traditional cost–benefit analysis that required practitioners to value costs and benefits that were not known with certainty.

DPC did, however, conduct an external cost review. 

External cost review

In 2015, DPC engaged a contractor to review the Reform Program business case's costs and benefits. This review found that the: 

  • assumptions underpinning the business case’s benefits were sound
  • the business case’s cost estimates (supplemented with a supporting methodology document) were traceable, reasonable, sound, and complete.

The review also noted that DPC could make improvements, including:

  • further quantifying the shift to digital channels, which allows for process improvements and transaction optimisation
  • improving the detail and accuracy of its cost estimates.

The review recommended that DPC undertake a sensitivity analysis of the Reform Program’s overall costs, benefits, and major risks. 

Sensitivity analysis

A sensitivity analysis helps decision-makers understand the impact a range of variables have on a given outcome. While there is no strict guidance on how to conduct a sensitivity analysis, the analysis typically isolates variables (such as volume) and then records the impact this has on the outcome (both positive and negative). 

DPC engaged a contractor to conduct a sensitivity analysis in March 2015. This looked at two alternative scenarios but did not clearly isolate variables that may impact on the outcome. DPC explored the impact of:

  • delays in the project but increased benefits through payments optimisation
  • a slowed ramp up of VicRoads and Land Victoria transactions with the substitution of two new transactions. 

The contractor's analysis found that under either of these scenarios, the Victorian Government would still achieve similar financial benefits to those outlined in the Reform Program business case. 

However, the analysis did not provide decision-makers with all the information needed:

The sensitivity analysis did not …

Instead, the analysis …

Provide clear information on the impact of a single variable (for example, lower than anticipated volume of transactions) on the overall outcome of the program.

Appeared to focus on demonstrating that even if certain cost and timeline assumptions changed, overall benefits expected could still be realised by implementing potential mitigations.

Considered only two alternative scenarios and tested each scenario against five risk mitigation options. However, it did not outline the feasibility of these mitigation options. 
 

 

Consider the possibility that SV would not onboard key transactions at all.

DPC's business case attributed $23.4 million in expected savings to Land Victoria transactions and $13.5 million to new vehicle registrations and VicRoads’ licensing transactions. As highlighted in the Gate 5 equivalent review, removing Land Victoria transactions from benefits realisations changed SV from an investment that would produce a return of $97 million over 10 years, to one that would cost the government money. This risk was not clearly costed and reported early in the implementation of SV.

3.2 Delivering SV

DPC's Reform Program business case stated that some details, such as benefits, costs and projects, would undergo further development as the program progressed. This meant that its scope, cost, and schedule estimates were only indicative. The Reform Program business case did not include the documents that DTF’s guidelines require, including a:

  • clear governance framework
  • procurement strategy
  • rigorous risk management plan
  • funding model 
  • stakeholder management plan
  • detailed project plan.

The use of an iterative approach to program delivery, while designed to maximise project efficiency and ensure continuous improvement, does not mean that key documents should be omitted. This is supported by the Gate 2 review, which highlighted a need for further development of the Reform Program business case to inform immediate government decision-making. 

The decision to invest in SV

The government recognised that the recommended reform was high risk, and that the Reform Program business case: 

  • had an ambitious project schedule
  • needed to establish long-term commitment across departments and agencies to be successful
  • was based on assumptions about costs and benefits. 

However, rather than revise the Reform Program business case, in 2015 the government approved $15 million of funding for further planning of SV and an additional $121.1 million to be held by DTF for SV. 

Under the HVHR Project Assurance Framework, infrastructure and ICT projects that are identified as high value or high risk are subject to more rigorous scrutiny and approval processes. This includes Gateway and Program Assurance Reviews, where an independent external reviewer provides advice about a project’s progress and likelihood of successful delivery.

This further funding was to be released in stages, contingent on SV successfully delivering program milestones and obtaining the Treasurer’s approval. In approving the funds, the government:

  • asked the then Special Minister of State to report back in late 2015 with a detailed program implementation plan before seeking approval to release funding for the next stage of works
  • noted that the Program Implementation Plan would be subject to the HVHR assurance process.

SV’s program implementation

SV's project management model is a blend of both the Agile and waterfall methodologies, which we discuss in Chapter 1. In February 2016, the Treasurer and the then Special Minister of State approved SV’s first Program Implementation Plan. SV revised this Program Implementation Plan on two subsequent occasions and provided this to the Program Control Board and DTF. We do not have evidence that the later plans were reviewed by the Treasurer or Special Minister of State, however, they were subject to review as part of the HVHR assurance process. Each of these Program Implementation Plans progressively defined SV's model and deliverables. 

SV's first Program Implementation Plan (2015) …

But it did not …

Included information missing from the Reform Program business case, such as a:

  • risk management plan
  • governance framework (the business case only outlined governance at a high level)
  • project timeframe and deliverables
  • benefit management reporting tool.
  • address baseline data limitations
  • detail the financial impact that agency digital initiatives may have on benefit realisation
  • specify how SV would sign up stakeholders
  • provide a detailed procurement strategy, but instead outlined it at a high level
  • detail how it would fund its operations after Horizon 1 (that is, government funding or self funded).
     

The Gate 3 (equivalent) review in 2016 highlighted that while the project approach of using Agile, waterfall and Lean project methodologies was sensible, the program team was inexperienced in their use. It also stated that this was the first application of these approaches to a program of this complexity within the Victorian Government. 

This approach also meant that SV’s scope was still only developed at a high level almost 12 months into its delivery. SV undertook an iterative approach to its delivery, which meant that it progressively defined requirements across interrelated workstreams such as customer need, developing the new service unit and delivering the technology. SV stated that the program implementation plans were ‛as complete as possible’ for each stage of development. 

This progressive approach allowed SV to adapt to change, but it also meant that: 

  • SV and DPC focused on delivering the current stage of the program and a specified product, not ensuring high-value transactions were onboarded and would deliver the intended benefits.
  • The decision to invest in SV was based on a vision that has changed over time. For example, DPC, in the Reform Program business case, envisaged that SV would set standards and help agencies drive digital reform. SV amended its business focus in mid-2016 to focus on competitive reform, disruption of current agency business models, and the provision of new assets to government. 
  • There was no clear line of sight of the changes to the Reform Program business case and their impact.

The business case lays the blueprint for the whole investment lifecycle. Over time, SV and DPC amended SV's scope, the transactions it would deliver, its targets and its vision. 

DTF Investment Lifecycle and High Value High Risk Guidelines—Business case states that if there are any material changes that affect the business case, the business case should be updated. Despite recommendations in Gateway reviews, DPC and SV did not update the business case, nor did they use DTF's Investment Management Standard to evaluate how well SV had delivered the expected benefits. Had they done so, it may have helped government and SV make decisions about whether to continue with the program, amend it or invest further to maximise benefit realisation.  

3.3 Benefits and performance measures

As discussed in Chapter 2, DPC's Reform Program business case listed three key benefits that SV would achieve. DTF guidelines outline that benefits should be real in nature, attainable and be a direct consequence of the proposed investment. Not all of SV’s intended benefits met these guidelines. 

In particular, the stated benefit of increased compliance with regulation was not:

  • clearly linked to the problems outlined in the business case (poor customer satisfaction, costly and fragmented services and the risk that the government may fail to meet increasing demand)
  • attainable—there was no evidence that simplifying and digitising transactions would improve compliance and regulation. 

Key performance indicators

DTF guidelines outline that a business case should include KPIs that are specific, measurable, achievable, relevant, and timely. DPC's Reform Program business case did not meet this requirement. 

DPC recognised this lack of detail and, in the business case, stated that:

'[d]ue to the complex and ongoing nature of the project, the identified benefits will be generated at differing times in the future and therefore some KPIs and measures can be more readily defined and measured than others’.

In 2018, SV revised its KPIs and targets. Despite this, the following limitations remain:

  • The benefit of increased compliance with regulation still cannot be effectively assessed through the measurements, and associated KPIs are inadequate to address this.   
  • The KPI measurements and targets are SV-centric, and do not reflect SV's performance in delivering the government-wide change envisaged by the Reform Program business case. 
  • SV does not have reliable baselines for all transactions, meaning it cannot accurately measure whether it has improved outcomes (see Section 2.5). 

SV's annual financial benefit

One of SV's anticipated benefits is improved productivity of government transactions. SV measures this through its 'annual benefit to government' KPI, which it introduced in January 2018. 

DPC's Reform Program business case predicted that SV would deliver an annual financial benefit of $61 million. However, in 2018, SV: 

  • revised the annual benefit figure down to $30.3 million, reflecting the removal of Land Victoria transactions and a lowered cost target
  • added several benefits that the Reform Program business case did not outline. 

These additional benefits were outlined in the program close report and are listed in SV’s benefit realisation framework. 

SV reports its achievement of benefits to its BoM, the DPC Secretary and the Minister for Government Services. SV reported that it achieved its total annual benefit in 
2018–19 and 2019–20. However, this is based on the additional benefits SV introduced in 2018. While an internal audit found that SV had a reasonable basis for reporting benefits across categories, we found limitations to these benefits, as illustrated in Figure 3C.

FIGURE 3C: Savings claimed by SV for additional benefits (2018–19 and 2019–20) and the limitations to these benefits

Benefit and description Savings claimed 2018–19 ($ m) Savings claimed 2019–20 ($ m) Limitations

Avoided costs

Cost savings to agencies that have chosen SV instead of a market solution, including from:

  • not undergoing procurement processes
  • using a solution that is cheaper than a market option.

SV estimates this value through market research or agency identified costs 

$45.54

$6.78

  • SV assumes the agency would have incurred the cost if SV was not available. This means the costs are largely theoretical—the government may not have approved funding for a market solution, or the agency may have used internal resources to deliver change. For example, in 2018–19, SV claimed $43 million for the avoided costs of developing software for Solar Homes transactions. This assumes that the government would have funded this alternative cost. 
  • It does not factor in the 'unavoided' costs to agencies and government such as the cost of maintaining backend systems or retaining their own online services. 
  • SV has refined its approach. Prior to 2020, SV did not have a consistent method for calculating avoided costs. 

Re-use benefits

SV can re-use its digital infrastructure, such as cloud based platforms and common or centralised capabilities (such as digital licences, payment gateways and digital support)

This results in a reduction in the cost and time it takes SV to set up and deliver digital transactions

$24.56

$25.55 

  • Including both the re-use benefit and avoided costs is double counting. For example, SV has claimed the avoided cost of building digital capability for the Solar Homes transactions as well as the savings from reusing its technology and processes to provide a solution. 
  • The reason SV can offer a solution for less than similar commercial alternatives is because it can re-use its technology and processes.
  • SV was set up to build common capabilities and its ability to deliver more services at less cost is reflected in the overall cost of transaction. 
  • The benefit is achieved when the capability is actually re-used. 

Reduced transaction cost

SV outlines that transitioning services to SV reduces the cost for agencies to provide transactions

$1.12

$5.80

This does not capture the true cost of the transaction, or whether SV has, in fact, reduced the overall cost of the transaction.  

  • For most transactions, SV does not capture the ongoing costs of the transactions such as agency staff time reviewing the relevant application, or other indirect costs. 
  • It also does not capture SV’s operating costs for delivering each transaction. Instead, the cost per transaction is based on SV’s overall budget at an aggregated level.

Time returned to customer

This is the financial value of a customer's time by using SV's digital transactions

$1.66

$4.56

  • While this is a valid benefit, it does not deliver a direct financial benefit to government and benefits reporting should make this clear. 

Payments

This reflects the cost per transaction saving to government from customers using PayPal over more expensive credit card transactions and savings from the renegotiation of a government contract

$1.98

$2.00 

  • The ability to achieve significant financial benefits is uncertain as it depends on agencies offering, and customers using, PayPal.
  • It may be relatively short term, as payment systems are continually evolving. 

Intellectual property

Reusing intellectual property across government such as SV templates and research

$0.37

$0.05

  • This benefit is relatively low value. SV looked at the cost of obtaining the research or documentation and apportioned a percentage of the value of providing this to another agency (up to 100 per cent). The true value is dependent on whether the agency uses such information and whether it falls clearly within the category of intellectual property. This is not clearly demonstrated by SV. 
  $75.24* $44.75*  

Note: *These figures do not add up due to rounding
Source: VAGO.

Most of the additional benefits that SV states it has achieved have been generated from a small number of transactions. For example, between 2017 and 2020, $65 million of SV’s reported annual benefit was from avoided costs and re-use benefits for Solar Homes transactions alone. We outline these in Appendix G. 

SV is not a commercial enterprise, and most services delivered to date have been free or low cost to agencies. This provides agencies who are unable to afford a market solution with an alternative option to digitise its transactions. 

This is one of the benefits of a centralised digital service. However, it does not mean that the government is saving money by using SV, as neither DPC nor SV:

  • measure the 'unavoided' costs of SV (the cost of agencies not using SV and instead developing their own systems)
  • calculate the cost of agencies running their own parallel systems. 

The fragmentation and duplication of systems was one of the problems SV was intended to address. There is no evidence that SV has done this.

Total financial benefit reported by SV

DPC predicted that most of the financial benefits SV would deliver would be transactional in nature. As Figure 3D shows, SV has not achieved these transactional benefits. 

FIGURE 3D: SV’s reported annual benefit by financial year

  Business case estimates SV reported benefits
2017–18 ($ million) 2018–19 ($ million) 2019–20 ($ million)
Transactional benefits 52.7 0.1 1.1 5.8
Other benefits(a) 7.8 3.5 74.1 38.9
Total annual benefit $60.5m $3.6m $75.2m $44.8m*

Note: (a) Other benefits include savings from streamlining service centres, avoided agency costs, re-use benefits, intellectual property benefits, time returned to customer benefits and payment benefits. 
Note: *This figure does not add up due to rounding
Source: VAGO analysis of SV data.

The annual benefit is also far short of its potential benefits of between $120 to $150 million per year that DPC predicted SV could achieve by year five if SV captured all Victorian Government transactions.

3.4 Governance and performance reporting

SV did not submit a new business case or clearly re-baseline its costs and benefits when it became clear that it would not deliver the original scope of transactions identified in the Reform Program business case. Instead, SV reported its implementation through:

  • Program Implementation Plans
  • end of stage reports and program close reports provided to DPC's Secretary, the then Special Minister of State, and audit committees.

As outlined in Chapter 1, SV had a mix of program and project governance. It was also subject to oversight through DPC's ARMC and gateway reviews.

Audit committees and Project Control Boards

Audit committees play a key accountability role in an organisation’s governance framework. While SV's management team is ultimately responsible for its operations, the audit committee independently reviews and provides assurance on key areas such as governance and risk. 

As outlined in Appendix H, DPC’s ARMC has overseen SV's performance since 2015, and had a dedicated SV subcommittee until May 2017. 

In September 2017, a separate PCB was established to ensure that SV: 

  • was on target to deliver its agreed outcomes and expected benefits
  • had appropriate risk management plans, which were followed
  • optimised its cost value and delivered the program in a timely manner.

Since 2018, SV has also had its own internal audit function, which provides assurance to SV, the DPC Secretary and the ARMC. 

The ARMC and PCB’s early meeting minutes show that these committees focused on establishing governance and ensuring that the program's technical component was delivered. It also noted that the original project timeframes had slipped and that these were replaced by new milestones in the Program Implementation Plans. 

SV did not begin realising benefits until its third year (2017–18) and even then, only with a small amount of transactions in a testing phase. However, it was clear that the benefits were at risk earlier in the program. 

In 2016, the PCB raised questions on benefit realisation and: 

  • approved SV's proposal to provide a paper on how benefits would be re baselined
  • requested further information to understand stakeholder reporting. 

The PCB did not adequately address SV's risk of achievement of benefits and particularly whether it remained economically viable with a changed transaction mix. It also did not recommend other options (such as further influence by DPC) to improve stakeholder engagement and onboarding. 

Reporting during the implementation phase

In 2016, the ARMC subcommittee and PCB recognised the need for SV to provide clear reporting on the benefits and challenges that were in focus. However, the PCB still had concerns about the level of detail reported to it and measurability of SV’s benefits in June 2017. The PCB also highlighted the need for re-baselining benefits and linking them back to the business case. 

SV completed a program close report when its implementation phase concluded in June 2018, which the PCB reviewed, and which was provided to DPC's Secretary and the Special Minister for State. This report did not provide a clear comparison of the benefits SV had delivered against the expected benefits outlined in the Reform Program business case. 

The program close report …

But it did not …

stated that ‛modelling shows that the business case financial benefits are achievable and will continue to grow as the program progresses into its second horizon’.

clearly highlight the shortfall between the forecast NPV ($5.96 million) compared to the NPV anticipated in the Reform Program business case ($97 million).

listed all benefits (including new benefits identified) and a projected 10-year NPV.

isolate the business case benefits and provide clear reporting against them, particularly with the changed transaction bundle that was identified. 

detailed its approach to customer feedback.

note how this was different to the central digital complaints channel—a project within scope of the Reform Program business case—which was designed to result in fewer complaints to the Victorian Ombudsman.

SV's current performance management framework

In 2018, SV became an administrative office and gained powers and responsibilities under the Service Victoria Act 2018. This means that SV's CEO reports to DPC's Secretary about the general conduct and management of SV, and to the Minister for Government Services for the exercise of its functions under the Service Victoria Act 2018

SV is not required to report publicly on its annual benefit, volume of services and transactions or cost to deliver transactions. Its only public reporting measure is customer satisfaction. This reduces transparency of SV’s performance.

SV’s benefits realisation plan sets out KPIs and measurement metrics for each of its identified benefits. SV monitors and reports on these benefits through a benefits realisation register. This is an Excel spreadsheet that tracks SV’s KPIs against its actual values for each financial year. SV uses the register to inform its BoM about significant deviations from planned targets and timelines. The register also supports decision making and risk management processes.

SV does not have an annual plan for its performance. Instead, it uses a scaled Agile planning framework. As Figure 3E shows, this framework incorporates six-monthly and quarterly plans with monthly, fortnightly and daily reviews. 

FIGURE 3E: SV’s performance management approach 

Frequency Type Forum and description

Annually

External reporting

SV reports its BP3 measure of customer satisfaction in DPC’s annual reports and the Budget Papers. 

SV is also obliged to annually report to the Victorian Information Commissioner and the Health Complaints Commissioner on its compliance with information and privacy legislation. 

Six-monthly

Transaction catalogue

The BoM assesses new opportunities and reviews items from the transaction catalogue every six months.

Monthly

Benefits review

The BoM meets monthly to track SV’s progress against its benefits using a benefits realisation register.

Fortnightly

Sprint planning

Teams meet fortnightly to undertake team-based activity planning and monitoring using Kanban boards(a)

Weekly

Priority and resource monitoring

SV’s Portfolio Steering and Program Committee, which includes its executive leadership team, reviews SV’s operational performance weekly.

Daily

Daily scrums and executive unblocking 

SV’s executive leadership team holds daily stand-up meetings using Kanban boards to review factors that are affecting SV’s performance and service delivery.

Other

Transaction dashboards

SV’s executive team uses Google Analytics to monitor transaction performance in real time.

Note: (a) Kanban boards are an Agile project management tool used to show work status, optimise workflow and highlight issues. SV uses a virtual board called JIRA. 
Source: VAGO, based on SV’s reporting framework.

SV's Board of Management

As outlined in Chapter 1, SV's BoM is responsible for overseeing SV’s operations and functions. The Gate 6 review noted that SV's governance structure is fit for purpose and highly focused on delivering benefits. We saw evidence of regular reviews of the program benefits and opportunities to onboard new agencies. However, given the lack of mandate to use SV, its approach is focused on smaller agencies commencing new digital transactions rather than the high-volume and high-value transactions originally intended by DPC.  

3.5 Risk management by DPC and SV

DPC outlined the top 14 risks to the program in its Reform Program business case. 

In the Reform Program business case, DPC presented, discussed, and assessed 32 risks. While it outlined mitigating actions for the top 14 risks, it did not include a rigorous risk management plan. SV’s initial program implementation plan in 2015 introduced its risk management plan and risk register. 

Top risks listed in the Reform Program business case included a lack of buy-in from agencies and that government leaders would not mandate the program. It also included a risk mitigation strategy that the project would not continue without a mandated vision. This did not occur.

One of the key risks of this program was the complexity and knowledge needed to deliver a project of this nature, particularly given the use of Agile and Lean methodologies. Gateway reviews found that the project team lacked a detailed knowledge of these methodologies, particularly in the implementation stage.

SV embedded its Agile and iterative approach in its risk management approach. It outlined risks in the most detail for the phase that the project was in. It provided less detail about risks associated with its later stages. This reduced DPC and SV's ability to clearly predict future risks and articulate plans to mitigate them in a timely way. 

Figure 3F shows how SV’s core and significant risks changed as the program progressed.

FIGURE 3F: SV’s core risks over time

FIGURE 3F: SV’s core risks over time

Source: VAGO, from SV’s risk registers and program implementation plans.

The 2018 Program Assurance Review (Gate 5 equivalent review) found that SV had an appropriate and active risk management approach, including for its implementation phase. SV’s current risk management framework, DPC’s risk management framework and the Victorian Government Risk Management Framework require agencies to outline the steps they will take to identify, monitor and treat risks. However, SV's risk register had several gaps. 

SV's risk register

Was flawed because it …

At the outset (in 2015)

Did not include the following risks:

  • The government could delay or reject legislative changes around the establishment of SV, transfer of functions or identity verification.
  • Agencies might not agree on who owns underlying customer or transaction data. 
  • The agency data that is needed by SV to onboard the transaction could be poor quality.
  • There could be delays engaging with agencies due to either agency resources or agencies’ business-as-usual operations and other projects. 
     

In 2017

  • Allocated some risks to a team (not an individual).
  • Identified risks that were broadly expressed.
  • Did not include detailed treatment plans and timeframes.

Current (in 2020)

Does not include the following risks:

  • Knowledge could be lost due to SV being a small team and dependant on individuals for key capabilities. 
  • SV's relocation to Ballarat may result in staff departures, be costly or delayed.
  • SV does not achieve benefits due to: 
    • not enough agencies using SV due to inadequate stakeholder engagement by SV or lack of a government supported mandate
    • customers not using SV due to insufficient brand recognition within the community.

SV's 2018 program close report suggested that only one strategic risk remained in October 2018—achieving the benefits targets. There were no stakeholder risks listed in SV’s enterprise risk summary despite this clearly being an ongoing risk to the SV.

DPC’s risk register also did not include a specific risk about SV until June 2019, which Figure 3G shows.

FIGURE 3G: DPC’s risk register—controls and treatment actions for SV risk

Risk: That SV might fail to fully realise the forecast benefits associated with bringing state government transactions together on a single digital customer platform
Existing controls Treatment actions
  • SV was established as an administrative office related to DPC, which provides greater certainty to its longevity. The Service Victoria Act 2018 also supports this.
  • SV's Executive Management Committee monitors SV’s benefits weekly.
  • SV implemented a customer research program of work, which ensures that SV continuously improves its product.
  • SV designs transactions with benefits in mind.
  • SV makes build decisions based on achieving benefits.
  • SV’s service engagement lead actively works with agencies to maximise throughput for SV’s transactions, while minimising their use of other channels.
  • SV continually reviews customers’ live transaction journeys to make sure that it is constantly meeting customers’ needs.
  • SV continuously improves its service offering by introducing new capabilities to its platform.
  • SV is developing its in-house capability to reduce overheads and increase institutional memory.
  • SV proactively seeks transactions from agencies by competing for tenders (for example, for the Solar Homes Program) to onboard.
  • SV has a business development team that proactively seeks the highest benefit transactions to onboard.
  • SV is conducting an audit of its benefits, including its benefits methodology, to provide greater certainty about its approach.
  • SV has partnered with DPC to strongly advocate for agencies to use SV and proactivity identify opportunities for departments and agencies to onboard their transactions.(a)

Note: (a) DPC added this treatment action to its register in 2020.
Source: VAGO, from DPC’s risk register.

Since SV's establishment, the Gateway reviews and SV’s risk management plans have identified its benefits realisation and stakeholder management as key risks. 

Benefits realisation

The Reform Program business case highlighted the risk that SV may not realise its expected benefits. SV included this in its risk management plans and implemented treatment plans to address it, such as:

  • identifying alternative sources of benefits and a decision-making framework for agreeing to future transactions
  • developing a benefits realisation plan and early warning measures to identify benefits that were not being realised.

However, despite early recommendations in its Gateway and Program Assurance reviews, SV did not make agreements with stakeholders to lock them in at an early stage. It also did not confirm its scope changes or outline how they would impact its benefits by amending the business case.

SV did not develop a revised business case to recalculate benefits given the altered project scope and advise government of the results. 

SV has taken a cautious approach to benefit realisation. Its rationale has been to prove its product prior to delivering at scale. This is designed to prevent typical ICT project failures where early high expectations are not met. This has meant that SV does not have a strong public profile. The documentation that agencies provide to customers concerning transaction options has limited SV branding. For example, VicRoads registration renewal notices do not list SV as a payment option. Feedback from stakeholder agencies supports the view that SV’s brand is ‛underdone’, with customers often regarding SV as an extension of the agency and not visiting SV as the first portal to find transactions. 

Gate 6 identified this lack of brand recognition within the community as a risk. Yet SV does not have a marketing or brand recognition strategy.

Stakeholder management

The volume of services that SV onboards and the number of transactions it delivers is crucial to it delivering its expected benefits. This requires agency commitment and buy-in, particularly in situations where it is not mandatory to use the service.

SV had high-level stakeholder support during its initial development. However, this did not translate to the scope or volume of transactions it onboarded.

In other jurisdictions, digital transactional services have developed at different paces and with different models. For example, Service NSW was announced in 2012 and leveraged existing technology and services, such as Transport for NSW and Births, Deaths and Marriages NSW. By its second year it was delivering 13.5 million transactions, but its development and overhaul of dedicated end-to-end transactions progressed over time.  

myGov is another example. While it is not mandatory for agencies to use myGov, it almost doubled its expected number of customers by its third year due to key large agencies such as Centrelink and the Australian Taxation Office coming on board. However, the Australian National Audit Office found that myGov’s effectiveness was hampered by government services not joining it. 

Gateway reviews between 2014 and 2018 identified concerns from stakeholders about SV’s benefits realisation, delivery model and achievability. While SV developed a stakeholder engagement plan in 2015, this was only a high-level strategic document. It did not outline how agencies would be signed up to SV or specific communication strategies. 

Initially, SV focused on engaging with VicRoads because its transactions were in Horizon 1's scope. It set up a VicRoads transition board, which allowed a dedicated forum of senior stakeholders to consider strategies for transitioning services to SV.

Despite this, VicRoads has not onboarded the scope or scale of transactions that the Reform Program business case predicted. As shown in Appendix E, SV did not revise the transactions that it would deliver until 2018. It has also not delivered all of these revised transactions. Issues that have impacted SV’s ability to onboard transactions included:

Issue

Result

It was not mandatory for agencies to onboard services with SV.

Agencies decided not to onboard all transactions initially scoped.

SV’s first formal agreements with agencies were not signed until late 2017.

SV did not lock in key stakeholders at an early stage.

SV did not develop detailed stakeholder communication plans early in the program.

SV’s communication with agencies was inadequate.

There were delays in SV's maturity and product development. 

Agencies continued with their own digital projects, which reduced their need to onboard transactions with SV.

Some agencies were concerned that using the new service would adversely impact their resources.

Agencies were reluctant to onboard.

SV is required to quote alongside the market in most circumstances.

SV's current approach to gaining new services is:

  • not efficient for the government or the open market
  • resource intensive and with increasing growth of services, would not be sustainable in the long term. 

SV's funding has been consistently short term. 

This has impacted stakeholders’ trust in relation to SV’s longevity.

One of the Information Technology Strategy 2016–2020’s priorities is to re-use technology where possible. SV provides an opportunity to deliver on this, but it requires clear engagement and buy-in from agencies. 

After onboarding Solar Homes transactions in 2019, SV focused on onboarding new services in addition to high-volume legacy services.

Identifying potential transactions

SV has developed a process for identifying suitable transactions and liaising with the relevant agencies. 

SV uses an Excel spreadsheet to identify transactions that have a high value to both SV and the agency and are suitable for use on its platform. This spreadsheet lists:

  • most Victorian Government transactions and the agency involved
  • each transaction’s annual volume (where available)
  • if the transaction includes identity verification or payment options
  • whether the service needs physical or digital evidence of transaction (for example a WWCC card or fishing licence) 
  • any key decisions or actions SV has made or requires.

SV then uses this list to develop a pipeline of opportunities that it prioritises and reviews every six months.

Once SV identifies a transaction or service, it follows its service improvement framework to transition from identifying opportunities to delivering transactions. This is a process where SV progressively obtains more information, such as the benefits, technical options, funding and timing before submitting a proposal to the agency, entering into a memorandum of understanding (MoU) and building the transaction. 

SV’s Excel spreadsheet contains 10 separate Victorian Government transactions that are high volume and involve identity verification (equating to almost 3.4 million transactions per year). Of these 10 services, SV has:

  • delivered one (new WWCC applications)
  • a further two on its delivery roadmap (Residential Tenancies Bond Authority and WorkSafe Victoria licensing applications)
  • six that are awaiting a government decision (all VicRoads transactions)
  • one that SV cannot deliver due to a decision by the agency (land tax payments and amendments). 

There are several other high-volume transactions, such as Fines Victoria, myki and local government transactions, that SV is not delivering. 

SV also relies on DPC to identify opportunities for delivering services. However, we saw limited evidence of DPC encouraging agencies to use SV prior to 2020, when it advocated for SV to have a role in the reform of the fines system. While it is an agency or department's decision to use SV, there is a benefit in DPC driving the government priority of reusing technology and services where possible. SV stated that it also:

  • checks if departments’ contracts for ICT systems are about to expire (if they are, SV approaches them to discuss opportunities for onboarding their transactions)
  • maintains contact with DTF to check which agencies are requesting investment in digital technologies and uses this to focus engagement discussions
  • monitors government agencies’ organisational charts. Changes to executive or leadership structures can create new approaches for digital delivery. If staff change, then SV approaches the new staff to present its offering and revisits any opportunities to work with the agency. 

However, decisions to invest in SV should be based on evidence of value and be driven by the Victorian Government strategy. This requires clear senior executive buy in and sponsorship to ensure that Victoria moves away from the fragmented and duplicated transaction delivery model first identified in 2015. 

Memorandums of understanding 

Platform availability reflects the amount of time that the service is online or 'available'. 99.99 per cent availability means that in any given month, a service can only be offline for four minutes.

Once an agency agrees to use SV to deliver its transactions, SV and the agency enter into an MoU. This is a non-binding document that records the parties’ intent to work together. It outlines their roles, responsibilities, and expectations. 

SV monitors compliance with MoUs and operating agreements through its transaction and operations stakeholder committees, which include representatives from the relevant agency and SV. These agreements outline a dispute resolution process, which, if unresolved, is escalated to the Executive Committee. 

All of SV’s MoUs with agencies (except for one) are supported by:

  • a non-binding operating agreement, which outlines the specific transactions that SV has agreed to deliver, timing and responsibilities
  • transactions journeys, which set out the steps involved in onboarding transactions. 

There is a risk that agencies may renege on an agreement if it is non-binding. 

We analysed SV’s MoUs with six agencies—Solar Homes, Victorian Fisheries Authority, VicRoads, Victoria Police, WWCC Victoria (two MoUs) and Ambulance Victoria. We made the following observations: 

Out of the seven MoUs …

had …

All

funding variations. Funding varied from a neutral funding position (where both the agency and SV are responsible for the cost of onboarding a transaction) to SV reimbursing the other agency for additional onboarding costs. These costs are not included in the benefits but are factored into the profit and costs of SV. 

Five

KPIs around the availability of SV’s service platform (99.99 per cent availability) and incident reporting.

Two

‛ramp-up’ criteria to provide agencies with information on how to increase or decrease the number of transactions SV delivers. This was a positive addition to the MoUs as it gives both parties a framework for increasing transactions and monitoring quality.

One

no KPIs. While it listed incidents and volume of requests, it did not place a target on this.

One

been signed after the ramp up of its transactions was actioned.

None

broader KPIs around SV’s performance, such as transaction completion rates, customer satisfaction or transaction completion times. However, SV provides this information to agencies on a regular basis.

Stakeholder feedback 

We spoke to four agencies that have services with SV and three that have either used an alternative service or not onboarded transactions with SV. Stakeholders advised us that their decision not to onboard with SV was influenced by various factors including the readiness of SV at the time, internal policy decisions and the availability of other suitable technology (such as myGov). 

Agencies that use SV gave the following feedback:

Agencies said that SV has …

but can improve …

because …

matured in its technical capability and stakeholder engagement

its branding and marketing

SV does not have a current public presence and it can be confusing for customers to understand the relationship between SV and the agency

reduced implementation and transaction costs 

its calculation of benefits

it is difficult for both the agency and SV to quantify direct and indirect costs and benefits

improved customer experience and usability

its reporting

more detailed project management reporting would help agencies’ internal reporting

no ongoing operational costs, licence costs or mark-up for services

the size of its team and the ability to take on extra work

this impacts SV’s responsiveness and ability to build new services

allowed them to process more transactions quicker and with less resources

how it shares learnings across projects with agencies

these learnings may improve overall Victorian Government digital delivery 

Stakeholders told us that many agencies want to test SV’s service before ramping up their transactions. They also said that they need to have confidence in SV’s ability to deliver a transaction before increasing the redirection rate. 

While SV sets up an engagement process and meets regularly with stakeholders when it onboards a service, it does not have a method for agencies to provide formal feedback, such as through an annual survey. This is a missed opportunity for SV to identify issues from a stakeholder’s perspective and track its performance. 

Performance reporting provided to agencies 

We reviewed a sample of reports that SV provides to stakeholder agencies. SV uses Google Analytics to give agencies information on SV's performance in relation to agency transactions. The reports outline the:

  • number of customer visits to the transaction page
  • number of transactions started and completed
  • median completion time
  • customer satisfaction score for each transaction.

SV's monthly performance reports to agencies do not include information about the KPIs outlined in five MoUs, such as incidents or its platform’s service availability. While meeting minutes suggest that SV and the agency discuss incidents, it should also form part of regular reporting to agencies. 

Solar Homes raised opportunities for SV to improve its reporting in August 2019 such as including actual performance against the service level agreements and severity one incidents. However, SV has not addressed this yet.

The broader stakeholder feedback that we obtained suggests that SV’s lack of structured project reporting during the development phase can make it difficult for agencies to ensure governance over new projects. One stakeholder opted to engage with an external contractor to bridge this project management gap. 

SV’s ability to market its services, onboard and retain new transactions and ensure faith in its delivery is central to its success and whether it is a viable service in the long term. 

Back to top

Appendix A. Submissions and comments

We have consulted with DPC and SV, and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments. 

Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

Responses were received as follows:
Response provided by the Secretary, DPC

DPC response letter

DPC action plan page 1

DPC action plan page 2

Response provided by the Chief Executive Officer, SV

SV response letter

SV action plan

Back to top

Appendix B. Acronyms and abbreviations

Acronyms  
ARMC Audit and Risk Management Committee
BoM board of management
DPC Department of Premier and Cabinet
DTF Department of Treasury and Finance
HVHR high value high risk
ICT information and communications technology
KPI key performance indicator
MoU memorandum of understanding
NPV net present value
PCB Program Control Board
SV Service Victoria
VAGO Victorian Auditor-General’s Office
WWCC Working with Children Check
Abbreviations  
BP3 Victorian Government Budget Paper No. 3
Preliminary business case Improving the efficiency and effectiveness of Victorian Government transactional services Preliminary business case
Reform Program Victorian Transactions Reform Program
Reform Program business case Improving the efficiency and effectiveness of Victorian Government transactional services full business case

Back to top

Appendix C. Scope of this audit

Who we audited What we assessed What the audit cost

DPC and SV

Whether digital delivery of government services has improved customer experience and reduced transaction costs. 

In particular, we assessed if:

  • the Reform Program’s business case and implementation followed better practice guidance and ensured an effective basis for SV's performance 
  • SV is achieving its intended benefits.

The cost of this audit was $646 000.

Our methods

As part of the audit we:

  • inspected and reviewed relevant documents, including:
    • legislation, policies and guidance
    • business cases and gateway reviews
    • business and strategic plans and frameworks
    • internal audits
    • performance data and reporting
  • reviewed and analysed SV’s performance data, including (but not limited to) its transaction volume, cost, and customer satisfaction 
  • interviewed relevant DPC and SV staff 
  • consulted with agency stakeholders who have used SV’s platform or have run their own digital initiatives.

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements. We also provided a copy of the report to DTF. Unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

Back to top

Appendix D. Outcomes of the Reform Program

FIGURE D1: Horizon 1 expected outcomes

Outcome Status VAGO comment

A consistent digital experience for the customer regardless of the transaction type and department. Government leveraging cloud and mobile technologies to improve customers’ experience, access and cost

Achieved

While SV does have a consistent digital experience for the customer, this only applied to 17 customer facing end-to-end transactions available at 30 June 2020. This is a very small subset of the Victorian Government’s transactions.

Multi-purpose ‘tokens’ (evidence of the transaction undertaken) that can be used by multiple departments and agencies and can be stored digitally on phones and the web

Achieved

SV’s ‛my account’, ‘digital wallet’ and ‘reminders’ functions allow customers to store their data and transactions online and access any digital licences. 

A new government unit that facilitates service delivery and sets standards with senior sponsorship and metrics fostering good service, reduced costs and increased pace and agility

Partially achieved

SV uses Google Analytics to measure its performance.

It has also developed customer service and identity verification standards for the transactions that it delivers. 

There have been good examples of reduced transaction costs (for example, for Solar Homes transactions). However, due to multiple systems being run by both SV and agencies, there is no evidence that this has reduced overall costs.

Standardised data verification and customer record sharing, leading to reduced form filling and fewer face-to-face interactions required to prove identity and apply for services

Partially achieved

This capability is available through SV. SV has released a digital identity verification function with its Solar Homes transaction. This has led to reduced form filling. However, few transactions use identity verification and as such few customers have an identification passport.

Efficient and effective development and maintenance of legislation and policy impacting transactions

Partially achieved

The Service Victoria Act 2018 was introduced, which includes the requirement that SV must establish and comply with customer service standards. However, there is not yet a consistent whole-of-Victorian-Government policy approach to digital transactions.

Consistent use of low cost, highly reliable payment methods across all Victorian Government

Partially achieved

While SV has achieved payment contract savings, its value is limited to SV’s transactions.

A significant increase in the number of transactions completed with little or no human interaction

Not achieved

SV provided 17 end-to-end customer-facing services and delivered 814 282 transactions in 2019–20. This is approximately 1 per cent of all Victorian Government transactions.

This is not a significant increase. Further, many transactions available online are also available through another digital platform (for example, registration via VicRoads’ website).

Foundations laid for an optimised physical footprint (retail outlets, contact centres etc.) enabled by a greater digital uptake

Not achieved

SV’s business case for additional funding was not approved.

Reduced complaints, which are captured and analysed from a whole of Victorian Government perspective

Not achieved

There is no evidence that SV has reduced complaints and no evidence that these are analysed from a whole of Victorian Government perspective. While SV provides support through its online chat function, ‘Ask Vicky’, and its internal complaints team, this does not equate to less complaints about government services. Feedback from stakeholders supports the view that SV has not reduced complaints. 
Lower operating expenditure and total costs of ownership, primarily through migration of transactions, to lower cost channels Not achieved

SV is yet to achieve an overall financial benefit.

SV’s annual benefit largely depends on the volume and value of transactions it delivers. It has not migrated many high-value end-to-end transactions. As a result, its current cost per transaction is high.

Source: VAGO analysis from SV data and Reform Program business case.

Back to top

Appendix E. Reform Program business case transactions

SV did not deliver all original transactions outlined in the Reform Program business case. Figure E1 outlines the changes to the transactions over time and whether these have been delivered. 

FIGURE E1: SV's transactional delivery

Reform Program business case intended transaction and potential volume Delivered by 
2018–19?
Revised/substituted transactions (2018) Delivered by 
2018–19?
VAGO commentary
  • VicRoads new licences (260 000)
  • VicRoads licence renewal (850 000)

  • Fishing licences

SV does not offer any VicRoads licensing services. VicRoads released its own digital licensing functionality in 2020. SV substituted fishing licences to prove capability.
  • VicRoads registrations (new and transferred) (565 000)
  • VicRoads registration renewals (5.3 million)

Partial

  • VicRoads check registration

SV does not issue new vehicle registrations.

Registration renewal payments can be made on SV's website (for annual payments or where customers have previously signed up for short-term registration).

  • Boat and vehicle registration renewal transactions

  • Firearm licences (new applications and renewals) (120 000)
  • Private security licences (new applications and renewals) (18 000)
  • Other Victoria police applications such as criminal records checks and National Police checks (490 000)

  • Victoria Police records check 

SV does not deliver any of the Victoria Police transactions anticipated by the Reform Program. Instead it delivered two online reporting transactions. 

In September 2020, SV and Victoria Police signed an agreement to digitise parts of firearm applications and renewals and private security licences.

Criminal record check transactions require legislative changes.

  • Victoria Police online reporting (two transactions)

  • Births, Deaths and Marriages registrations and certificates (295 000)

  • Births, Deaths and Marriages certificates  

SV does not offer birth registration or certificate applications. It commenced testing Ambulance Victoria transactions in 2018–19.

  • Proof of age card

  • Ambulance Victoria new subscriptions and renewals

Partial

  • WWCC applications (540 000)

 

Partial

SV offers five WWCC transactions on its website (apply or renew your WWCC, update your details, change from a volunteer to an employee or notify WWCC Victoria that you are a teacher engaging in additional child-related work). 

However, SV is only the sole or primary digital provider for one of these (the WWCC Victoria notification by teachers), which was a requirement introduced by government in 2019.

  • Land Victoria Registration services (750 000)
  • Land title searches (4.3 million)

  • 14 Department of Environment, Land, Water and Planning payments 

In 2016, the government established Land Use Victoria, which ultimately decided to commercialise its transactions and not use SV.  

Note: The potential volume includes digital and non-digital transactions and is based on 2014-15 data in the Reform Program business case
Source: VAGO.

Back to top

Appendix F. Summary of SV's performance against its KPIs

FIGURE F1: Summary of SV's performance against its KPIs

KPI Measure SV baseline SV target Achievement of this target for 2019–20
Benefit 1: Improved customer satisfaction and productivity from government transactions
Reduced effort to undertake a transaction Days returned to citizen (red tape reduction) No baseline in the Reform Program business case
SV amendment: baseline of agency time to service 
No target set in Reform Program business case.
SV target: 25%
Unable to assess
No reliable baseline data
Improved customer satisfaction Satisfaction survey (perceived effort removed in 2018) Baseline of agency customer satisfaction  No target in Reform Program business case. 
SV’s BP3 target: 95% (BP3 measure)
Yes
Achieved in 2018–19 and 2019–20
Baseline of agency customer satisfaction SV target: Improvement on baseline Unable to assess
No reliable baseline data
Benefit 2: Increased compliance with regulation and increased effectiveness of government policy and regulations
Increased compliance rates with regulation Increased revenue from greater compliance Unpaid fines (annual bad debts) Reduction in unpaid fines No
This benefit was removed in 2018 
Percentage of first time transactions completed Reform Program business case baseline: percentage of applications completed first time (baseline survey) 
SV amendment: baseline percentage of agency transactions completed first time
Increase % Unable to assess
No reliable baseline data
Improved awareness and take-up of government policy Use of government services in the longer term
Use of new and existing government services via SV 
55 million transactions per year in 2014
SV amendment: increase from baseline 
2% per annum Unable to assess
The increase in SV services does not reflect an increase in overall digital use for Victorian Government transactions or increased compliance 
Benefit 3: Improved productivity of government in providing transactions
Total cost of transactions to  government Reform Program business case measure: reduced transaction cost per capita 
SV amendment: Annual benefit 
Reform Program business case baseline: total cost was $461 million 
SV amendment: baseline total cost $398 million
Reform Program business case target: 10% reduction 
SV target: $30.3 million (7.6% cost reduction) 
No
Without SV's additional financial benefits including re-use benefits, SV has not achieved this benefit
Cost per transaction  Average cost for selected transactions  Reform Program business case: 20 to 40% reduction in cost to transact for selected transactions
SV target: 30% reduction in cost to transact for selected transactions
Unable to assess
No reliable data
Change in channel mix Percentage of low-cost self service channel use 32% digital for selected transactions 50% digital end-to-end
SV target: 50% end to end for SV transactions 
No
SV has a target to deliver 50% of the transactions that customers complete for each transaction type. SV has only achieved this target for seven of its 17 transactions
It has not demonstrated a broader move to digital for whole-of-government transactions

Source: VAGO. 

Back to top

Appendix G. Benefits by transaction type

Figure G1 shows SV’s reported avoided costs and re-use benefits by transaction type for 2018–19 and 2019–20.

FIGURE G1: SV’s reported benefits by transaction type for 2018–19 and 2019–20

FIGURE G1: SV’s reported benefits by transaction type for 2018–19 and 2019–20

Note: SV has not claimed avoided costs or re-use benefits for all transaction types. However, avoided costs and re-use benefits have been included for transactions that have been agreed but not yet delivered. 
Source: VAGO analysis of SV data.

Figure G2 examines SV's performance against its transaction time baselines.

FIGURE G2: SV’s transaction times and values

FIGURE G2: SV’s transaction times and values

Note: Value of overall time returned to customers is calculated by the overall time saved for customers multiplied by the value of that time ($53.84 per hour), which is then multiplied by transaction volume. 
Source: VAGO analysis of SV data.

Figure G3 shows SV’s reported performance for customer satisfaction against baselines and Figure G4 shows SV's completion rates on a per transaction basis. 

FIGURE G3: SV’s customer satisfaction scores against baselines

FIGURE G3: SV’s customer satisfaction scores against baselines

Note: Two transactions (find a kangaroo harvester and apply for new WWCC) are not included in the figure as they do not have 2019–20 data. Baseline data does not exist for five transactions (Ambulance Victoria new membership, Ambulance Victoria renewals, Victoria Police absence from residence, WWCC renewals and WWCC Victoria teacher notifications. 
Source: VAGO analysis of SV data.

FIGURE G4: SV's reported completion rate compared against baselines

FIGURE G4: SV's reported completion rate compared against baselines

Note: Agency completion rate baselines were only available for six transactions.
Note: Data for 2018–19 is not available for three transactions and WWCC new transaction has no 2018–19 or 2019–20 data.
Source: VAGO analysis of SV data.

Back to top

Appendix H. Gateway reviews

As the government classified the Reform Program as HVHR, DTF (in consultation with DPC) arranged six Gateway reviews of SV. Figure H1 provides more information about these reviews. 

FIGURE H1: Gateway reviews of SV

Gateway review Focus On track? Findings

Gate 1 
(2014)

The proposed investment’s (the Preliminary business case) outcomes and objectives

This review found that the Reform Program was established with a strong mandate and considerable work was undertaken to build cross agency rapport. The review recommended that the program develop a clear understanding about how it would sign up agencies.

Gate 2 
(2015)

Confirm that the business case meets the business need, is affordable, achievable, has explored appropriate options and is likely to achieve value for money

This review found that while the Reform Program was supported at a conceptual level, departments reserved their commitment until they received more detail on how the program would progress. 

The review also stated that some details that would normally be expected in a business case would undergo iterative development. However, there was a need for further development in some areas such as clarifying the delivery and funding models, benefit realisation plan and stakeholder management to inform decision-making. The review recommended incorporating strategies to achieve agreement with stakeholders. 

Program assurance review 
(Gate 3) 
(2016)

Provide assurance on the program’s progress so it can proceed to the next stage

This review found that the program was complex, its scope was only defined at a high level and it was still uncertain which agencies and transactions it would deliver and by when. The review commended SV on its customer focus though. 

It also highlighted risks to the program, such as SV and stakeholders’ skills in Agile methodologies and its lack of detailed plans and stakeholder engagement. The review recommended that SV enter into MoUs with agencies.

Program assurance review
(Gate 4)
(2017)

Provide assurance on the program’s progress so it can proceed to the next stage

This review found that with the program’s procurement process complete, it now needed to focus on delivering its solutions and onboarding agencies.

It also highlighted the need for SV to establish a clear mandate and structural framework, and for these to be supported and understood at a ministerial level. 

The review raised concerns about the lack of clarity and understanding around SV’s actual and intended scope and role. It also noted that SV’s legal status had reduced its stakeholder communications and progress. The review recommended that SV enter into MoUs with agencies. 

Program assurance review 
(Gate 5)
(2018)

Provide assurance on the program’s progress so it can proceed to the next stage

The review found that SV had made significant progress since the last review. It also stated that while the program was consistent with the Reform Program business case, the scope of its transactions had changed, which had impacted its expected financial benefits.

The review anticipated that the outcomes from Horizon 1 would be ‛substantially met' by June 2018. It also predicted that SV’s transaction mix would prove its ability to deliver the technical and functional capabilities expected in the Reform Program business case. 

The review was of the view that SV should consider developing addendums to the business case; however, it did not list this as a specific recommendation. Instead it recommended that SV agree, confirm and simplify the business outcomes for the Reform Program business case investment by government to ensure there is a line of sight between the business case and the outcomes and benefits tracking. 

Gate 6 report 
(2020)

To assess if the program is delivering its anticipated benefits and if its ongoing contractual arrangements are meeting the business need

The review found that SV has delivered a repeatable and scalable digital platform with proven customer service performance on time and under budget. 

It highlighted that SV should now have the opportunity to maximise this investment but that the program is at risk because it is unfunded. 

The review found that even with approved funding, a lack of mandate will continue to impact SV’s ability to address the problem statements described in the Reform Program business case. 

Consistent with previous reviews, the Gate 6 report noted that the Reform Program business case was not maintained. While SV has been consistent in reporting against the original benefits of the business case, the benefits themselves had not been revised to reflect the changes in the transaction mix or new benefits listed.

Source: VAGO, from Gateway reviews of SV.

Back to top

Appendix I. Oversight during implementation

FIGURE I1: Oversight committees during implementation (2015–18)

Committee Purpose and meeting frequency Membership 

ARMC 

2015 to current 

To provide independent assurance and advice to the Secretary on the effectiveness of the department's (and its agencies) financial management systems and controls, performance and stability, compliance with laws and regulations, and risk management.

SV's CEO has provided ARMC with regular reports on its performance and program risks. 

ARMC met four times a year.

Four independent members appointed by the Secretary with standing invitations given to key senior executives at DPC and VAGO. 

SV ARMC
Subcommittee

 

October 2015 to May 2017 

To ensure that SV’s governance was appropriate and provide assurance to DPC, ARMC and DPC’s secretary that SV’s level of governance was proportionate to its risks.

SV ARMC subcommittee met initially every six weeks before moving to monthly meetings. 

  • Independent chair
  • SV's Senior Responsible Officer
  • Independent member

In March 2016, membership was amended to include a DPC executive to provide central policy oversight, a DTF executive for HVHR oversight, a representative from DPC's ARMC and also SV's independent assurance adviser.

PCB 

September 2017 to June 2018(a)

To ensure that SV was on target to deliver its agreed outcomes and expected benefits. 

To ensure that:

  • an appropriate risk management plan was defined and followed
  • the cost value was optimised
  • the overall delivery was timely. 

If outcomes were not achievable at the cost, the PCB was required to make decisions about making the program more efficient or recommend ceasing SV.

The PCB met monthly.

  • SV's Senior Responsible Officer (chair)
  • Two independent members—one who is also on DPC's ARMC
  • A DPC and a DTF executive to provide central policy and HVHR oversight
  • Stakeholder representative from an in-scope department or agency

Note: (a) PCB’s first meeting occurred on 28 September 2016. However, it did not establish its terms of reference until 1 May 2017. 
Source: VAGO.

Back to top