Victorian Auditor-General’s report
23 March 2011

Effectiveness of Small Business Victoria’s Support Programs 
Audit summary
The small business sector makes a vital contribution to the state’s economy. There are over 470 000 small businesses which account for 96 per cent of all businesses. These employ 43 per cent of the private sector workforce. It is estimated that the sector accounts for 35 per cent of the gross state product.
In August 2006, Time to Thrive (TTT), the $98 million small business policy statement, outlined the overall framework to support the sector. Various departments and agencies across government were involved in the delivery and implementation of TTT programs.
Small Business Victoria (SBV), a branch within the Department of Business and Innovation (DBI), previously the Department of Innovation, Industry and Regional Development, provides advocacy for small businesses and develops strategies to address issues impacting on the small business environment. 
This audit focused on TTT’s eight programs delivered by SBV: the Business Victoria website also known as World Class Service (WCS); the Fair Payments Policy Advocacy; and a suite of six support programs designed to assist current and prospective small business operators to start or grow their businesses. Four-year funding for these eight programs comprised $38.4 million of the total $98 million TTT budget.
The objective of this audit was to assess the effectiveness of SBV’s support programs. The audit examined whether:
·	business cases underpinned the development of support programs, and whether these were effectively designed to address identified sector needs
·	monitoring and evaluation of programs were regular and results used to inform program management and delivery
·	the objectives of the support programs were being achieved.
Conclusions
The TTT and its two major support programs, the WCS and Fair Payments Policy Advocacy, were appropriately planned by SBV. They were underpinned by robust business cases which included an analysis of the options considered to address the identified needs. 
On the other hand, the six remaining programs did not have business cases analysing the options from which they were selected. This meant that there was limited assurance that these six programs represented best use of their $20.4 million budget allocation.
All programs were appropriately implemented by SBV. 
All programs were monitored and evaluated regularly to inform program management and delivery. SBV can improve its evaluations by using efficiency indicators and more objective effectiveness indicators, in addition to relying on the feedback of program participants.
The programs’ objectives were achieved over the four-year period 2006–10. Participating small business operators, both current and prospective, found that the programs assisted them in obtaining relevant business information and making better‑informed business decisions.
While the support programs were well received, participation rates were low. Given the significant infrastructure created through WCS, there is potential for the state to get better leverage from its investment through increased participation rates.
The low rates of participation means that a significant number of current and prospective small business operators have not benefited from subsidised assistance that could better prepare them to start or grow their businesses.
Findings
Effectiveness
The programs’ objectives were achieved over the four-year period 2006–10. The objectives were to:
·	equip small business operators with the skills and relevant information to better and more efficiently manage their business
·	assist those intending to start their businesses to be more informed of regulatory requirements
·	increase the proportion of small business invoices paid by government agencies within 30 days.
SBV evaluations revealed that:
·	the proportion of small business invoices paid by government agencies within 30 days increased from 77 per cent in 2004 to 90 per cent in 2009
·	small business operators who participated in the programs reported that they gained skills and relevant information to manage their businesses more efficiently
·	prospective small business operators who participated in the programs reported that these assisted them in gaining better understanding of regulatory requirements.
Australian and international studies indicate that there is a direct correlation between business growth and participation in support programs.
In terms of program uptake, less than 6 per cent of Victorian small businesses access SBV’s workshops, seminars, festival events and/or mentoring services. This participation rate, while comparable with those in other Australian jurisdictions, is low. A higher participation rate would not only benefit the sector, it would also provide a better return on government investment in the programs, such as the WCS website.
World Class Service estimate of reduction in regulatory burden
In 2006, the Department of Treasury and Finance (DTF) estimated the total regulatory burden across government at $1.03 billion per annum. 
In 2009, SBV reported that WCS’s annual contribution to the reduction of regulatory burden was $33.3 million. The Victorian Competition and Efficiency Commission (VCEC), the agency tasked to independently verify computations of this nature, was unable to verify this estimate because SBV did not use the endorsed computation methodology—the Standard Cost Model. VCEC noted that it ‘considers the savings claimed [to be] significantly overstated’.
DTF has since replaced the Standard Cost Model with Regulatory Change Measurement as the required methodology in computing reduction in regulatory burden. SBV is currently preparing a computation under the new Regulatory Change Measurement model.
Program planning and implementation 
Business case and implementation plan
The development of TTT was appropriately informed through an extensive consultation process which identified small business sector needs.
TTT and its two principal support programs, WCS and the Fair Payments Policy Advocacy, were underpinned by robust business cases. Of the $38.4 million allocated for all eight programs, WCS received $18 million for 2006–10 and an additional $10.75 million was allocated for 2010–13. Ongoing costs starting from 2013–14 were estimated at $4 million per annum. The Fair Payments Policy Advocacy program had no budget allocation.
The six remaining programs, however, did not have business cases analysing the costs and benefits of the range of options from which the programs were selected. SBV advised that other options were in fact considered, however, this analysis was neither documented nor included in the TTT business case.
The absence of documentation evidencing the consideration of possible options meant that during the planning phase of these programs, there was less assurance that the total of $20.4 million allocated over a period of four years represented the best and most cost effective way to address the identified needs.
Implementation plans developed for TTT and its eight support programs outlined how identified needs were to be addressed. These plans clearly addressed accountability arrangements, risk management, expected outcomes, communication, reporting cycle and performance measures.
Small business support programs by other agencies
As the lead agency advocating for the sector, it was important for SBV to have a good understanding of all support programs available to Victorian small businesses. However, as SBV had not conducted a thorough mapping and analysis of these programs across government, there is limited assurance that duplication and inefficiencies in the delivery of support programs were minimised.
Time to Thrive Performance Monitoring Framework
SBV developed the TTT Performance Monitoring Framework (the framework) in conjunction with DBI’s Strategy and Policy unit, to track how TTT is achieving its objectives. The performance measures identified in the framework included relevance, efficiency and effectiveness indicators.
SBV advised, however, that resourcing and operational issues did not allow it to use the framework to evaluate TTT as a whole. This was a missed opportunity. Using the framework would have complemented the program-specific evaluations, and provided relevant information on:
·	how the programs worked together to achieve TTT objectives
·	key lessons learned and opportunities for improvement in the design and delivery of future policy programs
·	how efficiently resources had been used in the delivery of TTT.
Program monitoring and evaluation
SBV regularly monitored and evaluated its programs and used evaluation results to inform program management and delivery. 
From 2006 to 2010, SBV planned and commissioned 21 evaluations across the eight support programs. With the exception of the eChange program which was evaluated once in its two years of implementation, all the other programs were evaluated at least twice between 2006 and 2010. 
SBV’s evaluations focus primarily on participant feedback as the measure for program effectiveness. While this is an appropriate evaluation approach, more objective performance indicators would allow SBV to better measure programs’ effectiveness. For example, the indicators could have included measuring the net gain in the number of small businesses correlated with the number of start-ups assisted by support programs.
Moreover, the TTT support programs are not measured against efficiency indicators that correlate outputs to the resources required to deliver them. Reporting on efficiency would provide assurance that the support services provided to small businesses are cost effective. 
Recommendations
Number	Recommendation	Page	
1.	Given the correlation between business growth and participation in support programs, Small Business Victoria should more actively promote these programs.	14	
2.	Small Business Victoria should collect and collate data on the demographics of registered users for the World Class Service website to identify small businesses from the rest of the users.	14	
3.	For future funding proposals for support programs, Small Business Victoria should include an analysis of the range of options from which the endorsed programs were chosen.	22	
4.	Small Business Victoria should establish efficiency indicators for monitoring and evaluating support programs.	29	
5.	Small Business Victoria should apply its evaluation framework to the overarching Time to Thrive strategy. 	29	
6.	Small Business Victoria should develop more rigorous performance indicators to measure the effectiveness of its support programs.	29	

Submissions and comments received
In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994 a copy of this report was provided to the Department of Business and Innovation with a request for comments or submissions.
Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments however, are included in Appendix A.