Victorian Auditor-General’s report
17 August 2011
 
Biotechnology in Victoria: the Public Sector’s Investment
 
Audit summary
Biotechnology is a collection of technologies, techniques, and tools that use living organisms and their components to make new products and to develop new industrial processes. More broadly, biotechnology is a ‘life science’—the study of the structure, behaviour and life processes of living organisms such as plants, animals and human beings.
Since 1999 to the end of the 2011 financial year, Victoria has invested $3.442 billion of public funds in the science, technology and innovation (STI) sector. Over the last decade the biotechnology sector, which is a part of the STI sector, has directly received some $722 million in public sector investment, or 21 per cent of total innovation funding. 
In 2001 the former Department of Innovation, Industry and Regional Development, now known as the Department of Business and Innovation (DBI), developed the first whole-of-government Biotechnology Strategic Development Plan for Victoria (BSDP). Two more plans followed in 2004 and 2007.
Many investments in the biotechnology sector are intrinsically worthwhile, because they are aimed at extending and improving human life. This does not, however, exempt these public investments from scrutiny of their rationale, efficacy, benefits or outcomes.
Although precise measurement can be challenging, due to the interaction of biotechnology with the innovation economy, it is reasonable to expect that:
·	the potential benefit from each public investment is identified
·	the methods chosen to achieve the benefit are relevant, appropriate and reliable
·	costs incurred are economical and represent value for money in the context
·	the outcomes of the investment are progressively measured and assessed so that achievements can be attributed, and lessons learned to inform future investment decisions.
The objective of this audit was to examine the efficiency and effectiveness of the Victorian public sector’s investments in the biotechnology sector. 
There were three sub-objectives to the audit, which were addressed by posing the following questions:
·	Is there clear alignment of programs and investments with the applicable BSDP?
·	Are program and investment outputs monitored, analysed, and reported to assess the achievement of desired strategic outcomes?
·	Have available benefits been realised?
The audit examined the application of the BSDPs as well as program design, program governance, investment evaluations, business case reviews, progress reporting, and benefits assessments. The audit also examined other documentation and data relating to DBI’s biotechnology activities, as well as advice provided to the government by the Department of Treasury and Finance and the Department of Premier and Cabinet.
The following public sector agencies were included in this audit:
·	Australian Synchrotron Company Limited
·	Australian Synchrotron Holding Company Limited
·	Bio21 Molecular Science and Biotechnology Institute (and its parent institution, the University of Melbourne)
·	DBI
·	Melbourne Health (as responsible entity for the BioGrid project).
Conclusions
In its role as the public sector agency responsible for administering the bulk of public investment in biotechnology, DBI has delivered a range of world class facilities and technology platforms for the biotechnology and broader life sciences sector. 
The eight biotechnology projects examined have directly supported life sciences researchers and practitioners and have the potential to:
·	stimulate scientific research and innovation
·	assist pharmaceutical and medical discoveries
·	attract more private investment 
·	develop highly skilled people 
·	extend existing scientific knowledge.
DBI has publicly reported that of the 16 high-level strategic goals and targets that the government set for the biotechnology sector over the last decade, ten have been met, and adequate progress has been made for targets that have not been fully met.
Notwithstanding the claimed achievement of high-level sector goals and targets, there are shortcomings in DBI’s approach to investment targeting, and gaps in its benefits measurement and monitoring approaches. In particular, DBI does not use recognised or accepted methodologies to systematically determine portfolio risk and inform investment allocations.
DBI’s administrative focus for the sector is also mainly on grant and contract compliance. Though useful in its own right, this focus does not produce performance data about whether the strategic aims of the public sector’s investment in the biotechnology sector are being delivered. Instead, the primary focus of DBI’s performance measurement is monitoring the impact of short-term stimulatory activity in the sector, without evident attention to the delivery of longer-term and strategic benefits.
As a result, after more than ten years of active investment and ‘market intervention’ in the biotechnology sector, DBI is not yet able to objectively demonstrate the development of a cause and effect relationship between its investments in the biotechnology sector and the results achieved in that sector. It is, however, reasonable to attribute at least part of the achievement of BSDP targets to investments made by the public sector.
Against a background of typically long lead times for the full benefits from such investments to become evident, it is now timely for DBI to address these issues so that progressive assurance can be provided that its portfolio of investments in biotechnology is both targeted to where it is needed and is achieving its intended purpose.
Further, because there are no measures or targets that could be used as a benchmark against other leading international biotechnology locations, it is not possible to conclude on the aspirational goal that Victoria is, or will be, ‘one of the world’s top five biotechnology locations’.
Findings
Based on our review of eight representative investments in the biotechnology sector, there are consistent systemic issues in relation to measuring benefits. These issues can be summarised as follows:
·	monitoring and evaluation of benefits is limited
·	benefits from investments are not always sustainable
·	DBI can not systematically demonstrate whether benefits have been realised.
Monitoring and evaluation of benefits
Because DBI’s benefits measurement and management framework is not comprehensive or robust, DBI is not able to objectively demonstrate whether benefits have been achieved from public sector investments in biotechnology.
Sustainability of realised benefits 
Notwithstanding the achievements recorded, DBI’s preferred funding models do not explicitly require the identification and pursuit of sustainable future benefits. 
Achievement of benefits
In terms of benefits measurement, we observed:
·	reporting is at a very high level and often based on grantee data that has not been validated
·	a baseline of a current or ‘before’ state is not set so that the ‘after’ state can be accurately measured
·	data collected by DBI has been focused on sector-level metrics that do not measure the efficacy of individual grants
·	retrospective studies commissioned by DBI have methodological weaknesses in that they use general economic ‘activity’ as a proxy for specific public sector investment ‘effectiveness’.
In consequence, reliable conclusions cannot be drawn regarding the extent to which intended objectives are being met in a timely and cost-effective manner.
Recommendations
Number	Recommendation	Page	
1.	The Department of Business and Innovation should develop an overarching biotechnology strategy to: 
·	guide whole-of-government activities and investments in the sector
·	link the various funding programs to optimise investments’ contribution to the agreed goals and targets for the sector
·	set cause and effect measurement criteria that will allow for progressively more objective assessment of the achievement of goals and strategies.	17	
2.	The Department of Business and Innovation should adapt an investment portfolio allocation model to determine whether there is alignment of investments in the biotechnology sector with the Department of Business and Innovation’s desired risk and return profile.	32	
3.	The Department of Business and Innovation should further develop its grant management processes for biotechnology investments by:
·	setting, for each of the investments that it funds, performance targets that relate to strategic objectives 
·	validating information provided by grant recipients
·	embedding benefits planning in the funding approval process.	32	
Submissions and comments received
In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994 a copy of this report was provided to the Australian Synchrotron Company Limited, Australian Synchrotron Holding Company Limited, Department of Business and Innovation, Melbourne Health, and the University of Melbourne with a request for submissions or comments.
Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments are included in Appendix C.