Victorian Auditor-General's Report 
12 December 2012 
Learning Technologies in Government Schools
Audit summary
Information and communications technologies (ICT) are now extensively embedded into our modern lifestyles. Many innovative technologies are emerging, and teaching practice and content is now increasingly expected to be delivered ‘anytime, anywhere’.
Governments across Australia and in most other developed countries invest heavily in ICT for use in schools. 
Over the past nine years, the Department of Education and Early Childhood Development (DEECD) has spent around $1.6 billion on the provision of ICT services for government schools. This amount excludes Commonwealth funding and schools’ own direct ICT expenditure. DEECD estimates that it will spend a further $2.6 billion over the next 10 years on ICT investment for government schools.
The term ‘learning technologies’ refers to a broad range of ICT tools that can be used to support learning, teaching and assessment. These tools include interactive boards and learning materials as well as networked technology such as online learning environments. Learning technologies also include devices such as handheld, desktop or laptop computers.
This audit reviewed the effectiveness of the application of learning technologies in government schools. In particular, the audit focused on two key DEECD learning technologies initiatives: VicSmart and the Ultranet.
The audit assessed in detail whether DEECD has:
·	conducted robust research on the most appropriate use of ICT in government school teaching environments
·	developed effective strategies to implement learning technologies in government schools and supported teaching staff to adopt these technologies
·	monitored achievement of desired outcomes, including realisation of purported benefits described in business cases used to support funding decisions for VicSmart and the Ultranet.
Conclusion
DEECD’s Digital Learning Statement (the Statement) does not provide a clear action plan or framework for investment in learning technologies. There is no supporting strategy or further detail to the Statement. This means that departmental staff and school leaders have little guidance on how future learning technology initiatives can be appropriately planned and integrated to build upon past and present ICT investments.
Planning for the VicSmart high-speed fibre-optic network for all government schools was underpinned by a robust needs and options analysis. Although the project was a less complex infrastructure rollout, it was well executed and is delivering its desired benefits. The high-speed connectivity that VicSmart provides is a key enabler of current and future digital learning in government schools.
In contrast, the Ultranet, the Statement’s key foundation plank and key enabler, was poorly planned and implemented. Six years after its announcement as a government priority, it is yet to achieve expected benefits for students, parents and schools. It is significantly late, more than 80 per cent over its first announced budget, has very low uptake by users, and does not have the functionality originally intended.
This audit identified a number of serious probity, procurement and financial management issues surrounding the Ultranet project. DEECD’s tender process lacked rigour and was seriously flawed. There is little confidence in the costing and financial management practices around the Ultranet project, and limited assurance that the selected outcome represented value for money.
Findings
Digital Learning Statement
The Digital Learning Statement—the government’s current policy document on the use of learning technologies—was not informed by robust and comprehensive research and does not make a clear and cogent case for government investment in learning technologies. 
The Statement does not deliver on the directive in the 2008 Blueprint for Education and Early Childhood Development to provide a plan of action to use learning technologies in teaching and learning. A review of DEECD’s advice to the Minister for Education shows that the government was not advised that the Statement did not comply with the Blueprint’s directive to develop and deliver a strategy. 
To date, there has been no accompanying detailed strategy developed to support the Statement, even though this was originally planned.
VicSmart high-speed broadband for schools
Planning for VicSmart was underpinned by a robust needs and options analysis, as demonstrated by its 2005 business case, which articulated the needs to be addressed and provided a clear rationale for the purchase of high-speed fibre-optic broadband connectivity. The business case provided confidence that the project was achievable and could be delivered as planned.
The VicSmart procurement process was streamlined by using a mandated 
whole-of-government single-source provider. The fibre-optic system is performing as expected and has been upgraded incrementally to meet emerging data and connectivity needs across the government school system.
Ultranet e-learning system
The Ultranet project was poorly planned and implemented. None of its three business cases had a well thought out needs analysis or gave considered options to deliver the project. The various business cases did not answer the ‘Why invest?’ question for the Ultranet, nor did they provide a sound basis for the project’s approval.
Some six years since its announcement as a government priority, the Ultranet has not delivered its main objectives: 
·	to improve responsiveness to individual learning needs
·	to provide better information to parents, the school system and government
·	to improve the efficiency of the learning environment and school administration.
Consistent with public sector practice, the Department of Treasury and Finance (DTF) and the Department of Premier and Cabinet (DPC) provided advice to government at key decision points over the life of the Ultranet project. 
The project continued despite advice from central agencies that it should cease or be delayed. Further, there is no trail of documentary evidence to explain whether or how DEECD addressed the many critical issues raised by DPC and DTF.
It is difficult to understand why the Ultranet procurement was able to proceed to contract execution, given the significant concerns raised by DPC and DTF, as well as the many adverse ratings that DEECD had received from various Gateway reviews since the project first commenced.
Further, this audit detected a number of serious process and probity issues in relation to tendering and procurement for the Ultranet. DEECD has advised that it has commenced a number of actions and further detailed investigations in response to these matters.
There is little confidence that the financial management practices relating to the Ultranet were sound and that full costs have been adequately recorded. VAGO estimates that actual capital and operating expenditure for the Ultranet was approximately $162 million as at June 2012, and by June 2013 it is likely to have cost approximately $180 million. DEECD has advised that it is currently investigating the financial practices in relation to this major ICT project.
Despite this significant expenditure, no cost-benefit analysis has been conducted to determine whether the Ultranet provides value for money, or whether the same functionality could have been delivered more cost effectively.
Performance indicators for the Ultranet have been revised down over time and do not provide appropriate measures of whether the Ultranet is achieving what the government expected when it funded the project. 
Use of the Ultranet is low, and declining. On average, only 10 per cent of students and 27 per cent of teachers logged in on a monthly basis from July 2011 to May 2012.
An underlying factor which has limited the effective implementation of the Ultranet is the significant discrepancy between the original scope of the project and expected benefits and what has actually been implemented and delivered. This underscores the urgent need for DEECD to review whether it should continue to invest in this project.
Further, DEECD did not adequately manage the change processes required to maximise the Ultranet’s acceptance and, therefore, the state’s return on investment. Teachers and parents were not appropriately trained and supported to use the Ultranet. Ultimately, the Ultranet is only a technology tool, and cannot by itself deliver the benefits intended from it.
Recommendations
Number	Recommendation	Page	
	The Department of Education and Early Childhood Development should: 		
1.	develop a comprehensive and evidence‑based strategy or plan of action for use of learning technologies to underpin and guide the significant investment in ICT for government schools	12	
2.	develop performance indicators that measure both VicSmart’s ongoing operational performance and its achievement of intended benefits	17	
3.	urgently review its investment in the Ultranet, with a particular focus on:
·	assessing whether the contractor has delivered all functionality as required by the contract and what action, if any, needs to be taken to enforce the state’s rights
·	rigorously assessing its financial management practices and identifying the real, current cost of the Ultranet to determine the extent to which further investment is warranted
·	identifying and addressing the underlying causes of low take‑up rates across the school system by teachers, students and parents
·	providing advice to government on the cost-benefit of decommissioning the system now against continuing to fund and rectify the system so that it can be implemented as originally expected	31	
4.	conduct an agency-wide review of its internal tendering, probity and financial management practices in light of the serious issues identified by this audit	31	
5.	expedite the provision of guidance to schools on the current status of the Ultranet as the department’s key learning technology investment, and clarify the policy context of schools’ autonomy in purchasing non-Ultranet learning technologies.	31	
Submissions and comments received
In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994 a copy of this report, or relevant extracts, were provided to the Department of Education and Early Childhood Development, the Department of Premier and Cabinet and the Department of Treasury and Finance, with a request for submissions or comments.
Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments are included in Appendix A.