Victorian Auditor-General’s report
9 June 2010

Administration of the Victorian Certificate of Education

Audit summary
The Victorian Curriculum and Assessment Authority (VCAA) is a statutory body established under the Education and Training Reform Act 2006 to develop school curriculum and assessments for all primary and secondary students. The VCAA also has primary responsibility for administration of the Victorian Certificate of Education (VCE). 
The VCAA spends around $28.6 million annually on assessment operations including administration of VCE assessment and reporting which comprises a series of in house and contracted tasks. These tasks include the development of assessment instruments, oversight of school based assessments and a wide range of checks and crosschecks on the reliability and validity of students’ results. The VCAA award more than 700 000 grades and issue 70 000 statements of results annually.
The objective of the audit was to determine the effectiveness of the procedures that VCAA has in place to achieve its organisational objective of high quality assessment in the VCE.
Conclusion
The VCAA is effectively administering the VCE providing a high level of assurance that the results students receive are accurate and have been marked consistently and accurately. It has established strong processes and quality controls, and has developed clear and comprehensive guidance to assist schools to administer school based assessments. 
There is, however, scope for VCAA to strengthen its security protocols and to introduce a more systematic and data driven process of continuous improvement.
Findings
The Australasian Curriculum, Assessment and Certification Authorities (ACACA), the national body of the statutory authorities responsible for awarding certificates of senior secondary education, has developed guidelines designed to improve the quality and fairness of assessments. As VCAA is a founding member, the principles of ACACA are comprehensively embedded in its assessment policies and in the guidance it provides to schools.
Developing exams for 130 VCE studies is a complex process involving the input of about 600 exam developers who work collaboratively as part of exam panels to produce exam papers. The VCAA has established sound processes to guide and support exam developers, which could be further enhanced by introducing formal training for all exam developers.
Schools are responsible for designing and administering accurate and fair assessments of student performance against their expected learning outcomes specified in VCAA’s study designs. The VCAA provides written guidance and periodic training to schools and audits the quality of school assessed coursework. There is scope for the VCAA to more efficiently conduct audits of school assessed coursework and to improve the feedback it provides to schools on the audit findings.
The VCAA has a large number of committees that comprehensively review the performance of its VCE assessment processes. However, VCAA could make better use of existing quantitative data to inform its review processes as part of a formal program of continuous improvement.
As exam errors, delays and security breaches are significant risks for the VCAA, it has developed clear and detailed security and exam administration processes. Although sound, these processes could be further improved by reviewing exam panel practices and introducing a regular program of real-time audits of printers’ premises and exam centres.
The VCAA has also developed an extensive range of quality control and quality assurance processes to mark exam papers accurately and consistently. This includes the use of trial marking, and using control exam papers and statistical data to monitor assessor reliability and performance. The ability of VCAA to monitor and improve assessor performance would be significantly enhanced by expanding its online marking technology to include all VCE studies.
The VCAA has implemented robust technical moderation and study score calculation procedures so that students receive fair and comparable results.
Recommendations
Number	Recommendation	Page	
1.	The Victorian Curriculum and Assessment Authority should:
·	improve its systems for managing coursework audits and responding to the findings
·	provide more structured training in assessment development techniques to exam panel members and develop a planned process of self review and evaluation for all exam panels
·	make better use of quantitative data to inform exam review processes as part of a formal program of continuous improvement.	11	
2.	The Victorian Curriculum and Assessment Authority should:
·	strengthen its security practices by introducing a program of regular real-time audits of printers’ premises and exam centres
·	increase the number of schools subjected to moderation by visitation
·	explore opportunities for the wider application of online marking.	21