Annual Report 2017–18

Tabled: 20 September 2018

4 Our audit quality: Improving performance

4.1 Audit quality outside VAGO

We aim to be a leader in public sector auditing, working also to improve the practice of our profession.

Engagement with the Australian Accounting Standards Board

This year, we increased our engagement with the Australian Accounting Standards Board (AASB), with the aim of ensuring that public sector reporting is considered more consciously in their deliberations and decisions.

VAGO staff were on the AASB Insurance Project Advisory Panel and the AASB Fair Value Project Advisory Panel. We also attended and contributed to the AASB Roundtable: Options for replacing Special Purpose Financial Statements.

Engagement with the Australasian Council of Auditors-General

ACAG is an association established by Auditors-General from around Australia, New Zealand and the Pacific nations, to share information about our unique public auditing role. Our participation in ACAG ensures that Victorians benefit from the most up-to-date public auditing approaches and knowledge.

Currently the Auditor-General is the chair of ACAG's Financial Reporting and Accounting Committee. During the year we streamlined ACAG's process for making submissions to accounting standard regulatory bodies.

4.2 Audit quality at VAGO

To continue providing value to the Victorian public through our insights we must maintain our high standards. This diagram illustrates our financial audit quality assurance and continuous improvement processes.

This diagram illustrates our financial audit quality assurance and continuous improvement processes.

 

Note: An accessible version of this diagram is available at the end of this page.

Some financial audit quality assurance processes do not occur at specific times during an audit, and may not occur for all audits:

An outline of some financial audit quality assurance processes do not occur at specific times during an audit, and may not occur for all audits.

Note: An accessible version of this diagram is available at the end of this page.

This diagram illustrates our rigorous framework of similar processes for our performance audits.

This diagram illustrates our rigorous framework of similar processes for our performance audits.

Note: An accessible version of this diagram is available at the end of this page.

Results of our recent quality surveys
Parliamentarians survey

We survey Parliamentarians every year. This year's results were positive overall, and broadly similar to our results over the past few years. Parliamentarians assessed the Auditor-General's reports and services as follows:

Provide valuable information on public sector performance

93%

down from 98%

 

Help improve public sector administration

93%

up from 89%

 

Performance audits address key areas of interest

62%

up from 54%

 

Performance audits address issues of significance

68%

up from 66%

Audit committee chair survey

We last surveyed the chairs of audit committees in 2014–15. This year, 75 per cent of chairs believed that VAGO adds value by helping public sector entities improve their performance (slightly up from 70 per cent).

A number of chairs made positive comments about improvements such as visibility, access and communication, and a stronger strategic focus. The changes were considered to be 'welcome', 'refreshing' and 'encouraged'.

Ninety per cent of chairs agreed that we are targeting the right audit topics (up from 64 per cent) and 85 per cent thought that we are addressing topics at the right time (up from 54 per cent). Some chairs commented that they are looking forward to seeing the impacts of our data analytics activities.

Chief financial officer survey

We conduct surveys of chief financial officers every two years, in two rounds, based on their organisation's financial year end. This year's results generally show a marked improvement on those from 2015–16.

Professional
conduct

98%

up from 93%

 

Professional skills and knowledge

96%

up from 91%

 

Understanding of the organisation

90%

up from 84%

 

Effective
communication

89%

up from 80%

Despite the improved results, there were some concerns about staff continuity, inadequate staffing, not enough time being allocated for audits, audit fees and excessive requests for information.

Chief financial officers valued the assurance provided by our financial statement audits (94 per cent, up from 91 per cent) and valued our recommendations for improvements (90 per cent, up from 87 per cent).

Performance audit surveys

Every year we survey the agencies involved in our performance audits about our process, reporting and value.

Process

 

Reporting

 

Value

Positives: the quality of the audit process, the professionalism and engagement of audit teams, sufficient opportunity to comment on issues prior to receiving a draft report.

For improvement: continuity of audit staff.

 

Positives: there were adequate opportunities to make comments, tabled reports were accurate and clearly communicated the issues.

For improvement: the relevance and practicality of our recommendations, the timing allowed to respond to report drafts.

 

Positives: audits were generally focused on the right areas.

For improvement: the depth and scope of audit recommendations, being clearer about the risk or issue driving the audit, providing best practice standards and benchmarks.

4.3 Improving our performance

Data analytics

This year, we have significantly expanded our data analytics capability. We developed a strategy for integrating data analytics into our audit work to support traditional auditing methods. To help public sector agencies see the power of the data that they hold, we have begun creating and providing dashboards to agencies.

One advantage of our dashboards is the ability to add data year on year and build up a complete picture of how a program or agency is performing. This will allow us, and agencies, to return to the data we've collated over time which will reveal long-term trends and better inform their planning and decision making.

The first phase of our strategy focuses on our data analytics staff building dashboards and other analyses and helping source the most appropriate datasets for our performance audits. Staff in our performance audit areas are being trained in how to use data analytics tools, which increases their skills and lifts the capability of the whole organisation.

We have begun accessing datasets from non-government sources to complement our own work, and have been piloting the use of basic text mining tools to assist with annual planning.

For our financial audits, we are streamlining the collection of data and building standardised dashboards that our financial audit teams can use and build on each year.

In the future, our staff will be skilled enough to build their own dashboards. We will expand our use of external datasets and introduce predictive modelling methods. We will also progress to using more sophisticated text mining tools for analysing unstructured data.

Financial audit

During the year, we developed a multi-year strategy to embed data analytics into our financial audit practices. We will be applying data analytic tools and techniques across many financial audits in the future, which will improve our audit quality and efficiency. We have been working to streamline the collection of financial data. We use this data to improve our understanding of our financial audit clients' business risks, to assess their internal controls, and to improve the focus of our audit testing.

Although we are still early in our journey, we have engaged with most of the entities identified for phase 1 of the strategy. We've gathered expenditure and payroll data from several of our larger audit clients and have begun building a customised analytics solution that will standardise their data into a uniform format for analysis. This will allow us to:

  • identify key audit risks and tailor our approaches to address them
  • effectively sample and select data
  • gain better insights and provide better business intelligence back to our financial audit clients.

Our first four dashboards were published in November 2017, along with our report Results of 2016–17 Audits: Public Hospitals. These dashboards are interactive visualisation tools that summarise the financial statement data of all Victorian hospitals. They allow stakeholders and the general public to refine and compare a huge range of financial information.

In May 2018, we published similar dashboards to accompany our reports on Technical and Further Education (TAFE) institutes and universities.

Image of a similar dashboard to accompany our reports on Technical and Further Education (TAFE) institutes and universities we published in May 2018..

These are examples of ways we are increasing the accessibility of our work through new interactive channels. Over time, we will add new content and insights, and will improve the usability of our dashboards by seeking feedback from our stakeholders.

Performance audit

We have been working to create a strategic three-year plan to successfully execute data-driven audits—an innovative way of ensuring our audits are based on persuasive, objective evidence. Some of the key steps we are taking to bring together our data analytics and performance audit capabilities include:

  • collaborating up to six months before an audit begins, to kick-start the audit's data strategy
  • planning together to more effectively use data analytics
  • co-sourcing rather than outsourcing when using external analytics services to ensure the skills and knowledge are transferred to VAGO.

Fifteen performance audit projects have been supported by data analytics in various ways, from assuring the quality of the data used for analytic work to creating interactive dashboards.

One dashboard that has been particularly valuable was created as part of our audit of Victorian Public Hospital Operating Theatre Efficiency. For this audit, we compiled a dataset using data from 23 health services. We created a dashboard to analyse health system performance and benchmark data in multiple ways not previously done in Victoria. The dashboard can be used to analyse the efficiency of operating theatres within and between health services and hospitals.

Creating dashboards and datasets not previously compiled and providing these back to agencies is an area of future growth for us. This will be a key way to help agencies make the most of their own information, and to achieve our strategic objective of being more influential by providing our clients with new insights they can use.

Audit mandate and process

Performance audit

This year, we tabled our first audit using our new follow-the-dollar powers, Safety and Cost Effectiveness of Private Prisons. These powers allow us to audit private and non-government organisations that are contracted to provide public services. This audit published our analysis of a large amount of data about how private prisons operate that was not previously available to the public. We found that private prisons are cheaper to run than publicly operated prisons, and that they largely meet their contract service and performance requirements.

We established a new Performance Audit Practice Governance Committee this year to ensure that performance audit practices are up to date, streamlined and consistent. The committee, made up of staff from across VAGO, aims to implement improvements and changes arising from audit debriefs, stakeholder feedback and internal staff suggestions. It considers technology tools, policy, guidance and manuals, training materials, templates and practice statements.

We have improved the quality of our planning by adopting a more risk-based approach. We have improved our understanding of agencies and audit subject matter to identify any risks and better tailor our audits. This approach is reflected in our updated methodology and our recent training.

Part of improving our understanding of agencies is working with them earlier in the audit process. On two recent audits, Effectively Planning for Population Growth and Community Health Program, we brought agencies together to discuss our recommendations jointly and earlier than usual. This has resulted in recommendations that are more useful and practical for agencies.

This has also led to better working relationships with departments. In our recent surveys, agencies rated the quality of our audit process highly, particularly the professionalism, skills and knowledge of our staff, and their ability to communicate effectively.

Financial audit

It has been 12 months since the new standard, ASA 701 Communicating Key Audit Matters in the Independent Auditor's Report, became effective. It requires us to include a description of key audit matters in our reports. These are matters that we determine to be the most significant. We must include a brief description, an explanation of why we consider them to be key audit matters and details of what we did to address them.

We have voluntarily adopted this new standard as we believe it will enhance the value of our auditor's report by providing greater transparency and insight to our audit process. In the Auditor-General's Report on the Annual Financial Report of the State of Victoria: 2016–17, tabled in November 2017, we disclosed key audit matters for the first time. We continued a staged implementation of this standard for other entities during 2017–18.

We have been working with entities to help them adopt streamlined reporting. The purpose of streamlined reporting is to make reports easier to understand. These reports still comply with all relevant standards and legislation, but remove any information that is immaterial. They are easier to read and more user‑friendly than traditional financial reports. Both the local government and university sectors have adopted streamlined reporting, with positive results.

We dedicated sustained effort at all levels to working with audit clients and other key stakeholders to improve the level of service we provide, including in our written communications. For example, we changed the look, feel and structure of key documents provided to stakeholders like audit committees to help them to focus on the most important insights we have to offer.

 

The below tables provide an accessible version of the images in Section 4.2.

To continue providing value to the Victorian public through our insights we must maintain our high standards. This diagram illustrates our financial audit quality assurance and continuous improvement processes.

BlueDownArrow.PNG

Audit planning, including risk assessment and agreeing with the audited agency to the terms of engagement through an arrangement letter

EQCR point

Audit strategy issued to the audited agency

Interim management letter issued to the audited agency

Interim audit activity

Year-end audit activity

Concluding the audit and reporting

EQCR point

Closing report issued to the audited agency

Audit opinions issued on signed financial statements and, for some agencies and councils, performance statements also.

Final management letter issued to the audited agency

PAFR point

Note:
Engagement quality control reviewer (EQCR)—Depending on the risk and complexity of the audit, selected audit files will have an EQCR appointed. The role of the EQCR is not to be a member of the audit engagement team, but to offer an objective evaluation and review of the significant judgements made by the team, to ensure the quality of the conclusions being reached.
Post-audit file reviews (PAFR)—PAFRs are performed on a selection of completed audit files. These 'cold reviews' assess the quality of the engagement against the requirements of applicable audit standards. They are performed by engagement quality reviewers who are objective and independent of the audit. As well as ensuring the quality of our audits, these reviews are used to measure our performance against one of our BP3 reporting measures, as shown on page 23.

Some financial audit quality assurance processes do not occur at specific times during an audit, and may not occur for all audits:

Technical panel

Audit report modification panel

The technical panel, which consists of the Auditor-General and the Assistant Auditors-General of the financial audit and technical audit teams, maintains and considers a register of significant accounting matters. The register of significant matters and the panel provide a forum for discussing and considering technical issues facing the public sector and audit teams.

A modified audit opinion is one that includes, for example, key audit matters, an emphasis of matter paragraph, a qualification or a disclaimer. Issuing a modified opinion requires the approval of the Auditor-General. The audit report modification panel considers and recommends whether these modifications should be made. It consists of the Assistant Auditor-General of financial audit and an independent member of the technical audit team.

Technical consultation

Client surveys

In our technical audit team we employ subject-matter experts. At any time during an audit, if the audit team requires impartial advice on an audit or financial reporting matter, they can consult with our technical audit subject-matter experts.

Each year, we survey our clients, alternating between chief financial officers and audit committee chairs, to measure our performance against their expectations. The results of this year’s surveys are discussed on page 30.

This diagram illustrates our rigorous framework of similar processes for our performance audits.

BlueDownArrow.PNG

Audit initiation

AAG/AG point

Audit planning

EQCR point

AAG/AG point

Audit conduct

EQCR point

AAG/AG point

End of conduct brief sent out to agencies for consultation

EQCR point

AAG/AG point

Provisional draft of Parliamentary report sent out to agencies for consultation

EQCR point

AAG/AG point

Proposed draft of Parliamentary report sent out to agencies for consultation

EQCR point

AAG/AG point

Preparation of final Parliamentary report

Report tabled

Back to Top