Key requirements for agencies to comply with the standing directions include:
- conducting an annual assessment of compliance with all applicable requirements, including the requirement to apply the AMAF
- attesting to financial management compliance and disclosing all material compliance deficiencies in their annual reports
- taking remedial action to address any compliance deficiency, whether material or not.
DTF also requires departments to report to it annually on their financial management compliance and that of their portfolio agencies.
Departmental secretaries and audit committees have key responsibilities to assure the reliability and accuracy of compliance assessments and attestations.
The standing directions instructions explain that these requirements are designed to 'improve rigour in compliance assessment, ensure action is taken to improve identified compliance weaknesses … and increase transparency'. The standing directions guidance explains that agencies are expected to 'take a practical, risk-based approach to demonstrating compliance' for their annual compliance assessments, provide the assessments to their audit committees and use the assessments to inform their annual compliance reports to DTF.
Departments need well-designed arrangements to assure compliance and support reliable attestations. In this Part, we examine the assurance approaches used by departments and their audit committees. Figure 3A highlights the main steps of the attestation process.
The main steps in the attestation process
Five departments used reliable approaches to assure their compliance and support their 2018 attestations. The remaining two departments do not have enough detail in their approaches to be able to assess whether they comply with the AMAF, given the criticality, complexity and risks related to their assets.
The attestation itself does not indicate how well departments are complying with the AMAF. This is because the standing directions require agencies to say they comply even when they have not met all mandatory requirements, unless the department considers that non-compliance for one or more mandatory requirements is significant or material.
All departments have weaknesses in the reliability or accuracy of their compliance assessments, such as gaps in their oversight arrangements or not having the right evidence to show compliance. Departments that involve senior leaders with asset management responsibilities in overseeing compliance are more likely to demonstrate better assurance practices and have more reliable and accurate compliance assessments.
Audit committees' limited record-keeping makes it hard for them to show how they apply a risk- and evidence-based approach to their review role or how they satisfy themselves that departmental compliance assessments and attestations are accurate.
3.2 Departments' 2018 attestation results
Two departments reported a material compliance deficiency for the AMAF in their 2017–18 annual reports—DET for school buildings and DHHS for the assets of its health building authority.
The standing directions instructions mandate that the attestation wording includes:
- a brief summary of the reasons for, or circumstances of, the material compliance deficiency
- details of planned and completed remedial actions.
The way DET and DHHS provided this information in their attestations differed:
- DET identified that its deficiency related to school assets was material because the AMAF had not yet been applied across all schools, but it had a five-year plan to address this.
- DHHS identified that it had a material deficiency and that this would be addressed by December 2018, but did not explain which asset class(es) this related to or what made it material.
The limited information in DHHS's attestation does not provide transparency about the nature of the deficiency.
Determining if material compliance deficiencies exist
The focus of the attestation is on identifying any material compliance deficiencies and actions to address them. DTF's guidance to agencies on materiality and the AMAF provides an example about a major telecommunications distribution cable—any failure is likely to disrupt service delivery over a wide area and damage agency finances and reputations, and would therefore be material. However, if the computer systems in the telecommunications retail provider failed, it would inconvenience customers by closing shopfronts, but the telecommunications service would continue, and would therefore be non-material.
All departments took steps to identify material compliance deficiencies, although only DEDJTR provided specific guidance on how staff should apply the department's risk management framework to determine this.
DET and DHHS documented their rationales for why those deficiencies were material. DPC was the only department to document a rationale for why its compliance deficiencies were not material.
Although all departments identified non-material compliance deficiencies, only DET and DHHS identified material compliance deficiencies in their 2018 attestations and only around five per cent of all public sector agencies reported material deficiencies. This is because the 'reasonable person' test in the standing directions deliberately sets a high bar for a deficiency to be considered material. Other regulatory bodies in Australia and overseas apply different but similarly high bars to define materiality, including the Australian Accounting Standards Board and the U.S. Securities and Exchange Commission.
Definitions of compliance and the relationship between compliance and the attestation
The purpose of including the AMAF in the standing directions and the attestation is to hold agency heads or governing boards accountable for applying the framework. The focus of the attestation is on identifying material deficiencies, to alert Parliament, public agencies and the community to any significant gaps in financial management.
The standing directions instructions state:
where the Agency has not identified a Material Compliance Deficiency that occurred during the relevant year, attest that the Agency has complied with the applicable Directions and Instructions, in the form set out in clause 2.2(a) of this Instruction.
However, this does not match the meaning of compliance under the standing directions. Although the standing directions do not define compliance, they define a compliance deficiency as:
an attribute, condition, action or omission that is not fully compliant with an applicable requirement in the FMA [Financial Management Act 1994], Standing Directions and/or Instructions.
The implication is that compliance with the standing directions means not having any compliance deficiencies, material or not. So, although the attestation usefully identifies agencies with significant or material financial management issues, its assertion that all other agencies comply with the standing directions is misleading. The five departments that did not have material deficiencies all still had compliance deficiencies with the AMAF, and that was for just one of 458 financial management compliance obligations.
Departments must report on their financial management compliance annually to DTF, separately to the attestation. DTF publishes a summary of these reports on its website, but does not make public the individual department results.
DTF advised us that these different definitions of 'compliance' reflect the fact that the public attestation and the compliance reporting are two separate, parallel processes. However, the AMAF describes the mandatory requirements as 'mandatory attestation requirements' that accountable officers 'must meet to allow for full attestation of compliance'.
This means agencies need to know their levels of compliance, not just their material compliance deficiencies, against all 41 mandatory requirements to inform the attestation.
Five departments recognised the benefit in using their compliance assessments to provide a systematic, evidence-based approach in determining whether they had material deficiencies.
3.3 Departments' checks on compliance
For departments, the Secretary is responsible for the accuracy and completeness of the attestation. In each department, the Secretary delegates this responsibility to one or more senior leaders—deputy secretaries or other executive officers—but still approves the final attestation.
Under the AMAF and the standing directions, agencies should identify responsibilities and processes for monitoring compliance and should ensure systems and processes to support it are in place.
All departments have arrangements to oversee compliance with the AMAF and follow a similar oversight model, as shown in Figure 3B. The extent of their oversight varies, with a key area of difference being the extent to which they assign oversight responsibilities to senior leaders.
The oversight model commonly used by departments
Figure 3C highlights the good practices we saw in departments' oversight arrangements.
Good practice elements in oversight arrangements
Most of the departments with large and complex asset portfolios and risks have arrangements that include senior leaders, but gaps exist in the way they are executed. The main gaps are:
- senior leaders not actively fulfilling their identified responsibilities
- the whole-of-department committee having limited or informal oversight.
In the departments that did not include senior leaders, the AMAF implementation responsibilities sat solely with their corporate finance areas. Although this enabled streamlined approaches to implementation and compliance assessments, it missed the opportunity to engage a senior leader with operational asset management responsibilities to 'champion' and drive the AMAF and asset improvement across the organisation.
Compliance assurance approaches
The AMAF identifies examples of the types of evidence that demonstrate compliance with the mandatory requirements, but it does not specify how agencies should assess compliance. The standing directions expect agencies to establish the processes they need to assess compliance. Departments also have a responsibility to provide high-quality information to their audit committees to support the committees to perform their review roles.
We assessed whether departments have reliable processes that are appropriate to their size and asset complexity and whether those processes are:
- communicated to those conducting assurance activities and approving the assessments
- applied consistently across different asset classes.
The requirement to publicly attest to all standing directions was new in 2018, and the AMAF and reporting requirements were also new. However, before this, agencies were required to publicly attest to their risk management compliance and to separately certify their financial management compliance to their responsible minister and report this to DTF. This means that assurance processes such as assessing compliance should be well established in departments.
Five departments advised us that they followed the centrally coordinated processes already established for their other financial compliance obligations. Departments' internal audit results show these processes are reliable.
The departments' processes all included similar assurance steps for the AMAF, and these aligned with guidance for the standing directions and the AMAF.
The departments that had the most reliable compliance assurance processes—DEDJTR and DET—communicate and apply well-designed approaches that include several of the good practices in Figure 3D. They paired these approaches with strong central coordination. DEDJTR also provided specific guidance to staff on aspects such as evidence requirements and material deficiency considerations, and verifies its compliance assessments and evidence.
Figure 3D identifies the main assurance steps and the good practices and weaknesses we found in how departments apply them.
Good practices and gaps identified in departments' assurance processes
Assurance process step
Good practices across departments
Identified gaps and issues
Develop process and communicate to staff
Make documented process available to all staff, e.g. on the intranet
Process not documented or only in internal audit papers
Complete separate assessments for important asset classes, or on a rolling program
Aggregated assessments that do not provide detail about important asset classes
Support consistent assessment across asset groups, e.g. by providing an assessment tool and guidance on aspects such as evidence requirements, assessing material deficiency, developing remedial actions
No common assessment tool
No guidance provided
Determine any material compliance deficiencies
Use corporate risk framework to assess material deficiency
Document rationale for whether there is a material deficiency
Rationale not documented when it is determined there are no material deficiencies
Develop remedial actions
Identify positions responsible for completing actions
Develop remedial action plan(s)
Monitor the progress of actions
Limited planning details to support implementation of remedial actions
Senior leader responsible for assets approves the assessments
Senior leader completes the accountable officer checklist from DTF's AMAF implementation guidance
No executive approval or approval not documented, i.e. no formal sign off
One example where the assessor was also the approver
Verify compliance prior to approval—can be risk-based, e.g. for higher risk asset classes or AMAF requirements
Draw on asset management expertise to verify compliance
No review or verification prior to audit committee review
Deputy Secretary signs declaration related to the compliance assessment/attestation
Inform audit committee
Ask audit committee to specify the information it needs to support its attestation role
Audit committee not consulted
Monitor remedial actions
Whole-of-department oversight and monitoring
No monitoring in place
The AMAF is one of up to 458 financial management directions that agencies must assess compliance for, which sets a very broad base for the attestation. The standing directions allow agencies to take a risk-based approach to assurance, so that they focus their compliance efforts and evidence on checking the areas of their operations that are higher risk or value. All departments would benefit from doing this and documenting their chosen approach and the basis for their risk approach.
Departments need to know their levels of compliance with the AMAF's mandatory requirements and be confident in their assessments, to support the attestation.
Room exists for departments to improve the accuracy and consistency of their compliance assessments.
The AMAF and the standing directions expect that agencies will take a 'practical, risk-based approach to demonstrating compliance'. This allows flexibility in approaches so that smaller agencies and those with lower risks can use proportionately less detailed assessments and evidence and avoid an unnecessary compliance burden, as Figure 3E indicates.
Relationship between compliance-related risks and evidence needs
Five departments align their compliance assessment approaches with their asset risk or complexity, although none clearly documented the rationale for the assessment approach. Examples of their approaches included:
- one department that matched its diverse, often complex and high-value assets with compliance and maturity assessments for every asset class and key portfolio agency
- another department that contracted asset management consultants to individually assess its major asset class, and compiled an aggregated assessment across its simple asset classes.
The remaining two departments also identified that they had higher risk or complexity associated with some asset classes, but only used aggregated, whole-of-department assessments. We consider that both these departments need specific assessments for their higher-risk asset classes to provide an accurate assessment and a sufficient baseline of compliance, and identify any deficiencies to help prioritise improvement and risk management efforts.
Figure 3F summarises the good practices we saw in the way departments assessed compliance.
Good practices for compliance assessment
DTF advised us that a flexible approach could involve assessing the compliance of lower-risk assets less frequently and requiring less evidence and verification efforts for those, but none of the departments did this and most had not understood that it was an option.
Adequate evidence of compliance
The standing directions guidance specifies the need for evidence to substantiate compliance. The AMAF implementation guidance provides examples of the types of evidence that could substantiate compliance with its different requirements for complex and simple asset portfolios. The AMAF and the standing directions identify that achieving the mandatory requirements and demonstrating compliance require that asset policies, systems and controls are in place and are being applied as intended.
Through our analysis and from the results of two departments' internal audits, we found numerous issues that reduced the accuracy of the compliance ratings and led to some overestimates of the level of compliance—that is, rating requirements as compliant when they are not. The main issues were:
- inadequate evidence of compliance across all departments' assessments
- missing evidence in some departments' assessments
- errors identifying whether requirements are applicable or not at different stages of an asset's lifecycle.
The most common evidentiary-related problem is that departments' assessments rated some requirements as 'compliant' on the basis that they have a policy or process, even though, for many of the mandatory requirements, compliance means also showing that these processes are followed.
Value in verifying compliance assessments
Compliance with the AMAF and standing directions is based on agency self‑assessments. The assessments do not require independent verification or validation. It is up to agencies to determine their approaches to assuring themselves about compliance.
Australian and international accounting policies and standards for attestations identify the need to test evidence that underpins the attestation, or to test the completeness of the explanation if the attestation is supported by a management assertion.
Only DEDJTR and DHHS showed good practice by verifying their compliance assessments. Figure 3G highlights the good practices we saw in how they verify compliance.
Good practices for verifying compliance assessments
These verification activities identified examples where insufficient or no evidence substantiated assertions of compliance, and where mandatory requirements had been incorrectly identified as applicable or not applicable to a particular asset class. They showed that one department had significantly overstated compliance for some of its asset classes.
This underscores the value of verifying evidence for significant asset classes or mandatory requirements to improve the reliability and accuracy of the information supporting the accountable officer's attestation. It also highlights the need for departments to guide and train their staff to conduct accurate assessments.
Consistency of assessments
DTF provides a compliance assessment tool template on its website. Although this is optional, it is a mechanism that can support consistency in and across departments. For the AMAF, the tool prompts agencies to assess:
- all 41 mandatory requirements
- compliance and evidence of compliance
- compliance deficiencies and actions to address them
- any material compliance deficiencies, justification for these and actions to address them.
Five departments based their approach on DTF's template, although three departments changed it. The main area missing from all departments' tools was the justification for whether deficiencies were material.
Four departments assessed compliance separately for individual asset classes. Each used a common assessment tool in the department but, despite this, their assessments were not always consistent or adequate. For example:
- they assessed compliance against different numbers of mandatory requirements for different asset classes
- they explained the reasons supporting 'compliant' and 'deficient' ratings for some asset classes but only explained deficiencies for other classes.
This meant that many departments received limited value from the assessments to understand their levels of compliance and compliance deficiencies, and to help prioritise future improvement actions.
3.4 Audit committees' checks on compliance
A department's audit committee needs to review and satisfy itself with the attestation statement before the Secretary signs off. The 2016 standing directions strengthened the audit committee's role and responsibilities to advise the Secretary on the attestation and exercise independent judgment in its decisions concerning the attestation.
The AMAF requires an audit committee to satisfy itself with the department's recommended attestation of compliance prior to finalising it for the annual report. Audit committees have other key obligations in the standing directions, instructions and guidance to:
- review the department's annual assessment of compliance with the AMAF
- provide the Secretary with assurance, advice and recommendations on the level of compliance attained, issues to be resolved and proposed mitigation plans
- review and monitor actions the department takes to remedy compliance deficiencies.
Appendix D provides the full list of audit committees' requirements for reviewing compliance under the AMAF and the standing directions. It is up to each committee to determine the information and evidence that it needs from the department to satisfy these responsibilities.
We examined how the audit committees fulfil these responsibilities by looking at:
- the processes they put in place to identify their information requirements and guide their review and monitoring activities
- the information they received
- their review actions.
Audit committees' approaches to reviewing the compliance assessment and the attestation
Reflecting the considerable volume and complexity of legislation, policy and directions that agencies must comply with, the standing directions allow agencies and their audit committees to undertake their roles and responsibilities in ways that are proportionate to the complexity and risks of their operations. This includes expectations that they:
- focus on the directions that pose the highest risks to them in managing their compliance responsibilities
- take a practical, risk-based approach to the evidence needed to demonstrate compliance.
All audit committees advised us that they take a risk-based approach to their AMAF responsibilities. The approaches vary, in part reflecting the distinct characteristics of their departments' asset portfolios.
While DEDJTR's audit committee documented elements that showed its approach, none of the audit committees transparently recorded that they plan or adopt a risk-based approach to their compliance and attestation-related actions, for example, in their internal audit plan, meeting papers or minutes. Only DTF's audit committee minutes identify the information that it expects the department to provide.
Documenting the approach is not a specific requirement of the AMAF or the standing directions. However, the significance of audit committees' obligations under the standing directions is high relative to the number of members and amount of time they have to fulfil them. Audit committees and their members also need to demonstrate accountability and transparency in performing their roles. Planning and articulating a risk-based approach—considering the nature of a portfolio's assets, value, materiality and significance to service delivery—would help committees avoid unnecessary compliance burdens and third-party costs.
Based on our findings, Figure 3H identifies a good practice model for audit committees' future review of the AMAF compliance assessment and the attestation. The extent to which all elements are relevant to each department will depend on the complexity and risks of their asset portfolios, the level of compliance and maturity, and the potential for material compliance deficiency.
A good practice model for audit committees' future review of AMAF compliance and attestations
Information received by audit committees
An effective approach to compliance enables a department to meet its obligations under the standing directions. The annual review and reporting against financial management obligations for the AMAF should provide rigour in compliance assessment, ensure action is taken to improve identified compliance weaknesses at the departmental level and increase transparency. Audit committees play a critical role in this.
Our August 2016 Audit Committee Governance audit found that providing effective operational support for audit committees is a significant role of departments. It also identified that an audit committee's operation is enhanced by having high-quality information. It recommended that departments work with their audit committees to better identify the committee's information needs, including whether reported information is reliable and understandable. All departments accepted the recommendation.
Departments provide varying levels of written information to their audit committees to enable them to review compliance, understand whether deficiencies are material or not, and satisfy themselves about the attestation. Figure 3I identifies good practices we saw in relation to this.
Good practices in information that departments provided to their audit committees
Departments provided to their audit committees:
These practices are relevant to audit committees of departments with simple or more complex asset portfolios, although the level of detail needed may vary in proportion to the criticality, complexity and risk.
There are gaps in the documentation departments provide to audit committees about AMAF compliance:
- The information provided to four committees only identifies compliance deficiencies, and does not include information to support areas assessed as 'compliant'.
- Where compliance information is included, two committees received limited information describing the basis or evidence for why individual requirements are rated as 'compliant'.
- Three committees received no information on why identified deficiencies were not material.
- One committee received no information related to compliance or compliance deficiencies—only a rationale for why there were no material compliance deficiencies.
The chairs of six audit committees and a committee member of the seventh said that their committees considered that the level of AMAF evidence and the information that they received from departments met their expectations. Most of the committee members we spoke to advised us that progressive briefings from departmental staff and the results of trial attestations undertaken in 2016–17 helped them fulfil their annual reviews of compliance and attestation responsibilities. One chair explained, 'we understood where we were going. We had the chance to ask questions and understand the requirements'. Another considered that the AMAF was 'a small part of the attestation' and did not need more detailed information.
Given the accountability focus of the AMAF, we consider audit committees need more information on the annual AMAF compliance assessments, at least for significant asset classes or where the department identifies significant risks associated with the AMAF. This would include information on which—if any—of the 41 mandatory requirements they are compliant with and why, in addition to information on non-material compliance deficiencies. It would also include a clear rationale for why any compliance deficiencies are not material, as well as for any that are deemed material. Although oral presentations, discussions and past results are important, they should complement the documented compliance assessments and determinations, and not be a substitute for them.
Audit committees' review of the attestation
We found that few audit committees checked evidence of compliance. Although they all discussed compliance, none recorded how they satisfy themselves about compliance and the attestation.
Checks on the AMAF requirements assessed as compliant
Only DEDJTR's and DHHS's audit committees could demonstrate that they reviewed evidence supporting the departments' compliance assessments. One committee chair explained that it is important to review compliance evidence because 'we need to satisfy ourselves … we want a second pair of independent eyes'.
Both committees commissioned internal audits of the departments' AMAF compliance assessments to check the evidence of compliance. DEDJTR's audit committee chair also reviewed the hardcopy compliance assessment sheets completed for the department's multiple asset classes and the attestations made by the senior leaders with delegated responsibility for them. The internal audits provided objective and independent assurance to the audit committees on the level of compliance with the AMAF mandatory requirements, and about whether material compliance deficiencies existed.
This is a sound approach for departments with complex asset portfolios or significant AMAF or asset-related risks.
The two internal audits found multiple examples where a compliant rating was based on inappropriate or inadequate evidence, showing that the departments had overstated their AMAF compliance. This shows the benefit of an internal audit to support an accurate compliance assessment and identify where staff making the assessments may need additional training and guidance.
DPC's audit committee recorded that its simple asset holdings did not warrant a review of evidence. This is consistent with a practical and risk-based approach.
The remaining four audit committees did not check evidence of compliance. They relied on two sources of information to satisfy themselves about the accuracy of the compliance assessments, which were:
- managements' assertions about the departments' AMAF progress and compliance, and sign off by delegated senior leaders on the compliance assessments—although the committees did not record how they tested the completeness of the explanations supporting the management assertions
- the positive results from internal audits of their controls over compliance.
They also advised us that they were aware that supporting evidence was available should the committee want to review it.
As three of these departments identified significant compliance risks related to the AMAF, their audit committees' reviews would be more transparent if they recorded how they were satisfied that the evidence provided justified the compliance assessment.
Processes audit committees use to satisfy themselves about compliance and the attestation
Audit committee chairs and departmental staff who attended committee meetings advised us that compliance and material deficiencies were discussed prior to the attestation. However, there are limited records in audit committee meeting minutes indicating how the committees considered compliance and compliance deficiencies, and the evidence for them.
No audit committee transparently recorded how it considered the department's information to satisfy itself about the AMAF attestation. While audit committee meeting papers indicate what information is presented to the committees, committee minutes are brief and decision-orientated and do not record the discussions held. They do not indicate how the information provided is used to satisfy committees about the departmental compliance assessments and attestations. For example, they do not identify how committees check the assertions that departmental staff make, how much weight they give to draft internal audit reports or how they gain assurance that material compliance deficiencies do not exist.
As a result, most audit committees could not demonstrate that they used risk- and evidence-based approaches to satisfy themselves about the level of compliance attained and the compliance attestation.
One audit committee only considered the overall potential for any material compliance deficiency, without receiving any ratings of compliance, evidence of compliance or information on compliance deficiencies prior to the 2018 attestation. If audit committees only consider the potential for material deficiencies, they are not fulfilling their compliance review role.
Audit committee chairs all agreed that balance is needed in the level of detail they record in their minutes on the considerations that inform key actions and decisions. The chairs or members of four audit committees we interviewed agreed that recording key discussions relating to the AMAF could be looked at for transparency.