Implementation of the Government Risk Management Framework

Conclusions

The Framework has helped public sector agencies improve their risk management, and they largely comply with the high-level requirements. However, risk management is not yet widely and consistently used as a proactive, outcome driven discipline because of weaknesses in the Framework and the practices used to translate it into action.

The most important gap in the Framework and the underpinning guidelines is in the management of interagency and statewide risks, and agencies have not found a way to work with each other to manage them effectively.

The information provided to government by VMIA and DTF does not adequately convey the key vulnerabilities for the state and the need for urgent and coordinated action because:

• the weight VMIA gives to these risks when assessing agencies' risk management does not reflect their importance or the state's vulnerability to their impacts
• the high level of reported compliance with the Framework gives government a false sense of security because the result is not checked, and achieving compliance is not sufficient to assure government that risks are well managed.

Progress to address this critical gap has been very slow. VAGO's 2007 audit reminded agencies of our earlier 2003 recommendation to fix this problem and warned that the Framework DTF was developing in 2007 would not fully address it. DTF missed this opportunity to modify the 2007 Framework to address this concern and has not remedied it in the six years since.

The formation of the IDC in December 2012 and DTF's planned update of the Framework in 2014 provide the opportunity to make significant progress. It is critical that DTF does the necessary groundwork to enable government to seize this opportunity by:

• supporting the IDC to help it provide government with practical, effective advice on how best to manage these risks
• incorporating a coherent and effective approach for managing these risks in the updated Framework.

Findings

The Framework is soundly based on the principles of the Australian/New Zealand risk management standard: AS/NZS 31000:2009, but is neither comprehensive nor sufficiently clear. Its introduction marked an important step in developing risk management in Victoria, providing a high-level approach consistent with the AS/NZ standard and making agencies more accountable for applying its principles.

However, it needs to be strengthened to address gaps and provide greater clarity to agencies about the minimum requirements for effective risk management.

VMIA does a reasonable job in helping agencies to apply the Framework. However, while its guidelines are extensive and useful, we found gaps that need to be addressed. VMIA's intention to better focus its training and support by developing a learning and development strategy is sensible.

The information provided by DTF and VMIA to government is not sufficient for it to understand the significance of the risks faced by the public sector.

Agencies are largely, but not fully, compliant in applying the Framework's requirements. However, their practices have only matured to the point where they are partially effective in managing the risks they face.

The audit found that the line agencies examined needed to improve how they:

• manage interagency and statewide risks
• use organisational objectives to drive risk management
• document the analysis underpinning risk management
• select, prioritise and apply risk treatments
• embed risk management and communicate good practice
• evaluate performance,including their use of risk performance indicators.

Recommendations

That the Department of Treasury and Finance:

1. works with the Victorian Managed Insurance Authority to update the Victorian Government Risk Management Framework to clearly articulate minimum requirements that agencies need to meet to demonstrate that they are effectively managing risk—including improving the coverage of interagency and statewide risks, updating the attestation requirements and better describing its intent, purpose and key risk concepts
2. review progress in applying VAGO's 2007 audit recommendations and address those recommendations that have not yet been fully applied.

That the Victorian Managed Insurance Authority:

3. update the detailed guidelines to reflect the Department of Treasury and Finance's updates to the Victorian Government Risk Management Framework, incorporating our recommendations to address the gaps—including most importantly how agencies should manage interagency and statewide risks
4. develop a learning and development strategy to clearly guide and focus its support and training activities, prioritising actions for addressing the most significant weaknesses and gaps in current risk management practices. This strategy should be informed by the Department of Treasury and Finance's review of agencies' compliance.

That the Department of Treasury and Finance:

5. work with the Victorian Managed Insurance Authority to develop, clearly communicate and monitor the effectiveness of a whole-of-government framework for managing interagency and statewide risks with the intended outcomes.

Submissions and comments received

In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994 a copy of this report was provided to the Department of Treasury and Finance and the Victorian Managed Insurance Authority with a request for submissions or comments.

Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments are included in Appendix B.

1.1 Introduction

The scale, complexity and likely impacts of the demographic, health, security and climate challenges facing Victoria requires a mature, well-coordinated approach to public sector risk management.

Falling short of achieving this threatens the government's goals for growing the economy in a way that protects and enhances the environment while improving the quality of life of all Victorians. The Victorian Government Risk Management Framework (the Framework) recognises the importance of good risk management:

'Effective risk management is regarded as essential for the development and delivery of quality services' and, '…the government seeks to embed risk management into planning, delivery and reporting processes within and across public sector entities.'

(Victorian Government Risk Management Framework, page 4)

It is timely for VAGO to re-examine the government's approach to understand how far it has progressed along the path to effectively managing the risks facing the Victorian public sector.

1.2 VAGO's previous audit findings

VAGO's 2007 performance audit, Managing Risk Across the Public Sector: Toward Good Practice found that government actions to promote and better support risk management had improved the situation since VAGO previously audited risk management in 2003.

The 2007 audit found that formal risk management had become an accepted and widespread practice for the 25 public sector agencies examined.

However, the report made it clear that agencies, and the government as a whole, needed to progress beyond this basic level of risk management if they were to effectively manage the short- and long-term risks threatening their success.

The report singled out the management of interagency and statewide risks, extending beyond the boundaries of individual agencies, as a significant area of weakness. VAGO recommended that:

• central agencies and the Victorian Managed Insurance Authority(VMIA) develop a framework and guidelines to help public sector agencies better manage risk
• all agencies improve how they manage and report on the risks they face
• all agencies better manage interagency and statewide risks where, for example cross-government programs affect multiple agencies or, where risks are so widespread or the consequences so large, that they are of statewide significance
• the Department of Treasury and Finance (DTF) develop, and all agencies apply, a whole-of-government claims model to better manage self-insured financial claims against public sector agencies.

1.3 Progress since the 2007 audit

1.3.1 Victoria's updated risk framework

In 2007 the government introduced the Framework. VMIA published detailed guidelines in 2010 and DTF revised the Framework in 2011 to align with the updated Australian/New Zealand risk management standard: AS/NZS 31000:2009 (the AS/NZ standard).

While these actions are positive and addressed a subset of VAGO's 2007 recommendations, performance audits since 2007 have highlighted many examples where risks—especially interagency and statewide risks—are not well managed. Part 5 of this report summarises this evidence and forms a powerful imperative for further change.

1.3.2 Informing government about statewide risks

The government established the Whole-of-Victorian-Government Statewide Risks Interdepartmental Committee (IDC) in September 2012. The purpose of the IDC is to support the identification of major interagency and statewide risks and to support the development, operation and effectiveness of the whole-of-government risk management frameworks related to those risks.

The IDC has met four times and will report to government in late 2013 on the gaps in managing these risks and the type of framework needed to address these gaps.

1.4 Agencies' roles and responsibilities

1.4.1 Department of Treasury and Finance

DTF's main focus is on the financial implications of insurable and non‐insurable statewide risks for the Budget and the state's balance sheet. DTF supports the Treasurer and Minister for Finance in administering the Financial Management Act 1994. This includes maintaining appropriate compliance and risk management frameworks for capturing, monitoring and reporting on financial risks.

DTF is responsible for:

• developing and updating the Framework, and communicating changes
• reporting to the Minister for Finance on compliance
• since2012, developing a whole-of-government approach to statewiderisks.

DTF's view is that it is not responsible for ensuring that agencies meet the Framework's requirements. This accountability sits with each public sector agency, with DTF collating and reporting to government on agency compliance.

To monitor compliance, DTF manages a program to understand how well the government's Standing Directions are being applied for a sample chosen each year.

A review of the risk management direction has not been included to date because DTF assessed it as low risk due to the high level of self-reported compliance. The first review is set for 2014.

DTF does not play a significant role in supporting agencies beyond formally communicating the Framework and any updates to agencies' Chief Financial Officers. This role is played by the VMIA.

1.4.2 Victorian Managed Insurance Authority

VMIA has legislative responsibilities in relation to public sector agencies, including:

• assisting them to establish programs to identify, quantify and manage risks
• monitoring their risk management
• acting as their insurer
• providing risk management advice and training
• advising the government on risk management—statewide risks and the application of risk management by public sector agencies.

VMIA takes the lead in guiding and supporting agencies to apply the Framework by providing risk guidelines, training and support and by assessing agencies' risk maturity and areas for improvement through Risk Framework Quality Reviews.

1.4.3 Public sector agencies and applying the Framework

The Framework states on page 6 that:

'All agencies should adopt the Framework as part of good governance and corporate planning processes. However, application of the Framework is required by departments and those agencies that report in the annual Financial Report for the State of Victoria.'

Approximately 280 state controlled agencies are required to apply the Framework, while the remaining agencies, including local councils, universities and denominational hospitals, should adopt the Framework as part of following good practice.

1.5 Audit objective and scope

The audit objective was to examine the Framework's effectiveness by assessing whether:

• it is sound and provides clear and comprehensive information and instructions to support good practice
• DTF and VMIA—the agencies responsible for developing and supporting the Framework—provide adequate support and guidance to agencies to implement and comply with the Framework
• agencies acquit their Framework responsibilities and effectively manage risk.

The audit focused on how well DTF and VMIA have acquitted their responsibilities for managing the Framework and providing support and guidance.

We examined how six line agencies had applied the Framework and used these results to inform our conclusions about DTF and VMIA.

VAGO will write to each of these agencies separately to advise them of improvement opportunities that they should act on.

1.6 Audit method and cost

The audit examined the roles of DTF and VMIA in developing and updating the Framework through documentary reviews and interviews and by also reviewing the application of the Framework in six line agencies—three departments and three smaller health, education and legal sector agencies.

VAGO contracted an expert consultant to assist us by examining—under the first sub-objective—whether the Framework provides clear and comprehensive information and instructions to support good practice.

The expert:

• reviewed the Framework against the AS/NZ standard
• examined how the six line agencies had applied the Framework by reviewing their documentation and interviewing their risk managers
• interviewed relevant DTF and VMIA staff
• documented draft findings before finalising these after discussion with us.

The audit focused on the development and updating of the Framework between our 2007 audit and the present. Compliance and practice issues in specific agencies were tested against their current approach to risk management. However, in testing their attestation compliance we examined their annual reports dating back to 2010–11.

The audit was conducted in accordance with the Australian Auditing and Assurance Standards. Pursuant to section 20(3) of the Audit Act 1994, unless otherwise indicated any persons named in this report are not the subject of adverse comment or opinion.

The cost of the audit was $369 000 including both direct and indirect internal labour, a share of corporate overheads, and the costs of report printing and distribution.

1.7 Structure of the report

The report has four further parts:

• Part 2 examines the adequacy of the current Framework
• Part 3 examines how well VMIA and DTF are supporting and guiding public sector agencies in applying the Framework
• Part 4 examines how well agencies are following the Framework and draws out common compliance and practice issues that need to be addressed
• Part 5 brings together the evidence and recommends a way forward for the management of interagency and statewide risks.

2.1 Introduction

The purpose of the Victorian Government Risk Management Framework (the Framework) is to provide a minimum common risk management approach for public sector agencies. The goal of the Framework is the improved coordination and effectiveness of risk management across the public sector.

This Part examines whether the Framework is sound, and provides clear and comprehensive instructions to support good risk management practices, by examining:

• the Framework's strengths and areas for improvement
• how well the Framework addresses VAGO's 2007 audit recommendations
• whether the Department of Treasury and Finance (DTF) appropriately updates the Framework to keep it aligned to the Australian/New Zealand risk management standard: AS/NZS 31000:2009 (AS/NZ standard) and to address emerging issues and weaknesses.

2.2 Conclusion

The Framework is soundly based on the principles of the AS/NZ standard, but is neither comprehensive nor clear.

The introduction of the Framework in 2007 marked an important step in the evolution of risk management in Victoria. It provided a high-level approach consistent with the AS/NZ standard and made agencies accountable for attesting that they had complied with its requirements.

The 2011 Framework update kept it aligned with the revised standard, and incorporated some of the recommendations from VAGO's 2007 performance audit on public sector risk management. However, it did not significantly alter its structure or content.

The update planned for 2014 is timely because the Framework needs to be strengthened if it is to help public sector agencies move beyond simply complying with the current high-level requirements, to embedding effective risk management practices throughout and between their organisations.

DTF should work with the Victorian Managed Insurance Authority (VMIA) to update the Framework so it more clearly articulates the minimum requirements agencies need to meet to demonstrate that they are effectively managing risk. This will involve:

• improving the Framework's coverage of interagency and statewide risks
• describing the Framework's intent and purpose
• updating the attestation requirements
• better describing key risk concepts.

Currently the Framework does not specifically link to the VMIA guidelines and it would be sensible to create that link in any update of the Framework.

2.3 Strengths

The Framework is aligned to the AS/NZ standard, and provides a high-level description of the risk management requirements expected of agencies.

The strength of this generic approach is that agencies can tailor the Framework to align with their specific circumstances in terms of the level of risk they are prepared to tolerate given the type of environment they operate in.

The potential weakness of this approach is that agencies technically comply with the Framework without applying the practices needed to effectively control risks. This weakness is compounded by a lack of clarity about which parts of the Framework are mandatory and which parts are advisory.

The VMIA guidelines are extensive, detailed and practical but not mandatory.

2.4 Improvement opportunities

The audit identified areas where DTF should strengthen the Framework by:

• clearly describing the minimum requirements for effectively managing risk
• clearly defining the Framework's intent and purpose
• updating the attestation process
• better explaining key risk concepts and how to effectively apply them.

2.4.1 Setting clear minimum requirements

The current Framework does not clearly communicate to agencies what they are mandated to do and what it means for them to deliver on this mandate. DTF needs to update the Framework in consultation with agencies so that they clearly and consistently understand what compliance means.

Agencies do not currently share this type of common and complete understanding because the document is open to different interpretations, and important aspects of risk management are not fully explained.

Ambiguous language—'must', 'are required to', or 'should'

The Framework uses 'must' five times, 'required' or 'requirements' 54 times and 'should' 48 times, without explaining how to interpret these words. One of the six line agencies interpreted 'must' and 'required' as conveying mandated actions, and 'should' as conveying preferable but non-mandatory practices.

This potential ambiguity is illustrated on page 6 of the Framework where it states, 'all agencies should adopt the Framework as part of good governance' and, the subset of agencies that report in the AFR, where, 'application of the Framework is required…' This implies that 'should' means advisable and 'is required' means mandatory.

It is unclear whether these interpretations are valid and DTF needs to provide greater clarity about the minimum, mandated requirements when updating the Framework.

Better explaining how to manage interagency and statewide risks

The most significant gap is in describing what agencies need to do to contribute to managing interagency and statewide risks. The Framework inadequately describes the minimum requirements and how agencies can fulfil these.

The Framework needs to be amended to adequately convey what agencies need to do and how their actions fit within a clearly defined whole-of-government approach to managing these risks.

2.4.2 Clearly defining the Framework's intent and purpose

The Framework should better define its intent and purpose by describing:

• the purpose of risk management in the Victorian public sector context
• the government's risk management strategy and how agencies contribute to its achievement
• the role central agencies play in managing risks across the Victorian public sector
• who is available to support agencies and how they should access this support.

2.4.3 Updating the attestation process

Agencies are required to attest that:

• their risk management processes are consistent with AS/NZ standard
• their processes are effective in controlling risks to a satisfactory level
• a responsible body or audit committee verifies this assurance
• their risk profile has been critically reviewed within the past 12 months.

DTF should review and update the attestation process by:

• better defining how agencies demonstrate that they have met the four attestation requirements of the Framework through supporting documentation and evidence of implemented practices—this includes setting out minimum standards for compliance based on the risk profile of the agency
• reviewing and better explaining the wording used in the attestation and what agencies need to do if they decide not to use this wording—some of the sampled agencies suggested alternative variations for organisations at different stages in their risk management maturity
• introducing process to verify the accuracy of attestations—for example by validating a sample of agencies' attestations each year.

VMIA's 2008 and 2009 surveys of agencies covered the attestation and raised these issues. However, the 2011 Framework update did not address them, and DTF should use the 2014 Framework update to do this.

2.4.4 Provide better implementation guidance

Our review of the Framework identified that DTF needs to better explain key risk concepts to help agencies successfully transfer these into successful practices. Figure 2A describes these recommended changes.

Figure 2A

Better explaining what risk management means

Source: Victorian Auditor-General's Office.

2.5 Addressing VAGO's 2007 recommendations

VAGO's 2007 report, Managing Risk Across the Public Sector: Toward Good Practice, had eight recommendations and based on our work in this audit, our assessment of progress against those recommendations is shown in Figure 2B.

Our previous recommendations remain valid. They should, where not fully applied, be completed as a matter of priority.

Figure 2B

Summary and acquittal of recommendations from 2007 audit

Source: Victorian Auditor-General's Office.

2.6 Keeping the Framework current

The Framework update in March 2011 realigned it with the revised AS/NZ standard and incorporated some of the recommendations from the 2007 VAGO report. The 2014 update is an opportunity for DTF and VMIA to:

• improve the Framework by taking into account the issues raised throughout this audit and incorporating information from VMIA's annual reviews
• review and fully address VAGO's 2007 recommendations
• correct parts of the Framework—for example, VMIA is not responsible for the register of statewide risks.

Recommendations

That the Department of Treasury and Finance:

1. works with the Victorian Managed Insurance Authority to update the Victorian Government Risk Management Framework to clearly articulate minimum requirements that agencies need to meet to demonstrate that they are effectively managing risk—including improving the coverage of interagency and statewide risks, updating the attestation requirements and better describing its intent, purpose and key risk concepts
2. review progress in applying VAGO's 2007 audit recommendations and address those recommendations that have not yet been fully applied.

3.1 Introduction

Part 2 of this report focused on how the Victorian Government Risk Management Framework (the Framework) needs to be strengthened if it is to provide a solid platform for effective risk management.

However, achieving sustained improvement requires more than an amended Framework because:

• public sector agencies need guidance and support to translate the requirements into effective practices
• government needs to understand how well agencies are managing the risks that affect its objectives, and how it can continuously improve by removing the barriers slowing progress towards maturity.

This Part examines how well the Department of Treasury and Finance (DTF) and the Victorian Managed Insurance Authority (VMIA) have supported agencies and also how they have informed government about agencies' performance and areas for improvement.

3.2 Conclusion

VMIA takes the lead in providing agencies with guidance and support to help them apply the Framework and does a reasonable job in doing this.

While its guidelines are extensive and useful, the audit found gaps that need to be addressed. We support VMIA's intention to better focus its training and support by developing a learning and development strategy.

VMIA needs to update the detailed guidelines to reflect our recommended changes to the Framework and to address gaps—most importantly about how agencies should manage interagency and statewide risks.

VMIA should develop its learning and development strategy in consultation with DTF and prioritise addressing the most significant weaknesses and gaps in current risk management practices.

Reporting to government through DTF compliance reports and VMIA's annual review is not sufficient to convey key risk vulnerabilities, and the impact of coordinated action to address these.

Of most significance is the absence of guidelines on how interagency and statewide risks should be managed, and the lack of information to government on the significance of these risks, and how well they are being managed.

3.3 Guiding and supporting agencies

VMIA takes the lead in this area by providing risk guidelines, training, support and reviews, and assessing agencies' maturity and areas for improvement. These are valuable activities—helping agencies that are committed to change, to improve their risk management.

However, VAGO identified the following areas that need to be improved:

• VMIA's risk guidelines need to be updated in parallel with the Framework to reflect and explain the updated requirements and to address gaps—most importantly the absence of guidance on how to manage interagency and statewide risks
• VMIA should develop a strategy to clearly guide its support and training activities in consultation with DTF, with one clear driver being to address significant and systemic weaknesses in risk management.

DTF does not play a significant role in supporting agencies beyond formally communicating the Framework and any updates to agencies' Chief Financial Officers.

3.3.1 VMIA guidelines

In March 2010 VMIA published the guide, Risk Management: Developing and Implementing a Risk Management Framework. This 180 page document provides detailed information on how to implement a risk management framework that is consistent with the Australian/New Zealand risk management standard: AS/NZS 31000:2009 and the Framework. It takes the reader through all the stages of risk management—describing the processes that should be followed together with tips, practical examples and the various tools agencies should consider using.

The guide is extremely useful for systematically managing risks. If followed it would help the six agencies reviewed to address most of the practice deficiencies found. For example, it describes how to develop and use risk performance indicators and how to strongly align risk management with organisational goals.

However, the audit found gaps, and the guide should be improved by:

• including information on how to manage interagency and statewide risks
• expanding the description of risk assessment criteria, beyond traditional measures of consequence and likelihood, to include additional criteria such as speed of onset and vulnerability
• describing how to complete a deeper, more extensive analysis of critical risks
• taking into account the changes flagged for the Framework in Part 2 of this report, including better defining key risk concepts.

The guide should be updated and published so that it aligns with and reflects the 2014 Framework update.

3.3.2 VMIA Risk Framework Quality Reviews

Completing Risk Framework Quality Reviews (RFQR) is part of VMIA's role of monitoring and helping agencies improve their risk management. They are an independent review of an agency's risk maturity. These reviews involve VMIA staff interviewing agency representatives and reviewing the documentation showing how agencies manage their risks.

The 2011–12 reviews identified overall areas for improvement consistent with VAGO's findings for the six line agencies examined.

Between 2006 and mid 2012 VMIA completed 340 reviews, with 56 happening in the 2011–12 financial year. VMIA aims to cover the larger agencies every three years while also targeting agencies carrying significant risks, for example regional hospitals with high medical indemnity risks. VMIA introduced a new RFQR model setting a higher standard in 2010.

VMIA summarises the findings from its reviews in its annual Risk Management Report to the minister.

Figure 3A shows how VMIA classified the maturity of the agencies reviewed in 2011–12. The 'integrating' category is equivalent to what the annual report to the minister terms as compliant—the Framework is complete. 'Effective' involves more than this, including the embedding of good risk practice as part of everyday management.

For 2011–12 VMIA rated the average maturity level of the 56 agencies reviewed at the lower end of the 'effective' category shown in Figure 3A.

Figure 3A

Survey assessments of agencies' risk maturity

Source: Victorian Auditor-General's Office based on information from VMIA.

VMIA identified the following areas that need to be improved:

• strengthening governance foundations—improving board-level capabilities and the quality of risk management information, better clarifying roles, responsibilities, risk appetite and tolerance, and formalising the processes supporting annual attestation
• getting the processes and outcomes right—improving the quality of the decision-making information in risk registers, linking data and performance indicators to risks, better balancing strategic and operational risks, driving awareness and practical application across the organisation
• better aligning risk management to sources of assurance such as internal audit
• looking for opportunities to better manage risks between organisations.

VAGO notes that the review puts a relatively small weighting on inter-organisational risk management—this category made up only 3 per cent of the overall maturity score. This weighting clearly underplays the imperative for improvement in this area in light of the findings of this audit about the inadequacy of current practices for managing statewide and interagency risks and the formation of an Interdepartmental Committee to address these weaknesses.

3.3.3 VMIA training and support

VMIA plays the lead role in educating public sector agencies about risk management by offering extensive training opportunities. However, these activities—while informed by the reviews it completes and its perceptions of client requirements—have not been guided by a documented strategy setting out its objectives, an analysis of need, proposed actions, and indicators for measuring success.

VMIA advised us that it is forming a learning and development strategy that will fill this gap and VAGO supports this initiative.

VMIA professional development and training programs include:

• workshops on a wide range of risk management topics in Melbourne and regional Victoria
• seminars, forums, round table discussions and a biennial two day conference
• tailored development programs including e-learning and a Diploma in Risk Management.

In 2011–12, VMIA trained more than 2 000 of its clients and over 450 organisations attended at least one VMIA training event. In 2012–13 VMIA held more than 40 workshops and seminars on risk management.

VAGO supports the development of a formal learning and development strategy to harness this activity in a way that is demonstrably effective. Targeted training is an essential and complementary activity to updating the Framework and the supporting guidelines.

3.4 Informing government about risk management

The formal reporting to the Minister for Finance by DTF and VMIA is not sufficient for government to understand the effectiveness of agencies' risk management and the potential implications for achieving its policy objectives.

Government does not have access to sufficient information to fully understand agencies' preparedness to manage significant interagency and statewide risks that pose immediate and longer-term threats to it achieving its policy goals.

Since our 2007 audit clearly highlighted this issue progress has been slow, with the most significant development being the appointment of a Whole‑of‑Victorian‑Government Statewide Risks Interdepartmental Committee (IDC) in September 2012.

The IDC needs to seize the opportunity provided by its terms of reference to advise government on how best to identify and manage significant interagency and statewide risks. There needs to be a sense of purpose and urgency about this task because of the evidence of a management vacuum across a range of significant risks.

3.4.1 Established reporting mechanisms

In terms of the reporting to the Minister for Finance:

• DTF communicates how well agencies comply with Standing Direction 4.5.5
• VMIA in its 2011–12 annual review for the minister:
  • described emerging risks—based on an international review
  • updated its assessment of risk management practices across the Victorian public sector
  • reported on strategies and initiatives to reduce the total cost of insurable risk
  • described the risk management implications of local and international events.

Taken together, DTF's reporting of 97 per cent of agencies fully complying with the Framework and, VMIA's assessment that on average agencies risk management is at the lower end of the 'effective' category of maturity—that is at just above the 'compliant' category—informs government's understanding about the state of risk management across the Victorian public sector.

The VMIA report also described the practice areas most needing improvement.

This information, however, does not adequately convey the key vulnerabilities for the state and the need for urgent and coordinated action to address these. The high level of compliance gives a false sense of security because the result is not checked, and achieving compliance is not sufficient to assure government that risks are well managed.

Of most significance is the gap in the management of interagency and statewide risks. A coordinated, purposeful approach to assessing, communicating, treating and monitoring a wide range of statewide risks is not evident. Consistently and effectively managing these cross-agency risks is an area requiring urgent improvement.

3.4.2 Whole-of-Victorian-Government Statewide Risks Interdepartmental Committee

The first report of the IDC will be provided to government at the end of 2013 as the first step towards developing a better appreciation of the statewide risks affecting Victoria.

Figure 3B describes the IDC's terms of reference and key deliverables.

The IDC includes deputy secretaries from each department, the Chief Executive Officer of VMIA and is chaired by a Deputy Secretary from DTF. The IDC is meant to meet quarterly, and over the past 12 months it has met four times—in November 2012 and February, June and August 2013.

The IDC is due to provide its first report to the Minister for Finance and Cabinet by the end of 2013. DTF advised us that in line with its terms of reference, the report will identify significant gaps in statewide risk management, and recommendations about how the risk Framework should be updated as a first step to addressing these vulnerabilities.

Figure 3B

IDC's terms of reference and deliverables

Source: Victorian Auditor-General's Office based on Department of Treasury and Finance information.

Recommendations

That the Victorian Managed Insurance Authority:

1. update the detailed guidelines to reflect the Department of Treasury and Finance's updates to the Victorian Government Risk Management Framework, incorporating our recommendations to address the gaps—including most importantly how agencies should manage interagency and statewide risks
2. develop a learning and development strategy to clearly guide and focus its support and training activities, prioritising actions for addressing the most significant weaknesses and gaps in current risk management practices. This strategy should be informed by the Department of Treasury and Finance's review of agencies' compliance.

4.1 Introduction

This Part assesses whether our sample of six line agencies are acquitting their responsibilities under the Victorian Government Risk Management Framework (the Framework) and applying practices that mean they are effectively managing the risks they face.

Standing Direction 4.5.5 requires the approximately 280 public sector agencies, reporting through the Annual Financial Report for the State of Victoria, to manage risks according to the Framework.

Accountable officers for these agencies attest that risk management processes:

• are consistent with the Australian/New Zealand risk management standard: AS/NZS 31000:2009
• are effective in controlling risks to a satisfactory level
• have been verified by a responsible body or audit committee.

4.2 Conclusion

Agencies are largely, but not fully, compliant with the Framework's requirements. However VAGO is not assured that agencies practices have matured to the point that they are effectively managing risks.

The Department of Treasury and Finance (DTF) has not detected and addressed clear areas where agencies have not complied with the Framework.

Again, the most important deviation is the absence of specific processes for managing interagency and statewide risks. Except for those risks which have materialised and provoked deep review and structural change, VAGO is not assured that a wide range of statewide risks are being effectively managed.

For the six line agencies examined, risk management is for the most part compliance driven, although the audit found examples of it being used as a proactive management tool, and evidence of agencies working to continuously improve their practices and performance.

However, the audit identified six important areas of risk management where most or all of the line agencies examined needed to improve.

DTF, the Victorian Managed Insurance Authority (VMIA) and individual line agencies have parts to play in delivering improved risk management.

DTF and VMIA need to help agencies better manage interagency and statewide risks by communicating a clear whole-of-government framework together with guidance so that agencies are very clear about how to apply its requirements.

Agencies should start to identify, analyse and jointly manage interagency and statewide risks.

4.2.1 Complying with the Framework

The six agencies reviewed complied with most of the requirements of the Framework and were able to demonstrate this because their risk documentation:

• adequately articulates their approach to risk management
• sets governance structures, defined roles and responsibilities
• clearly describes objectives and the principles and processes to achieve these
• commits to integrating risk management in all business processes
• prioritises risks through ratings about probability and consequence
• establishes processes for escalating risks within each agency
• demonstrates that the secretary or chief executive officer attested to the Framework, and audit committees verified these attestations.

While our findings on compliance are mostly positive, the audit identified gaps that agencies did not communicate in their returns to DTF.

Figure 4A summarises the results and shows that:

• no agencies implemented the interagency and statewide risk requirements nor met the requirement to regularly report on under-deductible claims
• four of the six agencies had amended the wording of the attestation without adequately explaining why they had done this and what they planned to do about their frameworks, processes and control systems to address these qualifiers.

In addition two of the six line agencies could not demonstrate that they had annually reviewed their frameworks over the past three years as required.

Figure 4A

Areas of noncompliance for sampled line agencies

Source: Victorian Auditor-General's Office.

Based on this sample, DTF's record that 97 per cent of agencies fully complied with the framework in 2011–12 is questionable. In response DTF advised that it:

• is aware of widespread noncompliance for the first two requirements and is addressing this by:
• developing an improved whole-of-government approach to statewide risks
• strengthening the Ministerial Directions so that agencies apply a more structured and evidence-based approach to judging what risks they can bear, and ensuring they are able to meet financial impacts from existing resources
• does not check the attestation wording against the Framework templates.

One line agency interpreted parts of the Framework that are required as mandatory, and those prefaced with the word 'should', as better practice advice. VAGO does not agree with this interpretation but does agree that DTF should provide greater clarity about mandated minimum requirements when updating the Framework in 2014.

4.3 Effectiveness of risk management practices

While the audit found examples of agencies applying the Framework in a more mature way as a proactive, outcome focused tool, our overall finding is that risk management is still at a basic, compliance-driven level of maturity. While the Framework's elements are in place, the practices driving and shaping how agencies manage these risks need to be deepened and improved.

This is particularly the case for interagency and statewide risks where the Framework is clearly inadequate and VAGO is not assured that these risks are managed effectively. The clear exceptions to this overall finding are where a risk has materialised, with major or catastrophic consequences, provoking in-depth review and cross-government action—as with the 2009 Black Saturday bushfires.

4.3.1 Agencies' practices and level of maturity

The audit found that the line agencies examined needed to improve how they:

• manage interagency and statewide risks
• use organisational objectives to drive risk management
• document the analysis underpinning risk management
• select, prioritise and apply risk treatments
• embed risk management, and communicate good practice
• evaluate performance, including their use of risk performance indicators.

VMIA provide reasonable guidance for agencies to follow in relation to these areas with the exception of managing interagency and statewide risks.

These weaknesses mean VAGO is not assured that line agencies are effectively identifying, analysing and treating the full range of risks likely to affect them achieving their stated objectives.

DTF and VMIA need to help agencies understand and address these weaknesses by updating the Framework and VMIA's guidelines, monitoring agencies' performance and providing the support they need to improve.

The following information provides more detail on the nature of these weaknesses.

Managing interagency and statewide risks

This is both a compliance and a practice issue. None of the line agencies are adequately managing these risks.

Their policies and plans do not adequately deal with the assessment and coordinated treatment and reporting of these risks. Risk registers do not identify interagency and statewide risks, or the other agencies that are critical in managing these risks and how action is to be coordinated.

This is a significant omission that needs to be addressed by identifying, assessing and appropriately incorporating and treating these risks.

Using organisational objectives to drive risk management

None of the line agencies could demonstrate that the identified risks have been distilled from a comprehensive and rigorous analysis of threats and opportunities to their strategic objectives. VAGO is therefore not assured that agencies have completely captured the risks that are likely to have a significant impact on them achieving their objectives.

Depth of documented analysis underpinning risk management

VAGO expected to see a depth of analysis linked into the Framework consistent with VMIA's guidelines. This involves describing threats and opportunities, and quantifying the likely short- and longer-term consequences with and without existing controls.

The audit did not find this depth of analysis in the risk documents reviewed for any of the six line agencies. If this material exists outside of the risk documentation—in the form of studies and data generated by parts of the agency—it needs to be effectively linked in, with risk-related decisions and priorities clearly based on this evidence.

One of the six line agencies had included inherent risk ratings in its risk register. Assessing inherent and residual risk after the application of controls is important because it helps agencies understand:

• the effectiveness and value of, and reliance on current controls
• the potential impacts if these controls fail.

Selecting, prioritising and applying risk treatments

Line agencies selection, prioritisation and application of risk treatments fell short of the better practice described in VMIA's guidelines.

Agencies' risk registers summarised proposed actions and responsibilities but omitted information on the monetary and resource costs, budget allocations, and how and when they planned to apply treatments. The audit found no comparison of the costs and benefits of alternative treatments to inform agencies' decision-making, or an articulation of the relative priorities across included treatments.

DTF and VMIA need to help agencies improve how they inform treatment decisions and priorities and the rigour applied to their management.

Embedding risk management and communicating good practice

The VMIA guidelines talk about embedding a risk management culture. This means incorporating good and consistent risk management behaviours and practices in all members of an organisation. Critical to this is effectively communicating good risk management practices throughout an organisation.

None of the line agencies were able to show that good risk management practices had been successfully embedded throughout their organisations.

Only two of the six agencies had surveyed staff about their awareness of, and support for, their risk policies and practices. While these surveys are a positive step forward, the results showed that neither agency had embedded good practices across the majority of their staff. A further two agencies included a small number of questions in their annual staff surveys which were not sufficient to understand risk behaviours across their organisations.

None of the agencies had developed a risk communication and training strategy as part of a structured approach to raise awareness and performance about day‑to‑day risk management.

DTF and VMIA need to guide and support agencies to better communicate and embed good risk management practices.

Evaluating performance using risk performance indicators

VMIA's risk guidelines advise agencies to define risk management performance indicators that align with organisational performance. Well defined, reliable indicators are critical if agencies are to understand whether they are successfully managing risks.

Two of the six line agencies had developed and applied performance indicators for their high-priority risks. The remaining four agencies relied on qualitative reports to understand the nature of the threats faced, and the effectiveness of controls. This latter approach is clearly inadequate, and DTF and VMIA should help agencies to develop the intelligence they need to assess performance and continuously improve.

4.3.2 DTF and VMIA's roles in addressing these issues

The planned revision of the Framework in 2014, and our recommendations for VMIA to update the supporting guidelines and develop a learning and development strategy provide DTF and VMIA with the opportunity to focus their efforts to improve practices in these areas.

5.1 Introduction

This Part of the report focuses on interagency and statewide risks because these represent critical vulnerabilities for Victoria's public sector. The earlier parts of the report have shown the significant gaps in the current Victorian Government Risk Management Framework (the Framework) and line agencies practices in relation to these risks.

The government's policy goals are especially threatened by interagency and statewide risks because:

• they represent a dimension of risk management not normally present in the private sector risk approaches that have been applied in the public sector
• as government increases its focus on joined-up program solutions and outcomes, this necessarily entails a whole-of-government approach to risk management
• past VAGO audits have shown that many of these risks are not well understood or effectively managed.

This Part of the report summarises these findings before describing the characteristics of an effective approach to managing these risks.

5.2 Conclusion

Victoria is vulnerable to the impacts of interagency and statewide risks and especially those risks where the full force and significance of the consequences are expected to materialise in the medium- to long-term—five years or more from today.

The state is not well prepared to effectively manage these risks because it does not have a framework and established practices for understanding and effectively responding to them.

The audit evidence summarised in this Part of the report—and described more fully in Appendix A—shows that agencies have not been able to manage a wide range of these risks in a coordinated and structured way.

The consequences of failing to manage these risks in a timely way are likely to be significant, involving less prevention, more treatment and expensive retrofitting rather than cost-effective planning.

The Department of Treasury and Finance (DTF) needs to do the necessary groundwork to raise awareness and develop a coherent framework for managing these risks and line agencies need to incorporate these risks in their frameworks.

5.3 VAGO audits on interagency and statewide risks

Over the past seven years VAGO has reported on how well agencies are managing the following risks:

• data security, including managing the threats of cyber-attack and the misuse and loss of data
• lifestyle-related disease, and specifically the threats posed by type 2 diabetes
• climate change and the state's preparedness to manage the emerging impacts
• demographic change and its potential impacts on traffic congestion and liveability.

In addition, VAGO collated the findings of more than 200 audits covering the period 2006–12, describing recurring themes that are relevant to developing more effective risk management.

The results of these audits are summarised in Appendix A and consistently show that these risks are:

• highly significant, especially in terms of their medium- to long-term consequences
• poorly understood by public sector agencies and by government
• managed in a disjointed way without the benefit of clear structures and detailed plans that effectively engage all relevant public sector agencies.

Continuing with this type of approach is likely to see these risks materialise in an uncontrolled way that significantly impacts on all Victorians.

5.4 Improving how statewide risks are managed

5.4.1 Introduction

VAGO is not assured that the management of these types of risks by central and line agencies is effective and this is a major area of exposure for Victoria.

There are clear cross-government responses to a range of emergency management hazards where significant risks have materialised and provoked action. However, for a range of significant statewide risks representing imminent and longer-term threats, the current Framework is ineffective and needs to be improved.

DTF understands this is a significant gap. It commissioned a 2011 review of the Framework focusing on the approach to statewide risks. This confirmed that the current approach to these risks is a long way short of the maturity needed to effectively manage them.

The following information describes what is needed if these risks are to be effectively managed.

5.4.2 Characteristics of an effective approach

From the evidence reviewed and our discussions with DTF, a mature and effective approach to managing interagency and statewide risks would involve:

• clearly identifying the full range of potentially significant risks
• objectively measuring the likelihood, and current and forecast consequences
• clearly communicating these to government and affected agencies
• clearly allocating ownership and leadership responsibilities
• achieving coordinated cross-government action to develop and apply credible, evidence-based treatments that are costed, timed and actively managed
• developing and reporting on indicators that reliably measure performance in controlling the current and forecast consequences flowing from these risks
• effectively responding to performance measures in adapting mitigation measures.

Achieving this type of maturity requires actions in three ways:

• raising government's awareness and objective understanding so it can own these risks and exercise leadership in making informed decisions about how they are prioritised and treated—the Whole-of-Victorian-Government Statewide Risks Interdepartmental Committee (IDC) is meant to raise government's awareness and understanding
• defining an overarching body responsible for managing statewide risks, as the 2011 review recommended—this goes beyond the current role of the IDC
• equipping, guiding and monitoring agencies so they effectively play their parts in managing these risks within a clear and comprehensive management framework—these roles encompass leading the management of specific risks where required across government or applying cross‑government treatments under the lead of another public sector agency.

5.4.3 Next steps

In 2014 DTF plans to review compliance with the risk management Standing Direction and update the Framework, providing opportunities to make inroads into the third set of actions about how public sector agencies deal with these risks.

For DTF, it is more difficult to prescribe tasks, time lines and a critical path for better informing government so it can own and better lead the management of these risks and define the whole-of-government body responsible for managing them.

Setting up the IDC in 2012 is a first step in raising awareness. This should provide more analytical depth and a greater understanding of these risks, helping government to become a fully-informed decision-maker.

The IDC is scheduled to make its first report to government by the end of 2013. This report is likely to focus on identifying significant gaps in the management of statewide risks. VAGO understands this will form the basis for a more rigorous analysis of these attributes and the designation of lead agency status for managing high-priority risks.

DTF's approach to advancing this process is 'one step at a time' to build understanding and confidence in the process and its direction. As such it has not prepared the type of detailed implementation plan recommended by the 2011 review nor put definite time lines on how long it will take to progress.

While VAGO understands the need to build support for change among the key players, this needs to be balanced against the state's significant vulnerability to these risks and the urgent need to start addressing these issues. The gaps in addressing these risks need to be addressed in the shortest time practical because agencies are vulnerable to statewide risks which:

• could materialise at any time, for example if data and IT systems are insecure
• are almost certain to result in significant future consequences where properly informed actions and effective implementation are critical in mitigating the impacts.

Accordingly, VAGO would expect to see significant progress over the two years following the publication of this report, substantiated by:

• documented evidence of the significant statewide risks facing Victoria
• a whole-of-government framework that clearly describes how interagency and statewide risks should be identified, evaluated, treated and monitored, together with adequate central agency oversight arrangements.

Recommendation

1. That the Department of Treasury and Finance work with Victorian Managed Insurance Authority to develop, clearly communicate and monitor the effectiveness of a whole-of-government framework for managing interagency and statewide risks with the intended outcomes.

From some 200 audits VAGO identified the following recurrent themes that referred to issues relevant to risk management (Source: VAGO, Auditing in the public interest—Reflections on audits 2006–2012, December 2012):

• Planning and delivery of services and infrastructure—for example, by failing to accurately predict and understand changing demand patterns as the basis for timely planning and service provision.
• Quality information for decision-making—for example, government's reliance on poor quality information makes it more likely that it will make major investment decisions without an accurate understanding of the costs, benefits and risks.
• Real governance and effective oversight—for example, our audits have found joined-up arrangements between agencies were often inadequate diminishing the potential benefits of these types of programs.
• Measuring and communicating performance—for example, the absence of the systematic and reliable measurement of the outcomes achieved by government agencies was widespread. It means agencies are unable to properly assess success or work out how to do things better in the future.
• Procurement and contract management—for example, VAGO found procurement shortcomings that threaten the transparency, fairness and value for money of major contractual arrangements.
• Managing information transparently and securely—for example, in both our performance and financial audits VAGO found systemic and significant areas of weakness including password security, access controls, monitoring user activity around sensitive datasets, insecure storage of sensitive information and inadequate change management.