Do health services' systems and processes assure quality and safe care?
Why this audit is important
The safety and quality of Victorian health services is vital to all patients and their families and carers. It is therefore crucial that health services have rigorous clinical governance systems and cultures to deliver safe, person-centred and effective care.
In 2016, the Victorian Government commissioned an independent review to assess how the then Department of Health and Human Services was overseeing the quality and safety of patient care across the state. This review, known as 'Targeting Zero', made many recommendations to improve the health sector and requested us to audit progress in addressing them.
Who we examined
Ballarat Health Services, Djerriwarrh Health Services, Melbourne Health and Peninsula Health as a representative selection of Victorian public health services.
What we examined
If the four health services:
- set clear clinical governance expectations
- have established a culture of patient safety
- understand and respond to quality and safety risks at the board and executive levels.
What we concluded
Health services' systems and processes do not consistently ensure they are providing high quality and safe patient care.
None of the audited health services investigate all serious incidents promptly, and only one acts on recommendations in a timely way to prevent safety risks recurring.
Over four years since Targeting Zero, some health services are still not fully 'living' their local clinical governance frameworks. By not prioritising and engaging in this work, they are not doing enough to improve patient safety.
Differences in progress between the four health services relate to the differences in their size and consequently, their resources and the maturity of their systems to deliver quality and safe care.
What we recommended
We made 18 recommendations in total across Ballarat Health Services, Djerriwarrh Health Services, Melbourne Health and Peninsula Health. These aim to ensure health services implement their local clinical governance frameworks, improve their patient safety culture and enhance the quality of safety and quality performance reporting to health service boards. The four audited health services have accepted all our recommendations.
Note: * The People Matter Survey is an annual public sector survey. ** We excluded Djerriwarrh Health Services as there were only five serious incidents during our sample period. *** The nature of recommended actions Djerriwarrh Health Services had to undertake were relatively simple.
What we found and recommend
We consulted with the audited agencies and considered their views when reaching our conclusions. The agencies’ full responses are in Appendix A.
Establishing and embedding clinical governance frameworks
Health services are a range of organisations that provide healthcare, including public hospitals, as defined by the Health Services Act 1988.
The Victorian Clinical Governance Framework (VCGF) sets the Victorian Government's expectations for the systems and processes that public health services need to deliver safe and high-quality healthcare. The VCGF requires each health service to establish its own clinical governance framework to support its staff to work towards common goals. While accounting for local needs, these individual frameworks must comply with the VCGF.
Melbourne Health (MH) and Peninsula Health (PH) have met this requirement. Both health services have developed clinical governance frameworks that comply with the VCGF and have embedded them through initiatives to translate their expectations into practice. Through these efforts, MH and PH are reinforcing a consistent message to their staff and keeping them focused on achieving high-quality care.
In contrast, while Ballarat Health Services (BHS) started developing its framework in October 2019, it only completed it in January 2021—three and a half years later than expected. It has only recently started promoting and implementing it.
Djerriwarrh Health Services' (DjHS) framework does not fully comply with the VCGF because it does not identify underlying priorities to help it achieve its framework goals, such as activities to ensure a visible and engaged executive leadership. To date, DjHS has not implemented its framework; it has not actively promoted or embedded it in its operations to drive quality improvement activities.
As a result, unlike MH and PH staff, staff we interviewed at BHS and DjHS did not have a good understanding of their organisation’s clinical governance framework or the priorities and expectations it contains.
While the Department of Health (DH) requires health services to comply with the VCGF as part of its Policy and Funding Guidelines, it does not assess this requirement.
Recommendations about establishing and embedding clinical governance frameworks
|We recommend that:||Response|
|Ballarat Health Services||1. continues to implement its clinical governance framework by promoting and embedding priority actions to ensure staff have a clear understanding of how they contribute to safe, high-quality care (see Sections 2.1 and 2.2)||Accepted by: Ballarat Health Services|
|Djerriwarrh Health Services||2. revises its clinical governance framework to ensure it complies with the Victorian Clinical Governance Framework and completes implementing it as a priority to ensure staff have a clear understanding of how they contribute to safe, high-quality care (see Sections 2.1 and 2.2).||Accepted by: Djerriwarrh Health Services|
Establishing and supporting a positive patient safety culture
Health services with a positive patient safety culture are more likely to detect clinical risks early, which allows them to act and prevent avoidable harm to patients. A health service has a positive patient safety culture when staff:
- are safe from bullying and harassment and do not fear reprisal or retribution when they speak up about personal or patient safety concerns
- are confident to speak up to their peers and managers about personal or patient safety concerns and are confident that management will act
- actively engage in activities that maintain or increase their focus on safe and high quality care.
Only MH and PH have established a positive patient safety culture.
People Matter Survey
The PMS measures different aspects of workplace culture, such as job satisfaction and career development, across the Victorian public sector. It has a specific section on 'patient safety climate' for health services only.
The Victorian Public Sector Commission’s annual People Matter Survey (PMS) captures staff perceptions of the patient safety culture at their workplace and how safe they feel to speak up. At MH and PH, staff perceptions have improved since 2016, which was when the Victorian Government released the Targeting Zero: Supporting the Victorian hospital system to eliminate avoidable harm and strengthen quality of care report (Targeting Zero). This means that staff at these health services are now more likely to report incidents, understand the importance of quality improvement activities and participate in them to reduce the risk of patient harm.
By contrast, BHS and DjHS have not improved their relevant PMS results since 2016. DjHS’s results for bullying and how safe staff feel to speak up have deteriorated.
All four health services, especially DjHS, scored low results in:
- staff feeling safe from reprisal if they report improper conduct
- staff confidence in the integrity of investigations into safety issues.
This suggests that the audited health services need to be more transparent about their investigation processes and address staff concerns about reprisal to strengthen their patient safety cultures.
MH and PH have a comprehensive suite of initiatives to build and maintain a positive patient safety culture. They directly support their staff to speak up and provide multiple avenues for them to do so. For instance, MH trains its staff to use a communication tool that uses a stepped approach to raising and escalating concerns with colleagues. MH and PH also promote and reward desired values and behaviours, and MH has evaluated its initiatives to identify barriers and enablers to patient safety.
BHS and DjHS have initiatives to increase staff awareness on patient safety and set expected values and behaviours. However, they lack initiatives to develop their staff’s skills and confidence to speak up. This is concerning especially for DjHS, whose PMS results indicate that its staff do not feel safe to do so.
Recommendations about establishing and supporting a positive patient safety culture
|We recommend that:||Response|
|Ballarat Health Services and Djerriwarrh Health Services||3. implement initiatives that strengthen their staff’s skills and confidence in speaking up (see Sections 3.2 and 3.3)||Accepted by: Ballarat Health Services and Djerriwarrh Health Services|
|4. design and implement targeted initiatives to improve their staff’s psychological safety (see Sections 3.2 and 3.3)||Accepted by: Ballarat Health Services and Djerriwarrh Health Services|
|Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||5. evaluate their initiatives to assess if they have been effectively improving their patient safety culture and apply learnings for continuous improvement (see Section 3.3).||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
A health service board consists of individual directors that the Minister for Health appoints. Each board is responsible for the performance of its health service and is accountable to the Minister for Health. Each board also establishes a quality and safety subcommittee, which focuses on overseeing quality and safety risks and performance.
Clinical incidents are classified using an incident severity rating (ISR). This rating scale ranges from ISR 1, which is death or severe harm, to ISR 4, which is no harm or a near miss.
In this report, we use the term serious incidents to refer mainly to ISR 1 and ISR 2 incidents, while less serious incidents refer mainly to ISR 3 and ISR 4 incidents.
Identifying and responding to quality and safety risks
Gaps in how health service boards monitor quality and safety
Health services identify quality and safety risks by monitoring clinical incidents and their quality and safety performance indicators. A health service should prepare regular reports for its board so the board can assure DH, the Minister for Health and its local community that it is providing high-quality and safe care.
The boards at all four audited health services receive regular reports on incidents and quality and safety performance indicators. However, there are gaps in these reports that limit each board’s ability to assure that their health service is promptly identifying and addressing quality and safety risks and areas of underperformance. Specifically:
Reports provided by the relevant health service/s to the …
do not provide …
detailed updates on the implementation status of recommendations and actions.
a comprehensive account of overdue serious incident investigations and recommendations because they:
the status of serious incident investigations.
consistent and clear information on reasons for delays in implementing recommendations, or what actions executives are taking to address these delays.
consistent and clear information on the status of serious incidents.
the implementation status of recommendations.
PH, BHS and DjHS boards
regular analyses on common contributing factors to serious incidents.
MH, PH, BHS and DjHS boards
regular analyses on common contributing factors to less serious incidents.
Recommendations about gaps in how health service boards monitor quality and safety
|We recommend that:||Response|
|Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||6. provide updates on the implementation status of recommendations and actions in response to incidents at each board quality and safety subcommittee meeting, which at the minimum, include:
||Accepted by: Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
|Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||7. analyse common contributing factors to serious and less serious incidents and report findings to their board quality and safety subcommittee at least every six months (see Sections 4.2 and 4.3)||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
|Melbourne Health||8. analyses common contributing factors to less serious incidents and reports its findings to its board quality and population health subcommittee at least every six months (see Sections 4.2 and 4.3)||Accepted by: Melbourne Health|
|Ballarat Health Services||9. reports the status of its serious incident investigations to its board quality and safety subcommittee (see Section 4.2)||Accepted by: Ballarat Health Services|
|Djerriwarrh Health Services||10. improves the consistency and quality of its regular incident summary reports to the board by clearly indicating the status of ongoing incident investigations, including if there are overdue investigations and reasons for delays (see Section 4.2).||Accepted by: Djerriwarrh Health Services|
Analysing and responding to performance indicators
To comprehensively oversee its quality and safety performance, a health service board must:
- monitor sufficient and relevant performance indicators
- understand when results represent a genuine change in performance
- know that its executives are acting when performance does not meet targets.
A SOP is an annual accountability agreement between a health service and the Minister for Health. It includes indicators and performance targets that cover service quality, accessibility and financial viability.
In this audit, we focused on indicators specific to service quality and accessibility and excluded the ones relating to financial viability.
Statement of priorities
At a minimum, health services should monitor their quality and safety performance by tracking it against the mandatory key performance indicators (KPIs) in their statement of priorities (SOP) to have a sufficient view of safety and quality. Beyond their SOP, health services should also monitor KPIs that are relevant to their clinical governance framework goals to assess if they are achieving them.
Of the four audited health services, only PH monitors all of its SOP quality and safety indicators in its routine internal board reporting. MH monitors most of its relevant SOP indicators and BHS and DjHS have significant gaps. Specifically:
Internal quality and safety KPI reports to the board at …
do not include …
that relate to…
3 per cent of its SOP indicators (one out of 31 indicators)
patients waiting for elective surgery for longer than clinically recommended. However, it includes this metric in quarterly reports.
81 per cent of its SOP indicators (26 out of 32 indicators)
mainly adult mental health services, maternity services, patient experience and access to emergency and elective surgery.
91 per cent of its SOP indicators (10 out of 11 indicators)
mainly infection control, patient experience and care-associated infections.
See Appendix F for a full list of SOP KPIs that each audited health service monitors.
VAHI is part of DH. It publishes monthly, quarterly and annual 'Monitor' reports on public health services’ performance against the targets agreed in their SOP. Specifically, VAHI provides quarterly Monitor reports for health service boards.
BHS and DjHS's boards rely on quarterly ‘Monitor’ reports from the Victorian Agency for Health Information (VAHI) to review their performance against most of the SOP quality and safety KPIs that are not reported in their bimonthly and monthly KPI board reports respectively. The frequency of this reporting may be appropriate for some indicators, particularly when considering the resources required to generate additional reporting. However, monitoring the large majority of SOP indicators only quarterly limits these health service boards’ ability to identify and address any underperforming areas in a more timely way.
Since feedback from the audit, DjHS has expanded its monthly activity performance report and now includes most of the unreported SOP KPIs.
Beyond their SOP indicators, all health service boards regularly monitor additional KPIs in their internal quality and safety KPI reports that they identify as a priority (see Appendix G for a full list of these KPIs). Some common additional KPIs include:
- timeliness in providing hospital discharge summaries
- common clinical incidents, such as pressure injuries, falls and medication errors
- unplanned readmissions for specific community groups or medical conditions.
For MH and PH, who already monitor most of their SOP indicators, these additional KPIs mean their boards have a comprehensive view of their quality and safety performance. Both health services have also aligned their internal quality and safety KPI reports to their clinical governance frameworks to allow them to track their progress against specific clinical governance goals.
In contrast, while BHS and DjHS’s boards monitor their additional KPIs, these KPIs do not address the gaps in their SOP KPI monitoring, which is the minimum monitoring requirement set by DH. This means that BHS and DjHS’s boards are not monitoring sufficient KPIs to have a comprehensive view of service quality and safety.
Additionally, BHS and DjHS do not group and report on KPIs against their local clinical governance framework goals, unlike MH and PH. This means that BHS and DjHS’s boards cannot easily assess if they are meeting their clinical governance goals.
Identifying and investigating poor performance
Health services need to promptly identify quality and safety risks so they can swiftly address them and prevent harm. To do this, health service boards need comprehensive reports that present clear analysis of quality and safety performance, risks and actionable insights. In particular, these reports need to include long-term trend analysis and reasons for underperformance.
Only two of the four audited health services are doing this:
Board-level reports at …
As a result, these health service boards …
comprehensive in highlighting emerging risks and underperformance
can identify emerging quality and safety risks and hold their executives accountable for implementing improvements.
PH has a low threshold for initiating further investigations, which it calls 'in focus’ analyses, into underperforming KPIs and includes these as part of its board reports. 'In focus' analyses are comprehensive because they identify longer-term trends, include reasons for underperformance and note actions management is taking to improve performance.
not comprehensive in highlighting emerging risks and accounting for underperformance
are not well equipped to identify and effectively respond to significant quality and safety risks.
Recommendations about analysing and responding to performance indicators
|We recommend that:||Response|
|Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||11. adopt more statistical approaches to identifying true performance variations, such as using run or control charts (or equivalent statistical approaches), to detect significant changes over time and departures from expected statistical variation (see Section 4.2)||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
|Ballarat Health Services and Djerriwarrh Health Services||12. provide more detailed accounts to their boards regarding performance issues, including at a minimum:
||Accepted by: Ballarat Health Services and Djerriwarrh Health Services|
Investigating and responding to clinical incidents
Incident management and open disclosure policies
All audited health services have incident management policies that provide staff with clear information on their processes for reporting, investigating and responding to incidents.
SCV is administrative office of DH and is responsible for leading quality and safety improvements in the Victorian health system.
However, BHS and DjHS’s policies do not clearly emphasise the support available to staff involved in incidents. DjHS's policy also does not:
- stress that staff must act immediately to contain risks and treat harm
- clearly indicate time frames for completing in-depth case reviews (IDCRs) and Safer Care Victoria's (SCV) recommendations and action plan document.
All audited health services have open disclosure policies and procedures that are broadly consistent with the Australian Commission on Safety and Quality in Health Care's (ACSQHC) Australian Open Disclosure Framework. However, only MH routinely monitors its open disclosure process to ensure each case is promptly and appropriately actioned.
A sentinel event is a wholly preventable adverse event that results in a death or serious harm to a patient.
Delays in completing serious incident investigations
All audited health services complete sentinel event investigations within SCV's time frames. However, none of them consistently complete investigations into other serious incidents within timeframes set in their own policies. This means that these health services are not consistently identifying causal and contributing factors early enough, which creates a risk that similar serious incidents will recur. All health services stated that these investigations are delayed because they lack the required staff capacity and capability.
Recommendation about completing serious incident investigations
|We recommend that:||Response|
|Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||13. increase staff capacity and capability to meet timeliness requirements for completing incident investigations (see Section 4.3).||Accepted by: Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
No regular thematic analyses of incidents
Health services should undertake regular, service-wide thematic analyses of all serious (ISR 1 and ISR 2) and less serious (ISR 3 and ISR 4) incidents to determine common themes or clusters relating to:
- clinical divisions or units
- common causal or contributing factors.
Not all health services undertake regular analyses of serious incidents. Only MH analyses serious incidents to identify any underlying themes every six months. While BHS analyses serious incidents monthly to identify themes, its analysis does not go into enough detail to identify common contributing factors. For instance, in February 2020, BHS identified that 70 per cent of medication errors involved its ‘administration process’ but did not further examine specific issues in its process for administering medication. BHS's individual monthly analyses also do not identify if common contributing factors recur across multiple months.
PH and DjHS analyse serious incidents for common themes on an ad-hoc basis. For DjHS, this also depends on it having sufficient data available for analysis.
None of the four audited health services undertake regular thematic analyses of less serious incidents. Consequently, they risk not identifying clusters of incidents or common factors underlying individual incidents.
Recommendations about thematic analyses of incidents
|We recommend that:||Response|
|Melbourne Health||14. undertakes thematic analyses of less serious incidents every six months at a minimum and includes them in its clinical incidents themes and trends report to its board quality and population health subcommittee (see Section 4.3)||Accepted by: Melbourne Health|
|Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||15. undertake thematic analyses of serious and less serious incidents at least every six months and report them to their board quality and safety subcommittee (see Section 4.3).||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
Delays in completing serious incident recommendations
If a health service identifies a risk through a serious incident investigation, it needs to quickly address it to prevent the incident from recurring.
Of the audited health services, only MH had no overdue recommendations from its serious incident investigations. PH, BHS and DjHS are not implementing their recommendations within their own specified time frames.
The reasons for PH and DjHS’s delays in implementing recommendations are unclear. Concerningly, at April 2020 during our audit conduct, the majority (70 per cent) of BHS’s recommendations were overdue, which put patients at risk of known and avoidable harm. BHS advised us that these delays are due to resourcing and skill deficiencies in its centre for safety and innovation team, which it has been working to build and address.
Time taken to implement recommendations
MH and PH were comparable in the typical amount of time they took to implement their recommendations—ten and six months respectively. Broadly, both health services took longer to implement more complex recommendations, which is reasonable.
In contrast, BHS typically took longer to implement its recommendations (16 months) and there is no distinct pattern as to why some recommendations took longer than others. This further indicates that BHS is not acting early enough to prevent future harm to patients.
DjHS took the least amount of time to implement its recommendations—about two months—but many of its actions were to mitigate basic risks, such as keeping walkways clear, rather than more comprehensive actions to address root causes.
Recommendation about implementing serious incident recommendations
|We recommend that:||Response|
|Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||16. identify and address factors contributing to delays in completing serious incident recommendations (see Section 4.3).||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services|
Assessing the impact of implemented recommendations
Following a serious incident investigation, a health service should identify how it will assess if its actions are effectively preventing similar patient harm.
None of the four audited health services consistently identify appropriate measures to assess the effectiveness of their implemented actions. Of the 16 serious incident action plans we assessed across these health services, only two identify measures to assess if a causal or contributing factor recurs.
Two factors contribute to the audited health services’ lack of effective monitoring:
health services do not routinely identify measures to assess the impact of their actions because …
while SCV requires health services to document a relevant 'outcome measure' to assess the effectiveness of their actions, it does not define or provide any guidance on what an outcome measure should be. As a result, health services tend to only record that they have completed an action and not how they will measure its impact.
other serious incidents
with the exception of BHS, their IDCR templates do not require staff to include measures to assess the impact of any actions. While BHS's template does require staff to include these measures, it does not provide guidance to staff on how to identify appropriate measures.
Recommendations about assessing the impact of implemented recommendations
|We recommend that:||Response|
|Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||17. include impact assessments as a standard requirement of action plans following serious incident investigations and provide guidance to staff on appropriate measures to assess impact (see Section 4.3)||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services
Partially accepted or accepted in principle by: Melbourne Health
|Melbourne Health, Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services||18. report the results of impact assessments in serious incident action plans to their board quality and safety subcommittees so they can be assured that recommendations have been effective. (see Section 4.3)||Accepted by: Peninsula Health, Ballarat Health Services and Djerriwarrh Health Services
Partially accepted or accepted in principle by: Melbourne Health
Health services provide care in complex and high-pressure environments where avoidable harm to patients can occur. Effective clinical governance cultures, systems and processes minimise this risk and reduce the potential for harm.
A health service's clinical governance framework describes the activities it will undertake to minimise harm and maximise the quality of patient care. Health services must meet national and state standards for clinical governance.
This chapter provides essential background information about:
According to ACSQHC, clinical governance refers to systems and processes that maintain and improve the reliability, safety and quality of healthcare provided to patients. Strong clinical governance results in healthcare that is safe, effective, patient centred and continuously improving.
During 2013 and 2014, there was a cluster of perinatal deaths at DjHS. Subsequent reviews found that there was inadequate clinical governance at the health service and it was not monitoring and responding to adverse clinical outcomes in a timely way.
Following these incidents, the Minister for Health requested the then DHHS to commission the Targeting Zero review.
Victorian health services must meet national and state standards for clinical governance. This includes the VCGF, which SCV developed in response to Targeting Zero's recommendations. This review primarily focused on how the then Department of Health and Human Services (DHHS) was managing, overseeing and monitoring quality and safety across the health system. Where relevant, it also briefly examined health services' clinical governance management, oversight and monitoring.
Targeting Zero's recommendations included establishing new agencies (see Section 1.3) and systems to support more effective clinical governance, including frameworks and projects to improve the practical capability of Victorian health services.
This is the first of two performance audit reports that follow up on the sector's progress since the Targeting Zero review. The review recommended that we assess the sector's progress in implementing recommendations in 2020. This report examines health services' clinical governance systems and processes, with a particular focus on actions taken at the board and executive levels. The second report examines DH's oversight of clinical governance across the health system.
1.2 Clinical governance standards and expectations
National Safety and Quality Health Service Standards
All Australian health services must be accredited against the National Safety and Quality Health Service Standards (NSQHS Standards) to operate. We outline them in Figure 1A.
FIGURE 1A: NSQHS Standards
|Standard||What it looks like|
|1. Clinical governance||Continuous improvement of the safety and quality of health services and ensuring that health services are patient centred, safe and effective|
|2. Partnering with consumers||Partnering with consumers to plan, design, deliver, measure and evaluate care|
|3. Preventing and controlling healthcare-associated infections||Systems to prevent, manage or control healthcare-associated infections and antimicrobial resistance to reduce harm and achieve good health outcomes for patients|
|4. Medication safety||Systems to reduce the occurrence of medication incidents and improve the safety and quality of medicine use|
|5. Comprehensive care||Systems and processes to support clinicians to deliver comprehensive care and establish and maintain systems to prevent and manage specific risks of harm to patients during the delivery of healthcare|
|6. Communicating for safety||Systems and processes to support effective communication with patients, carers and families; between multidisciplinary teams and clinicians; and across health service organisations|
|7. Blood management||Systems to ensure the safe, appropriate, efficient and effective care of patients’ own blood, as well as other blood and blood products|
|8. Recognising and responding to acute deteriorations||Systems and processes to respond effectively to patients when their physical, mental or cognitive condition deteriorates|
Source: VAGO, adapted from ACSQHC's NSQHS Standards (second edition), 2017.
Standard 1 specifically relates to clinical governance and requires health services to 'implement a clinical governance framework that ensures that patients and consumers receive safe and high-quality health care'. Health services must use their local clinical governance framework when implementing policies and procedures, managing risks and identifying training requirements for other standards.
Victorian Clinical Governance Framework
Under DH's Policy and Funding Guidelines, all Victorian health services must comply with the VCGF, which requires them to:
- establish a clinical governance framework that complies with the VCGF
- implement their framework by:
- socialising it with their staff
- using it
- improving it.
The VCGF sets out expectations regarding best-practice clinical governance for Victorian public health services. It describes the principles of effective clinical governance, which Figure 1B outlines, and identifies five domains required for implementing these principles, which Figure 1C shows.
FIGURE 1B: The VCGF’s clinical governance principles
|Clinical governance principle||What it looks like|
Excellent consumer experience
Commitment to providing a positive consumer experience every time
Clear accountability and ownership
Partnering with consumers
Consumer engagement and input is actively sought and facilitated
Effective planning and resource allocation
Staff have access to regular training and educational resources to maintain and enhance their required skill set
Strong clinical engagement and leadership
Empowered staff and consumers
Proactively collecting and sharing critical information
Openness, transparency and accuracy
Health service reporting, reviews and decision-making are underpinned by transparency and accuracy
Continuous improvement of care
Rigorous measurement of performance and progress is benchmarked and used to manage risk and drive improvement in the quality of care
Source: the VCGF.
1.3 Structure of Victoria's health system
Department of Health
From 1 February 2021, DHHS split into DH and the Department of Families, Fairness and Housing.
DH manages the Victorian public health system. It oversees and monitors the state's health services. According to the Victorian health services Performance Monitoring Framework 2019–20 (Performance Monitoring Framework), DH is responsible for:
- partnering with health services to identify and address performance concerns early and effectively
- supporting or intervening to ensure long-term and sustained performance improvement
- making use of available data and intelligence to maximise the depth and breadth of information used to assess health services’ performance
- enhancing health service boards’ skills and capabilities in clinical governance and other required information to ensure high-quality and safe care.
DH sets the rules for all Victorian health services through the Policy and Funding Guidelines. Annual service agreements, or SOPs, outline the Minister for Health's key performance expectations, targets and funding for public health services. DH monitors health services' performance against these expectations and targets using the Performance Monitoring Framework.
Victorian Agency for Health Information
VAHI is a business unit of DH that analyses and shares information across the health system. It does this by identifying measures of patient care and outcomes and using them for public reporting, oversight and clinical improvement. Its key functions include:
- collecting, analysing and sharing data so the community is better informed about health services and health services receive better information about their performance
- providing health service boards, executives and clinicians with the information they need to best serve their communities and provide better, safer care
- providing patients and carers with meaningful and useful information about care in their local community
- improving researchers’ access to data to create evidence that informs the provision of better, safer care.
VAHI provides regular quality and safety reports to public health services, DH and SCV.
Safer Care Victoria
SCV is an administrative office of DH and is responsible for leading quality and safety improvements in the state's health system. SCV’s key functions include:
An administrative office is a public service body that is separate from a government department but reports to the department’s secretary.
- supporting health services to prioritise and improve the safety and quality of patient care by, for instance, developing and providing best-practice resources
- implementing targeted improvement projects across the health service system
- providing independent advice and support to health services to help them respond to and address serious quality and safety concerns
- reviewing health services’ performance, along with DH, to investigate and improve patient safety and quality of care
- monitoring sentinel events reported by health services, as well as the quality of health services’ investigations and how successful their actions are to prevent similar events recurring
- undertaking reviews of systemic safety issues to identify areas for local and system-level improvement.
SCV partners with VAHI to monitor and review individual health services’ performance data and advises health services and DH about areas for improvement.
Public health services
Under the Health Services Act 1988, the Minister for Health appoints independent boards for health services, except for denominational and privately owned public hospitals. Boards must have effective and accountable risk management systems. This includes systems and processes to monitor and improve the quality, safety and effectiveness of the health services provided.
The Performance Monitoring Framework requires public hospitals and health services to:
- partner with DH and other agencies to improve performance at an individual and system-wide level
- promptly report any emerging risks or potential performance issues, including immediate action taken, to DH
- establish and maintain a culture of safety and performance improvement
- submit data and other information, including information about implementing agreed action plans and status update reports, in an accurate and timely way
- collaborate with other health services and system partners to maintain and improve performance and meet community needs.
1.4 Audited health services
This audit examined four public health services, which Figure 1D shows. These public health services differ in size, location and clinical capacity. This means they have different local contexts to consider as they adapt their local clinical governance frameworks to the VCGF.
FIGURE 1D: Locations and campuses of audited health services
Djerriwarrh Health Services' operating environment
DjHS has been operating under an administrator since October 2015. Under the Health Services Act 1998, the administrator has and may exercise all of the board’s powers and is subject to all of the board’s duties. Given this, we refer to the administrator as 'the board' in this report.
Since its serious patient safety issues in 2013 and 2014, DjHS has been operating in a significantly challenging environment due to:
- multiple senior executive changes, including its CEO, between 2015 and mid-2018
- lengthy proceedings with the Australian Health Practitioner Regulation Agency and the Victorian Civil and Administrative Tribunal
- the December 2019 announcement that it will potentially voluntarily amalgamate with Western Health, and the subsequent consultations as part of the potential amalgamation process.
Not all audited health services have embedded their clinical governance frameworks in their organisations. While their frameworks are generally consistent with the VCGF, only MH and PH use their frameworks to identify specific local quality and safety priorities, raise staff awareness and drive changes in organisational practices.
Our comparison of health services' progress in embedding clinical governance shows the difference between having a document and applying it in practice. Over four years since Targeting Zero, all health services should be 'living' their local clinical governance frameworks, but this is not yet the case.
This chapter discusses:
Two of the four audited health services—BHS and DjHS—are not yet meeting DH’s requirement to have an established and embedded local clinical governance framework that complies with the VCGF.
A health service’s local clinical governance framework complies with the VCGF when:
- its definition of high-quality care is consistent with the VCGF
- it includes activity domains that are consistent with the VCGF and has corresponding priorities and activities in each domain.
The VCGF defines high-quality care as:
- safe—'avoidable harm during delivery of care is eliminated'
- effective—'appropriate and integrated care is delivered in the right way at the right time, with the right outcomes for each consumer'
- person centred—'people’s values, beliefs and their specific contexts and situations guide the delivery of care and organisational planning. The health service is focused on building meaningful partnerships with consumers to enable and facilitate active and effective participation'.
As Figure 1C shows, the VCGF identifies five underlying activity domains that health services require to achieve high-quality care. It also recommends a range of activities for each domain. Using the VCGF as a guide, health services should:
- reflect on and identify their own priority activities for each VCGF domain
- articulate these activities as priorities in their local clinical governance framework.
Figure 2A shows the progress of each audited health service in meeting these requirements.
FIGURE 2A: Audited health services' progress in implementing clinical governance framework requirements as at June 2020
|Title of local clinical governance framework||Clinical Governance Framework||Peninsula Care Framework (Peninsula Care)||Quality and Safety Framework||Governance Framework|
|Definitions align with the VCGF||✓||✓||✓||✓|
|Domains align with the VCGF||✓||✓||✓||✓|
|Local priority activities to support achievement in domain areas are included||✓||✓||✕||✕|
|Progress status||Actively using the framework and undertaking ongoing monitoring to improve it||Completed and socialised the framework and is identifying how to monitor its implementation||Completed the framework but has not socialised and embedded it||Recently completed the framework and is starting to socialise and embed it|
For PH and BHS, connected care is primarily focused on providing an integrated care pathway. This means care that reflects the patient's various needs, is matched to the different clinicians and services required and works together in a coordinated way.
Three of the audited health services chose to expand their definition of high-quality care, with MH including 'timely care' and PH and BHS including 'connected care' in their frameworks. These additions prompt these health services to prioritise and monitor these aspects of safe and quality care.
Only MH and PH have identified priority activities within each of their activity domains to target their improvement efforts. BHS only identified its priorities in January 2021, and these priorities were still to be approved by BHS's quality care committee (board subcommittee) in early 2021.
DjHS has not identified priority activities in its Quality and Safety Framework. Instead, DjHS relies on a range of other policies and plans to operationalise clinical governance, such as its strategic and operational plans and Safe Practice Framework. Not having this information in a consolidated document makes it difficult for DjHS to identify gaps and for staff to easily understand the range of activities needed to provide safe, effective and person-centred care.
Catalysts and challenges
We found that the audited health services' progress in implementing the VCGF has been affected by:
- leadership, including leadership changes and if leaders champion patient safety and align their local clinical governance framework with the VCGF
- organisational culture, particularly the extent to which health services' executives and managers address bullying and harassment, occupational violence and aggression, and prioritise staff’s physical and psychological safety.
A major challenge for health services in implementing their local clinical governance frameworks has been staff lacking awareness and knowledge about:
- the importance of clinical governance
- how they and their work unit should contribute to delivering high-quality care to patients.
As Figure 2B shows, each of the four health services have experienced different catalysts and challenges associated with implementing the VCGF and establishing and implementing their own clinical governance frameworks.
FIGURE 2B: Catalysts and challenges associated with implementing the VCGF and local clinical governance frameworks
Keeping staff engaged and reminding them about the importance and impact of effective clinical governance in the workplace. MH addresses this through ongoing, multiple channels for raising awareness and feedback
A new CEO with a focus on clinical governance
New senior executives and board members recognised the need to improve organisational culture as a key priority
Lack of awareness across the organisation about the VCGF and its importance as a tool for prioritising activities to strengthen clinical governance
2.2 Embedding clinical governance frameworks
Clear staff roles and responsibilities
To meaningfully implement a clinical governance framework, a health service needs to express how the framework translates to staff roles and responsibilities. This enables staff to have a clear understanding about how they contribute to providing high-quality care.
All four audited health services' local clinical governance frameworks identify staff roles and responsibilities for achieving high-quality care that broadly align with the VCGF. Health services have also formalised their expectations of staff by adding quality and safety roles and responsibilities to staff position descriptions, which Figure 2C shows. When health services identify defined roles and responsibilities, staff have a clear and consistent understanding of their part in providing high-quality care. They can also be held accountable if their behaviour does not align with stated expectations.
FIGURE 2C: Examples of how audited health services have included quality and safety expectations in staff position descriptions
STEP is an acronym that MH uses to represent its high-quality care goals of safe, timely, effective and person-centred care in its clinical governance framework. MH staff commonly refer to STEP when talking about the health service's clinical governance framework.
|Health service||Position description updates|
Now includes a standard clinical governance framework section. This section states that employees are responsible for delivering safe, timely, effective and person centred (STEP) care and outlines a few ways that employees can achieve this, including:
Now includes a standard quality and safety section. This section emphasises that staff are responsible for ensuring:
Now includes an 'occupational health, safety and quality responsibilities' section for each position description. The content in this section varies for executive staff, managers and other employees. Examples of executive staff and the CEO’s responsibilities are:
Examples of managers’ responsibilities are:
Now includes a standard 'quality improvement' section that outlines a range of responsibilities, including:
Building expectations into day-to-day activities
As figures 2D, 2E and 2F show, MH and PH have implemented extensive initiatives to translate their local clinical governance frameworks into practice. These initiatives collectively reinforce a consistent message to staff across the health services and maintain their focus on achieving high-quality care.
In contrast, BHS and DjHS are yet to demonstrate the impact of their clinical governance frameworks on achieving high quality care. This is because they have implemented few initiatives to embed their frameworks. We note BHS and DjHS's initiatives in Figure 3D in Chapter 3.
FIGURE 2D: Common MH and PH initiatives to embed their clinical governance frameworks
|Ward or unit plans/noticeboards||Improvement boards||Peninsula Care placemats|
|Quality and safety huddles (based on clinical governance framework goals)||Ward staff improvement huddles||-|
|Reorganisation of board quality and safety subcommittee meeting agendas||-||In accordance with Peninsula Care goals of safe, personal, effective and connected care|
|Restructuring KPI reporting to prioritise focus areas within clinical governance framework||In accordance with MH's STEP goals||In accordance with Peninsula Care goals of safe, personal, effective and connected care|
|Staff orientation||Half-day presentation to introduce the five domains that support STEP||Introduces Peninsula Care and emphasises staff responsibility to speak up for safety|
FIGURE 2E: Case study: MH's approach to embedding its local clinical governance framework
Improvement boards and huddles
Each ward at MH has an ‘improvement board’ on display in its staff meeting room. These noticeboards present a range of information, such as:
- selected STEP performance indicators and corresponding data
- staff suggestions for improvements and their implementation status
- staff recognition.
The wards also hold weekly ‘improvement huddles’ where staff across all disciplines discuss the information presented on the boards.
MH acknowledges that there should be a level of consistent information recorded on improvement boards across wards and in improvement huddles. MH is still in the process of fully embedding this program and working to achieve consistency across its wards.
MH evaluated its improvement huddles pilot and found they increase team engagement because they involve staff across different clinical disciplines coming together. MH found that staff value the opportunity to communicate and exchange views with their colleagues.
Staff also noted that this approach has improved their and their patients’ experience.
The evaluation also found potential barriers to the huddles’ success, such as:
- variable attendance at huddles by the nurse unit manager or assistant nurse unit manager
- heavy reliance on the nurse unit manager or assistant nurse unit manager's capability and capacity to lead the huddles
- duplication with other huddles.
MH is aware that it needs to provide ongoing coaching to staff to embed and sustain this initiative.
The picture below shows a template of what an improvement board looks like.
Note: Parking lot is an area for staff to ‘park’ ideas to consider later.
Source: VAGO, based on information from MH. Image supplied by MH.
FIGURE 2F: Case study: PH’s approach to embedding its local clinical governance framework
Local Peninsula Care plans and placemats
PH has established a planning and goal-setting process across its local clinical units. Each unit develops a:
- local Peninsula Care plan
- Peninsula Care placemat.
Local units’ Peninsula Care plans identify the specific actions for each unit and categorises them by both the NSQHS Standards and Peninsula Care’s goals. These plans:
- state who is responsible for completing a specific action(s)
- state the action’s expected date of completion
- detail progress against specific actions
- evaluate and include evidence for completed actions.
To share goals and remind staff and visitors how they can contribute to Peninsula Care, local units summarise their Peninsula Care plans and major activities on placemats. Units display placemats publicly.
Peninsula Care plans are a useful tool for local units to identify improvement areas. However, the plans are currently administratively burdensome. PH is examining ways to automate reporting on local plans to reduce the burden on staff.
The following image is an example of a Peninsula Care placemat.
Note: CRP refers to community rehabilitation program. COPM refers to the Canadian occupational performance measure. FIM refers to functional independence measure. SDP refers to the stroke detours program.
Source: VAGO, based on information from PH. Image supplied by PH.
MH and PH have made greater improvements to their patient safety cultures since Targeting Zero, than BHS and DjHS. They have done this by embedding their clinical governance frameworks in their organisations and supporting staff to actively uphold patient safety.
In contrast, BHS and DjHS have not implemented an appropriate mix of quality and safety initiatives. In particular, they lack actions to build their staff’s confidence to speak up about safety concerns. By not prioritising and engaging in this work, these health services are not doing enough to ensure patient safety.
This chapter discusses:
If a health service has a positive patient safety culture:
- staff feel safe to speak up when they have concerns about patient safety
- the health service is committed to learning from errors
- the health service responds to warning signs early and avoids catastrophic incidents.
DH and health services annually assess patient safety culture through the PMS (this survey did not occur in 2020 due to the coronavirus (COVID-19) pandemic). The survey provides insights into how health service staff perceive their wellbeing in the workplace and the patient safety climate at their health service. Factors that threaten staff wellbeing, including a lack of psychological safety, contribute to a poor patient safety culture.
Three interrelated dimensions contribute to an ongoing positive patient safety culture:
The dimension of
means staff are …
and it is crucial for establishing and supporting a positive patient safety culture because it …
staff psychological safety
To promote a positive patient safety culture, health services need to implement initiatives that contribute to these three dimensions. Health services should also evaluate their initiatives to assess if they are achieving the intended outcomes.
3.2 Patient safety culture at audited health services
All Victorian health services should have strengthened their patient safety culture following Targeting Zero as part of the sector's renewed focus on improving care.
We analysed the audited health services' results for selected PMS questions between 2016 and 2019 on:
- staff experience of bullying
- staff confidence in speaking up
- patient safety climate.
A 95 per cent confidence interval is the range of values that you can be 95 per cent confident that the true value lies within.
We looked for changes in the results between 2016 and 2019 and analysed the data to assess if these changes were statistically significant at a 95 per cent confidence interval. We also looked at the raw results to identify areas where further improvements are necessary.
Improvements since Targeting Zero
Of the four audited health services, only MH has consistently improved its staff’s perceptions of patient safety culture across the PMS measures since 2016. While PH’s results have declined for a minority of patient safety culture measures, it has improved its performance overall. This means that since 2016, MH and PH staff:
- better understand the importance of quality improvement activities
- are more likely to participate in these activities to reduce the risk of patient harm
- are more likely to report incidents.
In contrast, BHS and DjHS have not made substantial improvements since 2016, and for DjHS, some results have deteriorated.
Figure 3A summarises these changes between 2016 and 2019.
FIGURE 3A: Changes in staff’s perceptions of workplace wellbeing in the four audited health services from 2016 to 2019
Source: VAGO, based on PMS data between 2016 and 2019.
Areas for further improvement
The PMS data in Figure 3B shows that the percentage of staff who have experienced bullying at MH, PH and BHS has declined since 2016. All four health services also have fair to strong results regarding:
- how confident staff feel about reporting concerns to their manager
- staff feeling encouraged by colleagues to report concerns
- how management responds to and handles safety issues.
However, DjHS has seen an increase in staff reporting that they have experienced bullying. All four health services, but especially DjHS, also have significant room to improve:
- staff feeling safe from reprisal if they report improper conduct
- staff confidence in the integrity of investigations into safety issues.
This indicates that the audited health services can do more to transparently demonstrate the quality of their investigation processes to staff and address their concerns about reprisal, which the low results strongly suggest is occurring. Unless directly addressed, these factors will continue to act as barriers to these health services developing and maintaining a strong patient safety culture. We discuss the persistent patient safety culture issues at DjHS in more detail in Figure 3C.
FIGURE 3B: The four audited health services’ PMS performance results for 2016 and 2019
|MH (per cent)||PH (per cent)||BHS (per cent)||DjHS (per cent)|
|Staff who personally experienced bullying in the last 12 months||26||15||28||18||29||17||20||22|
|Safe to speak up|
|I am confident that I would be protected from reprisal for reporting improper conduct||50||59||43||54||47||53||54||35|
|I would be confident in approaching my manager to discuss concerns and grievances||74||77||68||73||76||75||78||69|
|I am confident that if I raised a grievance in my organisation, it would be investigated in a thorough and objective manner||53||57||49||51||50||49||58||31|
|Patient safety climate|
|My suggestions about patient safety would be acted upon if I expressed them to my manager||70||75||67||72||72||69||73||67|
|I am encouraged by my colleagues to report any patient safety concerns I may have||76||81||76||80||78||79||82||77|
|Patient care errors are handled appropriately in my work area||70||76||68||74||73||71||68||67|
Note: Green = improved, orange = improved between 2016 and 2018 then declined in 2019, red = declined and black = no statistically significant change.
Source: VAGO, based on PMS data between 2016 and 2019.
FIGURE 3C: Case study: DjHS’s challenges in creating a positive patient safety culture
Since its serious patient safety issues in 2013 and 2014, DjHS has undergone lengthy proceedings with the Australian Health Practitioner Regulation Agency and the Victorian Civil and Administrative Tribunal, as well as significant governance and personnel changes. Within this challenging context, DjHS has not yet made significant progress in establishing a positive patient safety culture.
The 2019 PMS found that only 31 per cent of DjHS’s staff are confident that it would investigate grievances in a thorough and objective manner. This is significantly lower than the median for its peer health services (health services of similar size and patient types), which is 57 per cent.
Interviews we conducted with DjHS staff corroborated the PMS results. Staff expressed reluctance to report issues relating to safety and wellbeing, including inappropriate behaviour like bullying and harassment, because they had previously received poor responses from executives.
DjHS advised us that its staff have a poor understanding of bullying, harassment and their investigation processes and that management is taking steps to address this through extensive staff education.
DjHS also noted that its PMS results for patient safety culture may be influenced by more general staff sentiment. For example, the results could possibly reflect staff unhappiness with an administrative decision to move some staff to ensure teams are co-located.
However, the PMS questions are very specific and the significance of DjHS's variance from its peers’ results means the results warrant acceptance and effort to understand the root causes of staff concerns and respond to their feedback.
As the table below shows, DjHS’s performance has been declining and moving further away from its peer group’s median results since 2016 for PMS measures on:
- staff confidence that they would be protected from reprisal for reporting improper conduct
- staff confidence that grievances would be investigated in a thorough and objective manner.
|I am confident that I would be protected from reprisal for reporting improper conduct||Peer median||56||58||60||58|
|I am confident that if I raised a grievance in my organisation, it would be investigated in a thorough and objective manner||Peer median||59||64||65||57|
DjHS must do more to provide a workplace culture where staff feel safe to speak up. DjHS has informed us that its People and Culture team are undertaking staff focus groups in May and June 2021 to better understand staff experiences.
3.3 Initiatives to create a positive patient safety culture
To promote a positive patient safety culture, health services need to implement initiatives that increase their staff’s psychological safety, confidence to speak up and engagement.
As Figure 3D shows, MH and PH have implemented an extensive mix of initiatives that address these three dimensions. This means that staff at these health services are more likely to be vigilant in detecting and reporting patient safety issues.
However, MH is the only audited health service that has evaluated its initiatives to understand barriers and enablers to fostering a positive patient safety culture. Figure 3E explains how MH identified ways to improve one of its initiatives. PH advised us that it will be evaluating its initiatives to assess their impact on raising staff awareness of bullying and harassment.
BHS and DjHS have not developed or implemented sufficient initiatives across all three dimensions. This is because they have not developed and/or implemented a comprehensive local clinical governance framework. Without a comprehensive mix of initiatives, BHS and DjHS are not doing enough to promote a positive patient safety culture.
Figure 3D provides an overview of the audited health services’ current initiatives categorised by the three dimensions. Appendix D contains further details about these initiatives.
FIGURE 3D: Audited health services’ initiatives for each patient safety culture dimension
|Health service||Safety culture dimensions|
|Psychological safety initiatives||Confidence to speak up initiatives||Engagement in safety and improvement activities initiatives|
||Confidential feedback email address||
Note: Comparisons should not be made between health services having or missing specific initiatives because each health service's circumstances are different. Instead, we assessed if health services have appropriate initiatives against the patient safety culture dimensions.
The initiatives BHS has implemented to date have focused on increasing its staff’s psychological safety. BHS has only recently started to actively engage staff in quality and safety activities. Figure 3D shows that BHS has implemented a pathway for staff to raise concerns confidentially. However, it has not implemented initiatives to equip staff with the skills and confidence to speak up about safety concerns with their peers and managers. BHS advised us that its workplace safety committee will consider possible actions to address this.
DjHS's initiatives focus on increasing staff awareness of patient safety in wards. Like BHS, DjHS has not implemented initiatives to increase its staff’s skills and confidence in speaking up. This is concerning given that DjHS's PMS survey results show that the percentage of staff who feel safe to speak up has declined and DjHS does not have any initiatives in place to address this.
DjHS advised us that while it had developed a ‘safe to speak up’ program, it did not fully implement it due to competing demands associated with:
- a restructure of its people and culture team
- the potential voluntary amalgamation with a larger health service
- the COVID-19 pandemic.
DjHS noted that instead, its health and safety representatives and managers promote the message that it is safe to speak up through various formal and informal meetings with staff. Nonetheless, DjHS recognises that it needs to take a different approach to improve its PMS results.
DjHS also advised us that it is revising its safe to speak up program, which will include staff focus groups to identify the root causes of its cultural issues.
Figure 3E highlights MH's use and review of its weCare system as an example of how a health service can assess the effectiveness of its cultural initiatives.
FIGURE 3E: Case study: how MH evaluated one of its safety culture initiatives
In May 2018, MH reviewed its weCare system to identify opportunities for improvement.
weCare is a system that enables staff to:
- nominate colleagues for specific staff awards
- raise issues about a colleague's behaviour when they do not feel able to or safe to speak up.
In May 2018, staff provided a range of feedback about weCare through MH’s ‘speaking up for safety survey’. The survey found that:
- staff were concerned about the anonymity and confidentiality of the system
- staff used weCare as a first resort instead of attempting to address issues through their line manager
- staff felt distressed after being notified of incidents via weCare if they did not have prior knowledge of the incident and were unable to apologise or resolve the issue
- leadership lacked confidence in the system
- there were deficiencies in weCare’s triage and escalation processes.
After receiving the feedback, MH undertook a number of steps to improve staff confidence in weCare and encourage appropriate use of the system. For instance:
- MH developed a weCare dashboard for senior leaders to promote transparency and build confidence in the system
- leaders started discussing weCare with their teams to demonstrate their confidence in the system and raise awareness of improvements. At the same time, they also started emphasising other methods for raising concerns that can be effective and that weCare is a ‘safety net’.
From MH's perspective, weCare has been effective as a pathway for staff to raise concerns. However, MH also recognises that it needs to continue to monitor use of the system to drive improvements and prevent misuse.
Source: VAGO, based on information from MH.
While health services act when they identify underperformance or emerging risks, they do not consistently identify and respond to quality and safety risks in a timely way. Significant delays in completing serious incident investigations and resulting actions to address underlying issues mean that patients remain at risk of known avoidable harm for too long.
Health service boards, especially at the audited regional and rural health services, are not consistently monitoring enough information across incidents and KPIs to have a comprehensive view of quality and safety risks.
This chapter discusses:
A clinical incident is an event or circumstance that results in unintended or unnecessary harm to a patient.
DH requires health services to classify all clinical incidents according to the degree of harm that occurred to the patient using the ISR scale, which Figure 4A shows. Sentinel events are a subset of ISR 1 incidents because they are the most serious incidents that are wholly preventable and involve death or serious harm.
FIGURE 4A: ISR categories of clinical incidents
|ISR||Degree of impact|
|1||Severe or death|
|4||No harm or near miss|
While there are multiple layers of governance within a health service, the board and its quality and safety subcommittee are accountable for assuring DH, the Minister for Health and the community about their health service’s quality and safety.
Board quality and safety subcommittees:
- include board members
- provide their meeting minutes to the board
- can escalate risks and issues to the board.
As such, throughout this section we refer to both boards and board quality and safety subcommittees as 'boards'.
To have effective oversight, boards should:
- receive regular reports on clinical incidents and quality and safety KPIs
- understand current and emerging risks
- ensure actions occur to address underperformance and clinical risks.
At the four audited health services, the boards' oversight of quality and safety risks is not consistently adequate. There are various gaps at each audited health service. Overall, there are more significant deficiencies in oversight at BHS and DjHS relating to incident investigations, implementing recommendations and holding executives accountable for underperformance against KPIs. None of the four boards receive regular thematic reports on less serious incidents to check for underlying systemic issues.
Monitoring incident investigations and responses
Health service boards need assurance that their executives are quickly identifying and addressing issues that have caused avoidable serious incidents (mainly ISR 1 and ISR 2 incidents) to minimise the chances of them recurring. For less serious incidents (mainly ISR 3 and ISR 4 incidents) that may occur more frequently, they should receive regular thematic reports that identify any common contributing factors and actions taken to address them.
None of the audited health service boards receive comprehensive information on incidents and responses to them. In particular, none of the health services undertake systematic analyses of less serious incidents to gauge emerging risks. Figure 4B shows these gaps.
FIGURE 4B: What board incident reports contain at the audited health services
|Review all new serious incidents||✓||✓||✓||✓|
|Monitor the status of serious incident investigations||✓||Inadequately||✕||Inconsistently|
|Assure actions are implemented to address risks||Inadequately||Inadequately||Inadequately||Inconsistently|
|Regularly review common underlying themes across all incidents||Serious incidents only||✕||✕||✕|
Serious incident investigations
MH's board regularly oversees the status of serious incidents but inadequately oversees how the health service implements actions to address them.
MH's oversight of actions is limited because it only reports on the overall proportion of completed recommendations. There is no rule or requirement for when MH needs to account for overdue recommendations. While MH had no overdue recommendations to report during the course of this audit, MH should ensure it has a clear process to do so to avoid a potential gap in its board’s oversight.
PH’s board regularly oversees the status of serious incident investigations and actions to address risks. However, its oversight is inadequate in the following ways:
- PH's reports do not include overdue investigations and recommendations for incidents that occurred outside of the reporting period. This means that the board does not have a full account of overdue investigations and recommendations.
- PH does not account for why investigations or recommendations are overdue and what rectifying actions are in place. Without this information, the board cannot be assured that the executive is taking steps to minimise delays in completing overdue investigations and recommendations. PH has advised us that it will address this gap.
While BHS’s board does not regularly oversee the status of serious incident investigations, it does regularly oversee the status of actions to address risks. However, its oversight is not adequate. BHS's reports on implementing actions identify overdue actions but do not consistently account for why they are overdue and what actions are needed for improvement. Hence, the board does not know if these delays are reasonable.
DjHS does not provide its board with clear and consistent information on the status of its investigations and recommendations. DjHS provides an ISR 1 and ISR 2 summary report to its board at each meeting. While the intent of this report is to inform the board of serious incidents that occurred during the reporting months, it only sometimes includes the status of investigations and implementation of actions for some incidents. Hence, the board does not receive consistent and clear information on whether there are overdue investigations and actions.
Monitoring quality and safety KPIs
To have a comprehensive view of quality and safety performance and risks across a health service, in addition to monitoring incidents, the board needs to:
- assess quality and safety performance against appropriate KPIs as part of its regular board KPI suite, including:
- at a minimum, the mandatory quality and safety SOP KPIs
- any additional KPIs it sees as necessary to meet obligations under the Health Services Act 1988
- receive reports that:
- clearly identify areas of significant underperformance
- include reasons contributing to underperformance
- include information on actions taken to address underperformance and assure that these actions have been effective
- preferably group and report KPIs against the health service's clinical governance framework goals to enable clear assessment of its progress.
As Figure 4C shows, not all audited health service boards receive sufficient information to adequately understand quality and safety performance.
FIGURE 4C: What audited health services report to their boards about quality and safety performance
|Monthly or bimonthly SOP KPI results
(Percentage of total relevant quality and safety SOP KPIs included)
(97 per cent)
(100 per cent)
(19 per cent)
(9 per cent)
|Grouping of KPI performance against local clinical governance framework goals||✓||✓||✕||✕|
|Explanations of significant variations in performance||✓||✓||✕||✕|
|Updates on actions to address performance issues||✓||✓||✕||Inconsistently|
Monitoring SOP quality and safety-related KPIs
Only PH's board monitors all of its SOP quality and safety KPIs as part of its monthly and bimonthly KPI reporting suite. MH's board monitors almost all of its SOP KPIs. Except for minor gaps in MH’s reporting, the boards and executives at PH and MH therefore have a good understanding of performance issues.
However, BHS and DjHS's boards only monitor 19 per cent and 9 per cent (respectively) of their SOP quality and safety KPIs as part of their monthly or bimonthly KPI reporting suite.
There is no distinct pattern across the four health services for which SOP KPIs they do not monitor (see Appendix F for full list of SOP KPIs that each health service monitors). Specifically:
The board of …
in its monthly or bimonthly reports does not monitor…
that relate to …
3 per cent of its SOP quality and safety related indicators (one indicator).
patients waiting for elective surgery for longer than the clinically recommended time frames.
81 per cent of its SOP quality and safety related indicators (26 indicators)
mainly adult mental health, maternity, patient experience and access to emergency and elective surgery.
91 per cent of its SOP quality and safety related indicators (10 indicators)
mainly infection control, patient experience and care associated infections.
Unreported SOP KPIs
MH reports on its KPI for elective surgery wait times in a separate quarterly SOP report to the board, and therefore excludes it from its regular board KPI reporting suite to reduce duplication. However, the less frequent reporting may reduce the board’s opportunity to address any issues in a timely way.
Both BHS and DjHS rely on VAHI’s quarterly Monitor reports to review their performance for the SOP KPIs excluded from their regular bimonthly and monthly reporting suite, respectively. This means there is a risk that both boards do not become aware of underperformance and take action in a timely manner.
BHS advised us that clinical directorates monitor the 26 SOP KPIs omitted from its board reports that are relevant to their clinical area in their respective quality and governance committee meetings. These committees can escalate issues to BHS’s board quality subcommittee through the patient safety and innovation committee, which is BHS's peak executive quality committee. However, we did not see evidence of this escalation in the meeting minutes we reviewed across a four-month period (2 committee reporting cycles). There is also a two to three month lag in escalating issues through this process.
DjHS advised us that as a small health service, recreating and analysing the 10 SOP KPIs omitted from its regular internal reporting is not a priority given other pressing matters.
Despite the reasons described above, BHS and DjHS's reliance on VAHI's Monitor reports to inform their boards on SOP KPIs is problematic because they only receive them quarterly. This is not frequent enough to allow these boards to identify and address underperforming SOP KPIs within the annual agreement period.
Since feedback from this audit, DjHS has expanded its monthly activity performance report and now includes most of the unreported SOP KPIs.
Monitoring other quality and safety KPIs
Beyond their SOP KPIs, all health service boards regularly monitor additional KPIs they identify as priorities (see Appendix G for a full list of these KPIs). Some common KPIs cover:
- timeliness in providing hospital discharge summaries
- common clinical incidents, such as pressure injuries, falls and medication errors
- unplanned readmissions for specific community groups or medical conditions.
For MH and PH, who already monitor most of their SOP KPIs, these additional KPIs mean their boards have a comprehensive view of quality and safety performance. Both health services have also grouped their SOP and additional quality and safety KPIs to align with the goals in their clinical governance frameworks for their internal reporting. This allows the boards and internal stakeholders to understand performance results against organisational quality and safety goals.
In contrast, BHS and DjHS’s boards do not sufficiently monitor their KPIs to have a comprehensive view of service quality and safety. As mentioned above, BHS and DjHS only review 19 per cent and 9 per cent (respectively) of all mandatory SOP KPIs. While both boards monitor additional KPIs they identify as priorities, these KPIs do not address the gaps in their SOP monitoring, which is the minimum set by DH.
Additionally, unlike MH and PH, BHS and DjHS do not group and report on their KPIs against their local clinical governance framework goals. This means that the boards at these health services cannot easily assess if they are meeting their clinical governance goals.
Identifying and accounting for poor performance
Health services need to clearly highlight any areas of underperformance and emerging areas of performance risks to their boards so they can make further queries and informed decisions on actions needed for improvement.
Not all health services have a comprehensive approach to identifying and accounting for underperformance to their boards, which Figure 4D shows.
We reviewed board quality and safety subcommittee performance reports for the four audited health services between July 2019 to February 2020 to assess if they:
- identify changes in performance
- detect deteriorating performance early using clearly established criteria
- summarise their achievements against benchmarks or targets
- provide adequate accounts for poor performance
- compare and analyse performance across relevant patient cohorts and/or service streams to identify areas of strengths and weaknesses across the health service.
Figure 4D outlines the outcomes of our analysis.
FIGURE 4D: Does reporting to audited health services' boards show underperformance and emerging trends?
|Identifies changes in performance||✓
Monthly results for the last two years
Result for the current month against the previous month and same month last year
Against the previous month and same month last year
Monthly over the current and previous calendar year (12 to 24 months)
|Defined method/rule for identifying change in trend/s||✓
Statistical approach using run charts
Did not meet the target or benchmark at least once in last two months
Did not meet the target in reporting month
Did not meet the target for two or three months, depending on the variation from the target
|Identifies statistically significant changes in results||✓||✕||✕||✕|
|Colour codes performance achievement||✓||✓||✓||✓|
|Adequately accounts for underperformance||✓||✓||✕||✕|
|Comparative analysis across the health service||Different campuses only (for selected indicators)||Different campuses only (for selected indicators)||✕||Different campuses only (for selected indicators)|
Identifying performance issues
All audited health services clearly identify poor performance using colour codes to bring them to the board's attention in their KPI reports. However, only MH takes a statistical approach to identifying significant performance changes over time for every board report. This means that MH’s board knows when real performance changes occur, as opposed to insignificant levels of change associated with expected variation. This allows the board to focus its attention on genuine issues.
Figure 4E highlights why MH's statistical approach to identifying significant emerging trends using run charts is better practice for discovering current or emerging performance issues.
FIGURE 4E: Case study: MH's approach to identifying significant performance variation using run charts
In November 2018, MH implemented a new approach to identifying and responding to emerging quality and safety risks before performance deteriorates below targets.
This approach means that MH’s board and executives have a thorough understanding of emerging quality and safety issues and can proactively address them.
MH's board reports include comprehensive analysis of and information on:
- monthly or quarterly quality and safety trends over 12 to 24 months
- quality and safety performance changes
- actions MH has already taken to improve its quality and safety performance
- responsible executive staff.
MH's board-level performance reports also provide a summary of its monthly quality and safety performance changes and detailed run charts that highlight statistically significant performance changes over time. This enables MH's board and executives to easily identify quality and safety issues and improvements.
Run charts present performance results over time and compare performance against the median. A trend forms when there are:
- multiple data points clearly running together on one side of the median line
- multiple data points with very few crossing the median line
- major departures from the median line (for example, three standard deviations).
Run charts, as shown below for sepsis mortality per 1 000 separations, allow users to tell the difference between 'statistical noise', performance result changes within an expected range of variation and real differences. This is useful to detect not just emerging risks but also to know if an improvement initiative is actually working.
Source: VAGO. Run chart image provided by MH.
Areas for improvement
PH's approach to identifying emerging risks is adequate overall but can be improved. PH has a low threshold for initiating investigations into poor-performing KPIs (not meeting the target for two consecutive months). These investigations, which we discuss further below, include reviewing longer-term data to identify any concerning trends and issues. Because of the low threshold, PH is still be able to identify emerging risks early. Nonetheless, we note that PH has a strong data analytics capability and could take a more statistical approach to identify emerging trends in its performance data.
Sparkline plots are small line charts without axes to succinctly visualise data.
BHS's approach to identifying emerging risks is inadequate. It lacks trended performance information, which means it may not identify emerging performance risks. BHS now includes sparkline plots to identify trends over the last three months in its board KPI reports. However, it is unlikely that three months' worth of data is sufficient to show a reliable trend.
None of the audited health services' board reports provide a comprehensive view of whether particular areas of a health service are underperforming. MH, PH and DjHS’s boards only receive a breakdown by campus for selected indicators. None of the audited health services consistently examine KPIs that are performing below target by campus and clinical unit to identify and promote learning between those that are performing above expectations and those that are underperforming.
We note that DjHS only improved its data collection, analysis and reporting to its board in the last three years. It introduced monthly activity performance reports in 2018 and a DjHS dashboard in February 2020. DjHS advised us that its improvements since 2018 are from a ‘zero base’, where reporting was previously a manual and fragmented process.
Despite this work, gaps remain. Given Targeting Zero’s imperative, DjHS has not acted fast enough or taken sufficient action to improve its performance analysis and reporting.
Accounting for performance issues
MH and PH investigate and provide clear accounts of underperformance to assure their boards about steps taken to improve performance. Specifically, PH provides detailed 'in-focus' analyses, in addition to its board KPI report, that discuss reasons for underperformance and actions to address them.
In contrast, BHS and DjHS's regular board reports on quality and safety do not consistently provide:
- detailed explanations of underperformance
- actions taken to address performance issues
- information on who the responsible executive is to address the matter.
This means that BHS and DjHS’s boards cannot ensure that their health services are addressing quality and safety issues. For example:
- BHS's board KPI reports have an indicator to measure the percentage of patients who have comprehensive medication assessments (also known as best possible medication history) documented at admission. In October 2019, BHS achieved 57 per cent against its target of greater than 80 per cent for this KPI. The only comment reported against this result was that the national average is 62 per cent.
- In its KPI reports, DjHS provides details on medication errors and reasons for why they occurred, such as drug charts being read incorrectly, when this KPI does not meet its target. However, DjHS does not provide a similar level of explanation for its underperforming KPIs on falls at Grant Lodge.
DjHS only established criteria for when executives need to provide an action plan to the board to address KPI underperformance in April 2020. Our review of a sample action plan showed that it focused on actions taken to improve performance, rather than first identifying the factors that contributed to underperformance. This means the actions taken may not address the root cause.
All Victorian health services must meet SCV's Policy: Adverse patient safety events, which sets out health services' responsibilities in relation to clinical incidents. According to this policy, health services must:
- have clear incident management polices
- openly disclose with patients and carers when harm has occurred
- investigate all incidents
- identify and implement actions
- monitor the impact of actions.
Figure 4F sets out these steps.
FIGURE 4F: Requirements for responding to clinical incidents
Source: VAGO, based on SCV’s Policy: Adverse patient safety events.
Incident management policy
As Figure 4G shows, all audited health services have incident management policies and procedures that provide staff with clear processes for reporting, investigating and responding to incidents. However, BHS and DjHS could further improve theirs.
We assessed these health services' incident management policies against expectations and principles set out in SCV's Policy: Adverse patient safety events and the NSQHS Standard 1.11 under incident management systems and open disclosure.
FIGURE 4G: Assessment of audited health services' incident management policies and procedures
|Staff obligations to speak up, report and act||✓||✓||✓||✓|
|Immediate containment of active risks||✓||✓||✓||✕|
|Immediate treatment of harm to patients and staff||✓||✓||✓||✕|
|Pathways for notifying senior clinical staff and management||✓||✓||✓||✓|
|Identify available pathways for providing support to staff who are involved in an incident||✓||✓||✕||✕|
|Openly disclosing incidents to patients and carers||✓||✓||✓||✓|
|Timely reporting to relevant authorities (for example, SCV)||✓||✓||✓||✓|
|Systems-based approaches to incident investigation||✓||✓||✓||✓|
|Developing and implementing actions to prevent future incidents||✓||✓||✓||✓|
|Organisational commitment to learning and improving||✓||✓||✓||✓|
|Roles and responsibilities for incident investigations||✓||✓||✓||✓|
|Investigation types based on incident severity||✓||✓||✓||✓|
|Time frames for reporting and investigating incidents||✓||✓||Recently
Note: ~While DjHS’s policy does not include time frames to complete IDCRs and SCV’s risk reduction action plan, its document templates do.
Following feedback from this audit, DjHS is updating its incident management policy to address the gaps shown in Figure 4G.
All audited health services also have policies and procedures that detail the required reviews for incidents of different severity levels:
- All ISR 1 incidents, including all sentinel events, require comprehensive root cause analysis (RCA) followed by a detailed report that specifies the full range of causal and contributing factors that led to an incident.
- All ISR 2 incidents require an IDCR and a report that identifies any breakdowns in care processes and how these affected an incident’s occurrence.
- All ISR 3 and 4 incidents are subject to reviews by senior clinical staff in the relevant units.
During this audit, we notified BHS that its ‘adverse event/incident management non clinical protocol’ did not specify time frames for completing IDCRs. BHS promptly addressed our feedback and is now able to assess the timeliness of IDCRs and reports to improve quality and safety.
Openly disclosing clinical incidents
Patients and their carers may only become aware of a clinical incident when health service staff openly disclose the event to them. All Victorian health services are obligated under Victoria's Charter of Human Rights and Responsibilities Act 2006 and the NSQHS Standards to have a formal process for openly disclosing clinical incidents.
Open disclosure involves a meeting in which clinical staff provide patients and their carers with an:
- expression of regret or apology that an event occurred
- explanation of the event and its impact on the patient and their carer/s
- invitation for the patient to account for their experience of the event
- outline of the intended steps to manage any consequences of the event
- outline of what will be done to prevent the event from recurring.
ACSQHC's Australian Open Disclosure Framework sets out minimum standards for conducting open disclosures.
As Figure 4H shows, all audited health services have open disclosure policies and/or procedures that are broadly consistent with the Australian Open Disclosure Framework. This supports a consistent approach for staff undertaking open disclosures. While DjHS has a minor gap in its open disclosure policy, which concerns the principle of 'good governance', it is addressing this.
Across all four audited health services, only MH routinely monitors its open disclosure process to ensure each case is promptly and appropriately actioned. The other audited health services have gaps in monitoring their open disclosure processes.
The Victorian Health Incident Management System is Victoria's statewide incident management system managed by DH.
PH staff can record open disclosures in two separate locations—the Victorian Health Incident Management System, the patient’s medical record or both. As a result, there is not one consistent record of open disclosures that have occurred, and PH’s board and executives cannot easily monitor or assess compliance. PH advised us that its executives are aware if open disclosures have occurred for ISR 1 incidents because this is discussed during the initial incident review meeting for these incidents.
BHS does not have a process for monitoring open disclosures. Its clinical governance committee is aware of this issue and is considering changes to its incident management system to record staff compliance.
DjHS staff use the Victorian Health Incident Management System to record open disclosures. However, its board and executives do not routinely monitor relevant data to ensure that staff are undertaking open disclosures as required by its open disclosure policy.
FIGURE 4H: Assessment of health services’ open disclosure policies and/or procedures
|Highlights the importance of open disclosure||✓||✓||✓||✓|
|Includes a detailed process guide or checklist||✓||✓||✓||✓|
|Describes the Australian Open Disclosure Framework’s eight principles:|
|1. Open and timely communication||✓||✓||~||✓|
|3. Apology or expression of regret||✓||✓||~||✓|
|4. Supporting and meeting the needs and expectations of patients and their families and carers||✓||✓||~||✓|
|5. Supporting and meeting the needs and expectations of those providing healthcare||✓||✓||Included in
|6. Integrated clinical risk management and systems improvement||✓||✓||~||✓|
|7. Good governance||✓||✓||~||✕|
Note: ~The policy or procedure generally references a principle but does not provide a specific description of the expected activity or behaviour.
*DjHS has updated its open disclosure policy to include this principle and anticipates that the board will endorse this policy in late May 2021.
Investigating clinical incidents
Health services need to investigate clinical incidents in a timely manner to quickly identify causal and contributing factors, learn from the incidents and implement improvements to prevent future avoidable harm.
Completing timely serious incident investigations
In 2019–20, all audited health services performed better than the statewide average (40 per cent) for reporting sentinel events to SCV within 30 days. For example, MH and PH reported almost all sentinel events within 30 days (eight out of nine events for MH and 12 out of 14 events for PH).
While these results are comparatively better than other health services, it is still concerning that health services, including the audited health services, are not always meeting requirements for reporting sentinel events in a timely way.
Aside from sentinel events, DH has not set specific time frames for completing investigations into other incidents. Health services should set their own clear time frames for investigating ISR 1 and 2 incidents to ensure they respond to patient safety risks in a timely way. As Figure 4I shows, the four audited health services have set varying periods for completing RCA and IDCRs.
FIGURE 4I: Audited health services’ time frames for investigating serious incidents
|Health service||Time frame for RCA (business days)||Time frame for IDCR (business days)|
Despite health services setting their own time frames for completing serious incident investigations, none of the audited health services were consistently meeting them. This risks health services not identifying causal factors of incidents early enough to prevent similar harm from recurring.
As Figure 4J shows, we assessed the median proportion of overdue serious incident investigations at MH, PH and BHS. We excluded DjHS from this analysis because there were only five serious incidents reported during our sample period of board papers between October 2019 and February 2020. Nonetheless, we note that three out of five ongoing investigations at DjHS were overdue. For MH, PH and BHS, the median proportion of overdue serious incidents investigations ranged from 17 per cent at MH to 82 per cent at BHS over respective sampling periods.
These three audited health services attributed delays in serious incident investigations to a lack of staff capacity and capability.
MH also advised us that it aims to meet reporting time frames, but this can be difficult to achieve and sometimes depends on appropriate clinicians and consumers being available.
PH advised us that it prioritises new incidents and focuses on the quality of investigations rather than meeting internal reporting time frames. While its focus on quality is understandable, we did not find evidence of PH staff’s obtaining approval to extend timelines for quality purposes or assessing the risks of delays.
FIGURE 4J: Median proportion of overdue serious incident investigations at MH, PH and BHS
Note: We determined sampling periods based on the maximum number of months of valid data to enable calculations. DjHS is excluded from this analysis due to insufficient data.
Analysing incident themes
Health services should undertake regular and service-wide thematic analyses of all serious (ISR 1 and ISR 2) and less serious (ISR 3 and ISR 4) incidents to determine common themes or clusters relating to:
- clinical division or units
- causal or contributing factors.
As Figure 4K shows, not all of the audited health services do this. Consequently, they risk not identifying clusters of incidents or factors underlying incidents. Figure 4K shows the frequency of thematic analyses at each audited health service.
FIGURE 4K: Frequency of thematic analyses of serious and less serious incidents across the whole organisation at the four audited health services
|Health service||Serious incidents
(ISR 1 and ISR 2)
|Less serious incidents
(ISR 3 and ISR 4)
|PH||Ad hoc||Ad hoc|
|DjHS||Ad hoc and depending on data availability||✕|
Victorian Health Incident Management System Local Solution is a local system for health services to report incidents to the Victorian Health Incident Management System dataset.
PH’s Incident Explorer, which is an online Power BI reporting dashboard, enables users to undertake long term and real time analysis of incidents. It enables users to analyse and identify themes, such as incident types and clinical units, and drill down to individual case records for further investigation.
MH advised us that it will examine regular thematic analyses of less serious incidents after implementing DH's Victorian Health Incident Management System Local Solution.
While PH has not conducted regular thematic analyses, we have seen evidence that it has undertaken this type of analysis on request. For instance, following a serious incident review committee request, PH reviewed its medication error data over a three-month period and analysed it by clinical craft group and 'reason for the error'. PH could improve its thematic analysis of incidents through its ‘Incident Explorer’ dashboard.
BHS analyses its serious incidents monthly by key incident types, such as falls and medication errors, within clinical directorates and by clinical units to identify if there are any concerning themes and trends. However, these analyses are not detailed enough to identify common contributing factors. For instance, in February 2020, BHS found that 70 per cent of medication errors involved an ‘administration process’ but did not further explain what specific issues led to the errors. BHS's individual monthly analyses also do not identify if common contributing factors recur across multiple months.
DjHS advised us that as a small health service, thematic analyses are only possible when there is sufficient data. We note some examples of DjHS conducting thematic analyses in response to multiple clinical incidents to determine common underlying causes, such as analyses of postpartum haemorrhage, medication errors and falls.
Figure 4K also shows that none of the audited health services undertake regular thematic analyses of less serious incidents to identify potential themes across the health service.
For BHS, while its risk and compliance team review all incidents across the health service to identify themes, there is no documented, standard approach to do this so staff may be using inconsistent methods.
For MH and PH, thematic analyses primarily occur at the local unit or ward level. PH’s analyses also occur on an ad-hoc basis by its NSQHS Standard committees. However, similar to BHS, MH and PH do not have clear guidelines for local managers to ensure that they adopt a consistent approach. Hence, there is a risk that they may miss early warning signs.
PH does analyse themes for specific less serious incidents that are part of its regular KPI suite, such as patient identification and wrong site or procedure. It also investigates contributing factors as part of its variance-reporting process. However, PH does not complete these analyses regularly over standard time periods and across similar incident types to identify and track issues.
Implementing actions to prevent future incidents
Health services should implement recommended actions from incident investigations in a timely manner to prevent events recurring. For serious incidents, we expect health services to implement recommendations as soon as possible after finalising the RCA or IDCR.
The amount of time it takes for a health service to implement actions depends on the action’s nature and complexity. Actions such as updating current procedures or providing staff training are quicker to complete compared to more complex actions, such as purchasing new medical equipment or coordinating intricate multidisciplinary protocols across an entire health service.
- the proportion of overdue actions at each audited health service
- how quickly they implemented actions between February 2019 and February 2020.
As Figure 4L shows, only MH completed all actions within its time frames and consistently took prompt action to address serious patient safety issues. PH and DjHS did not implement actions within their own estimated time frames and could not provide clear reasons for delays.
BHS had not implemented the majority (70 per cent) of its recommended actions as of April 2020, which allows risks to patient safety to remain. BHS advised us that delays were due to a resourcing and skills deficit in its centre for safety and innovation team and it had been working to build capacity in 2020.
FIGURE 4L: Audited health services’ delays in completing serious incident recommendations
|Health service||Timeliness in completing serious incident recommendations|
|MH||As of October 2019, MH did not have any overdue recommendations.
We note that MH sought and got extensions for two recommendations that were due in October 2019. This was the first time staff sought extensions for these recommendations.
|PH||As of February 2020, 40 per cent of recommendations were overdue.|
|BHS||As of April 2020, 70 per cent of recommendations were overdue, with due dates as far back as July 2019.|
|DjHS||As of February 2020, DjHS reported no recommendations to the board. We note that between September 2019 and February 2020 DjHS implemented five recommendations, two of which were overdue and had been for two months.|
Note: The time periods differ between health services due to the staged nature of our audit conduct work and the availability of health service reports.
Figure 4M shows the median amount of time that health services took to complete actions across three or four reporting periods between February 2019 and February 2020.
FIGURE 4M: Median time to implement recommended actions from February 2019 to February 2020
Note: 'Outliers' refer to incidents that reflect a statistically significant level of variance from the norm.
MH and PH implemented recommended actions within approximately six to 10 months from the date of the original incident. We found this reasonable given the complexity of the recommended actions. For example:
- MH changed ward-level fittings and fixtures and conducted staff education within four to seven months.
- PH purchased new equipment and introduced new staff positions within 10 months.
However, we note that PH took at least 14 months longer than its median time taken to implement recommended actions for 10 incidents. PH was unable to provide reasons for the delays.
BHS took longer than the other health services—between 20 to 26 months—to implement a quarter of all its recommended actions. BHS could not provide reasons for these delays due to a lack of documentation and there were no obvious patterns associated with the types of actions.
DjHS took the least amount of time to take recommended actions, but many of these were immediate actions to mitigate basic risks (such as keeping walkways clear) rather than more comprehensive actions to address root causes.
Assessing the effectiveness of actions to prevent incidents
Health services should monitor the effectiveness of actions they implement to ensure they are addressing underlying causes of serious incidents and preventing future harm. Health services should use better-practice approaches to monitor the effectiveness of their actions, including:
- identifying the intended result to be assessed
- setting a target
- specifying the sample group and sampling period
- identifying how data will be collected.
None of the four audited health services have consistently identified appropriate measures to assess the effectiveness of implemented actions. Out of a sample of 16 serious incident action plans across the four health services, we were only able to identify two instances where the health service set measures to specifically assess the impact of the actions taken to prevent the same incident from recurring, which Figure 4N shows.
FIGURE 4N: Instances of appropriate measures that assess the impact of actions from 16 serious incident action plans
|Incident type||Appropriate measure reported in incident action plan|
|Incorrect application of stroke patient clinical pathway||100 per cent compliance with stroke pathway|
|Retained products post surgical procedure||Zero future incidents of retained surgical products|
Two factors contribute to the audited health services' lack of effective monitoring:
- While SCV's RCA recommendations and action plan template requires health services to document an 'outcome measure' (which evaluates the impact of an intervention), it does not define what this is and how it differs from an output measure (which might, for example, just note that the planned action was completed).
- IDCR templates designed and used by MH, PH and DjHS do not require staff to identify outcome measures.
While BHS requires its staff to specify measures to assess the impact of recommended actions, it does not provide guidance on how to identify appropriate measures.
We have included examples of better measures that the audited health services could consider using in Appendix E.
We have consulted with BHS, DjHS, MH, PH and DH, and we considered their views when reaching our audit conclusions. As required by the Audit Act 1994, we gave a draft copy of this report, or relevant extracts, to those agencies and asked for their submissions and comments.
Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.
|ACSQHC||Australian Commission on Safety and Quality in Health Care|
|BHS||Ballarat Health Services|
|CEO||chief executive officer|
|DH||Department of Health|
|DHHS||Department of Health and Human Services|
|DjHS||Djerriwarrh Health Services|
|IDCR||in-depth case review|
|ISR||incident severity rating|
|KPI||key performance indicator|
|PMS||People Matter Survey|
|RCA||root cause analysis|
|SCV||Safer Care Victoria|
|SOP||statement of priorities|
|STEP||safe, timely, effective and person centred|
|VAGO||Victorian Auditor-General's Office|
|VCGF||Victorian Clinical Governance Framework|
|NSQHS Standards||National Safety and Quality Health Service Standards|
|Performance Monitoring Framework||Victorian health services Performance Monitoring Framework 2019–20|
|Targeting Zero||Targeting Zero: Supporting the Victorian hospital system to eliminate avoidable harm and strengthen quality of care|
|Who we audited||What we assessed||What the audit cost|
We assessed whether health services have adequate systems and processes in place to assure the quality and safety of their services.
The cost of this audit was
As part of the audit we:
- audited four health services, including reviewing their:
- clinical governance frameworks
- policies and procedures on quality and safety
- documents on their patient safety culture improvement initiatives
- board reports on incidents and KPIs
- interviewed various staff across each health service from board members and executives through to clinical directors and nurse unit managers
- visited clinical wards at all health services except PH due to the COVID 19 pandemic.
We selected the four health services as a representative spread of health services by location and size.
We undertook the majority of audit conduct work from January 2020 to April 2020. The COVID-19 pandemic resulted in some delay to the completion of the audit.
We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements.
We also provided a copy of the report to the Department of Premier and Cabinet and the Department of Treasury and Finance.
We briefly discuss the audited health services' initiatives to promote a positive patient safety culture below. PH's placemats and MH's improvement huddles and improvement noticeboards are excluded here as these are covered in Section 2.2.
- weCare system: provides an anonymous avenue for staff to raise concerns about colleagues' behaviour.
- Progressive communication tool: for staff to communicate and escalate concerns about patient safety.
- ‘Good catch’ awards: recognises staff for intervening and speaking up about patient safety.
- ‘You made a difference’ awards: recognises and celebrates staff and volunteers for building on MH's safety culture by living MH's values in their work.
- Safety champions: staff are role models for peers on speaking up for MH's values and use the progressive communication model to communicate and escalate concerns about patient safety.
- Safety huddles: routine, structured short clinical unit briefings designed to give nurses and other caregivers opportunities to stay informed, review events, and make and share plans to ensure well-coordinated safe care.
- Independent facilitator: an informal, independent, neutral and confidential resource for staff to raise concerns, build capacity to speak up and seek support on conflict resolution.
- Leadership program: to develop leadership capabilities in areas such as self awareness and developing others and leading in a complex work environment.
- Values integration program: staff workshops that assert expected and acceptable behaviours.
- Internal human resources advisory line, and an external and independent whistleblower alert service: to encourage staff to report any matters that may adversely affect the organisation, its employees or other persons.
- Safer care consultants: provides advice and support to clinical and operations leaders of all clinical areas on patient safety, continuous improvement and risk management.
- Daily operating system meetings: to discuss various operational aspects of the health service and quality and safety issues, such as clinical incidents that needed escalation in the previous 24 hours. There are three tiers of daily operating system meetings—tier 0 occurs at the local unit or ward area, tier 1 occurs between managers and their director, and tier 2 occurs between the CEO, executive directors, clinical and operational directors and all department directors.
- ‘Ask the chief executive’ portal: online portal for staff to raise concerns directly with the chief executive anonymously and in confidence.
- ‘Know better be better’ campaign: DH and WorkSafe Victoria's bullying and harassment awareness campaign that has been rebranded and incorporated as part of PH's ongoing ‘Safe culture’s campaign to improve staff wellbeing.
- ‘Above and below the line’ behaviours initiative: across all levels of BHS from the board and executive down to departments, each staff group identifies a range of acceptable and not-acceptable behaviours.
- Values-based training: BHS's expectations on staff behaviour and professional conduct.
- Confidential feedback: an email address monitored by the CEO's office where staff may raise concerns.
- ‘BHS together’ values awards: recognises and rewards staff who exhibit behaviours that align with the health service's values.
- Ward noticeboards on safe, effective, person-centred and connected care: raises staff awareness on what high-quality care means and how staff are contributing to these goals of care.
- Daily operating system meetings: daily operational initiative that cascades from the local department level up to the executive level to discuss current and emerging patient safety issues and incidents.
- Safety crosses at some wards: staff use a colour-coding system on a calendar to record when and what type of clinical incidents have occurred.
- ‘Clinical incident champions’: ward-based roles that provide leadership and assist ward staff with education on various incidents, such as falls and blood management.
- Management development program: builds the capability of managers in a range of areas including having difficult conversations and driving change in teams.
- Annual staff awards: from 2020, DjHS shifted the focus of these awards from years of service to performance that exceeds expectations over a range of areas that demonstrate quality and commitment to the health service spanning:
- improvement and innovation
- patient-centred care and safety
- diversity and inclusion
- clinical excellence
- commitment to DjHS and the community.
- Safety crosses: at each ward or clinical unit staff use a colour-coding system on a calendar to record when and what type of clinical incidents have occurred.
- Local ‘Know how we are doing’ noticeboards: raises staff and consumer awareness on incidents related to specific NSQHS Standards, such as medication safety and comprehensive care, using safety crosses and also notes what staff are doing to improve and how consumers can play a part in preventing incidents.
- Local area safety huddles: local huddles to engage staff on information and data presented on safety crosses and local ‘know how we are doing’ noticeboards.
Figure E1 shows a selection of measures out of a sample of 16 serious incident action plans we reviewed across the four audited health services compared to better practice examples. The selection captures a spread of incident and measure types from the sampled action plans. There were two incidents where health services used relevant measures to assess the impact of implemented actions. We denote these measures with a ‘^’.
Click the link below to open a PDF copy of Figure E1: Reported and proposed outcome measures for selected serious incidents.
Click the link below to open a PDF copy of Appendix F: Audited health service monitoring of SOP indicators for 2019–20.
Click the link below to open a PDF copy of Appendix G: Additional quality and safety performance indicators.