Follow up of Residential Care Services for Children

Conclusions

The department has initiated adequate action to address all of the recommendations in our 2014 report. In most instances, this has meant addressing systemic issues that require a long time to overcome.

Given the change in government on 30 November 2014 and the strategic approach to addressing recommendations, the department's progress acting on the recommendations is adequate and has transitioned smoothly between governments.

The actions are part of a broader reform program of the child protection system that has been a departmental focus under both the current and former governments. In August 2015, the current government announced the development of a Roadmap for Reform: strong families; safe children. One of the reform directions is to reorient the out-of-home care system to reduce the use of residential care, increasing funding and support to home-based care.

It is positive that the department is taking a strategic approach to addressing longstanding issues, but it will need to continue to provide appropriate oversight to ensure the timely completion of the actions still in progress and to evaluate the impact of the actions on systemic issues.

Findings

Meeting the needs of children

The 2014 audit found that the residential care system had been operating over capacity and was unable to meet the needs of children. As a result, the department had to buy additional capacity or contingency places known as 'unfunded contingency placements' at a cost of $24 million in 2012–13.

The department has acted to address systemic issues in its residential care model through developing new models of care to address the needs of the children and young people with the most complex needs in residential care, and to gradually move them out of residential care.

This is a change from the previous government's policy, which focused on increasing the capacity of the residential care system. Therefore, the department has not specifically addressed our recommendation to review approaches to forecasting demand for residential care and prepare plans for the capacity that is required. Instead, in line with current government policy, the department has focused on managing demand for out-of-home care by increasing the capacity of the home-based care system.

Investments in moving children out of residential care have resulted in a reduction in the daily average occupancy of unfunded contingency placements from 141.9 places in 2012–13 to 38.7 places at the end of February 2016. Total spending on unfunded contingency placements fell from $24 million to about $11 million during the same time.

The department is working to improve its complaints processes and assess the feasibility of an independent advocacy role for children and young people in residential care, which will contribute further to meeting their needs. This has involved engaging with children and young people in residential care to include their voice in identifying the issues and the possible actions to address these.

Staffing capacity and capability

The 2014 audit found that the staffing models used were not providing sufficient quality and stability of care for children in residential care. Since then, the department has acted to address both staffing capacity and capability through:

• a pilot program that provides an additional overnight staff member
• developing a capability framework for residential care staff
• planning to introduce a minimum qualification for residential care workers.

The development of a capability framework will also include consultation with children and young people in residential care to find out what skills and capabilities they want from residential care workers.

While the pilot program continues, findings have already been used to improve services through a budget bid for ongoing funding for an additional overnight staff member in all residential care units. Findings from work on the capability framework have also informed the development of the department's 2015–18 strategy for its existing residential care staff learning and development program.

Performance monitoring and reporting

The 2014 audit found that the department's monitoring of how service providers performed in meeting contractual and legislative requirements, and in achieving outcomes for children, was inadequate.

The department has since introduced tools and processes that have improved how it oversees the performance of the department and service providers by providing more accurate and comprehensive information. Also, the rollout of an annual survey measuring performance against the out-of-home care Outcomes Framework, over the next three years, will allow the department and service providers to measure the impact of their services on people's lives for the first time.

This new focus on outcomes is being extended to work on a department-wide outcomes framework, and to getting planning, funding and accountability policies and mechanisms in line with achieving the intended outcomes for clients.

Audit committee oversight

Currently, the audit committee is updated on the progress and completion of departmental actions in response to VAGO performance audit recommendations through biannual status reports. However, the audit committee has not been provided with information about whether these actions have addressed the issues identified in our performance audits. Recent changes to the department's internal processes could address this.

Audit method and cost

The audit was conducted under section 15 of the Audit Act 1994, and was performed in accordance with the Australian Auditing and Assurance Standards. Pursuant to section 20(3) of the Audit Act 1994, unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

The total cost of the audit was $130 000.

Submissions and comments received

We have professionally engaged with the Department of Health & Human Services throughout the course of the audit. In accordance with section 16(3) of the Audit Act 1994, we provided a copy of this report to the department and requested its submissions or comments.

We have considered those views in reaching our audit conclusions and have represented them to the extent relevant and warranted. The full section 16(3) submissions and comments are included in Appendix A.

1.1 Introduction

At 31 January 2016, there were 442 children in residential care in Victoria. These children are among the most vulnerable in the community, and have complex needs. They are in the out-of-home care (OOHC) system because, in most instances, the Children's Court has decided they are at significant risk of harm, abuse or neglect from their families and cannot remain at home.

The Children's Court places these children under the protection of the Secretary of the Department of Health & Human Services (the department), which funds community service organisations to provide the placement services for children.

Residential care is the least preferred of three main OOHC options available to the Secretary. Preferred options are kinship and foster care, which are home-based placements. Children will be placed in residential care if they cannot stay in other home-based placement options either because their needs are too great or because the kinship or foster care options are not available or cannot keep them safe.

In March 2014, our audit report into Residential Care Services for Children (our 2014 report) was tabled in Parliament. The 2014 audit looked at the effectiveness of the then Department of Human Services' residential care services for children and young people.

Our 2014 report made six recommendations to:

• improve the then Department of Human Services' residential care services model to better meet the needs of children and young people in residential care
• improve performance monitoring in OOHC
• improve reporting of outcomes for children in OOHC
• strengthen the capacity and capability of residential care services staff.

Since our 2014 audit, machinery-of-government changes have seen the former Department of Human Services (which was subject to the original audit) merge with the former Department of Health to form the Department of Health & Human Services.

This follow-up audit looked at the progress the department has made in addressing the recommendations of the 2014 audit. Figure 1A provides an overview of the department's progress.

Figure 1A

Progress addressing and acting on recommendations

Summarised recommendation		Status
That the Department of Human Services:
1.	establishes alternative affordable models of care	✔
2.	promotes independent advocacy in a child's best interest	✔
3.	develops outcome performance measures	✔✔
4.	ensures staffing capacity and capability	✔
5.	identifies systems and processes for collecting and analysing information	✔✔
6.	reviews forecasting to meet placement demand	✔

Note: ✔ = commenced, ✔✔ = completed.

Source: Victorian Auditor-General's Office.

1.2 Meeting the needs of children

Our 2014 report concluded that the residential care system was unable to respond to the demand and growing complexity of children's needs and made three recommendations to address this inability, as outlined in Figure 1B.

Figure 1B

Recommendations 1, 2 and 6

That the Department of Human Services: establishes alternative affordable models of care with sufficient flexibility to cater for the varying and complex needs of children actively promotes to children in residential care the processes for making a complaint and investigates the feasibility of establishing an independent advocacy role to support children in residential care reviews existing demand forecasting approaches and develops plans for capacity that is actually required.

Source: Victorian Auditor-General's Office.

1.2.1 Alternative models of care

Our 2014 audit found that the residential care system had been operating over capacity because the needs of children in residential care were becoming increasingly complex and children were staying longer in residential care. As a result, the department had to buy additional placements (unfunded contingency placements) at a cost of $24 million in 2012–13. During this period, the key drivers of individual placement cost were demand (57 per cent), client complexity (36 per cent) and keeping siblings together (7 per cent).

Our 2014 report noted that the therapeutic residential care model had been found to have better outcomes than the standard residential care model because of planned placement decision-making and additional funding per placement to provide access to specialised support staff. A key feature of therapeutic residential care is the availability of a specialist to work with residential care staff to build confidence and expertise to manage children's needs and challenging behaviours. However, this type of care accounted for only 17 per cent of residential care placements at the time of the audit.

The department has acted to address the first recommendation in our 2014 report by developing alternative models of care. One of these models, Targeted Care Packages (TCP), introduced in April 2015, is reducing the number of children in residential care and unfunded contingency placements, while providing more affordable and flexible care options.

At the end of February 2016, the daily average occupancy of unfunded contingency placements had reduced from 141.9 places in 2012–13 to 38.7 places. Total spending on unfunded contingency placements fell from $24 million to about $11 million in the same period.

In addressing our recommendation, the department's policy changed from a focus on increasing the capacity of the residential care system, to residential placement prevention and quality initiatives. This was due to a commitment by the current government to significantly reduce the number of children in residential care.

This commitment was made in response to a recommendation made in the Commission for Children and Young People (CCYP) report, As a good parent would: Inquiry into the adequacy of the provision of residential care services, tabled on 18 August 2015. The commitment led to greater investment in the home-based care system to reduce the size of the residential care system, through TCPs and the Foster Care Recruitment and Retention Strategy.

Under the previous government, the focus was on increasing the capacity of the residential care system, and the department acted to achieve its long-term vision that all residential care placements would become therapeutic placements. This involved a tender process that was to provide some further therapeutic residential care in Victoria.

After the tabling of As a good parent would: Inquiry into the adequacy of the provision of residential care services, the department did not proceed with this initiative and reallocated the funding to TCPs.

Funding for support packages was allocated under the previous government as one of its immediate reform actions under Out-of-home care: a five year plan. However, TCPs began under the current government in April 2015. This new approach funded OOHC services that are focused on achieving outcomes for children and encouraged service providers to innovate.

TCPs are individualised and funding follows the child or young person throughout their OOHC journey. TCPs are targeted at those children in residential care where all attempts to move them into an existing form of care or to return home, with the current range of placements, supports and funds, have been exhausted.

At 31 January 2016, a total of 90 TCPs had been allocated. Of these, four were complete, with three young people returning home and one young person entering therapeutic residential care.

Of the 86 children and young people with active TCPs, about:

• one-third have been supported to return home to their family
• one-third are with a home-based carer
• one-third have been supported to begin transition to independence.

The department has provided case studies demonstrating the innovative thinking and flexibility of TCPs to meet the complex and varying needs of children and young people in residential care, as well as costings showing TCPs to be a more affordable model of care. Examples are provided in Figures 1C and 1D.

Figure 1C

Targeted Care Package case study—sibling group returns home

A group of five siblings with a long history of needing protection were experiencing poor educational, developmental and wellbeing outcomes. Child protection services removed the siblings from their home and placed them in unfunded contingency residential care. The siblings have since returned home with the support of a TCP to provide intensive support to improve their outcomes. This support includes increased in-home and personal care support to improve the living environment and hygiene, therapeutic support sessions to help address their significant trauma and additional support to help encourage community engagement. This support has allowed the children to remain with their family while helping to address the significant harm and neglect experienced. The TCP has improved outcomes for the siblings and returned them home at about one‑eighth of the cost of the unfunded contingency placement.

Source: Department of Health & Human Services, October 2015.

Figure 1D

Targeted Care Package case study—placement with a registered carer

Before entering an unfunded contingency residential care placement, the young person was in a long-term foster care placement, which broke down. Her former carers were emotionally invested and had maintained their long-term commitment to the young person while she was in residential care. The young person regularly absconded and engaged in inappropriate behaviour while in residential care. She was assessed as having an intellectual disability and had not attended school in a long time. The TCP enabled the young person to return to the care of her previous carers with whom she shares a strong connection. It provided necessary supports including respite to help ensure the ongoing stability of the placement. Therapeutic support is also provided to engage the young person with a focus on developing her relationship skills. The TCP has enabled the young person to return to a home-based placement with supports to improve outcomes and ensure the ongoing stability of the placement at about 40 per cent of the cost of the unfunded contingency placement.

Source: Department of Health & Human Services, October 2015.

The department is engaging an external contractor to evaluate the TCP program, with the first report due in June 2016, and a long-term report due in December 2016. The aim of the evaluation is to assess the effectiveness and efficiency of TCPs. This will include determining if TCPs are improving outcomes for children and young people in OOHC and identifying areas to improve. Findings will be used to improve the current program.

In October 2014, the department began work on a contained therapeutic model of care. This involved engaging KPMG and the Australian Childhood Foundation to develop an evidence paper based on:

• analysing the case files of 30 children who were frequent users of the department's Secure Welfare Services in 2013–15
• a literature review of therapeutic care for children in OOHC
• a review of the current OOHC options in Victoria.

The evidence paper identified the need for a service response that sits between the short-term Secure Welfare Services response, available for a maximum uninterrupted period of 21 days, and the longer-term therapeutic residential care service that can focus on stabilising a young person's situation enough to enable them to transition to a stable therapeutic placement option. The proposed contained therapeutic model of care is aimed at supporting this small cohort of young people whose complex needs are not being adequately met by the OOHC system.

In March 2016, the department received the draft final report outlining model options for a contained therapeutic model of care. The department is currently considering these options in the context of the Roadmap for Reform: strong families; safe children.

To address our recommendation on establishing alternative affordable models of care, the department had stated that it would carry out a major review of OOHC funding models as part of Out-of-home care: a five year plan. The aim of the review was to establish a system where funding to agencies was more focused on outcomes rather than almost exclusively on outputs.

The current government is pursuing the intent of the review, with the department engaging KPMG to carry out outcomes-focused performance framework research looking at ways to align planning, funding and accountability policies and mechanisms to achieving outcomes for clients. A final report that presents a recommended performance framework option is expected to be completed on 23 June 2016.

1.2.2 Increasing the voice of children in residential care

Our 2014 audit found that there was no clear independent advocacy on behalf of individual children in residential care. Although the department, the Victorian Ombudsman and service providers could all receive complaints about care from or on behalf of children in residential care, the audit found that service providers and current and former residents showed little awareness of these avenues. In addition, not all children in residential care knew how to report an incident or to whom to report it.

The department is addressing this recommendation through:

• working with the CCYP's Independent Visitor Program for residential care, which uses trained community visitors to listen to children and young people's experiences of being in residential care
• a project to promote and improve complaints processes
• scoping the feasibility of an independent advocacy role.

The CCYP informed the department in August 2014 that it would be piloting the Independent Visitor Program for residential care from February 2015 to June 2016 in the South Division. Its purpose is to provide an independent mechanism for children and young people to have their voice heard, provide feedback, raise matters of concern, and have problems resolved locally. Visitors can also help residents to lodge a matter externally if required, such as with Victoria Police or the Victorian Ombudsman.

After this advice, the department appropriately decided to defer finalising any action to establish an independent advocacy role until it receives the CCYP's final evaluation of the pilot. The department has been working with the Independent Visitor Program through being represented at quarterly meetings, receiving quarterly updates from the CCYP, and being responsible for identified placement and case planning matters.

In January 2015, the department began its complaints processes project. This has involved a co-design process, with sector representatives and children and young people in residential care being consulted on current processes and on the development of actions to address findings.

Findings from these consultations have been integrated into a final report with recommendations for promoting awareness of complaints processes in residential care, and for providing residents with appropriate support from an independent service, such as Kids Helpline, to help them prepare to make a complaint.

The OOHC Program Control Group endorsed the project's recommendations and prioritising of actions in April 2016 and is finalising time frames for implementation.

To address our recommendation on increasing the voice of children in residential care, the department had also originally stated that it would launch a targeted campaign, directed at children and young people in residential care, to increase their awareness of complaint mechanisms.

This has involved increasing accessibility to resources developed by the CREATE Foundation, explaining the rights of children and young people under the Charter for Children in OOHC. This was achieved through a one-off distribution of these resources to all residential care units in mid-2015 and by making them available on the department's website.

It was envisaged that the distribution of these resources to residential care units would provide an opportunity for young people to engage with clubCREATE, a program for young people aged 13–25 years who are or have been in care, and would raise awareness of the resources among children, young people and staff. The intended impact of this action has not been assessed.

However, the department stated that it intended to seek feedback from service providers in mid-2016 about the usefulness of the resources and the responses of children and young people in residential care to them. This would be timely, because evidence provided by the department shows both older residents and residential care staff members have questioned the accuracy of some aspects of the resource for younger children.

Other departmental actions promoting the voice of the child in residential care include:

• unannounced audits in residential care units that include discussions with children and young people about their experience living in that residential care unit (see section 1.4.2)
• considering potential opportunities for using a computer-assisted self-interviewing tool in ways other than for the National Survey of Children and Young People in OOHC, such as for evaluating TCPs.

1.2.3 Forecasting to meet demand

Our 2014 audit found that the residential care system had been operating over capacity since at least 2008, with the department paying a premium to place children in residential care outside the funded system capacity (unfunded contingency placements). In 2012–13, a 10 per cent shortfall in available residential care placements was mainly a result of:

• the increasing number of children entering the OOHC system
• lack of capacity and ineffectiveness of home-based care options
• the length of time children were staying in residential care.

The department spent $19.9 million in 2011–12 and $24 million in 2012–13 buying additional capacity to meet this shortfall.

Our 2014 audit also found that although the previous government had preferred to work on diversion strategies, these had had mixed results. Although they had gradually reduced the proportion of children in residential care aged under 12, this had resulted in a larger proportion of adolescents in care with more complex behaviours who were less likely to be moved to home-based care placements and were staying longer in residential care.

We expected the department to have developed plans to address the residential care deficit and the resultant need for unfunded contingency placements. Although the department has not addressed our specific recommendation, it has demonstrated action to address the underlying issue in line with current government policy—to reduce demand for residential care, the least preferred OOHC option.

To address our recommendation on forecasting to meet demand, the department advised that it had developed several models to look at demand for child protection, which had been used to inform State Budget bids. For example, in 2015–16, the department successfully asked for funding for an extra 85 full-time equivalent staff to meet demand for child protection.

However, increasing the number of child protection staff does not affect—and so has not addressed—the underlying problem of the residential care system operating significantly above its funded capacity, resulting in the need for unfunded contingency placements. In 2014, we recommended increasing the capacity of residential care to address this.

However, current government policy is to move away from building the capacity of the residential care system to increasing home-based care placements to reduce the number of children in residential care over time.

In line with this, the department prepared a 2016 Budget proposal using its existing OOHC demand forecasting model, the daily average occupancy by OOHC placement types, to request additional funding for forecast growth in home-based care placements. The department used the upper limit of growth of its demand modelling forecasts in response to recent trends in the number of actual placements consistently exceeding forecasts.

The department estimates that its residential care deficit will reduce by 30 per cent from 2013–14 to 2015–16 due to its investing in the home-based care system, such as through TCPs.

It is expected that moving children and young people from residential care to home‑based care will eventually allow for the closure of unused residential care units, so reducing the overall size of the residential care system. However, TCPs are not recurrently funded and will depend on children and young people leaving the OOHC system or alternative sources for future growth.

The department should continue to monitor the impact of its investment in home-based care placements and TCPs in reducing the residential care structural deficit in a timely manner.

Under-used capacity

Our 2014 audit found that some service providers were rejecting placements while receiving funding to provide services in the future, and that the department was unable to readily recoup or redirect funding in instances where placement capacity was not used for operational reasons. We considered this situation to be untenable if residential care was to remain sustainable.

Using the Service Delivery Tracking tool, described in further detail in Section 1.4.2, the department can now estimate the value of undelivered services at the end of the financial year. At 31 December 2015, the forecast statewide value of undelivered OOHC placement services at the end of 2015–16 was $21.2 million, of which $13.7 million was attributed to undelivered residential care services. The reasons for this include:

• the responsive nature of residential care requires a system that, at any one time, has some available capacity to be able to respond to urgent demand for placements
• the various factors that have to be considered when placing a child means that an available vacancy might not be an appropriate placement because of its location or the needs of the children in the residential care unit.

In line with the current government's focus on home-based care, the department's Home‑based care performance and reporting requirements policy, which took effect on 1 January 2015, outlines the process for reviewing home-based care providers' performance. These reviews have resulted in target adjustments or reallocations where there is underperformance, and reimbursements to recognise over‑performance.

1.3 Staffing capacity and capability

Our 2014 audit found that current staffing models were not providing sufficient quality and stability of care for children in residential care, with casual staff making up 55 per cent of the residential care workforce, and most service providers using a 'single parent' staffing model.

Also, no formal qualifications were required to work in residential care services although staff were employed to care for children with complex needs, including violent and risk-taking behaviours, substance abuse and mental health issues.

Figure 1E shows the audit recommendation to address staffing capacity and capability issues.

Figure 1E

Recommendation 4

That the Department of Human Services ensures that residential care staff have the necessary skills, qualifications, training and support to work effectively with children and their families.

Source: Victorian Auditor-General's Office.

The department has addressed this recommendation through:

• commissioning a residential care workforce census in 2014
• starting the Workforce Quality Initiative (WQI)
• introducing a $16 million 'funding uplift' to the overnight staffing model of 145 residential care units for 16 months
• submitting a State Budget bid for ongoing funding to roll out the overnight staffing model throughout the residential care system.

The 2014 residential care workforce census updated the 2012 census profile of the workforce and its qualifications. The department used the findings to identify areas where staff required support and development and to allocate funding to specific residential care training. The census was also used to inform the WQI, which focuses on determining which skills and training are appropriate for residential care staff.

The WQI consists of two main components:

• a future capability framework, including considering the introduction of a minimum qualification
• a coaching program that comprises training, coaching and case consultation to embed theory in practice.

To date, the department has commissioned scoping work for the capability framework and a literature review on the types of capabilities and competencies required of residential care staff, which was completed in February 2016. The next step involves arranging focus groups to obtain feedback from children and young people in residential care to find out what skills and capabilities they want from residential care workers. This work is expected to be completed by July 2016.

Findings from the initial stages of the WQI were used to develop the existing Residential Care Learning and Development Strategy for 2015–18, which aims to develop a skilled workforce through training and developing relevant qualifications for residential care staff. Future work under the WQI will continue to inform the Residential Care Learning and Development Strategy.

In addition to these activities to enhance the capability of residential care staff, the department also piloted an initiative to enhance their capacity. From March 2015, the department introduced changes to the overnight staffing model to directly address issues with placing a single staff member on duty overnight. This meant adding a staff member per four-bed house to provide greater support overnight.

A preliminary evaluation of the initiative found that it had improved the capability and capacity of community service organisations, including helping staff to prevent safety issues through better engagement and stability.

With this early evidence, the department has sought ongoing funding to provide for additional staffing overnight to improve safety, supervision and support throughout the residential care system. The department plans a final evaluation of the initiative in late 2016, towards the end of the funded period.

1.4 Performance monitoring and reporting

Our 2014 audit concluded that children's safety, stability and personal development are compromised by system constraints, including the inadequate monitoring of overall outcomes. Figure 1F states the audit recommendations to address these issues.

Figure 1F

Recommendations 3 and 5

That the Department of Human Services: develops performance measures for delivery of residential care services for children to meet legislative requirements and outcomes for children identifies systems and processes for collecting and analysing information which better meet the department's compliance, assurance and reporting needs.

Source: Victorian Auditor-General's Office.

1.4.1 Outcomes performance measures

Our 2014 audit found that the department did not systematically monitor outcomes for the whole residential care population, with service providers required only to report outputs, not outcomes.

Under the previous government, the department committed to establishing an Outcomes Framework for children and young people in OOHC in Out-of-home care: a five year plan, as well as to investing in a system that would enable tracking and reporting on outcomes. This commitment has been carried through under the current government.

To date, the department has addressed our recommendation by developing an Outcomes Framework through consultation and co-design processes with the wider community services and OOHC sector. The department engaged an external contractor to develop an online tool to enable outcomes tracking with an annual survey. The first survey took place in April 2016. Surveys will take place every year from 2016 to 2018, and the data used to improve services.

This is the first time that there has been an OOHC Outcomes Framework, which will allow the department and service providers to measure whether the services provided make a difference in people's lives. It will provide an evidence base that demonstrates the real value of the services provided.

The OOHC Outcomes Framework consists of three domains and seven outcome areas. The outcome areas represent key areas of disadvantage experienced by individuals and families. Each outcome area is underpinned by indicators, measures and questions that have been validated and tested by stakeholders from the department, service providers and relevant agencies such as the Centre for Excellence in Child and Family Welfare and the Victorian Aboriginal Child Care Agency. The OOHC Outcomes Framework is outlined in Figure 1G.

Figure 1G

The Out-of-Home Care Outcomes Framework

Source: Department of Health & Human Services, March 2016.

Questions for each outcome area have been incorporated in a survey of 100 questions, which service providers are required to complete for each child in OOHC in Victoria as a condition of their funding agreement. Service providers will now be able to use this survey tool to analyse what is working well for children and young people in their care compared to the rest of the state.

The department will use comparative analysis of service providers, survey questions and outcome areas to identify where follow-up is required. This will also be determined by comparing with findings from the National Survey of Children and Young People in OOHC that Community & Disability Services ministers agreed to in March 2012, and which the Australian Institute for Health & Welfare collates and analyses.

The department has advised that it intends to develop a similar survey tool for completion by children and young people to allow for validation of workers' views, subject to consideration by the OOHC Outcomes Framework Working Group.

The department is also addressing our recommendation more widely through:

• developing a Social Services System Outcomes Framework, which will build on and sit above the OOHC Outcomes Framework—this will involve raising outcomes from the client level to the social service system level
• a research and development project to look at ways of getting planning, funding and accountability policy and mechanisms in line with achieving the intended outcomes for clients.

1.4.2 Monitoring and reporting systems and processes

Our 2014 audit found that the department's data information systems were ineffective, with the incident reporting system being time-consuming and inefficient and the Client Relationship Information System being cumbersome and disorganised. Also, record‑keeping practices and case-plan implementation and monitoring were not subject to sufficient and appropriate quality assurance mechanisms.

The department has addressed this through identifying and implementing systems and processes for collecting and analysing information. This has allowed the department to better hold senior managers and service providers to account for their performance.

The department has introduced four tools and systems, which have improved its oversight of department and agency performance by providing a more accurate and comprehensive picture of performance than was previously available:

• The Qlikview analytic application is an interactive business intelligence tool allowing for analysis of the current range of performance measures throughout the department. The department has used this analytic application in developing tools to improve departmental monitoring and reporting on critical incidents and performance, that is the Critical Incident Analysis tool and the electronic Performance Assurance and Compliance tool (ePAC).
• The Critical Incident Analysis tool uploads data from TRIM, the system used by the department for manual incident reporting, to a central database and allows for analysis of data previously unavailable to agencies or the department.
• A cloud-based analytic application and monitoring tool holds monthly data on over 300 pre-existing key performance indicators and enables comparing trends over time, and agency performance by division down to area level. Data sets available for comparison include child protection demand, child protection compliance, OOHC contingency placements and OOHC placement types.
• The Service Delivery Tracking tool requires funded organisations to account for monthly service delivery against their service agreement target.

Regular analysis of performance information in ePAC is provided to senior leadership across the department, including to inform discussion at the Divisional Strategic Oversight and Coordination Committee, which comprises the Deputy Secretaries of Operations and the four divisions. The committee drives the department's strategic direction across the four operational divisions and is responsible for identifying and addressing emerging issues and risks related to service delivery performance and client outcomes. They do this through:

• problem-solving of significant issues for operational divisions
• promoting the development of strategies and actions to address emerging issues and risks
• advocating for service delivery and policy reform when gaps are identified.

ePAC operational data is also used as part of a biannual Divisional Performance and Compliance process whereby senior divisional and area managers are questioned about and held accountable for their progress towards meeting performance targets.

ePac data has also been used to develop Issues Papers for other committees as requested. For example, papers were developed for the OOHC Strategic Oversight Group and the Client Safety, Experience and Quality Committee in June 2015.

On 30 March 2015, the department began a program of unannounced audits of residential care units, which complements the existing accreditation process—a major formal review every three years by an independent body. Organisations are audited using the Human Services Standards—the quality standards for department-funded service providers and department-managed services.

The audits include discussions with children and young people, personal observations, interviews with staff and checking formal records. Auditors assess several aspects including residential care units' processes for complaints, incident reporting, supporting children and young people from Aboriginal and other cultural backgrounds, and developing and reviewing care plans. Service providers are required to provide the department with an action plan to respond to any problems identified.

A follow-up audit is carried out about three months later to review evidence that the service provider has satisfactorily implemented the actions and recommendations outlined in the initial audit report. Any outstanding issues identified at the follow-up audit are conveyed to the service provider's independent review body.

A process is in place for escalating matters that require urgent action, through a conference between the chief executive officer of the organisation and the divisional Deputy Secretary. Only one such conference has taken place to date.

At 31 January 2016, 80 of 240 residential units had been audited, with all units to be audited by September 2016. The department has produced quarterly reports for service providers from July 2015 on completed audits, identifying areas of strength and opportunities for improvement.

Since November 2015, these reports have included information on how to meet compliance requirements for common areas of noncompliance, such as medication administration. Specific areas of significant noncompliance have also been analysed and reported to the Divisional Strategic Oversight and Coordination Committee, following specific requests.

Critical Incident Reporting System

Our 2014 audit found that the department's incident reporting system relied on manual data entry using TRIM, which means that the department cannot be assured that its response to critical incidents is efficient or timely.

The Critical Incident Analysis tool described above is an interim solution to overcome the limitations of TRIM. In November 2015, to address systemic issues, the department engaged KPMG to develop and implement a new Client Incident Management System (CIMS). This built on KPMG's August 2014 review of the department's end-to-end critical incident response and management approach.

The proposed new CIMS will focus on the most serious incidents and increase local service-level accountability for non-critical incidents. It will include a model for:

• the approach to providing immediate support to clients and prioritising actions after an incident
• categorising incidents (to ensure a stronger focus on the most serious incidents)
• the approach to investigating incidents
• how the department and service providers review actions taken after an incident to ensure that clients, their families and carers receive the right support
• how the department and service providers learn from incidents to improve services
• accountability of the department and service providers.

The department has advised that the new CIMS is on track to be completed by 30 June 2016, with implementation to commence from July 2016.

Client Relationship Information System

Our 2014 audit found that the department's Client Relationship Information System operated mainly as a depository, with case-file information being difficult to find in the system. Consequently, the department failed to meet legislative requirements, including to complete and review case plans within stated time frames.

Also, the department did not actively monitor or report on compliance with the requirement for each Aboriginal child in OOHC who is subject to a Guardianship to Secretary Order to have a cultural support plan. This requirement became law on 1 March 2016.

The department has addressed our recommendation through implementing the Qlikview and ePac tools that have allowed it to actively monitor these requirements and hold divisions to account. ePAC contains a monthly update on statewide and divisional performance against six key compliance child protection measures, including the percentage of statutory case plans completed and the proportion of cultural support plans completed in draft or final version.

Since our 2014 audit, the department has improved its statewide statutory case plan completion performance from 76.6 per cent (at 30 June 2013) to 85.5 per cent (at 30 December 2015). On the other hand, compliance with cultural support plan requirements has not improved. However, the department has submitted a Budget bid for funding to meet this recently legislated requirement, and to reform the current approach to cultural support plans.

1.5 Audit committee oversight

This audit assessed whether agencies had allocated resources to regularly monitor progress against performance audit recommendations, and their impact.

Agency audit committees are required to monitor audit recommendations and enact planned responses through Standing Direction 2.6(f) under the Financial Management Act 1994. This includes requirements for audit committees to:

• recommend how the responsible body should act on advice received from external auditors
• monitor actions taken by management to resolve issues that external auditors raise
• monitor whether management addresses the accepted recommendations of external auditors in a timely manner
• review the impact of actions that management takes to resolve issues.

We expect agencies to have a process in place for demonstrating to audit committees the impact of their actions in addressing the issues underlying audit recommendations.

The department has a process in place for reporting to its audit committee through twice-yearly updates from program areas on the status and completion of actions taken in response to VAGO recommendations. However, these updates do not inform the audit committee about the impact of actions the department has taken to address performance issues.

In March 2016, the department's internal audit report, Follow Up of VAGO Performance Audit Recommendations, recommended that divisions should be responsible for assessing the impact of their actions in responding to VAGO recommendations and obtaining senior-level endorsement of this assessment before saying that actions have been completed.

Providing these assessments to the audit committee would address the issue identified here because it provides the audit committee with a more robust basis to understand the impact of actions that management has taken in response to our recommendations. Following endorsement by the audit committee in April 2016, the department advised that it would be implementing this new approach.

Introduction

In accordance with section 16(3) of the Audit Act 1994, a copy of this report was provided to the Department of Health & Human Services.

The submissions and comments provided are not subject to audit nor the evidentiary standards required to reach an audit conclusion. Responsibility for the accuracy, fairness and balance of those comments rests solely with the agency head.

RESPONSE provided by the Secretary, Department of Health & Human Services

RESPONSE provided by the Secretary, Department of Health & Human Services – continued

[ Contents of this report | VAGO Home | VAGO Publications ]