Ravenhall Prison: Rehabilitating and Reintegrating Prisoners

Tabled: 19 March 2020

3 Monitoring and evaluating Ravenhall’s performance

Ravenhall is the first Australian prison to have key performance measures and payments linked to its reintegration and reoffending outcomes. If successful, this approach could be applied more broadly throughout Victoria’s private prison system.

In this Part, we assess if CV and GEO have designed appropriate performance monitoring and evaluation frameworks that can attribute outcomes to Ravenhall’s unique programs and interventions. We also consider GEO’s early performance outcomes.

3.1 Conclusion

While both KPIs were well-intentioned, they have design limitations that impair their ability to definitively measure Ravenhall’s reintegration and reoffending outcomes.

KPI 15 measures prisoners’ outcomes following their engagement in post release reintegration services. It is not working as intended and has proven difficult for GEO to design and implement.

KPI 16 is not a valid or useful measure to determine performance payments for recidivism outcomes because it does not measure Ravenhall’s impact on reintegration or reoffending. It does not determine if reoffending outcomes are linked to GEO’s interventions at Ravenhall or if they should be attributed to other factors, particularly where prisoners released from Ravenhall have spent time at other prison facilities.

CV has no evaluation or research projects planned to explore the relationship between Ravenhall and its reoffending outcomes. This means that if Ravenhall is successful and has a lower rate of recidivism than other prisons, CV does not have a method to determine if it was Ravenhall’s programs and interventions that caused it.

3.2 Performance indicators

GEO will receive performance payments of up to $1 million per year for achieving each of its KPI 15 and 16 targets. This collective $2 million is a small percentage of the overall service payment available to GEO. While KPIs 15 and 16 measure outcomes, several of GEO’s other performance measures also directly relate to rehabilitation and reintegration services.

KPIs 15 and 16 are stretch targets, which means that they were designed to be challenging to achieve. Like any KPI, they need to be practical, clearly defined, understood and, given the contractual arrangement, agreed to by both parties.

It is important to measure and monitor rehabilitation and reintegration outcomes. However, CV’s financially incentivised KPI 16 is not the best mechanism to do this because it does not solely describe the impact of GEO’s interventions at Ravenhall.

KPIs 15 and 16 only measure outcomes for sentenced prisoners. Ravenhall’s increased proportion of remandees has significantly reduced the number of prisoners who are eligible to be included in these KPIs. This has further reduced their effectiveness as performance measures.

The changes to Ravenhall’s prisoner profile limit CV’s opportunity to observe how successful GEO’s Continuum of Care model is. Ravenhall’s reduced numbers of sentenced prisoners could also impact GEO’s ability to achieve its targets for KPIs 15 and 16. This is because it is providing criminogenic interventions to fewer prisoners.

Key performance indicator 15—reintegration services

KPI 15 measures how successful GEO is at supporting sentenced prisoners who have high reintegration needs across the five key reintegration domains of education, employment, housing, alcohol and drug use, and mental health. Because KPI 15 measures if prisoners successfully reintegrate, it may provide an early indication on whether Ravenhall will achieve its recidivism target for KPI 16.

To date, KPI 15 has not functioned as intended. Some challenges around this KPI include GEO’s and CV’s different views about which prisoners it counts, and its challenging information collection requirements. CV and GEO are working together to refine the KPI’s definition, which is likely to resolve most, if not all, of these issues.

Once the new definition is finalised, CV will review all of GEO’s earlier reported results. This may trigger retrospective deductions to KPI 15 performance payments.

Refining KPI 15

As of December 2019, GEO and CV are finalising revisions to KPI 15’s definition. They have been working together to refine the KPI for more than six months. CV and GEO are also drafting a reintegration-based KPI for remand prisoners because KPI 15 does not apply to them.

It is important that operational and strategic staff from GEO and CV are involved in revising the KPI. Many of the staff who manage the contract and report against KPI 15 were not involved in its early development. Revising the KPI presents an opportunity to refine its function with added insight about its practical application.

1. Does KPI 15 apply to remand prisoners?

It was initially unclear if KPI 15 included remand prisoners. KPI 15 was developed during the procurement of Ravenhall. At this time, Ravenhall was intended to hold sentenced prisoners only.

Why was this a problem?

Resolution

This was part of CV and GEO’s broader negotiation about Ravenhall providing intensive pre and post release reintegration services to remand prisoners, and the funding associated with this.

When remand prisoners were introduced as part of contract change 7, GEO and CV disagreed about whether KPI 15 applied to them or not.

GEO and CV are currently developing a service package of reintegration supports for remand and short-stay prisoners. Remand prisoners will not be measured under KPI 15. Instead, CV and GEO are developing a new KPI.

2. Who is a KPI 15 pathway participant?

Prisoners measured under KPI 15 are known as ‘pathway participants’. According to the KPI’s definition, a prisoner becomes a pathway participant if they have ‘an identified post-release need’ and are a ‘person who would benefit’ from post-release services.

GEO interpreted ‘a person who would benefit’ as prisoners who consent to engaging with GEO’s reintegration services. As a result, GEO did not report on prisoners with an identified post release need if they did not consent or willingly engage.

Why was this a problem?

Resolution

CV considered that GEO’s interpretation did not capture the intent of the KPI, which is to successfully engage prisoners in post-release services. Because GEO reported less pathway participants, its final KPI 15 results, once finalised, are likely to be worse than initially reported.

GEO is required to resubmit all KPI 15 data and results, including prisoners who did not consent to be pathway participants. Prisoners who did not provide consent will now be listed as a ‘fail’.

GEO and CV are currently refining KPI 15’s definition, prisoner eligibility, and valid exceptions.

3. Which pathway should prisoners participate in?

KPI 15’s current definition does not allow GEO to exclude prisoners who will derive limited value from participating in multiple pathways.

In some situations, participating in multiple pathways does not add value or benefit for the prisoner. For example, a participant who has established full time employment is unlikely to benefit from participating in a full-time education or training program that conflicts with their work, even if education is one of their identified needs.

Why was this a problem?

Resolution

Measuring the KPI with this definition may provide a more negative picture of prisoner engagement or progress than intended.

This issue is being considered by CV and GEO in their current revision of KPI 15.

4. Challenging information collection requirements

GEO is required to provide the following evidence for pathway participants’ success against KPI 15:

  • education/training—academic providers need to provide written confirmation
  • employment—case managers need to routinely contact employers to verify ongoing employment
  • housing—copies of rental agreements, confirmation from public housing representatives, proof of property ownership or utility bills need to be provided by prisoners
  • AOD treatment—treatment providers need to give written confirmation of attendance
  • mental health treatment—treatment providers need to give written confirmation of attendance.

GEO advised that it has found it difficult to obtain the required evidence from third-party service providers, including private employers and medical, healthcare and AOD service providers. This is because service providers are not obligated to provide information to GEO. Others refuse to provide GEO with information due to privacy concerns, even when the prisoner has given their consent.

Why was this a problem?

Resolution

KPI 15’s information collection requirements can be difficult to comply with, which may be impacting GEO’s results.

In instances where GEO cannot obtain the required evidence, the pathway participant is recorded as a ‘fail’. GEO is aware of instances where pathway participants had successfully achieved the KPI but were recorded as failures because GEO could not obtain the required evidence. This not only affects the reliability and accuracy of the data used to calculate Ravenhall’s results against the KPI, but limits CV’s understanding of how successful GEO’s interventions are.

CV and GEO may address this in their revision of KPI 15.

After attempts to obtain the required evidence fails, CV may accept a file note from GEO’s reintegration officers instead.

5. Target outcomes are not practical

Some of the targets for KPI 15’s pathways are extremely specific and impractical.

For example, the target outcome for KPI component 15.2 is for pathway participants to maintain employment of at least 20 hours per week for two months following their release.

Why was this a problem?

Resolution

This target is restrictive because it does not consider that a participant could work less than 20 hours per week, or for non consecutive weeks. A participant with casual employment of less than 20 hours a week would fail to meet the target outcome despite having successfully gained employment. This undermines the intent of the KPI.

CV states that it has validated pathway participants as a ‘pass’ in some instances where continuous employment under 20 hours has been documented.

CV and GEO may make this more explicit in their current revision of KPI 15.

Payments by results

In April 2019, CV requested GEO to resubmit all KPI 15 data and results to count prisoners who did not consent as pathway participants. All KPI 15 results, and GEO’s associated performance payments, are currently under review and marked as pending. If the KPI results are changed to a fail, CV will make a retrospective deduction from GEO’s performance payments.

Key performance indicator 16—reducing recidivism

KPI 16 is an outcome-based measure to test if the steps GEO has taken to rehabilitate prisoners and prevent future reoffending have been successful in the long term. This KPI compares Ravenhall’s recidivism rate against the average rate of other Victorian prisons.

CV modelled KPI 16 on the RoGS rate of return to prison measure. This measure attributes a prisoner’s reoffending outcome to the prison they were discharged from. Based on the available data, this is the easiest way to attribute a prisoner’s reoffending outcomes to a specific prison. However, it is not a reliable indicator of GEO’s impact on recidivism through Ravenhall’s model of interventions.

The Victorian prison system, which includes public and private prisons, is highly integrated, and prisoners frequently move between prisons for varying reasons. With this much movement, it can be misleading to attribute a prisoner’s reoffending outcome to the prison they were discharged from. KPI 16 does not consider the amount of time a prisoner spends at Ravenhall compared to other prisons they served part of their sentence at. It ignores the degree of influence that time spent at other prisons may have on a prisoner’s reoffending outcome.

Most of Ravenhall’s sentenced prisoners are transferred from another prison or correctional centre. Hypothetically, prisoners may serve most of their sentence at another prison and then transfer to Ravenhall for the last few weeks or months. These prisoners may not be at Ravenhall long enough to benefit from its Continuum of Care model, but their reoffending outcomes would still be attributed to Ravenhall rather than the broader system.

Conversely, prisoners may serve most of their sentence at Ravenhall and benefit from its interventions before transferring elsewhere prior to their release. These prisoners could have their reoffending outcomes attributed to the broader statewide system, which could potentially reduce Ravenhall’s rate of success.

A prisoner is eligible to be counted under KPI 16 if they are a sentenced prisoner who is released from Ravenhall. The KPI does not consider a prisoner’s length of stay at Ravenhall. From January 2019 to December 2019, 1 316 prisoners discharged from Ravenhall (or 51 per cent) were sentenced prisoners. Of these 1 316 sentenced prisoners, 59.1 per cent spent less than 90 days at Ravenhall. These prisoners were not eligible to engage in Ravenhall’s intensive programs (including OBPs, intensive reintegration services and FISPs) or engage in the full Continuum of Care model. For this reason, they did not spend enough time at Ravenhall to benefit from its interventions.

While KPI 16 is intended to capture the positive impact of GEO’s model, it is insensitive to the relationship between the amount of time prisoners spend engaging with Ravenhall’s programs and services and the likelihood of reducing recidivism. Research shows that taking the appropriate amount of time for treatment and interventions is an important factor in successfully rehabilitating prisoners.

Performance payments and attributing outcomes

KPI 16 is a blunt performance measure. While KPI 16 measures the rate at which prisoners released from Ravenhall return to prison, it cannot determine if Ravenhall’s interventions, services or supports have influenced the outcome. This means that if Ravenhall has a lower rate of recidivism than other prisons, then the KPI measure cannot attribute this to its unique programs and interventions.

As we note in the Context chapter, a range of external societal conditions can correlate to, or be a greater contributing factor on, recidivism rates.

As GEO will receive performance payments of up to $1 million if Ravenhall achieves KPI 16, it is concerning that a successful outcome cannot be directly attributed to its programs and interventions. While measuring and monitoring prisoners’ rates of return to individual prisons is valuable, it is not an appropriate way to determine performance payments. This is because it cannot fairly attribute results to any individual prison.

3.3 Evaluating outcomes

An evaluation project would be better equipped to establish causality between individual prisons and recidivism rates.

Unlike Ravenhall, CV has access to both Ravenhall’s data and data from other prisons. For this reason, only CV, or an independent research body it commissions (such as a university), can undertake system-wide analysis.

GEO has appropriate processes to evaluate Ravenhall’s program outcomes. It also has several planned research and evaluation projects, although it is not clear how or when it will complete some of these.

Evaluating Ravenhall’s interventions

CV commissioned the AIC to develop a research and evaluation agenda for Ravenhall in 2018. CV now accepts that this research agenda must be updated to reflect the introduction of remand prisoners. However, CV states it has capacity limitations and has no current plans to undertake this work.

Even so, much of the AIC’s suggested research aligns with the type of research and evaluation activity that we would expect CV to undertake. The AIC’s suggested research questions relate to reducing reoffending (see Figure 3A), and to outcomes for each of Ravenhall’s key focus areas. Evaluation and research projects could help CV to causally determine if introducing remand prisoners to Ravenhall impacted the success of its rehabilitation and reintegration model.

Figure 3A
AIC-suggested research project: reducing reoffending

Research question:

Across the whole cohort of prisoners, do prisoners released from Ravenhall differ from those released from other Victorian prisons in their likelihood of reoffending and time taken to reoffend?

  • What factors in Ravenhall’s programmatic responses influence observed differences in outcomes?

Source: AIC research agenda.

CV does not currently have plans to evaluate Ravenhall’s outcomes beyond the KPI measures. Better indicators and a strong research and evaluation project is required to meaningfully compare Ravenhall’s performance to that of other prisons.

Evaluating and comparing system-wide outcomes

CV’s research projects are guided according to its overarching research and evaluation framework.

CV has completed and commissioned multiple reviews and evaluations of specific cohorts, interventions, program and services that have included rehabilitation and reintegration. These include reviews and evaluations of high risk parolees and serious violent offenders, CV’s reintegration pathway, case management of sex offenders, reoffending risk assessment tools and prison parenting programs.

However, CV does not have any current or planned review of system wide reoffending and recidivism outcomes.

Data analysis

Evaluating and analysing system-wide outcomes is a challenging task. It requires an appropriate and ethical research methodology, available and accurate data and an IT system that can capture the data necessary for research.

CV produces some analysis to inform decision-makers about prisoner outcomes. This includes analysis of return to prison rates, trends for specific cohorts and the impact of policy changes such as parole and bail reforms. However, CV’s IT systems and datasets cannot support regular and complex analysis of reoffending, such as that required for evaluation studies. This limits CV’s ability to evaluate and understand prisoner outcomes.

GEO’s research and evaluation

GEO is contractually required to research:

  • the effectiveness of Ravenhall’s pre and post-release reintegration programs
  • the validity of KPI 15’s methodology to indicate reintegration outcomes and link this to recidivism rates.

CV requires GEO to annually submit a report outlining how it will conduct this research, including its research methodology. GEO has developed appropriate research questions that could potentially add value to its understanding of the effectiveness of Ravenhall’s rehabilitation and reintegration services. However, GEO’s current research and evaluation agenda for Ravenhall is general and does not include information such as:

  • the timing for research projects (including planned start or finish dates and duration)
  • required resourcing or staffing numbers
  • a scope or methodology to answer the research question.

CV advises that it is working with GEO to improve its research and evaluation agenda for Ravenhall.

GEO’s plan does not specify its timelines to complete the research. However, it is currently undertaking two evaluation projects that align with its research agenda. These evaluations examine Ravenhall’s remand service pathway and the effectiveness of KPI 15.

Other significant external academic research includes:

  • plans for an academic review of Ravenhall’s Continuum of Care model
  • an evaluation of Ravenhall’s forensic mental health services commissioned to Swinburne University of Technology and Forensicare. The university is due to complete this in December 2021.

3.4 Ravenhall’s performance outcomes to date

Ravenhall has produced mixed performance results to date. In some instances, its performance may be related to implementation issues, as it is still a relatively new prison. GEO has been making improvements to address these performance issues. Additionally, CV and GEO have revised and refined several of Ravenhall’s SDO and KPI benchmarks and definitions to ensure that they are fit for purpose and reflect the cohort change.

Performance measures

Beyond KPIs 15 and 16, a number of Ravenhall’s other SDOs and KPIs are relevant to its reintegration and rehabilitation outcomes. We analysed GEO’s performance against several indicators to undertake an early calculation of its outcomes. We considered Ravenhall’s performance from its opening in November 2017 until December 2019.

As Ravenhall is a relatively new prison, we expected that it would take time to meet its performance targets. Ravenhall initially did not meet its performance targets for the following measures:

  • SDO 14—prisoner engagement in purposeful activity
  • SDO 15—vocational education and training
  • SDO 23—case management
  • KPI 15.1 to 15.5—reintegration.

As outlined in Figure 3B, GEO has implemented improvements to address these performance issues. While GEO has not achieved its performance targets for SDOs 14 and 23, it has achieved improved outcomes, notably for SDO 15 in the second half of 2019.

Figure 3B
SDOs 14, 15 and 23

SDO and measure

Status at Dec 2019

Issues

Improvements and rectification strategies implemented by GEO

SDO 14 Engagement in purposeful activity

How many prisoners complete the required weekly hours of ‘purposeful activity’. This includes working in prison industries and/or participating in programs, training, education and reintegration activities.

Not yet passed

GEO stated that its failure to meet this SDO has been influenced by:

  • increases in prisoner numbers, particularly as the prison’s industries building struggled to meet demand
  • the introduction of remand prisoners, who cannot be required to work or participate in programs or education
  • different methods of record keeping and data storage—some records are kept in hard copy and others digitally in Gateway.
  • splitting industries into morning and afternoon shifts to accommodate more prisoners
  • training Gateway ‘champions’ to assist staff with scheduling activities
  • strengthening induction processes to encourage prisoners to work within their first week
  • strengthening the prisoner incentive scheme.

Note: If contract change 9 is implemented, or Ravenhall’s prisoner places increase beyond 1 300, CV will decrease the SDO 14 benchmark by 2.5 hours per week.

SDO 15 Vocational education and training

If prisoners participate in approved vocational education and training courses.

Not passed until July 2019.

Since July, passed every month.

The state’s 2018–19 review of SDO definitions and benchmarks found that the SDO 15 counting rule had historically been applied differently than intended. Due to this, the SDO benchmark was reduced by 2 per cent at 1 January 2019.

  • having monthly meetings with Alliance Partners
  • creating improvement plans for underperforming Alliance Partners
  • rectifying IT access issues for Alliance Partners.

Note: As at 1 July 2019, GEO and CV agreed to exclude prisoners with significant mental health issues from being counted under this SDO. This is because these prisoners are too unwell to participate in education and training programs.

SDO 23 Case management

Ravenhall’s average case management audit score. To calculate this, CV reviews GEO’s case management files and scores them against a case management audit tool using a point system.

Not yet passed, but improving*

GEO stated that its failure to meet this SDO has been influenced by:

  • having new staff
  • adjusting to new processes
  • settling into the new prison.
  • having peer auditors conduct monthly audits and provide feedback
  • delivering additional training to GEO clinicians by CV’s Offender Management Branch
  • having Ravenhall staff self-audit files
  • delivering additional training to Ravenhall clinicians from staff at GEO’s Fulham prison.

Note: This is one of the most commonly unmet SDOs across the Victorian prison system.

*In July 2019, the state agreed to lower Ravenhall’s benchmark for this SDO by 0.25 points. In October 2019, the prison met this SDO. However, it recorded an overall ‘fail’ for the October–December 2019 quarter.

Source: VAGO, based on CV and GEO documents.

Back to Top