Victorian Auditor-General’s report
6 April 2011

Problem-Solving Approaches to Justice 
Audit summary
Problem-solving approaches to justice have been developed over the last decade to address the underlying causes of crime and to find solutions to the complex problems facing offenders, victims and the community. These approaches aim to stop, or at least slow, the ‘revolving door’ where some offenders move in and out of the criminal justice system. The Department of Justice (the department) and the Magistrates’ Court (the court) have pioneered problem-solving approaches to justice.
In 2005, two new programs were funded based on problem-solving approaches: the Neighbourhood Justice Centre (NJC) and the Court Intervention Program—later to be known as the Court Integrated Services Program (CISP). The 2009–10 Budget, extended commitment to both programs, bringing total planned expenditure for NJC to $50 million and CISP to $28 million.
Both NJC and CISP are the first such programs in Australian jurisdictions. NJC was established in February 2007 at a Collingwood site within the City of Yarra. It is a multi‑jurisdictional venue, presided over by a single magistrate, and provides a range of onsite services to victims, offenders, civil litigants and the local community. These services include drug and alcohol, and mental health counselling, legal advice, and housing support. NJC also works with government, service providers and the local community to address local community problems that may contribute to crime.
CISP began in November 2006 and operates at three Victorian Magistrates’ Court venues: Melbourne, Sunshine and Latrobe Valley. Defendants who have been charged but have not yet been sentenced can be referred to CISP. The program provides short‑term assistance to individuals with health and social needs with the aim of reducing their likelihood of reoffending. CISP case managers link clients with community support services, including drug and alcohol treatment, crisis accommodation and mental health services. 
This audit examined whether NJC and CISP are achieving their intended outcomes. It also assessed whether the programs were based on sound evidence and research and whether the department and the court are effectively managing the programs. 
Conclusions
The initial development of NJC and CISP was based on the best available evidence and research. The department’s funding submissions adequately included the key elements required. However, the program objectives for NJC and CISP could have been clearer and this affected the ability to evaluate whether programs had been effective in meeting their intended objectives. In preparing the funding submission for CISP, the department could have better consulted with the court, in particular, on key program components such as referral targets. This would have made CISP implementation more timely and effective.
The introduction of NJC and CISP was both challenging and complex. While NJC's implementation was well managed, except for a delay in developing the client database system, the implementation of CISP initially encountered problems which resulted in underperformance in its early operations. Subsequently, the court has been effective in managing emerging issues. Both programs benefited from the sound management and ongoing monitoring of service delivery. CISP also benefited from a process evaluation which examined if it was implemented in line with its design.
In evaluating whether NJC and CISP have achieved their intended outcomes, the methodologies developed were sound but the limitations in available data reduced the level of reliance that could be placed on the evaluations. 
Both NJC and CISP have shown positive indications of achieving their client and community outcomes. Each program supported its intended client groups and provided high‑quality reports to the judiciary to assist their decision making. 
Both NJC and CISP showed positive indications of reducing reoffending. While CISP demonstrably reduced reoffending for offenders examined in the evaluation, definitive conclusions cannot yet be made for NJC. This is due to limitations in available data, primarily due to the small number of offenders who had completed the program, but also because of difficulties in accessing data without expending considerable manual effort. 
Findings
Developing the programs
Both NJC and CISP’s conceptual development were sound and underpinned by solid research and best practice in other jurisdictions. The programs’ concepts were also supported by relevant stakeholders. 
The site selection for each program was appropriately informed by thorough research into profiles of offending and local disadvantage. NJC’s site selection at Collingwood was also consistent with key success factors—identified from overseas experience—for establishing a community justice centre, namely, locating it in a densely populated area that has high levels of crime and disadvantage.
Given these programs had not been trialled elsewhere in Australia, the department’s 2005 funding submissions contained the key elements required, such as the rationale for funding, the degree of alignment with government policy, and the best available information on offending patterns and future demand for services. However, there was room for improvement in articulating clear and measurable program objectives and relevant and appropriate performance indicators that could be used to measure the programs’ performance.
Specifically, two of NJC’s objectives were not clear or concise and the CISP funding submission did not clearly state the program objectives. In addition, most of the objectives in the funding submissions did not have relevant and appropriate performance indicators. 
In its 2009–10 funding submissions, the department modified the objectives of both NJC and CISP. The new NJC objectives were, however, still problematic for the purpose of evaluating NJC’s effectiveness and the CISP submission did not specify performance indicators for the three new objectives included. 
Implementing the programs
Overall, the department successfully managed NJC’s implementation. NJC’s project governance was particularly effective because of high-level stakeholder involvement, timely and comprehensive reports on progress, clear roles and responsibilities in decision-making and reporting, and the active involvement of the department’s senior management.
The one shortcoming in NJC’s implementation was the delay in developing a client database system. This adversely affected the subsequent evaluation, due to the lack of complete demographic data on clients. As a result, NJC’s objective of better linking disadvantaged groups to particular services could not be evaluated. 
For CISP, while the department and court worked through many issues collaboratively, their overall performance in the initial implementation of this program was mixed. One of the main problems was the extent and timing of the consultation between the department and the court. 
During development of the funding submission, the department did not adequately consult the court on key elements of the program. These included the program’s referral targets, accommodation needs at the three program sites, and the use of a corrections-based assessment tool which the court considered to be unsuitable. This cast doubts as to whether the department could deliver the commitments made in its funding submission. Although these issues were subsequently addressed during the pre-operational planning, additional time and effort had to be devoted to their resolution.
After the funding submission was approved, the lack of consultation with the court also resulted in the risks of implementation not being adequately identified or managed, in areas such as negotiating with service providers. In addition, the implementation was further hampered by ineffective project oversight—the project control group was not receiving progress reports against milestones or actively monitoring and managing risks. Consequently CISP was launched four months later than planned and underperformed in its early operations. 
Since these initial difficulties, CISP has been managed effectively. The court is actively monitoring its operations, supported by ongoing reviews and assessments. The program has also benefited from strong leadership from some magistrates who actively support the program. For CISP, two areas remain as priorities for further attention; high staff turnover, and the management of services contracts.
Achieving outcomes
Both NJC and CISP have been evaluated by the department. 
The methodologies used to evaluate the criminal justice outcomes of NJC and CISP were sound. The lack of ready access to data and limited client throughput in NJC resulted in a sample size too small to definitively conclude how effective NJC was in reducing reoffending. The evaluation was required, however, as part of the new funding cycle and this affected the extent to which it was able to make definitive findings and conclusions. 
The CISP evaluation showed that the program had reduced reoffending for the sample of offenders included in the evaluation study. However, care needs to be taken not to generalise this conclusion to all who have completed CISP. The current analysis did not include ‘age of first offence’ which is one of the most important risk indicators for reoffending. Research shows that the younger a person begins offending, the greater the likelihood of reoffending. However, this data was not available when evaluations were conducted, as the data was not captured in the court’s database. 
Improving compliance with bail conditions, and completions of court orders, was also a program objective for NJC and CISP. However, it was not possible to conclude whether this objective was met due to the lack of data. The manual extraction of this data from Corrections Victoria was time consuming and as a result it was not available for the evaluation.  
Both NJC and CISP show positive indications of achieving their client and community outcomes. CISP has supported its intended client groups and provided high-quality reports to the judiciary to help them in their decision making. 
NJC is having a positive impact on its clients and on the local community by addressing factors contributing to local crime and disadvantage in the City of Yarra, for example, through targeted crime prevention initiatives. However, it is not possible to conclude at this early stage whether most of NJC’s community goals have been met. This is due, primarily, to a lack of baseline data to compare subsequent performance against.  
The department and the court need to improve data collection and analysis so that intervention programs can be better evaluated, particularly given the level of public funding provided. This would include recording the effect of intervention programs such as NJC and CISP on judiciary decisions, and better tracking defendants as they move through various parts of the criminal justice system. 
The department’s public reporting of both programs has not fairly represented the evaluators’ conclusions about program outcomes. The department has published the success of both programs in reducing reoffending without qualifying these conclusions. The reporting also cited the improvements in compliance with bail and court orders, which again need to be qualified due to limitations of the methodology and data. 
The department maintained that their public reporting of evaluation results was accurate but, based on Plain English principles, was simplified and condensed. The application of these principles should not, however, exclude important information, the absence of which could misinform the public. 
Recommendations
Number	Recommendation	Page	
1.	That the Department of Justice when developing problem‑solving programs: 
·	define clear and measurable objectives and relevant and appropriate performance indicators in funding submissions to support the case for funding and to allow the subsequent assessment of outcomes against objectives 
·	better consult with the Magistrates’ Court on the implementation plan including risk identification and mitigation.	16	
2.	That the Department of Justice and the Magistrates’ Court: 
·	better consult with service providers in implementing new programs and in risk management
·	have effective governance arrangements to actively monitor progress supported by timely and comprehensive reporting.  	28	
3.	That the Magistrates’ Court conduct staff exit surveys to assist in developing a long-term resourcing strategy for CISP, particularly to address staff turnover issues. 	28	
4.	That the Department of Justice and the Magistrates’ Court manage service contracts with clearer and more enforceable qualitative and quantitative performance measures.	28	
5.	That the Department of Justice evaluate NJC’s outcomes in reducing reoffending when sufficient data becomes available.	38	
6.	That the Department of Justice present research findings fairly and qualify them when appropriate in reporting publicly or seeking program funding.	38	
7.	That the Magistrates’ Court record the impact of interventions by problem-solving programs on judiciary decisions	38	
8.	That the Department of Justice and the Magistrates’ Court give priority to the use of unique identifiers for tracking defendants’ contact within the criminal justice system	38	
Submissions and comments received
In addition to progressive engagement during the course of the audit, in accordance with section 16(3) of the Audit Act 1994 a copy of this report was provided to the Department of Justice and the Magistrates’ Court with a request for submissions or comments.
Agency views have been considered in reaching our audit conclusions and are represented to the extent relevant and warranted in preparing this report. Their full section 16(3) submissions and comments, however, are included in Appendix A.