Connecting Courts—the Integrated Courts
Management System
1.1 Introduction
The Integrated Courts Management System (ICMS) is a major
Information Technology (IT) initiative of the Department of Justice
(DOJ).
The objective of the program is to establish a single integrated
technology platform and a set of applications for all Victorian
courts and tribunals to:
- enable the courts and tribunals to deal
effectively with the increasing volume and complexity of
cases
- improve justice system efficiency in case
management
- improve the experience of community and
legal practitioners in dealing with the justice system.
The 2005–2006 State Budget allocated $45 million to the ICMS
program, which began in July 2005 as a four-year program. The
budget allocated $32 million for capital and $13 million for
operational expenses.
The ICMS program is highly complex, and involves a number of
legal jurisdictions. It is being implemented in a period of
significant reform in the Victorian courts and tribunals, which
adds to its complexity.
At the core of the ICMS program is a common case management
system for all Victorian legal jurisdictions. The case management
system is an electronic system designed to manage cases in the
justice system from start to finish.
The ICMS program has a number of associated deliverables,
including:
- establishing and enhancing the
videoconferencing and electronic evidence management capabilities
in a number of court locations
- introducing internet-based services such
as payment facilities, document lodgement and information
management
- establishing a courts data warehouse
system for collecting and storing data to facilitate analysis for
effective decision making.
It affects the following court and tribunal jurisdictions:
- Supreme Court of Victoria
- County Court of Victoria
- Magistrates’ Court (including Coroner's
Court, Victims of Crime Assistance Tribunal)
- Victorian Civil and Administrative
Tribunal
- Dispute Settlement Centre
Victoria
- Children’s Court of Victoria.
It involves major change for these jurisdictions to improve and
integrate their justice services by implementing a common
technology platform and harmonising processes.
1.1.1 Objective of this audit
The objective of the audit was to assess whether the progress of
the ICMS program has conformed with its original investment
objectives.
We examined:
- program planning, including the
development of the funding proposal
- procurement strategy, planning and
process
- program controls, including program
monitoring and review.
1.2 Conclusion
There has been a significant delay in the ICMS program and
implementation costs have risen. Unsatisfactory supplier
performance has been a major factor in the time and cost overruns.
However, inadequacies in the planning and management of the ICMS
program have also contributed to the program’s difficulties.
The rationale for the investment, nevertheless, was and remains
sound. ICMS promises to allow the courts to be managed and run as a
coordinated system. DOJ has placed a high emphasis on stakeholder
management, and is putting significant effort into engaging
stakeholders, as their participation is crucial to the program’s
success. Encouragingly, we found a number of instances where
program participants from various jurisdictions were already
pooling their resources to coordinate their systems and
processes.
The business case presented to government provided a sound
argument based on service need and policy drivers for the
investment in ICMS but there were critical gaps in the assessment
of the proposed case management solution.
The program underestimated the risks involved in its approach
to:
- assessment of program costs
- analysis of industry capability to meet
the business requirements
- procurement strategy.
After a prolonged and complex procurement process a different
case management system was pursued to the one proposed in the
funding submission. The implications of the changed approach were
not thoroughly reassessed.
A non-traditional approach to managing the relationship with
multiple suppliers was adopted and the consequent risks of this
choice were underestimated. DOJ’s supplier coordination ability was
less than robust and there were gaps in the risk management
approach that resulted in the realisation of some key risks.
Nevertheless, DOJ has addressed significant challenges
encountered by the ICMS program, mainly due to supplier issues. In
spite of these challenges, DOJ has been able to maintain the
program momentum.
By their very nature, transformational IT-driven programs are
complex and risky. A program such as ICMS, which is being
implemented in a period of significant reform in the justice
system, is even more complex.
This audit found a number of effective practices adopted by the
ICMS program to deal with such complexity. These practices
include:
- strong articulation and communication of
vision
- strong belief in the value of the outcome
from all the stakeholders
- real commitment from the steering
committee and program management team
- active involvement of stakeholders from
the judiciary
- effective use of resources from
jurisdictions within the program team
- focused effort on dealing with change
management
- strong contract management
procedures
- sustained effort from the steering
committee and program management team to see the program through
difficult periods.
Nevertheless, the audit also found some major flaws in program
planning and control from which important lessons need to be
learned. Consequently, this report focuses on these program
management shortcomings.
1.3 Findings
1.3.1 Current status of the ICMS program
The ICMS program commenced in July 2005 as a four year program
scheduled to complete in June 2009.
Two of the five program components have been delivered, but the
case management system, the largest and most complex component, has
been delayed by at least fourteen months with the completion date
revised to August 2010.
The capital cost of the program has increased from $32.3 million
to $44 million, an increase of $11.8 million, or 36 per cent. As at
March 2009, $28 million of the allocated $32.3 million in capital
funds had been spent.
Due to delay, the ICMS program’s operational spend has been
deferred. For the four year period covered in the allocated budget,
operational costs have reduced from $12.8 million to $7
million.
Software for the case management system began to be delivered in
February 2009. This software release will be configured by DOJ for
the use of the Supreme Court, the first jurisdiction in DOJ’s
planned roll-out schedule. Incremental delivery of software is
planned for other jurisdictions.
DOJ bought a case management system supplied by a United States
based company, with the contract for implementation signed with its
fully owned Australian subsidiary. The supplier’s poor performance
has significantly contributed to the program’s delay.
In September 2008, the supplier advised DOJ that it had sold its
software development division, including the ownership of its case
management solution, to another company based in Canada. This sale
led to significant contractual negotiations between DOJ and the
supplier. These negotiations were effectively settled in April
2009, to DOJ’s satisfaction.
More positively, the ICMS program has commenced planning for
service readiness, in association with the jurisdictions. This is
important, as realisation of most of the benefits associated with
the ICMS program is contingent on the successful roll-out of the
case management system.
DOJ believes that the completed projects, Judicial Officers
Information Network (JOIN) and Smart Courts, are starting to
deliver some benefits. For example, online resources available on
JOIN have saved judicial users many hours of legal research time.
DOJ is in the process of collecting the relevant benefits
measurement data to validate these early observations.
1.3.2 Planning for ICMS
The need for the investment in ICMS was soundly established. The
ICMS business case thoroughly described its objectives, scope, the
service need for ICMS, and identified key stakeholders. It had a
robust plan for stakeholder engagement and communication.
The business case for ICMS was built on the premise that DOJ
would enhance the existing case management solution used in the
County Court and implement it in other jurisdictions. This was
expected to provide value for money, with lower risks compared to
the alternatives.
However, there were critical uncertainties in DOJ’s assessment
of system costs and supplier analysis when developing the business
case.
The procurement strategy for ICMS was not well defined. A robust
analysis was not undertaken to establish the implications of
pursuing a non-traditional method of using a systems integrator,
which is a supplier that takes products from multiple sources and
integrates them into a complete working solution.
During the procurement process DOJ found that its preferred
option of enhancing an existing solution was not viable due to gaps
between DOJ’s technical expectations and supplier capabilities.
In the end, DOJ selected a different case management system
solution from the one recommended in the funding proposal. The
selected solution involved a higher degree of delivery risk. DOJ’s
assessment of the implications of the changed solution approach for
the program, however, was not robust.
DOJ introduced strong measures to the contractual model to
manage payments for deliverables such as staged payment
commitments, payments based on the measured value of deliverables,
and risk sharing mechanisms.
DOJ followed the appropriate procurement and probity processes.
The steering committee and DOJ’s senior management ratified the
program’s procurement, satisfying their accountability
obligations.
1.3.3 Program control
The ICMS program has faced significant challenges due to
supplier related delivery and contractual issues.
DOJ has been able to maintain the program momentum despite these
issues and has secured an appropriate contractual arrangement with
the suppliers involved.
DOJ has thoroughly engaged program stakeholders and has designed
the ICMS organisation effectively. It has addressed the issues that
arose outside of its direct control.
DOJ assumed an increased responsibility for supplier
coordination due to the supplier engagement model it pursued.
However, its supplier coordination ability was not robust due to
its use of an ineffective delivery monitoring mechanism.
DOJ’s risk management approach in the ICMS program had gaps,
particularly in the early stages. It underestimated key risks which
later materialised with adverse effects. For a large part of the
ICMS program, lack of up to date program control documentation has
hindered DOJ’s ability to critically assess the program’s ongoing
viability.
The ICMS organisation and governance model reflected industry
best practice in project and program management. Notwithstanding,
this model would have benefited from the inclusion of independent
representative with specialist expertise to assist with challenges
associated with a major IT-driven transformation program.
1.4 Recommendations
For the ICMS program, DOJ should:
- produce, and communicate to stakeholders,
a comprehensive service readiness plan, as per the guidelines
available from program management best practice and Gateway, to
assist delivery of ICMS benefits (Recommendation
4.1).
- use the Gateway Review Process to
assess:
- readiness for service
- benefits realisation
(Recommendation 4.2).
- put in place a mechanism for monitoring
and reporting the status of supplier deliverables using the ‘effort
remaining’ measure and clearly allocate associated responsibilities
between DOJ and the Systems Integrator
(Recommendation 6.1).
- appropriately document and maintain the
business case and program management plan as per the project and
program management methodology adopted by the ICMS program
(Recommendation 6.2).
- clearly report the progress of the ICMS
program against the costs and schedule reference data outlined in
the business case (Recommendation
6.3).
For large IT transformation programs, DOJ should:
- review the lessons learned from supplier
delay, and apply these to current and future due diligence
assessment of supplier capability (Recommendation
4.3).
- incorporate a thorough assessment of the
supplier market, as per DTF’s business case development guidelines,
in its funding proposals (Recommendation
5.1).
- review its approach to supplier
coordination in the ICMS program to guide the development of future
procurement strategies (Recommendation
6.4).
- incorporate the use of system sizing
analysis tools used in the IT industry as an objective basis for
assessing:
- the fit of candidate solutions to business
requirements
- system development costs
(Recommendation 5.2).
- undertake periodic reviews by external
organisations specialising in IT program health-checks
(Recommendation 6.5).
- align its risk management approach with
DTF’s recently launched project risk management guidelines
(Recommendation 6.6).
- incorporate independent representatives
with specialist expertise into governance bodies
(Recommendation 6.7).