Crime statistics are one measure the community uses to judge our relative safety and security. They show the rate of recorded crime across local government areas or regions and can highlight trends such as increases or decreases in types of crimes, and how and where they are committed.
The government and the community also use crime statistics as one of several indicators to assess the effectiveness of criminal justice policies. Victoria Police uses crime statistics to help determine its resourcing needs.
Member—any sworn member of Victoria Police.
Given the importance of crime statistics in managing public safety and the high level of public attention and scrutiny they receive, it is important that they accurately reflect reported crime. The public also needs to be able to access crime statistics and understand what the results represent. Crime statistics that are reliable and robust are a useful tool for making evidence-based decisions, for developing justice policies, and for informing resourcing of police and other emergency services.
Offence—any criminal act or omission by a person or organisation for which a penalty could be imposed by the Victorian legal system.
Victoria Police is responsible for collecting, reviewing and recording information on criminal offences. Members use the Law Enforcement Assistance Program (LEAP) database to record reported crimes as part of an incident—a distinct event that can be made up of one or many sub-incidents or offences occurring at the same time and location. An incident can include multiple victims and alleged offenders. Members are required to record complete information about the incidents reported to them, including details of all offences, victims and offenders involved.
Victoria Police provides raw crime data to the Crime Statistics Agency (CSA). CSA is part of the Department of Justice and Regulation (DJR) and is responsible for publishing Victoria's crime statistics.
In this audit, we assessed whether crime data is reliable and effectively used for decision-making. We focused on the completeness, accuracy and timeliness of Victoria Police data and the methodology used by CSA to analyse and publish crime statistics.
Victoria Police accurately records the number of offences that have originated from divisional van reports in LEAP . Its members by and large correctly report on jobs dispatched from the Triple Zero hotline, and the information in LEAP is a reasonably materially complete record of reported crime.
Victoria Police's measures included in the Victorian Budget Paper No. 3: Service Delivery (BP3 ) accurately reflect the level of crime recorded in LEAP and Victoria Police's progress in addressing it. We did not detect any manipulation of crime data or cases falsely recorded as resolved.
However, risks to reliable reporting remain:
- Victoria Police has done little to improve its members' understanding of prima facie reporting since 2013 and does not know if members comply fully with the Victorian Crime Recording Standards.
- Members often fail to record sufficient detail in their divisional van reports, and van technology makes it difficult to track job outcomes, so there is a risk of important detail being lost between an offence being detected and recorded in LEAP.
- Victoria Police has no clear targets for timely recording of incidents and has data quality issues from incorrectly recorded date fields in LEAP records. This means Victoria Police cannot measure how quickly its members create reports after they are aware of incidents. Without a benchmark, there is little incentive for members to report in a reasonable time frame, so members may be unable to access crime information when they need it.
- Victoria Police can improve the usefulness of data available for internal purposes and decision-makers if members consistently record specific location information and the country of birth of victims.
- Supervising sergeants can play a stronger role in enforcing Victorian Crime Recording Standards.
The CSA methodology for transforming Victoria Police's raw crime data into published crime statistics is transparent and produces a reliable set of statistics for the benefit of policymakers and the wider community. The methodology is also publicly available, so the public can understand how to interpret the statistics.
Recording crime data
Prima facie reporting
Victoria Police can improve the consistency and reliability of prima facie reporting by coaching junior members and more closely monitoring reports before approving them.
The Victorian Crime Recording Standards are based on the principles of the National Crime Recording Standard (NCRS ), which Victoria Police adopted in 2009. The principles include the requirement to report a crime if, on the face of it, it appears that a crime has occurred. This is known as prima facie reporting, and its introduction was a change in approach for Victoria Police. Previously, members were accustomed to investigating an incident before deciding whether to record it in LEAP.
Prima facie crime recording—the requirement to report a crime if, on the face of it, it appears that a crime has occurred, as distinct from investigating further to determine if it occurred before reporting.
The Australian Bureau of Statistics (ABS) does not include reporting on assaults in Victoria in its national dataset because it states that Victoria Police does not comply with prima facie reporting. This makes Victorian data inconsistent with that of most other Australian jurisdictions. A 2013 Victoria Police internal review, 'Assessment of the Collection Recording and Reporting of Crime Data', found that 61 per cent of members would not report a minor assault, which contravenes the Victorian Crime Recording Standards. Victoria Police has not reviewed the progress of prima facie reporting since 2013.
Prima facie reporting should result in more information about offending, such as minor assaults, being captured on LEAP. If members do not consistently comply with prima facie reporting, it is unlikely that this is occurring. Recording family violence incidents has additional requirements—members must record criminal matters such as assault but also verbal disputes and other psychological and controlling behaviours that are considered civil matters. This additional reporting provides members with more comprehensive information about this type of offending and can be used to better assess risk.
Four of the 14 constables we interviewed during this audit spoke of investigating an incident further before reporting it, even if, on the face of it, a crime had occurred. This approach does not support the intent of prima facie reporting, which is to ensure that crime data in LEAP is complete and that this valuable information is quickly accessible without the need for further investigation.
Completeness of reporting
In the sample we examined, almost all reported crimes received by members in divisional vans and captured in Electronic Patrol Duty Returns (ePDR) from 1 July to 30 September 2017 were satisfactorily recorded in LEAP (96.1 per cent) or had valid reasons for not being recorded. Our analysis of unexplained omissions detected no systematic under-reporting of crime.
Electronic Patrol Duty Returns—capture information from callers to Triple Zero operators about crime, generated from information transmitted to mobile data terminals in divisional vans. They include essential information about the reported crime and the member's actions in response, as well as speed checks, breath tests and licence plate checks.
Completeness of demographic data
Collecting demographic data allows police and policymakers to identify emerging trends such as whether groups of people are increasingly becoming victims of crime. It is not mandatory for members to ask a person's gender or nationality, but they are expected to record as much relevant detail as possible. Asking about Indigenous status is mandatory, but it is possible to leave the field blank or unknown in reports.
We looked at how completely and consistently members record demographic information and found that information on offenders was more complete than information on victims. Country-of-birth data was complete in 27.9 per cent of victim records and was consistently recorded across different LEAP tables and reports 25.8 per cent of the time compared with offender records, which were 92.7 per cent complete and 78.9 per cent consistent.
Victoria Police has made great improvements in gender data and has reduced the number of gender fields marked 'unknown' in offender data from 339 in December 2017 to 37 in March 2018. Data about Indigenous status was missing for only 4.2 per cent of offender records but was missing for 41.2 per cent of victim records. Victoria Police indicated that asking these questions can be awkward for members—but without this information it cannot fully understand who is most affected by crime.
Quality assurance of LEAP data
Victoria Police's central Records Services Division (RSD) conducts checks of LEAP data guided by its data quality framework. The framework guides how RSD's data operators review LEAP reports for high-risk errors such as:
- incorrectly selecting or updating name information
- incorrectly entering member or station details
- failing to add most recent address or contact details
- incorrectly adding offender details or not adding an offender
- not linking charges to the correct sub-incident (the specific offence within the crime event).
Other checks involve running data analysis queries across the entire LEAP dataset to identify missing and incomplete data and other issues that prevent LEAP meeting the Victorian Crime Recording Standards.
This checking approach works well considering the complexity of LEAP and the manual processes it requires.
Accuracy of recording
Some high-volume crimes, such as theft, do not have a high clearance rate and members continue to get reminders to keep investigating as long as it remains unsolved. In such circumstances, there is a risk that members will artificially clear cases to improve the clearance rate—the time it takes to clear or solve crimes. Changing an open case (which would be categorised as 'unsolved') to a status like 'complaint withdrawn' changes it from unsolved to cleared crime.
We looked at the outcome of a selection of cases with the status 'complaint withdrawn'. We found that the narrative description did not always clearly indicate why the complaint was withdrawn, but we found no evidence of manipulation in the selection of cases we reviewed.
Another risk is that a serious offence, such as aggravated burglary, is not recorded accurately and downgraded to the less serious offence of theft. Such inaccuracies could mislead the community about crimes and provide a false picture of police success in addressing crime.
We assessed whether there had been shifts from more serious offences to less serious ones from 2016 to 2017. The shifts we observed involved small numbers, and we found no patterns in the data that would indicate intentional downgrading.
To test whether common errors might exist in LEAP, we looked at the expected sequence of dates. For example, a correct sequence might show a home burglary that occurred on 30 April being reported to police on 1 May, with police creating a LEAP record on 2 May and resolving the crime on 1 June. If the sequence of dates showed the crime resolved on 1 April but reported on 1 May, it would indicate an error. In checking for such errors, we found that for intervention orders it was standard practice for members to record the date the order was 'resolved' as occurring before the offence was committed. LEAP is not set up for intervention orders, so members regard the 'committed' date as the date the intervention order was in place. Our analysis found errors in less than 1 per cent of the data we examined, which means these inaccuracies will not materially impact published statistics. Victoria Police is reviewing and amending this data. Date errors are discussed further under timeliness of recording.
Members can improve the accuracy of location information in crime data. Members with responsibility for analysis and intelligence gathering reported to us that location information is often poor—addresses can be entered with a street name but no street number, or one location can be reported in multiple ways. This means that police analysis may be incomplete or inaccurate.
To identify quality issues caused by members, RSD runs data queries across LEAP to identify missing and incomplete data and other quality issues. Where possible, RSD then develops fixes—an example of this is identifying and improving the completeness of LEAP's gender data. RSD does not oversee or provide data quality advice to members. Supervising sergeants are responsible for checking members' reports for errors before they are approved in LEAP.
Timeliness of recording
Victoria Police's timeliness standards are not specific, stating that 'in the majority of instances, the details of a crime should be submitted for approval by the end of the shift on which the reporting member was informed of the crime'.
Most members complete reporting within a reasonable time frame, but Victoria Police can enforce tighter time frames to reduce the amount of reports completed more than two days after members become aware of an incident. This will allow members to have confidence that they are fulfilling their duties based on the most complete and accurate information.
Of the 48 jobs we examined, 40 were entered into LEAP within 24 hours (83.3 per cent), and this increased to 93.8 per cent within a 48-hour period. All jobs were reported within 3.5 days.
The two date fields used to measure the timeliness of crime reporting are:
However, the result from our timeliness test sample—that members recorded 93.8 per cent of jobs in LEAP within 48 hours during selected days in August 2017—is inconsistent with the average time between all 'report date' and 'create date' information entered in the system between January and September 2017, which is much higher (eight days). We found 557 744 sub‑incidents (16 per cent) with a gap greater than five days between the 'report date' and 'create date'.
Several issues affect the overall average:
- RSD data operators inadvertently save over the correct 'report date' when adding charges to existing sub-incidents.
- Members may record the date the alleged offence took place as the date reported to police, when it is more likely that there is a delay between the alleged offence occurring and its being reported to police.
- Offences such as the manufacturing of drugs, incest and manslaughter can involve information that needs to be restricted from LEAP users, and members may be unsure when it is appropriate to report the offences in LEAP.
- The average gap is also affected by some offences with disproportionately long gaps between the 'report date' and 'create date'.
Over the five years to 2017, the number of sub-incidents with a gap of over 100 days between 'report date' and 'create date' reduced by 15 per cent, from 9 402 in 2012 to 7 991 in September 2017.
So, while there is some improvement, Victoria Police members still make errors in recording dates, despite clearly defined standards. If this confusion is not corrected, the cleared crime could be reported in the wrong quarter. The number of errors will not have a material impact on CSA's quarterly published data, or on trends over time, but they should be rectified to ensure crime data is as accurate as possible. Victoria Police advises that it has started to address the incorrect LEAP reports.
The time taken to clear a crime is one of Victoria Police's BP3 measures, and its performance against this measure is published annually in the State Budget. Victoria Police uses CSA statistics to report on all crime performance measures except this one, which it generates using internal data. Victoria Police also now uses the 'report date' to calculate its performance results, which is likely to add additional time to the 30-day clearance rate reported in BP3 and provides the public with results that are 'worse' than if measured from the 'create date'. This demonstrates that Victoria Police has implemented the Victorian Ombudsman's recommendation to measure performance using the 'report date' field; but it now needs to improve the quality of the 'report date' data so that this result is more reliable. Working with CSA to use its statistics on crime clearance will help Victoria Police to avoid any perception of bias, as it is CSA's role to provide verified figures.
System and control effectiveness
Process noncompliance and control deficiencies can place critical business functions at risk. Victoria Police can improve its processes to ensure the reliability and integrity of its key systems and underlying data.
Quality control over data input
The quality of data input is paramount for accurate crime data. This means that reports need to be scrutinised before they are approved.
Supervising sergeants use the approval process to ensure members adhere to the Victorian Crime Recording Standards and include the correct information. We spoke with 18 members who felt this process needed to be more thorough to improve the quality of location information, identifying details and the description of the investigation.
Local area commanders for police service areas (PSA) also complete monthly workplace inspections to check compliance with a range of requirements, including investigation and reporting. A 2016 Victoria Police internal review found that workplace inspections—which include LEAP—are rudimentary, with assessments scored as either satisfactory or not satisfactory. This does not provide local area commanders with sufficient insight into reporting issues, and there is no evidence that this process has improved the quality of reporting. Victoria Police needs to make the methodology for the assessment more transparent so that reporting is meaningful and properly tracks performance. Victoria Police has delayed addressing the recommendations of the internal review until June 2019.
LEAP has operated since 1993. LEAP remains the repository for crime data, but members now enter data using a modern interface known as LEAP Electronic Data Recorder Mark 2 (LEDR Mk2). This interface allows members to enter incidents themselves, rather than sending forms to RSD to complete. This promotes more accurate and timely reporting. RSD data operators still enter data on offenders into LEAP from faxed forms.
Other reporting and case management systems—such as the mobile data terminals (MDT) used in vans, and Interpose , Victoria Police's case management system—are not integrated with LEAP. Interpose contains restricted information. As a result, crime reporting is onerous and repetitive, with members retyping the same information in multiple systems, which increases the risk of error.
The MDTs in divisional vans can only capture limited information, and members do not always capture enough information to document events. Further, the format of ePDR reports makes it difficult for supervisors to track the sequence and outcome of events.
The lack of interoperability between crime recording systems continues to be a major source of frustration for members and adds to their time spent on administration. It limits the ability of members to easily add in offences containing sensitive information, such as drug crime, incest and manslaughter. This impacts the quality and hence usefulness of crime data.
Information technology general controls
We assessed the key information technology general controls (ITGC) that underpin the LEAP and LEDR Mk2 systems. They are operational but we identified a number of opportunities to increase their robustness. These include:
- reviewing and updating all LEAP and LEDR Mk2 policies and procedures on an annual basis to ensure processes and controls are implemented consistently
- expanding and performing regular system security reviews to ensure security risks are identified and responded to
- performing regular user access reviews to make sure access is appropriate and anomalies are addressed
- complying with Victoria Police's change management administration review process to provide assurance that documentation processes comply with the requirements of the change management process
- performing regular information risk assessments as per Victoria Police's Risk Management Framework requirement to ensure information assets are appropriately protected.
Published crime statistics
After extracting LEAP data, CSA completes a series of 'transformation' steps before publishing crime statistics. This involves reasonableness checks, such as seeing if increases or decreases in crime are in line with expectations. CSA also organises individual offence types into meaningful groupings, such as assigning 'theft of a motor vehicle' to the 'theft' category. CSA ensures the information is statistically significant and poses no risk of identifying people. It applies counting rules to derive specific populations. We found that the transformation processes are transparent and appropriate.
CSA applies another counting rule to Indigenous status data. It uses the 'most frequent' rule to improve reported data—for example, where a person has described his or her Indigenous status differently on separate occasions, CSA uses the most frequent response to assign the offender a status in its reports. CSA’s method may reduce the impact of any data entry error.
CSA categorises offences into groups that largely reflect the Australian and New Zealand Standard Offence Classification (ANZSOC), developed by ABS. CSA has modified ANZSOC where required to match Victorian criminal law. This is an appropriate adaptation to make the classifications relevant to Victoria. The location classifications are also useful, as they allow searches by police zone, suburb and local government area.
CSA does not audit, nor does it have any powers to direct improvement of, the quality of LEAP data. CSA has access to the descriptions of the investigation proceedings for LEAP incidents, known as narratives, which may help to identify missing or obviously incorrect data, but, as yet, Victoria Police and CSA have not developed guidelines for the use of this information.
CSA reports seven data tables on crime. We focused on two:
- The Recorded Offences table, which provides quarterly updates on the number of offences committed in Victoria and includes offences where no offender is identified or charged. The crime rate is often derived from this data.
- The Criminal Incidents table, introduced in December 2017, which is a count of all the criminal events rather than the individual offences and designed to more closely represent crime as it is experienced in the community.
The Criminal Incidents dataset only includes the offence type that is determined as the principal offence. If there are several offences, CSA selects the most serious as the principal offence for reporting. We assessed how CSA selected the principal offence by comparing its classification system to the National Offence Index (NOI). We found that CSA's methodology is sound and can be used to fairly reflect the principal offence in a crime.
We used LEAP data and CSA's methodology to replicate the CSA crime statistics for the Recorded Offences and Criminal Incidents tables and determine their reliability. We noted no material variance for the data from 2008 to 2017. This means that CSA's crime statistics are an accurate reflection of Victoria Police's crime data.
Using data for decision-making
Senior members of Victoria Police regularly meet to discuss crime data and participate in briefings on local crime clearance rates, shifts in offences, and comparative data. In addition, these members can use data analysis tools to perform a range of common data queries on the Victoria Police intranet at any time. This makes it easy for members to quickly access changes in offending that need to be discussed.
We recommend that Victoria Police:
1. conduct a training needs analysis to determine members' understanding of prima facie reporting and address any gaps in targeted training (see Section 2.2)
2. establish a centralised monitoring and reporting mechanism to ensure management reviews and updates policies and procedures in a timely manner, including:
- for crimes where a victim can be identified, mandating that members ask if the victim is comfortable disclosing their country of birth and, if so, that this data be recorded in the Law Enforcement Assistance Program (LEAP)
- requiring supervisors to routinely check Electronic Patrol Duty Returns and other crime and investigative reporting in line with Rule 2 of the Victorian Crime Recording Standards before they approve the reports
- setting a benchmark for timely input of data into LEAP and reporting against this annually
- reviewing and updating the policy and procedural documents for LEAP and the LEAP Electronic Data Recorder Mark 2 (LEDR Mk2) systems in accordance with the Information Management Policy (see Sections 2.3, 3.2, 3.6, 3.7 and 3.9)
3. improve the accuracy and timeliness of recording in LEAP, and monitor and reduce errors in LEAP that cause long gaps between the date the offence is reported and the date it is created on LEAP (see Section 3.9)
4. develop a data dictionary that defines the requirements for LEAP and LEDR Mk2 fields (see Section 3.3)
5. adopt a consistent approach to using performance measures computed by the Crime Statistics Agency in Victorian Budget Paper 3: Service Delivery reporting and document the methodology (see Section 3.9)
6. set a benchmark for what constitutes satisfactory LEAP and LEDR Mk2 reporting in workplace inspection reporting (see Section 3.3)
7. strengthen the information technology general controls for LEAP and LEDR Mk2 by:
- removing inactive user accounts
- ensuring compliance with Victoria Police's change management administration review process for documentation requirements (see Section 3.5)
8. perform regular information risk assessments as per Victoria Police's Risk Management Framework requirements and, based on the results, redesign and implement security controls to remediate the risks identified (see Section 3.6)
9. implement an appropriate compliance management framework and processes for monitoring, measuring, evaluating and reporting on the performance of crime data reporting and information technology general controls (see Sections 3.4, 3.5 and 3.6).
Responses to recommendations
We have consulted with DJR, Victoria Police, CSA and the Emergency Services Telecommunications Authority (ESTA), and we considered their views when reaching our audit conclusions. As required by section 16(3) of the Audit Act 1994, we gave a draft copy of this report to those agencies and asked for their submissions or comments. We also provided a copy of the report to the Department of Premier and Cabinet.
The following is a summary of those responses. The full responses are included in Appendix A.
Victoria Police acknowledged and accepted the report’s findings. It also accepted all of the recommendations and advised that it will develop an action plan within three months to detail how it will address them. We will follow this up with Victoria Police.
DJR and CSA thanked VAGO for the report’s valuable insights and remain committed to providing transparent and reliable crime statistics. ESTA did not respond.