Crime Data

Tabled: 5 September 2018

3 Data quality and controls

Police members need to create incident records with the most complete and accurate information available so that crime is detected early and victims and perpetrators are treated fairly. Inaccurate or incomplete records in the LEAP database will affect the quality and reliability of published crime data.

This part of the report examines Victoria Police's control environment for ensuring it maintains the quality and security of crime data and assesses completeness, accuracy and timeliness of crime reports.

3.1 Conclusion

Victoria Police's central quality assurance mechanisms are targeted at the highest-risk errors. The central quality assurance process—including RSD and CSU—strengthens the reliability and accuracy of crime data but needs to incorporate regular monitoring of date entries to improve overall quality. RSD and CSU have limited control over the quality of data that members create, and these errors can impact the accuracy of publicly reported statistics.

We found no evidence of manipulation or intentional misrepresentation in the selection of data we reviewed. This shows that Victoria Police has improved its processes since the Victorian Ombudsman found evidence of falsely resolved crime in 2011.

However, problems with data quality persist. Some data fields—such as 'country of birth'—are not consistently completed and a lack of specific location information and investigation narrative means important details that could assist with intelligence operations or analysing crime trends are lost.

Victoria Police does not have clear expectations for timely reporting in LEAP, which reduces the incentive to report crime in a reasonable time frame. We could not determine whether Victoria Police records crime in LEAP in a timely way. On average, there is an eight-day gap between the date a crime is reported and the date its corresponding record is created in LEAP. However, in a limited test we found 93.8 per cent of potential crimes reported to Triple Zero were recorded in LEAP within a 48-hour period, far better than the average.

Without clear benchmarks for timely recording and better monitoring, timeliness data will remain unreliable and important incident information may not be available for members when they need it.

Victoria Police has ITGCs for system security, data protection and management of user access, and changes to its LEAP and LEDR Mk2 systems. However, it needs to further strengthen these controls to ensure the control environment is adequate for the level of risk associated with securing sensitive data and maintaining integrity.

3.2 Quality checking at police stations

Supervising sergeants monitor the tasks and activities allocated to patrol teams by reviewing ePDRs. This process allows senior sergeants to check whether junior members are complying with the requirement for prima facie reporting. However, there are no other checks or processes to verify whether members understand the standards and apply this style of reporting.

Once the supervisors approve ePDRs, they ensure members record any incident that is required in LEDR Mk2. According to the Victorian Crime Recording Standards, the checking supervisor ensures that:

  • the report time and date is when the police were informed of the crime
  • the 'crime incident from' and 'to' dates are before or equal to the 'report time and date'
  • the victim's full personal details are recorded—for example, name and date of birth
  • the full location of the crime is recorded—for example, street number, street name and suburb
  • the offence is correctly coded—that is, that the member has not confused similar types of offences such as theft with burglary and that the correct offence code is recorded
  • the modus operandi (the particular way a person commits a crime) is recorded
  • for crimes against a person, the victim notification details are completed.

Once approved by the supervisor, the records are transmitted from LEDR Mk2 to LEAP and are available for internal review. These quality assurance processes are discussed in Section 3.4.

We interviewed 18 detectives and members who use crime data to analyse and report on crime trends and patterns. In these interviews:

  • seven stated that supervising sergeants need to more thoroughly check the quality of information reported in LEAP
  • five stated that it was not errors that were the problem but a lack of information—check boxes are skipped and the quality of the narratives in describing the steps taken during the investigation could be improved
  • eight were frustrated with errors in recording location and name details.

Supervising sergeants indicated that time pressures often meant that they could not spend as much time doing quality reviews as they would like. Without detailed information about crimes, the accuracy and value of Victoria Police's crime data may be reduced.

Training and development

Victoria Police does not provide sufficient crime reporting training, nor does it set out clear expectations for staff regarding requirements for capturing accurate data.

Training manuals for LEAP and LEDR Mk2 provide guidance on how to report a range of scenarios and are available on the Victoria Police staff intranet. The manuals are clear and detailed. Victoria Police advises that members or staff are not permitted to access LEAP until they have completed training which lasts around half a day. They refer to the requirements in the VPM and the Victorian Crime Recording Standards for report quality and emphasise the importance of accurate data entry. These manuals are formatted as introductory modules for each system but are not intended to be a reference manual for recording investigations that comply with the VPM.

New members training at the Victoria Police Academy receive around one hour's training on data recording. This is inadequate for the significant amount of time members spend entering, updating and revising crime data. Training in the Victorian Crime Recording Standards needs to be included in data recording training to help members to understand their obligations. As it stands, the training will not provide members with sufficient understanding of how to complete reporting that meets the required standard.

Victoria Police runs other online training courses as necessary to introduce new requirements such as changes to crime recording. Members advised that it can be hard to fit online training into busy schedules and indicated that training was most effective when conducted in an 'on the job' environment—this is a reasonable approach given the range of scenarios that police encounter. However, the quality of on-the-job training varies from station to station, depending on the thoroughness of the member and the supervising sergeant.

To ensure all members understand the requirements for capturing accurate data, Victoria Police should provide additional training and define clear expectations. Further, senior members should reinforce the key messages with ongoing regular coaching, and follow up junior staff members' progress and understanding of the training material.

3.3 Regional oversight

Four regionally based teams have overall responsibility for undertaking appropriate audits of the crime-recording process. They may conduct audits when new senior members commence, to assist them in their new role, or when a risk has been identified. The teams look at a range of risk factors, including investigation workload and timeliness, and they conduct audits in accordance with the Victoria Police Workplace Inspection Manual .

The Workplace Inspection Manual guides the monthly Workplace Inspection Reports (WIR), which are completed at police stations to monitor compliance, workforce and workload. The regional audit teams collect data from LEAP and other Victoria Police systems and, if they identify risks, they may conduct training to rectify issues.

Monthly inspections

WIRs provide an indication of a PSA's workload, its progress against this workload, the quality of its reporting on investigations and the reasons for outstanding items—and they may prompt senior members to intervene. They also assess compliance with financial requirements, occupational health and safety, staff leave and investigations with an 'active' status for long periods.

WIRs include a review of all LEDR Mk2 entries with the status 'pending approval', 'draft' and 'reworked' to determine if they have been appropriately reviewed and actioned. Any entries with a 'pending approval', 'draft' or 'reworked' status for more than seven days are considered deficient. Local area commanders are expected to report on overdue, complex or deficient entries in the 'comments' section .

The scoring—'satisfactory', 'close to standard' or 'unsatisfactory'—for LEAP, family violence, LEDR Mk2 and other sections in WIRs is subjective. These descriptors are not based on a quantifiable measure but on judgment alone.

A 2013 internal Victoria Police report found that supervisors do not review LEDR Mk2 reports against computer-aided dispatch data from Triple Zero calls to identify reported incidents that should be in LEAP. The internal report noted the need to establish a coordinated approach for checking the completeness of crime records, also noting that station-level checks rely on members' diligence and the availability of resources. In response to this finding, Victoria Police updated the Workplace Inspection Manual to include this check in monthly reporting.

A further internal audit of the WIRs in 2016 found that Victoria Police assesses family violence and LEAP by the volume of the workload rather than the actions taken. This approach limits the WIRs' use for assessing compliance. For example, a PSA's LEAP reporting can be marked 'satisfactory' but there is no way to measure the checks that were undertaken or what the result means. The internal audit also found no evidence that WIRs are improving data quality or timeliness and that there is confusion as to who ultimately signs off WIRs. Many were left unsigned or were not submitted to the correct tier of management, despite being completed. The purpose and actual utilisation of WIR results for improving LEAP is unclear.

Victoria Police has not yet implemented the recommendations of the internal audit, including that it adopt a consistent methodology for assessing performance, but it intends to complete a compliance project to address these recommendations in June 2019.

Quality controls

Database dictionary

Database user manuals or dictionaries assist users to analyse data and are essential for maintaining data integrity during periods of technological and process change—particularly when data is input by many users locally and reported centrally. These manuals often describe the full set of fields that the system users see and the related technical field names. They also define the information that is captured within fields and include other technical information used to link and match data for analysis. A comprehensive database dictionary acts as a 'single source of truth'.

Victoria Police does not have a data dictionary for LEAP that defines all of its data fields and, as a consequence, there is significant reliance on the knowledge and experience of key staff for this important information. If these key staff leave the organisation, there is a risk that this knowledge will be lost and that LEAP data could be incorrectly captured or interpreted.

Preparing a comprehensive database dictionary would assist Victoria Police to maintain consistent and accurate data, particularly when field definitions change, or new fields are added. A database dictionary can be updated and communicated to all users.

A guide to navigating and searching LEAP is especially valuable for new users who may have difficulty navigating LEAP's old technology. While the LEDR Mk2 interface simplifies the process by allowing members to enter crime information with drop-down boxes and limited valid entries, newer members we interviewed struggled with using LEAP, not only because they were unfamiliar with the style of the old system but also because they use it infrequently. Without a database dictionary, there is a risk that information will be recorded, amended or approved inconsistently or inaccurately.

3.4 Central quality assurance

Data quality framework

Figure 3A shows how Victoria Police structures its data quality framework. Review of data quality is mostly completed at stations and PSAs by local supervising sergeants.

Figure 3A
LEAP data quality framework

Figure 3A shows how Victoria Police structures its data quality framework.

Note: SAS is data storage and analytics software used by Victoria Police. CSS is Crime Scene Services which investigates property crime.
Source: Victoria Police.

RSD is responsible for adding offenders to incidents and performing quality checks. It also clears theft of motor vehicles, theft from motor vehicles (number plates) and missing-person reports based on a phone call from the member creating the report. RSD applies a risk-based approach to its quality checks. Data operators perform checks on each record for which they are responsible, and a supervisor checks the quality of the data. RSD supervisors select a random date from the previous month and perform quality checks of the work completed by the relevant data operators for that shift, equivalent to around 5 per cent of the data operators' completed work for the month. RSD supervisors monitor data operators closely and record and analyse errors to identify areas for further development and training.

The high-risk errors that RSD's data operators check for include:

  • incorrect name record selected
  • most recent address or telephone number details not added
  • charges not linked to correct sub-incident
  • member or station entered incorrectly on a charge record
  • details of the accused's involvement added incorrectly
  • duplicate charge added
  • offender involvement not added.

In addition to the RSD supervisor checks, DQU runs daily queries across the dataset to review reports made by members via LEDR Mk2 and information entered by RSD staff from forms faxed by members.

In conjunction with the regional audits, CSU also checks compliance by comparing a division's reported crime against predetermined tolerance levels. It also completes a data analysis check to identify illogical, missing or duplicated entries—for example, a missing person's report with no person identified, missing victim information, ages more than 100 years or less than zero years recorded, and sub-incidents that appear duplicated.

The quality assurance conducted by RSD, DQU and CSU is reasonable, but the assurance that these mechanisms provide depends on the extent of quality control in the initial input of data. Ensuring that members understand how to complete reporting and that supervisors complete adequate checks is critical to improving data quality overall.

3.5 Information technology general controls

ITGCs are policies, procedures and controls to ensure the confidentiality, integrity and availability of systems and data—for example, an ITGC might require management to properly review and authorise access requests to the systems.

Ineffective ITGCs may affect the reliability and integrity of a system's underlying data and programs, and may affect management's ability to rely on underlying business and process controls.

Information technology general controls policies and procedures

Victoria Police's Enterprise Information Security Policy requires system-specific access control policies, standards and guidelines, and procedures that are documented, implemented and reviewed annually. Victoria Police has established enterprise-wide and LEAP and LEDR Mk2 specific policies and procedures, but it has not consistently reviewed and maintained them.

Figure 3B shows our testing results of 34 policy and procedure documents.

Figure 3B
Review of enterprise-wide and LEAP and LEDR Mk2 policy and procedure documents

Figure 3B shows our testing results of 34 policy and procedure documents.

Source: VAGO based on Victoria Police data.

Out-of-date policies and procedures may result in poor governance and inconsistent processes and controls implemented within the IT and business‑related applications. This may affect the confidentiality, integrity and availability of underlying systems and data.

Assessment of information technology general controls

User access management

User access management focuses on managing access to systems, including how access is approved, removed and periodically reviewed to ensure it aligns with staff roles and responsibilities. The main objective of managing user access is to maintain the confidentiality, integrity and availability of systems and data. Weaknesses in user access management controls can result in inappropriate and excessive privileges assigned to users and hence a risk of unauthorised access to systems and data.

Inactive user accounts

As employees take leave or resign, management should deactivate or remove user accounts in a timely manner to restrict access to systems and data. If not managed, these user accounts accumulate and can pose a security risk.

We identified a high number of enabled LEAP and LEDR Mk2 user accounts that had not been used for more than 90 days. LEAP and LEDR Mk2 systems have settings for user passwords to expire after 90 days to limit the security risk.

Victoria Police should improve its periodic user access management process to better identify, restrict and manage which users can access the LEAP and LEDR Mk2 systems.

Password configuration

The Australian Signals Directorate produces the Australian Government Information Security Manual—the standard that governs the security of government IT systems. The manual provides recommended password configuration guidelines for agencies.

Victoria Police has chosen not to fully comply with the recommendations of the Australian Government Information Security Manual for LEAP and LEDR Mk2 systems based on a risk assessment and system limitations. We noted that the systems' password configurations are robust. However, there are opportunities for improvement. Victoria Police has commenced an identity and access management project to review and improve the systems' password configurations.

System security review

Agencies periodically use vulnerability assessments and penetration tests to ensure continuous improvement of system security. Vulnerability assessment is a process that defines, identifies and classifies the security weaknesses in a computer system, and penetration tests simulate the actions of a hacker seeking to breach system security.

Victoria Police recently established vulnerability assessment and penetration test programs in 2018. The programs contain varying levels of system security reviews for LEAP and LEDR Mk2 systems. Victoria Police should improve its vulnerability assessment and penetration test programs to increase the security of LEAP and LEDR Mk2.

Program change management

Change management aims to ensure changes to an IT system or its environment are appropriate and preserve the integrity of the underlying system and data.

Figure 3C summarises a generic program change management life cycle.

Figure 3C
Generic program change management life cycle

Figure 3C summarises a generic program change management life cycle.

Source: VAGO based on information provided by Victoria Police.

Weaknesses in change management can lead to an increased risk of unauthorised changes being made to systems and data, which could affect their availability and integrity.

Open change requests

We analysed LEAP and LEDR Mk2 change requests for the period 4 February 2016 to 6 February 2018 and identified that 65 per cent are still recorded as open in the IT service desk system. We tested a sample of 25 change requests and verified that management had approved them for implementation into the production systems. Victoria Police indicated that the change requests have been implemented. However, it has not performed the administrative review, and the reviewer has not closed the change request records in the IT service desk system.

Victoria Police should improve the administrative review process within its compliance activities for the change management process, to ensure it records and updates change request records as and when the administrative review is completed.

3.6 Information management

Victoria Police's Information Management Policy mandates the protection of information. The policy provides a framework for assessing the impact if the confidentiality, integrity or availability of information is compromised.

Figure 3D shows the four-step process in the information protection process.

Figure 3D
Victoria Police information protection process

Figure 3D shows the four-step process in the information protection process.

Source: Victoria Police.

Information identification and assessment

Victoria Police performs information risk assessments to:

  • identify information assets used as part of business and police operations
  • assess the potential consequences if those information assets are compromised.

Victoria Police performed information risk assessments on the LEAP and LEDR Mk2 systems in February 2015 and April 2017 respectively. The risk assessment of the LEAP system used the version of the Information Management Policy relevant at that time, but a risk assessment has not been performed again since the policy was updated. This increases the risk that the assessment may not appropriately identify risks and that the systems may not apply relevant processes and controls to protect information.

Victoria Police indicated that it intends to refresh the LEAP and LEDR Mk2 information risk assessments in 2019 and, based on the results, redesign and implement security controls to remediate the risks identified.

Information marking and protection

Encryption is a form of information protection—it prevents data from being readable or usable in the event of unauthorised access to the system. Data encryption occurs at two stages—data in transit (electronic transfer of or access to data) or data at rest (electronic storage of data).

Affected family member—family members affected by events occurring during a family incident.

Victoria Police's existing processes only recommend the encryption of LEAP and LEDR Mk2 data when transmitted to a non-government agency. Victoria Police should improve its encryption controls over LEAP and LEDR Mk2 data. It has indicated that it intends to review its encryption controls for protecting sensitive personal data in its planned 2019 refresh of the LEAP and LEDR Mk2 information risk assessments.

3.7 Completeness

Master Name Index—numbers assigned to identify each individual who has interacted with police. Through these unique identifiers, Victoria Police can trace each individual's involvement with various incidents.

Consistent and missing data

Inconsistent identity data occurs when information about a person is recorded in LEAP differently for the same person across multiple interactions with police. As data can be re-entered without cross-referencing existing data, there is potential for conflicting data. Missing data occurs when a member skips a non‑mandatory field in LEAP. The more consistent and complete the data is, the more reliable it is.

To ensure crime data is accurate and can be fully utilised, LEAP should include reliable data on a person's country of birth, Indigenous status, gender and date of birth. This allows police, criminal justice researchers and policymakers to identify trends in crime such as the profile of victims of crime, offenders and affected family members.

When people interacting with police are recorded in LEAP in connection with an incident, they may already have an existing unique identifier known as a Master Name Index (MNI) number. LEAP assigns these unique numbers to identify each individual so that there is less risk of confusing individuals with a similar or the same name and identifying information. If an MNI does not already exist for an individual, LEAP generates a new one when they record the person's interaction with police.

'Unknown' and 'not stated' are options in some data fields for members to use if they cannot determine the answer. For non‑mandatory fields, the response can also be left blank.

To further mitigate the risk of incorrectly identifying a person, Victoria Police completes secondary checks of names against drivers' licence numbers, last known addresses and dates of birth. For other demographic information, however, it is not possible to reliably verify if it is correct. People interacting with police can provide contradictory advice, and police may also be hesitant to probe for personal information. We assessed the extent of inconsistent and missing data for critical data fields in LEAP. Figures 3E, 3F and 3G show the results of our analyses of key characteristics.

Figure 3E shows how well members record country-of-birth information.

Figure 3E
Consistency and completeness of country-of-birth data in LEAP, 2012 to 2017

Individual

Consistent

Inconsistent

Missing

Offender

78.9%

13.8%

7.3%

Victim

25.8%

2.1%

72.1%

Affected family member

59.1%

5.2%

35.7%

Other parties

72.1%

10.9%

17.0%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Victim of crime—a person who suffers harm as a direct result of an act committed by another person in the course of criminal conduct. Harm may include physical, psychological or psychiatric harm or may occur when property is deliberately taken, destroyed or damaged.

It is not mandatory for members to enter country-of-birth data, but for offenders they record it consistently 78.9 per cent of the time and it is completed 92.7 per cent of the time. Increasing the amount and consistency of country-of-birth data for victims of crime would help Victoria Police to better understand if people from different countries are targets of crime and could help members to better identify whether crime is racially motivated. The current lack of consistent data limits this kind of analysis.

Victoria Police is concerned that making more fields mandatory may decrease the quality of the data. As members already consider reporting requirements to be high, additional mandatory fields might lead to attempts to skip or rush the input of data.

Figure 3F shows how well members record gender.

Figure 3F
Consistency and completeness of gender data in LEAP, 2012 to 2017

Individual

Consistent

Inconsistent

Missing

Offender

95.9%

4.0%

0.1%

Victim

91.1%

1.0%

7.9%

Affected family member

97.4%

2.5%

0.1%

Other parties

96.1%

3.8%

0.1%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

The rate of gender information is largely complete and has higher rates of consistency compared with other categories. However, victim gender data is missing 7.9 per cent of the time. Victoria Police completed a backlog of updates to gender data, which has improved the reliability of this information.

Figure 3G shows how well members record Indigenous status in LEAP. Once again, there is a larger amount of missing information about victims (41.2 per cent) compared with offenders (4.2 per cent). We discuss CSA's method for counting Indigenous data in Section 4.3.

Figure 3G
Consistency and completeness of Indigenous status data in LEAP, 2012 to 2017

Individual

Consistent

Inconsistent

Missing

Offender

87.1%

8.7%

4.2%

Victim

57.2%

1.6%

41.2%

Affected family member

82.4%

5.6%

12.0%

Other parties

85.2%

8.5%

6.3%

Note: 'Other parties' can include a witness or a family member or person present at the incident. Field responses selected as 'unknown' or 'not stated' were not considered in this analysis. Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

3.8 Accuracy

Downgrading offences

There is a risk that members may downgrade the seriousness of an incident if there is internal pressure to reduce serious crime, or members feel they need to 'improve' the rate of serious crime data by recording a less serious offence code.

Downgrading can also occur if a member decides to record a lesser offence when first detecting and reporting an incident. If this first decision to downplay the seriousness of an incident is not corrected by another member or supervising sergeant, it is difficult to detect unless victims or witnesses successfully challenge the facts of the incident.

Until a report is approved and committed to LEAP, members can save changes over the top of the original offence code. Such changes to the incident report may be necessary before approval—for example, if new information becomes known or errors need to be corrected—downgrading or updating offences occurs. LEAP allows offences to be downgraded for legitimate purposes. Police often communicate such changes in the 'narrative' field—for example, a sergeant might send a report back to a member with a note in the LEAP narrative or via LEDR Mk2 to change the offence code. In this case, the field will only record the last change made, not the original code. In this way, improper downgrading is also possible and difficult to detect on LEAP.

Victoria Police cannot easily search audit logs for changes made to offence codes because the logs contain so much information. LEAP audit logs show each screen a user has accessed, and each keystroke made, but not the actual changes made. The logs are used to try to back up claims of misuse once a mistake or manipulation has been alleged or discovered.

To see whether shifts exist in the prevalence of offences over time or movements indicate downgrading of offences, we looked at movements in the following crimes:

  • robbery compared to theft
  • burglary compared to theft
  • burglary compared to trespass
  • actual offences compared to attempted offences.

We used the following offences to see whether shifts were significant:

  • trespassing—entering a private property without permission
  • burglary (split by attempted and actual)—similar to trespass but with intent to commit a crime
  • theft (split by attempted and actual)—taking someone's property with the intent to permanently deprive him or her of it
  • robbery (split by attempted and actual)—similar to theft but with the use of physical force or fear.

Looking at the number and the percentage of movements from 2016 to 2017, it was difficult to draw any conclusions. The shifts in offences involve small numbers that are not material. One PSA had a 6 per cent reduction in theft and a 3 per cent increase in attempted theft. However, the count shows that 442 fewer thefts were recorded and only 11 more attempted thefts for the period. This is not a material shift, and it is not possible to conclude that the reduction in recorded theft offences correlates with the increase in recorded attempts.

Another PSA had a 1 per cent reduction in recorded theft and a 5 per cent increase in recorded attempted theft. Looking at the actual count, there were 17 fewer thefts and an increase of 16 attempted thefts. The numbers are too small to definitively say that there has been a downgrading of offences.

Crime clearance

Clearing or resolving crime—the investigation into the entire incident has been completed and all crime records have been marked with one of the following:

  • 'offender processed'
  • 'intent to summons'
  • 'intent to summons not authorised'
  • 'complaint withdrawn'
  • 'no offence disclosed'
  • 'other' (offender deceased, underage).

Cleared incidents do not always result in an offender being apprehended. A member can request to clear an offence if, after some investigation, the member determines that no offence occurred and evidence is insufficient to proceed to court. If approved, the incident will be counted as cleared.

Irregular outcomes

In May 2011, the former OPI found evidence of police clearing cases inappropriately, particularly for 'high volume crime such as theft, which has a low clearance rate'. We looked at incidents with outcomes that members could use to artificially clear unresolved crime. First, we looked at the 70 063 incidents from 2012 to 2017 with one of the following outcomes:

  • 'no offence disclosed'
  • 'complaint withdrawn'
  • 'other'.

We reviewed the reasons for members recording these outcomes for 30 incidents recorded as 'other' and 30 recorded as 'no offence disclosed' and found that the outcomes were reasonable.

We also looked at incidents with 'complaint withdrawn' outcomes to assess whether a sample of the reports contained information to support that outcome. Inappropriate use of 'complaint withdrawn' could include a member failing to properly interview and obtain a statement from a victim, to verify the complaint is withdrawn or coercing the victim to sign such a statement. This will reduce the member's number of unsolved or open investigations and improve the clearance rate that Victoria Police is measured against.

We also checked a sample of cases open for long periods before being cleared with 'complaint withdrawn' to see whether there had been some incentive to reduce backlogs of old unsolved cases. There were 29 859 sub-incidents with 'complaint withdrawn' outcomes, and we reviewed a sample of the 12 per cent (3 583 incidents) that were resolved more than 120 days after being reported. These incidents represent less than 1 per cent of incidents for the period, so would not impact the clearance rates. We found that the LEAP narratives did not always sufficiently state how the outcome was reached. This is likely to be poor record-keeping rather than deliberate manipulation.

For crimes such as sexual offences, police manage the case through a dedicated case management system that has higher security protections, and a lot of information related to the case will not be recorded in LEAP. However, LEAP needs to remain a standalone reporting system. We found it was not always clear in the LEAP narrative how the member reached the stated outcome. It should be possible to record this without including sensitive information.

Victoria Police states that it is possible that members left Victoria Police without the incidents being resolved or that inexperienced staff were unsure how to record the information properly. Narratives should explicitly state how police reached outcomes so that information remains useful for other members and for compliance audits.

In our review of a selection of sub-incidents, we found no deliberate attempts to falsely clear reports. Victoria Police provided reasonable possible explanations for the outcomes, and the number of sub‑incidents that are affected is not material, so it does not significantly impact published data.

Illogical date sequencing

We tested whether LEAP data contained logical dates. The order in which we expected to see dates typically occur was:

  • 'commit from' date—the date that an incident began
  • 'commit to' date—the date that an incident ended
  • 'report' date—the date that Victoria Police became aware of an incident
  • 'create' date—the date that an incident was submitted to LEAP
  • 'sub-incident result' date—the date that a result was recorded against a sub-incident.

Using this date sequence, we identified illogical date orders in 377 706 sub‑incidents, such as incidents created in LEAP before they were committed or where the 'sub-incident result' date is earlier than the 'commit to' date. This illustrates the potential for RSD staff and members to mix up key dates in LEAP and the magnitude of the problem.

We found 375 154 sub-incidents with intervention orders attached where the date the offence was committed was later than the date the sub-incident was resolved, or a result achieved. LEAP is not set up to record intervention orders. Victoria Police advised that it records the dates the court decides the intervention order is in place in the 'commit to' and 'commit from' fields'—which does not match the expected date sequence—but that this approach still allows members to understand how long the intervention order is active.

LEAP does not contain controls to prevent all possible illogical date sequences. LEDR Mk2 does use some date logic controls when members enter information, but once the data is transferred to LEAP, the dates can be edited. Incorrect dates create confusing or incorrect reports—this can obscure the number of incidents reported as being committed within a certain period, which means reporting on the time it takes to clear an offence will be wrong. However, the errors represent less than 1 per cent of the dataset. Therefore, the impact on public reporting is negligible. Victoria Police states that, since our audit, it has developed a date-checking tool to prevent this happening in future.

Location information

Street number is not a mandatory field in LEAP. When it is missing, it is difficult for members investigating crime to pinpoint exactly where an incident occurred. In some instances, members input a default value—for example, recording the street number as '1' when they are unsure of the actual address. For long streets, a missing street number may place the incident in the wrong suburb and distort reporting against a location, making the data inaccurate.

It is also possible to enter multiple address configurations for one location, such as a shopping centre, a school, a building taking up a large area or a site located on a corner, whose address might be recorded as either of the intersecting roads. Inconsistently recorded locations make it more difficult to gather reliable intelligence on sites of interest or high activity.

In our interviews, members who use LEAP data for intelligence purposes expressed frustration that location information is incomplete or inaccurate. They advised that it limits their ability to collect all available information on locations of interest. If the locations are not specific, there is a risk that locations may be under-reported or that incidents may be reported in the wrong location.

Detectives from one PSA spoke about their frustration with incorrect location information in LEAP:

A serious error is incorrect recording of location or addresses—it skews the data and affects intelligence products looking at trends across different locations. You need to do a manual search of narratives to link information together and you need knowledge of different names for places. We found three versions of the same school address with multiple offences and needed to manually match them up. Without the prior knowledge of the incident, we wouldn't know to look for it. The result was there were eight rather than three incidences at the address. We then mapped a sample of incidents using SAS [analysis software]—only 66 per cent had a correct address.

Another member spoke of missing data for theft of motor vehicles—the address where a vehicle is recovered may include a suburb or a street name with no street number. The member also described the challenge of identifying addresses for shopping centres, for which up to 50 registered address configurations can exist for a single location. Five members spoke about location information being incorrect or incomplete, limiting the usefulness of the data.

Information technology upgrades

The intelligence management system, one of the projects in the BlueConnect program, is intended to assist members to search narrative text in LEAP, including sites of interest and other important details. This should significantly reduce the time it takes to perform analysis. However, even with enhanced search functionality, user input still needs to be of a high standard so that information is as complete and relevant as possible.

Victoria Police plans to update LEDR Mk2 to introduce offender processing for members. Currently, RSD processes offenders, which may occur a considerable time after a sub-incident is recorded. This separate process may cause delays in LEAP being updated, but is likely to result in more accurate data because it is one of RSD's main responsibilities and RSD staff are trained to look for errors. Victoria Police needs strong controls over date inputs and quality assurance to make sure the data is correct.

3.9 Timeliness

As discussed in Part 2, Victoria Police's standard for timely input of crime data after members become aware of a crime is vague. This makes it difficult to hold members to account. We reviewed ePDRs from randomly selected police stations to understand how soon members were creating LEAP reports.

For the 48 ePDR jobs we tested, we found that:

  • within 24 hours, 83.3 per cent (40 jobs) had been created in LEAP
  • within 48 hours, 93.8 per cent (45 jobs) had been created in LEAP
  • the remaining 6.2 per cent (three jobs) took up to 3.5 days to be created in LEAP.

Therefore, the majority of records in our selection were entered in LEAP in a timely fashion.

Measuring timeliness

To track how long it takes to clear a case, a member needs to input a start date. In LEAP, the 'report date' field records the date when police first knew of the incident. Another field—'create date'—generates automatically when the report is first created in LEAP. Victoria Police reports on timeliness of clearing cases by measuring the proportion of incidents resolved with 30 days. It uses the time elapsed between the report date and the date an incident is resolved.

In 2011, the Victorian Ombudsman recommended that Victoria Police should use the report date for case clearance reporting rather than the create date—which was previously used due to its higher reliability—and should work to improve the reliability of the report date field.

'The report time and date is when the incident is reported or attended by members and not the time and date details are entered on LEDR Mk2.'

LEDR Mk2 training manual

Prior to 2012, the report date was not used for reporting against BP3 measures because it was a non-mandatory field and the data was unreliable—our analysis confirmed that it was blank in a high proportion of entries. Since 2012, 'report date' has been a mandatory field.

Victoria Police now uses the 'report date' to calculate its BP3 timeliness measure results. However, there is no evidence that Victoria Police has attempted to improve members' understanding of the purpose of the field or improve the accuracy of data recorded in it.

Issues that impact timeliness data

We identified a number of issues that impacted our ability to accurately measure the timeliness data.

Long lag times between the 'report date' and 'create date' can indicate errors. RSD can inadvertently change dates in LEAP when adding information. Investigations can proceed over a long period of time and require many LEAP updates. When a witness is found after a sub-incident is approved or a suspect is re-interviewed, RSD may inadvertently enter these subsequent dates in the wrong field and accidentally override a reported date.

Members can also misunderstand the date concepts. Our analysis of irregular date patterns showed that members incorrectly enter the date an incident was committed as the date reported to police, particularly when people report historical offences.

Some offences, such as manufacturing drugs, involve information that needs to be restricted from LEAP, or require a longer period of time to establish that an offence occurred. Victoria Police needs to review whether these offences are being entered correctly and ensure processes involve close monitoring to avoid errors and improve the quality of the dataset.

Victoria Police advises that it has developed a program to scan LEAP for date irregularities, which should help to resolve this kind of error in future.

Time between the 'report date' and 'create date' fields

Timeliness of recording crime in LEAP

We analysed the timeliness of data entry by comparing 'report date' and 'create date' data for sub-incidents. Figure 3H shows the average number of days between the two dates by year since 2012. The average number of days has generally reduced during this period. However, it remains much higher than the VPM expectation that members complete the majority of reports within the same shift.

Figure 3H
Average number of days from 'report date' to 'create date', 2012–17

Figure 3H shows the average number of days from 'report date' to 'create date', 2012–17

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Figure 3I shows that entries for 45 per cent of sub-incidents were created on the same day they were reported to police. This proportion meets the VPM expectation. However, comparing the results in Figure 3I to the eight‑day average gap in 2017 shown in Figure 3H suggests that either a large portion of incidents are being created well after they are reported to police, or that 'report date' is still not a reliably completed field.

Figure 3I
Proportion of sub-incidents entered in LEAP by number of days from the date they were reported to police, 2012–17

Figure 3I shows the proportion of sub-incidents entered in LEAP by number of days from the date they were reported to police, 2012–17

Note: Data for 2017 includes records for January to September only. Excludes 4 per cent of sub‑incidents that had a 'create date' recorded as prior to 'report date'.
Source: VAGO based on Victoria Police data.

We analysed PSA data to see if some PSAs had particularly high averages that may be skewing the analysis. We found that all PSAs had an average of more than 6.8 days between the 'report date' and 'create date', ranging up to 14.5 days. Therefore, no PSA is performing well on timeliness, and it is likely that RSD and members across all PSAs are, at times, inaccurately recording reporting dates.

We also looked at the top five offences with the longest gap between the 'report date' and 'create date' to understand the impact offence type might have on the average. Figure 3J shows that offences such as manufacturing drugs, incest and manslaughter—which can all involve sensitive information that needs to be restricted from LEAP users—are created in LEAP after a substantial period of investigation.

Figure 3J
Top five offences with the longest gap between report and create dates, 2012 to 2017

Figure3J shows the top five offences with the longest gap between report and create dates, 2012 to 2017

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

In cases relating to drug manufacturing, manslaughter and incest that involve sensitive information that may need to be restricted, members may be unsure when to record the crime in LEAP. Victoria Police needs clear guidance on how to populate LEAP records correctly for these types of offences, so they do not distort timeliness data.

Older dates

Between 2012 and 2017, there were 3.5 million sub-incidents recorded in LEAP, and more than 500 000 of these (16 per cent) were created more than five days after the 'report date'. We performed further analysis of these sub-incidents and found significant differences between the 'report date' and 'create date', including:

  • 788 records that had differences of five years or more
  • 59 records that had differences of more than 20 years
  • one record that had a difference of 47 years.

We asked Victoria Police to clarify how these large gaps in time points had occurred and whether they represented errors in the data. We provided a list of 59 210 sub-incidents for which the 'create date' was more than 100 days after the 'report date'.

Victoria Police advised that this likely occurred due to RSD incorrectly saving over the original report date when adding supplementary sub-incidents, as described above. Approximately 67.3 per cent of the records on our list might have had supplementary sub-incidents added to an incident at a later time. RSD's business rules for entering dates when members have failed to record them can also contribute to illogical date sequencing. There has been some improvement in the incidence of long gaps—from 9 402 sub-incidents in 2012 to 7 991 in September 2017.

Active—an investigation that is ongoing and further leads are to be followed up.

Pending—an investigation that has been carried out as far as possible and all avenues of investigation have been exhausted with no final result. It is possible that crime incidents can remain in a pending state for many years.

During this audit, we saw no evidence that Victoria Police has attempted to improve the reliability of 'report date' recording since the Victorian Ombudsman reported the issue in 2011. Until the use of the 'report date' field is consistent, there is no way to understand whether members are meeting the standards or community expectations for crime recording. These issues are present in 1.6 per cent of the dataset so will have little impact on published crime statistics. However, Victoria Police can strengthen reporting training to reduce the risk of this occurring in future. RSD also needs to review how additional incidents are added to LEAP to make sure they do not contribute to the problem. Victoria Police advises that it intends to review these business rules.

Time taken to resolve cases

Police members clear crimes in a number of ways, such as by charging offenders, by investigating and finding that no offence has occurred, or by recording that a complaint has been withdrawn. In other instances, members may identify perpetrators but, for legal or other reasons, they cannot be charged—for example, if they are underage or deceased.

In 2011, the Victorian Ombudsman referred some allegations of police corruption relating to clearing of crime records to the former OPI for investigation. These allegations involved the mishandling of crime records, with the result that the data showed more crimes being resolved or 'cleared' than actual results.

In our interviews, members spoke of 'top-down' pressure that they or their superiors felt when there were increases in crime rates or breaches in time lines. Some saw this as indirectly encouraging members to artificially 'improve the numbers'. This pressure, if it does exist, will negatively impact on the completeness and accuracy of data. However, in this audit, we did not detect any manipulation of data or downgrading of offences to improve performance measures.

BP3 measures

Figure 3K shows the timeliness benchmarks that Victoria Police reports against in BP3.

Figure 3K
BP3 performance measures for Victoria Police—timeliness

Performance measures

Unit of measure

2017–18 target

Timeliness

Proportion of crimes against the person resolved within 30 days

per cent

>42

Proportion of property crime resolved within 30 days

per cent

>22

Source: VAGO based on Victorian Government Budget Papers, 2017–18.

Victoria Police publishes its performance against these measures in its annual reports. Over the past three years, it has requested data from CSA for the six crime-related BP3 measures—see Figure 1C in Part 1. However, during this period, Victoria Police used its own data to calculate the timeliness measures rather than using the figures provided by CSA.

Victoria Police used the 'report date', as recommended by the Victorian Ombudsman—rather than the 'create date'—and calculated timeliness based on all crime for the relevant period rather than just a proportion of solved cases. Victoria Police's results therefore include open cases that were taking longer than 30 days to close and provide a slightly less favourable but more transparent picture of timeliness.

We replicated the BP3 performance measure results based on the Victoria Police methodology and compared them to the published results for 2015–16 and 2016–17. Victoria Police uses statistics provided by CSA to report on the BP3 results, with the exception of the timeliness measures. We found that we could accurately replicate the results, which means that we have confidence that Victoria Police is reporting its performance results correctly.

To improve transparency, Victoria Police should work with CSA and ask CSA to compute timeliness measures for the Budget Papers. The Victoria Police methodology is sound, and our analysis showed that its results are accurate. However, it is CSA's role to provide verified figures, and Victoria Police can avoid any perception of bias if it uses CSA data.

Analysing case clearance times

Using LEAP data from the 'report date' and 'result date' fields, we analysed how long it takes to resolve or clear incidents. We found no evidence that police clear large volumes of reports close to their 30-day benchmark or at any other time point that would indicate manipulation of the data.

We also assessed individual PSAs for isolated pockets of case closure. We found that 43 per cent of cases were resolved within 10 days of the 'report date', and there were no spikes in case closures at the 30-day mark or any other points. This was true for all PSAs.

Figure 3L shows the number and proportion of cleared sub-incidents by days taken for the years 2012 to 2017. The chart shows that 84 per cent of sub‑incidents were cleared within 30 days, and 7 per cent were resolved over 120 days.

Figure 3L
Proportion of cleared cases within bands of days taken to clear, 2012 to 2017

Figure3L shows the number and proportion of cleared sub-incidents by days taken for the years 2012 to 2017.

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Victoria Police advised that there are common reasons for older cases remaining open, such as inexperienced members failing to clear cases in LEAP when they were resolved. Another cause may be intervention orders left open with no date recorded against the 'resolved date' field—this occurs because LEAP is not set up to record intervention orders, particularly where the order is in place indefinitely. Victoria Police now enters a proxy end date for intervention orders to overcome this issue. Some cases take longer to resolve due to prolonged investigations.

Figure 3M shows the average days taken to clear cases for the years 2012 to 2017. The number of cases cleared on the same day they are reported has steadily risen over the five-year period.

Figure 3M
Average days to clear cases (banded) by year, 2012 to 2017

Figure 3M shows the average days taken to clear cases for the years 2012 to 2017

Note: Data for 2017 includes records for January to September only.
Source: VAGO based on Victoria Police data.

Back to Top