The Effectiveness of Victoria Police’s Staff Allocation
Does Victoria Police understand its frontline staffing needs and use this information to make resourcing decisions aligned to its strategic objectives?
Why this audit is important
Victoria Police plays a critical role in ensuring a safe and secure society. Victoria Police's spending for staffing is significant and the majority of this is for sworn officers.
Given the large, ongoing government investment into its staffing, it is important to assess whether Victoria Police is using this funding effectively.
Who and what we examined
We examined Victoria Police’s understanding of its staffing needs. We also looked at how Victoria Police uses this understanding to make strategic resourcing decisions. As part of this assessment, we looked at how it developed and uses the Staff Allocation Model.
What we concluded
Victoria Police’s staffing needs are not clear because it does not have a strategic workforce plan. It does not have any modelling or forecasting to understand or plan for its staffing requirements—both current and long-term.
Without knowing its future staffing needs, Victoria Police continues to rely on what it receives from the government, rather than providing evidence-based advice.
Victoria Police also cannot assure itself or the community that the 2,729 additional police officers it received as part of the $2 billion Community Safety Statement (CSS) program delivered community safety outcomes.
This is because the CSS program only had a short-form business case which was inadequate for the size of the investment. This diminishes transparency and information for decision makers. Victoria Police has not since attempted to assess the incremental benefits realised from the investment.
What we recommended
We made 6 recommendations to Victoria Police about understanding and documenting its staffing needs, benefits management and data management.
Note: *VPS = Victorian Public Service; PSO = protective services officer; PCO = police custody officer.
What we found and recommend
We consulted Victoria Police and considered its views when reaching our conclusions. Its full response is in Appendix A.
Victoria Police’s assessment of staffing requirements is not supported by robust evidence or analysis
Victoria Police has no modelling and forecasting capabilities to have a complete understanding or clear plan for its staffing requirements—both current and long term. There is no strategic workforce plan for future sworn officer positions and proposals submitted to the government are not evidence based.
In November 2016, Victoria Police drafted a short-form business case to support its request for 2,729 additional police officers. We found no evidence of how Victoria Police arrived at this number. Victoria Police told us that this number originated from a government decision.
Victoria Police tells us it undertook a consultative process with the government and other key stakeholders to refine its 2022–23 Budget bid for an additional 1,489 staff. In its Budget submission, Victoria Police noted that it needed more than 2,300 staff, which was also determined via a consultative process, but only asked for 1,489.
Without a strategic workforce plan or robust modelling or forecasting, Victoria Police cannot fully understand its staffing needs.
Benefit realisation reporting is inadequate
Monitoring government investments is critical to determine the impact of public funds. The Department of Treasury and Finance's guidance notes that without a benefits management plan (BMP), agencies focus less on benefits once a project has started and faces the challenges of business as usual.
Victoria Police cannot adequately measure and report on whether the 2,729 staff increase is improving the effectiveness of Victoria's frontline police response and reducing harm by meeting demand for police services. It did not complete or implement its initial BMP or the enterprise benefits framework it intended to develop. Its most recent project benefits plan is focused on inputs rather than outcomes.
Victoria Police drafted a BMP in 2018 but did not finalise it or track any benefits from the deployment of 2,729 police officers, which was part of the $2 billion CSS investment. There is no documentation to support why Victoria Police did not finalise the 2018 plan. Instead, it then decided to develop an enterprise benefits framework to report on the wider benefits of the Community Safety Statement (CSS) initiatives. Almost 5 years on, Victoria Police is yet to finalise this framework.
In 2020, Victoria Police implemented a project benefits plan that focused on monitoring the recruitment and deployment of new staff. This plan is inadequate to track the benefits of a $2 billion government investment because it does not show the actual benefit to the community.
Recommendations about Victoria Police's staffing needs
|We recommend that:||Response|
|Victoria Police||1. develops a long-term strategic workforce plan of its staffing requirements including sworn police officers and uses this to inform future budget requests (see Section 2.1)||Accepted in principle|
2. produces a benefits realisation plan for projects that aligns with the Department of Treasury and Finance's guidelines and:
Accepted in principle
Staff Allocation Model governance and reporting structures
In November 2016, Victoria Police commissioned the Staff Allocation Model (SAM) to help it to allocate staff resources across the state. SAM is a modelling system that calculates, compares and prioritises demand based on various input data. It then recommends how Victoria Police should allocate new policing resources to different regions and commands.
Victoria Police did not document why it changed staff allocations
Victoria Police uses its professional judgement to adjust SAM's allocation recommendations. It told us it does this because of limitations in measuring emerging risks, trends and business improvements in the SAM.
For example, Victoria Police's Executive Command adjusted the SAM recommendations in Generation 3 (Gen 3) and Generation 4 (Gen 4) based on its professional judgement. It also diverted 200 staff from CSS positions to fill unfunded and discretionary positions.
However, Victoria Police has not documented why it has made these changes. Inadequate documentation makes decision-making less transparent. It also makes it harder to assess the limitations in SAM’s recommendations and reduces opportunities to continuously improve the system.
Management of SAM data inputs
Victoria Police does not comply with government data quality standards
All government departments, including Victoria Police, must meet the requirements of the government’s Data Quality Guideline Information Management Framework (Data Quality Guideline). However, Victoria Police's management of SAM data inputs does not comply with the Victorian Government's data quality standards.
To comply, Victoria Police should:
- have a custodian responsible for data quality management
- have a data quality management plan and statement
- assess its data regularly.
It has not done this.
SAM does not have a data dictionary
A data dictionary is a centralised repository of information about a dataset, such as meanings, relationships to other data, origins, usage, and formats that are used to define conventions of a project and drive consistency.
Victoria Police does not have a data dictionary to define and simplify how it manages SAM data. In 2019, Victoria Police started developing the dictionary as a solution to its inconsistent data capture methods but has not progressed it since.
Without a data dictionary, Victoria Police continues to have a higher risk of:
- inconsistent data capture
- mismatched understanding of technical requirements
- heightened risks to business continuity.
SAM's further development
Systems like SAM need key documentation to guide their development and ensure organisations are clear about their use. Victoria Police has not developed some key documentation for SAM, including:
- its business rules
- the data needed for the level of demand that Victoria Police wants to include in SAM over time
- ongoing monitoring and review for continuous improvement opportunities.
Victoria Police also lacks a long-term planning document, such as a strategic plan, to guide SAM’s ongoing development.
Victoria Police could better guide the SAM development team by setting the level of demand it wishes to capture through SAM. While Victoria Police has made new plans for SAM each year, it now plans to treat all SAM-related work as a business-as-usual activity. With SAM transitioning to a business-as-usual tool, Victoria Police will not be producing formal project plans or closure reports to guide its ongoing use. A lack of business rules, strategic monitoring and review documents creates a risk that SAM will continue to operate with the issues already identified in internal and external reviews.
External review of SAM
In 2019, Victoria Police engaged a university to undertake a review of SAM Gen 3. This found that while SAM's allocation approach produced reasonable results, it could not guarantee the best results because of weaknesses in the SAM algorithm.
The review recommended that Victoria Police should investigate more sophisticated mathematical methods to enhance the model for potentially better results. However, Victoria Police chose not to pursue other methods because it could be too difficult to explain the complex process internally.
Recommendations about governance and reporting structures
|We recommend that:||Response|
3. comprehensively and clearly documents:
4. complies with the Victorian Government's Data Quality Guideline Information Management Framework to:
5. documents for the Staff Allocation Model:
6. revisits the finding in the external review to investigate and assess alternative methods of allocating resources within the Staff Allocation Model algorithm (see Section 3.3).
1. Audit context
Victoria Police is responsible for preserving a safe, secure and orderly society. It does this primarily by employing sworn police officers based at police stations that operate 24 hours a day, 7 days a week.
To achieve its goals to improve community safety, it is important that Victoria Police effectively allocates and deploys newly sworn members to frontline policing positions where they can deliver reactive and proactive policing.
This chapter provides essential background information about:
1.1 Victoria Police
Victoria Police's role is to serve the Victorian community and uphold the law to promote a safe, secure, and orderly society. It does this by providing policing services to the Victorian community across 54 police areas within 21 divisions and 4 regions.
Figure 1A shows the regional breakdown of Victoria and the number of divisions per region.
FIGURE 1A: Police regions in Victoria
Source: VAGO, based on Victoria Police.
Victoria Police has around 21,600 staff. Sworn police officers, protective services officers and police custody officers account for about 18,100 staff.
Victoria Police also employs around 3,500 public service professionals. Its workforce budget makes up about 72.7 per cent of the annual funding that the government invests in Victoria Police, which was $3.8 billion in 2020–21.
Victoria Police's senior leadership
The Chief Commissioner of Victoria Police is the head of Victoria Police. The Victoria Police Act 2013 allows them to make decisions on the recruitment, allocation and deployment of Victoria Police employees, including sworn police officers. The Chief Commissioner is also responsible for implementing policing policies and priorities of the Victorian Government.
Victoria Police’s senior leadership group is known as Executive Command. It is the primary advisory and decision-making body for Victoria Police. It is made up of the Chief Commissioner, all deputy commissioners and deputy secretaries and an independent member.
1.2 Deploying police officers
Regional and divisional police commanders make decisions to deploy sworn police officers for proactive and reactive operations:
- Proactive policing aims to deter criminal activity by showing police presence, such as road policing.
- Reactive policing is when officers respond to service needs reported in the community, such as traffic accidents, 000 calls or incidents of family violence.
1.3 Community Safety Statement
The Victorian Government launched the CSS in December 2016. This agreement between Victoria Police and the Victorian Government establishes a commitment to the community to set priorities to reduce crime and keep the state safe.
Figure 1B shows CSS’s 5 priority areas.
FIGURE 1B: CSS priority areas
Source: VAGO, based on Victoria Police.
To meet these priorities, the government provided Victoria Police $2 billion in additional funding over 4 years, starting in 2017‒18. This funding was for an additional 2,729 sworn officers.
Victoria Police also received $596 million for 406 new recruits in the 2016‒17 state Budget under the Public Safety Police Response (PSPR) package.
These 2 funding packages were the largest recruitment investment in Victoria Police’s history.
Implementation of the CSS
In 2017, the government appointed the Community Safety Trustee to report on the implementation of CSS actions and the delivery of key parts of the investment.
The trustee published 6 reports during their 4-year appointment that concluded in April 2021. Their final report, which was published on 3 May 2021, assessed the recruitment of 2,729 new sworn officers as ongoing but progressing. It reported that:
'Victoria Police advised that despite transient delays due to the introduction of COVID-19 [coronavirus] pandemic-related restrictions, the recruitment program is on schedule to complete full recruitment by 30 June 2021’.
1.4 Staff Allocation Model
Staff allocation in the modern policing environment is complex. Victoria Police commissioned the development of SAM in November 2016 as an input into its planning and decision-making. SAM helps Victoria Police allocate staff resources to meet demand across the state. SAM is intended to calculate, compare and prioritise demand across different regions and commands based on various input data. The inputs to SAM include factors such as:
- baseline staffing requirements for stations and patrol vans
- demand factors, such as calls for assistance, reported crime, event management and the time required to provide these services.
Victoria Police takes SAM’s outputs as recommendations on how to allocate the staffing numbers from the CSS bid. Executive Command then reviews, amends and endorses allocations. SAM is not used to roster staff.
Figure 1C lists Victoria Police’s structures that govern SAM.
FIGURE 1C: Governance arrangements for SAM
|Executive Command||Approving the ongoing development of SAM, including data inputs and the final SAM allocations|
|2,729 Steering committee||Prioritising tasks, issues and risks in relation to SAM|
|Executive sponsor||Overseeing SAM development and demand data inputs|
|Project advisory group (March to July 2020)||Advised on the scope of work for SAM Gen 5, including how to capture demand and reflect modern policing more accurately|
|Service Delivery Transformation Command (from March 2022)||Will lead SAM's future use and development|
Source: VAGO, based on Victoria Police.
SAM’s development over time
As shown in Figure 1D, Victoria Police has been developing SAM through various ‘generations’, which are successive releases that add enhancements and features. In 2016, Victoria Police used Gen 1 to allocate new staff under the PSPR. It then used Gen 2 to Gen 5 to allocate new staff under the CSS.
FIGURE 1D: Timeline of SAM generations
Note: After Victoria Police releases a SAM generation, the actual deployment of new staff happens the following financial year when police academy graduates are ready to be deployed to police stations.
SAM has 4 modules, which inform how it allocates:
- sworn officers for regional operations
- sworn officers for specialist operations
- Victorian Public Service (VPS) staff
- staff in corporate departments.
Regional operations cover policing activities across Victoria Police's 4 regions that involve interactions with the public. These include:
- general patrolling
- traffic management
- public order management
- highway patrol
- criminal investigation units
- family violence investigation units
- sexual offences and child-abuse investigation teams
- divisional intelligence units.
Specialist operations refer to Victoria Police's commands and departments that oversee functions, such as:
- forensic services
- legal services
- intelligence and covert support
- family violence
- professional standards.
SAM Generation 5
Victoria Police released SAM Gen 5 in November 2020. It committed Victoria Police to including demand-and-supply modelling for regional VPS employees as well as bringing a revised methodology for specialist police and a maturing of the inputs for regional police.
Victoria Police has been expanding the demand data inputs SAM uses over time. Its SAM Gen 1 had only 8 data input categories. Victoria Police’s SAM guidance for Gen 5 includes 22 input categories. We list these categories in Appendix D.
Earlier staff allocation models
SAM is Victoria Police's fourth staff allocation model. Victoria Police has told us it incorporated learnings from the earlier models to help it develop SAM. Figure 1E shows the earlier models.
FIGURE 1E: Staff allocation models used by Victoria Police before SAM
|Police Allocation Model (PAM)—2005||PAM used 13 variables, including estimated levels of crime against the person, crime against property and road trauma, and overlaid this information with special events affecting demand (such as major events), dispersed populations and custodial management.|
Resource Allocation Model—2010
Heavily influenced by the preceding PAM, this model included multiple demand ratios, such as the number of:
Victoria Police applied these demand ratios across the police service areas to identify state averages and relative rankings of demand levels. It then used this information to determine how many resources it needed to equalise these demand levels.
Demand Allocation Model—2011
Developed specifically to allocate 1,700 new resources in 2010–11, this model worked to ‘top up’ resources and help smooth out the resource spread. It used state demand indicators (that is, computer-aided dispatch (CAD) events) to establish the average workload.
Source: Victoria Police.
2. Victoria Police’s staffing needs
Victoria Police does not have a strategic workforce plan or well developed forecasting to understand its current or future staffing requirements. It therefore cannot tell the government how many police officers it requires. Despite this, Victoria Police has received more than 3,200 new police officers since 2016.
Victoria Police's $2 billion CSS funding was supported only by a short-form business case. This reduces transparency and information for decision-makers. Victoria Police tracked whether it deployed 3,135 new police officers as part of the CSS and PSPR packages. However, this is inadequate to track the benefits for the community of this significant investment.
This chapter discusses:
2.1 Victoria Police’s staffing needs
Victoria Police does not have a strategic workforce plan. It does not have a current and accurate understanding of its current or future staffing requirements. As a result, it cannot show if the number of police officers it has meets demand. Without a strategic workforce plan supported by forecasting of staffing it needs, Victoria Police cannot advise the government of its long-term staffing needs. Victoria Police advised us that it is developing a workforce plan as a priority for 2022–23. However, it was unable to provide any project scope, terms of reference or draft work plans to us.
Figure 2A shows the 2 significant increases in sworn police numbers approved by government since 2016.
FIGURE 2A: Government announcements of new police officers from 2016 to 2022
Short-form business case for the $2 billion recruitment of 2,729 new police officers
Victoria Police only had a short-form business case in November 2016 to support the $2 billion CSS investment in 2,729 new police officers. It did not have a full business case. The short-form business case does not provide adequate information to decision-makers for such a large investment. Victoria Police cannot show that the 2,729 new police officers it said it needed were supported by any modelling of its future staffing requirements.
Victoria Police advised us that the process for building a funding submission is complex, requiring input and consultation with the government and key stakeholders to reach the final funding submission. Victoria Police advised us that the number of new police officers originated from this process and included this in its draft business case. Victoria Police should document this process, including consultation from future business case negotiations.
Without any workforce planning strategies or modelling of need, Victoria Police does not fully understand its staffing needs and could not provide reliable advice to the government of its staffing requirements.
2022 bid for new officers
In the 2022–23 Budget announced in May 2022, the government committed funding for 502 additional police officers to be recruited over two years. Victoria Police produced a business case for the government in February 2022 arguing the case for an additional 1,489 police officers because the 2,729 recruits do not meet current and future demand. The business case said that Victoria Police needed more than 2,300 additional police to meet its requirements. It said this figure was determined via consultation. Victoria Police told us it moderated this figure down to 1,489 staff over 4 years as a result of consultation with the government and key stakeholders.
We found 69 per cent of the total number requested by Victoria Police, or 1,030, were either supported by outdated information or no information was provided to support allocation under professional judgement.
Victoria Police’s bid used …
We found …
professional judgement to decide on 403 new police officer positions (27 per cent)
Victoria Police has not documented its rationale for using professional judgement to support this need.
a 2016 staffing benchmark to decide on 408 new police officer positions (27 per cent)
the issues register noted this benchmark has the potential to negatively impact outcomes. Victoria Police has tested other benchmarks and has told us the 2016 version is the most appropriate.
We also found an error where Victoria Police double counted the requirements of one new police station.
specialist operations module (SOM) modelling to decide on 219 new positions (15 per cent).
the SOM modelling was completed in 2020 and not representative of demand in 2022.
The detailed SOM modelling is different from what Victoria Police has used. Victoria Police has no explanation for why it changed the modelling.
Victoria Police’s business case also planned for 408 new police officers to improve its capacity to meet baseline minimum service levels (BMSLs). In March 2020, Victoria Police and the Police Federation of Australia (Victoria Police branch) committed to BMSL obligations.
Victoria Police acknowledges it is currently operating below those BMSL requirements. Victoria Police has advised us that this will be addressed through the additional 502 new police officers from the 2022–23 Budget announcement.
2.2 Benefits realisation reporting
For the first 3 years of the CSS, Victoria Police did not have a BMP. In 2020, Victoria Police implemented a project benefits plan, which focuses on inputs not outcomes. The project benefits plan does not measure and report on if and by how much the increase of 2,729 staff is reducing harm. As a result, Victoria Police cannot assure itself or the community that the $2 billion CSS investment is meeting its aim.
Monitoring government investments is critical to understanding the impact of public funds. The Department of Treasury and Finance's guidance on benefits realisation states that without a BMP, agencies lose the focus on benefits once a project begins and starts to face the challenges of business as usual. A BMP will specify:
- the identified problem
- intended benefits of the investment
- measures of success.
Victoria Police’s aim for the 2,729 police officer investment was to meet current and future demand for police services. Its draft business case in November 2016 stated that the staff investment would contribute to Victoria Police meeting its performance measures.
The PSPR package was an initiative the government announced in the 2016–17 Budget for 406 new police officers and 52 support personnel.
In March 2020, an internal Victoria Police brief noted that:
‘a Benefits Management Plan (BMP) is required to determine return on investment for the additional 3,135 sworn resources (comprising 406 under the Public Safety Police Response package and 2,729 under the CSS)’.
However, after reviewing the draft BMP, Victoria Police chose not to finalise the BMP or report on the benefits of these staffing investments. Instead, it decided to develop an enterprise benefits framework to report on the wider benefits of CSS initiatives. As of July 2022, this framework had not been finalised.
In 2020, Victoria Police implemented a ‘project benefits plan’ that focused on monitoring the recruitment and deployment of the 3,135 new staff. The key deliverables this plan identified were to:
- attract, recruit and deploy 3,135 police
- increase cultural and linguistic diversity and gender representation in the police force.
This project benefits plan is inadequate to track the benefits of this significant investment because it does not measure how adding 3,135 more police impacts community safety.
3. SAM’s governance and reporting structures
Victoria Police’s Executive Command uses its professional judgement to adjust the SAM allocation recommendations. However, it has not documented any details of the rationale of these adjustments. This means Victoria Police risks losing decision-making corporate knowledge and valuable information for improving SAM.
Victoria Police’s management of SAM does not comply with the Victorian Government’s 2018 Data Quality Guideline. There is a risk SAM data collection is not systematic and consistent.
Victoria Police also lacks a long-term planning document to guide SAM’s ongoing development. Plans to make SAM-related work a business-as-usual activity risk making improvement of the model a lower priority in the future.
This chapter discusses:
3.1 Final staff allocations and SAM recommendations
Victoria Police makes its final staff allocation decisions for divisions and regions after reviewing SAM recommendations. We found that Victoria Police does not always follow these recommendations and that Executive Command uses its professional judgement to reallocate resources. However, Victoria Police has not documented the rationale for most of these decisions.
For example, Victoria Police changed SAM recommendations in 15 out of 21 divisions for Gen 3 and changed 14 out of 21 divisions for Gen 4 with no supporting rationale.
Victoria Police also made significant manual adjustments to SAM’s Gen 5 recommendations. Executive Command did document some reasons why it made these adjustments. The main ones were to:
- address supervision shortfalls
- address unsupported demand
- support north-west metro division 1 (CBD division) requirements.
By not documenting its rationale for amending SAM recommendations, Victoria Police has failed to retain corporate knowledge of its rationale and decision-making. Victoria Police also lost opportunities to better refine SAM according to factors that affect demand for police services.
CSS allocations diverted to fill other positions
Victoria Police diverted 10 per cent, or 200, of the CSS positions to fill unfunded and discretionary positions that it had not modelled as part of the CSS program. Victoria Police did not use SAM to allocate these positions, instead using professional judgement. Victoria Police's Executive Command did not document its rationale in choosing to fill these positions over the CSS allocations. It is unclear:
- why there was an ongoing need for these positions
- how Victoria Police determined that need was greater for these discretionary positions than the planned CSS positions
- how Victoria Police determined which positions should be filled.
3.2 Management of SAM data inputs
Victoria Police’s management of SAM data inputs does not comply with the Victorian Government’s 2018 Data Quality Guideline. Gaps in its guidance and processes mean that despite improving data collection across SAM’s different generations, some data issues remain unaddressed. There is a risk that the data collection for SAM is not systematic and consistent.
Compliance with the Victorian Government Data Quality Guideline
The Data Quality Information Management Framework Standard defines a data asset as a dataset containing data that can be organised and stored in fixed fields, such as in a relational database record or spreadsheet
The Data Quality Guideline gives government departments, including Victoria Police, high level advice on how to assess, maintain and improve the quality of data assets. The guideline is a part of the whole-of-government information management framework.
The Data Quality Guideline requires Victoria Police to meet the 2018 Data Quality Information Management Framework Standard (Data Quality Standard). Our assessment in Figure 3A shows that Victoria Police’s management of the SAM data inputs does not meet the standard’s minimum requirements.
FIGURE 3A: Assessment of SAM data inputs against the Victorian Data Quality Standard
|Data Quality Standard requirement||Does Victoria Police meet the requirement for SAM?|
|Ensure the accountable data asset custodian is responsible for data quality management||No. Victoria Police has not nominated a data asset custodian for SAM.|
|Develop and maintain a data quality management plan for each critical data asset||No. Victoria Police does not have data quality management plans for any SAM data assets. These are critical to Victoria Police's ability to objectively assess demand and allocate new frontline staff.|
|Create a data quality statement for all critical data assets, datasets to be shared with other departments or external parties, and datasets to be released to the public||No. Victoria Police has not identified any critical data assets and does not have any data quality statements for any of the datasets that SAM uses.|
|Assess data assets against the data quality dimensions (accuracy, validity, completeness, representation, timeliness, fitness for purpose, collection method)||No. Victoria Police has not assessed the SAM input datasets against the 7 data quality dimensions.|
Source: VAGO analysis using the Victorian Government Data Quality Standard.
Data collection practices
Victoria Police does not have an overarching data policy, strategy, framework or business rules governing the SAM inputs. Instead, 4 documents serve as the guiding framework for its SAM data inputs, as Figure 3B shows.
FIGURE 3B: The documents in Victoria Police’s guiding framework for SAM data inputs
|Project plan||Describes the project method that will be used to deliver each generation of SAM, including the project's scope and limitations, governance structures to monitor project delivery, resource requirements and project risks|
|Inputs master||Identifies the input datasets and explains how the model treats them, such as what aspect of demand the data measures and how much police time that aspect of demand requires (for example, 45 minutes for a CAD job)|
|Regional operations module (ROM) data collection instructions (from SAM Gen 5)||Documents information on the Gen 5 input datasets, including some collection information, such as data sources and datasets used|
|SAM/ROM modelling documentation||Primarily describes the model components and how the model works|
Collectively, these documents have many of the elements needed to support consistent data collection. However, Victoria Police does not have any overarching guidance explaining how the documents:
- relate to and complement each other
- should be used and updated over successive generations.
Gaps in the guidance for these documents means the information does not fully support systematic and consistent data collection.
None of the documents …
Which means …
record the location of input data or the file structure in Victoria Police’s information technology systems
it is harder for new staff working with SAM to access data.
provide a current list of known errors or issues, information on corrections, or a consolidated record and history of improvements and updates
there is no structured way to record and track errors and concerns with SAM’s operation and progress in resolving these matters. This can create inconsistencies and lead to errors in selecting the right dataset to use.
describe how data derived from consultation processes is generated, collected and validated
Victoria Police is less able to ensure that the consultation data it relies on is consistent between years.
A SAM data dictionary is needed
In 2019, Victoria Police’s Gen 4 closure report recommended Victoria Police develop a data dictionary as a solution to inconsistent data capture methods. Victoria Police has not developed one and its data collection documents only contain some of the information that a data dictionary includes.
A data dictionary catalogues and communicates the structure and content of data and gives meaningful descriptions of individually named datasets. The Victorian Government's Data Quality Guideline explains that a data dictionary is used to increase understanding about the data and make this information readily available, including through:
- documenting data formats and standards
- defining the intent of each data field.
Victoria Police developed terms of reference for the dataset dictionary working group in August 2019. However, no meetings have occurred and Victoria Police has made no progress advancing the data dictionary.
Without a data dictionary, Victoria Police continues to risk:
- inconsistent data capture
- mismatched understanding of technical requirements
- heightened risks to business continuity.
3.3 SAM’s further development
SAM is critical for Victoria Police’s staff allocation and decision-making. Victoria Police has invested resources into creating SAM but its development as a modelling system is still very fragmented. Victoria Police needs to better plan for SAM’s future development.
Strategic direction for SAM’s development
Victoria Police does not have any planning documents, such as a strategic plan, to guide SAM’s ongoing development. Such a document should include what aims Victoria Police wants to achieve in terms of modelling accuracy and what further data should be incorporated into SAM. This would help Victoria Police guide its ongoing development.
Victoria Police’s SAM team develops annual project plans for each SAM generation. These outline the objectives and scope of that year’s improvement and use of SAM. This planning is focused on the short term and does not consider a long-term vision of what SAM should be. This creates the risk that SAM does not develop to its full potential or evolves too slowly to provide the most useful allocation advice.
Victoria Police told us that SAM transitioned to the Service Delivery Transformation Command in March 2022. SAM has now become a business-as-usual allocation tool and will no longer have formal project plans or closure reports, but there may be informal planning discussions to guide its ongoing use.
Victoria Police does not have any finalised documentation on what it intends to do with SAM for further generations. It is unclear how Victoria Police will develop SAM Gen 6.
External review of SAM
In April 2019, Victoria Police engaged a university to undertake a review of SAM Gen 3. It found that while SAM’s allocation approach produced reasonable results, the model could not guarantee the best possible allocations because of the nature of the algorithm SAM uses. The review said that more sophisticated resource allocation methods are available that Victoria Police should investigate for potentially better results.
The review made 19 recommendations focused on the structure and format of the code to prevent errors, and some improvements and alternatives to the overall methodology. Victoria Police’s response did not make clear which of these recommendations it accepted.
Victoria Police did not take up the review’s recommendation to investigate alternative methods of allocating resources. Victoria Police said that due to the possible difficulty in explaining the complex process to its workforce, it would not accept the recommendation.
This means Victoria Police did not fully utilise the expert advice it paid for and has missed an opportunity to potentially improve its allocation processes.
Appendix A. Submissions and comments
Click the link below to download a PDF copy of Appendix A. Submissions and comments.
Click here to download Appendix A. Submissions and comments
Appendix B. Acronyms, abbreviations and glossary
Click the link below to download a PDF copy of Appendix B. Acronyms, abbreviations and glossary.
Click here to download Appendix B. Acronyms, abbreviations and glossary
Appendix C. Scope of this audit
Click the link below to download a PDF copy of Appendix C. Scope of this audit.
Click here to download Appendix C. Scope of this audit
Appendix D. Gen 5 ROM data inclusions
Click the link below to download a PDF copy of Appendix D. Gen 5 ROM data inclusions.
Click here to download Appendix D. Gen 5 ROM data inclusions