Collecting appropriate data on public dental service patients is critical for monitoring and reporting on whether services are improving oral health outcomes.
Our 2016 audit found that DHSV collected limited data about the clinical oral health of eligible adults on entry to services. It does not collect this data when care had been completed, or when a patient returns to the service to determine whether their oral health has improved.
DHSV's SoP reports dental health program KPIs, against which DHSV and CDAs are held accountable. Our 2016 audit found that none of these indicators showed whether access to care had improved oral health outcomes. In addition, reporting by DHHS on the dental health program through the BP3 was output‑focused. This meant it did not provide information on the outcomes of service provision. We concluded that neither DHHS or DHSV provide a comprehensive picture of program impact in their reporting on public dental services.
In this Part, we look at current measurement and public reporting on the performance of public dental services.
DHSV have developed oral health outcome indicators with an international consortium, but they are not yet implemented. If adopted, these outcome measures will enable reporting about the extent to which access to care improves oral health. DHHS updated both the SoP and BP3 measures and while they are an improvement, they do not yet provide a comprehensive picture of the impact of programs.
4.2 Measuring performance
Our 2016 audit made one recommendation in relation to oral health data on people who are eligible for public dental services. This concerned a lack of data to enable assessment of whether access to care has improved oral health outcomes.
Collect data on eligible patients
That the Department of Health and Human Services and Dental Health Services Victoria work with community dental agencies to collect data on people who are eligible for public dental services as a subset of its broader oral health outcomes measures based on the whole population.
DHHS and DHSV have focused their efforts on participating in two population‑based surveys:
- The annual Victorian Population Health Survey (based on a random sample of 7 500 adults) added three oral health questions to the 2016 and 2017 surveys—self-reported dental health, when they last visited a dental health professional, and whether cost had an impact on this.
- The Victorian component of the National Study of Adult Oral Health (based on a random sample of adults), which was most recently conducted in 2016–18, has only been conducted twice before (1987–88 and 2004–06). It comprises a telephone survey and oral epidemiological examinations. Of the national sample of 10 220 adults, DHSV managed oral examinations of 1 421 Victorians, which were conducted in CDAs.
These population surveys do not target individuals who are eligible for public dental services—they provide information about the pattern of oral disease and use of dental services in the general adult population.
Data collection by CDAs
The DHSV dental health program dataset defines the data that CDAs are required to record for patients accessing their services. Data on patient clinical oral status includes the number of decayed, missing and filled teeth, and a rating of prosthetic status. CDAs are not required to collect data on returning patients, such as whether they have less tooth decay or improved oral health.
In future, it will be important that DHHS and DHSV enable CDAs to collect relevant data from patients pre and post access to services. CDAs will need support and resources to put the required processes and systems in place to collect the data. This data will provide an assessment of whether the oral health of patients accessing public dental services has improved.
In two of the CDAs we visited, we observed that they had begun to collect some patient self-reported data on the impact of some of its services and activities. This includes pre and post assessment of patient-reported levels of service and oral health literacy, as well as oral hygiene habits and health behaviours. This information is used to inform ongoing improvement of service provision. We do not know the extent to which other CDAs across the state have adopted such practices.
International Consortium for Health Outcomes Measurement
In our 2016 audit, we reported that DHSV had begun work on developing oral health outcome indicators through its participation in the ICHOM oral health working group. This group was established to develop an internationally agreed standard set of oral health outcome measures.
Since 2016, this work has continued to progress. The ICHOM working group identified 23 key outcome measures to include in an adult oral health standard set. DHSV is leading the implementation of the ICHOM standard set consumer validation survey in Australia. Based on feedback from clinicians, work is now underway to determine the most important questions in the adult oral health standard set to develop a shorter set.
While the ICHOM adult oral health standard set has not yet been released, DHSV plans to incorporate its use as part of its transition to MoC based on VBHC principles.
Oral health data management
DHSV recognises the need to improve its information and communications technology (ICT) platform to support data collection to monitor and track the experiences and oral health outcomes for each patient.
DHSV's Digital Strategy 2018–21 sets out its plan to put in place the required ICT platform to help enable its transition to VBHC.
In June 2019, consultants completed a business case for a solution to improve DHSV's oral health record management. The preferred option is to adopt a statewide electronic oral health record that has the capacity to both analyse and report and track patient experience.
The business case sets out a program of work scheduled to begin in January 2020 and finish by February 2022. DHSV advised that the business case is yet to be presented to its board.
4.3 Reporting performance
Our 2016 audit made two recommendations in relation to reporting on the performance of public dental services. These concerned the usefulness, relevance and appropriateness of KPIs in the BP3 and in DHSV's SoP.
Statement of Priorities
That the Department of Health and Human Services, in consultation with Dental Health Services Victoria, review the relevance and appropriateness of current key performance indicators in the Statement of Priorities and identify more relevant indicators for providing a comprehensive picture of the impact of the dental health program.
The SoP is a document signed by DHSV and the Minister for Health that holds DHSV accountable for achieving KPIs for the dental program and targets set by the government in the financial year.
DHHS did a review of the SoP KPIs as part of its review of KPIs in the 2017–18 Budget Papers. This identified two new performance measures, both of which are included in the 2018–19 SoP. This review is discussed further below.
The 12 KPIs in the 2018–19 SoP that focus on access and timeliness of care include measures related to treatment of emergency triage patients, recall interval and waiting times for general and denture care, and numbers of patients treated. The 2018–19 SoP does not include any performance indicators for improved oral health outcomes.
Given DHSV's transition to a VBHC approach, and its work with ICHOM to develop a standard set of oral health outcome measures, it should consider including appropriate outcome performance measures in its SoP. This will help improve the comprehensiveness of its reporting on the impact of the dental health program.
State Budget Paper 3
That the Department of Health and Human Services, in consultation with Dental Health Services Victoria, review the usefulness of the current key performance indicators in the State Budget Paper 3: Service Delivery and identify more relevant indicators for providing a comprehensive picture of how public dental services are delivered.
The BP3 KPIs include the actual and targeted annual performance of the dental health program against specified activity-based measures.
Following our 2016 audit, DHHS undertook two reviews of the dental health program performance measures, one of which was externally commissioned, followed by an internal review. These reviews informed changes to the 2017–18 BP3 measures, including:
- discontinuing the measure 'ratio of emergency to general courses of dental care', because it does not show whether emergency care has been timely, as reported in our 2016 audit
- adding a replacement measure, 'percentage of dental emergency triage category 1 clients treated within 24 hours'. While this measure only includes one of three categories of emergency patients, it is the most important performance measure as it represents the most urgent need
- adding a new measure, 'number of priority and emergency clients treated', which shows the extent to which public dental services are achieving the policy intent of treating priority patients. DHHS considers that this helps improve the comprehensiveness of reporting on public dental service delivery.
The 2019–20 BP3 measures also include the 'number of children participating in the Smiles 4 Miles oral health promotion program'. This measure increases the profile of oral disease prevention activity by CDAs. It is also aligned with the new VBHC approach to public dental health service delivery.
Compared with earlier reporting periods, the current BP3 measures have a greater focus on the timeliness of public dental service for the most urgent emergency patients and the extent of service delivery to priority groups. This enhances the usefulness of the suite of KPIs. However, as reported in our 2016 audit, the current BP3 measures still do not provide a comprehensive picture of public dental service delivery because there are no indicators that enable an assessment of how well services objectives have been met. We note that DHHS is continuing to review and trial further measures.