Effectiveness of the Navigator Program

Tabled: 23 March 2022

Snapshot

Is the Navigator program effectively re-engaging students in education?

What we examined

  • Department of Education and Training (DET’s) management of Navigator
  • Delivery of Navigator in four DET areas by service providers Jesuit Social Services, Berry Street, Mission Australia, and the Northern Mallee Local Learning and Employment Network
  • student outcomes.

Why this audit is important

Students who are disengaged from learning are at high risk of leaving school early. The Navigator program is designed to support Victoria’s most disengaged students aged 12–17 years. These students are often highly vulnerable with complex barriers to re engaging with school.

The program aims to reduce disengagement for students whose attendance was below 30 per cent in the previous school term, and to re-engage most of them in mainstream education with sustained attendance above 70 per cent.

DET contracts specialist youth services with expertise and resources not available in schools. These providers work with young people and their families to return students to education or training. 

A Navigator pilot commenced in 2016 and it was rolled out statewide in 2021.

What we concluded

DET cannot demonstrate Navigator is an effective intervention at a program level or that it is delivered equitably.

DET’s data collection means that it cannot clearly demonstrate Navigator’s effectiveness over time. This can be improved through better data linkage and analysis.

Students do not have equitable access to Navigator. Students' access to Navigator varies depending on where they live—referrals of eligible students vary across the state, as does the support students receive at school before being referred to Navigator.

Our recommendations

We made four recommendations to DET:

  • three about access to the Navigator program
  • one about improving the program’s effectiveness.
     

Video presentation

Video transcript

Key facts

Selected key facts from the report.

What we found and recommend

We consulted with the audited agency (Department of Education and Training) and associated entities and considered their views when reaching our conclusions. Their full responses are in Appendix A. 

Not all students have equitable and timely access

Referrals to Navigator vary across Victoria

Some areas refer about 40 per cent of eligible students and some refer less than 15 per cent. Statewide, the proportion of eligible students referred to Navigator is about 21 per cent.

This indicates inconsistent school practice in referring students to Navigator.

However, DET does not communicate to schools whether it expects them to refer all eligible students. DET also does not monitor referral rates to help it understand this variation.

Only a quarter of students received specialised DET support before referral

DET expects that schools will support students who disengage from their learning. It provides guidance and resources to schools to do this. 

DET expects that schools will increase their support as a student’s absences increase. By the time a student is eligible for Navigator, DET expects that they have been given individualised support. Schools can use their own wellbeing resources or DET's area based and specialised support, including Student Support Services, to do this.

We found that for students referred in 2019, three-quarters had not received individualised support from DET's Student Support Services, which includes social workers, visiting teachers, psychologists and other allied health professionals. This indicates that not all schools make full use of DET’s student support programs and workforces.

It is likely Navigator is less effective when students do not receive earlier individualised support for their disengagement. 

DET does not manage Navigator to meet variation in demand

Not all referred students receive timely access to Navigator. Demand for Navigator exceeds the number of available places in the program across Victoria and some students wait longer for services, depending on where they live. 

Active hold means that a student is on a wait list and receiving limited support.

Areas with much higher student numbers and with higher prevalence of chronic absenteeism have more demand for Navigator. When areas have a high number of referrals, students wait longer for their referrals to go to the service provider. It can take up to six weeks for a referral to be finalised. Once the service provider receives the referral, students in areas of high demand can be on active hold for between four and six months.

This means that students may wait a very long time before they receive Navigator’s intensive case management services. Navigator representatives at schools reported in our survey that long wait times reduce the effectiveness of Navigator and its ability to meet the needs of students. This wait time may affect a school's decision about whether to refer a student to Navigator.

DET does not use data on demand and service provider capacity to identify likely demand for Navigator or to adjust substantially how many students Navigator can support in each area.

Recommendations about access to Navigator

We recommend that the Department of Education and Training: Response

1. develop a Navigator engagement strategy so that:

  • schools understand and consistently apply the Department of Education and Training's tiered system of support for highly disengaged students
  • area teams support and monitor schools to provide individualised support for highly disengaged students
  • students receive a timely referral to Navigator after receiving individualised support, whether school-based or using the Department of Education and Training's area teams

Accepted

2. improve oversight and follow-up of schools to ensure consistent application of the Department of Education and Training's tiered system of support and referral practices (see Part 2)

Accepted

3. monitor program demand and uses this information for continuous program improvement, including to:

  • reduce waiting times for intensive case management services 
  • address variation in service access across the Department of Education and Training areas (see Section 2.1).

Accepted

DET is not able to demonstrate how effective Navigator is at returning students to education

Very few students achieve the program target of a return to education at 70 per cent attendance for two terms. However, many Navigator students do re-engage with education at attendance rates lower than the program target and achieve a range of other positive outcomes.

We used a range of data to test Navigator’s effectiveness. Limited data collection over time and no linking between Navigator participant data and DET’s record of student attendance and achievement meant that we could not clearly assess Navigator’s effectiveness.

Limited historical data means DET does not capture all the positive outcomes Navigator students achieve

Navigator achieves other positive outcomes. These include students who increase their attendance at a lower attendance rate than Navigator's target, or older Navigator students who may exit the program to a positive pathway of training or work.

The service providers who support Navigator students see other improvements in students' wellbeing. Service providers observe that students improve their social and emotional wellbeing and have the support they need to manage difficult circumstances. They see that students who were not able to leave their bedrooms or connect with family and friends are able to socialise and begin attending school.

DET's data collection for Navigator has not been able to capture all these outcomes in detail.

DET can do more to monitor and evaluate Navigator’s effectiveness

DET does not have a clear monitoring and evaluation framework for Navigator or performance benchmarks for service providers. This means that DET also does not have a way to review program implementation to ensure that it is consistent across schools.

DET does not routinely link data from its Navigator Data Management System (NDMS) with its student attendance and achievement data. This means that DET does not have a detailed understanding of program performance over time. It also means that DET does not understand the characteristics of students more likely to be referred to Navigator and those more likely to be helped by it.

DET did not have an efficient way to collect information about student progress and outcomes before December 2019. It introduced the NDMS in 2020 but while this increased oversight it still did not capture all relevant outcomes, such as increases in attendance below the 70 per cent target. 

DET has recently upgraded the NDMS and it now collects more information about students' progress and the different outcomes they achieve. This means it can better understand what Navigator is able to achieve.

It is important for DET to understand Navigator service delivery and program performance over time. It can do this by regularly analysing the full range of data collected by the NDMS and linking data with its student attendance and achievement data.

Recommendations about Navigator's effectiveness

We recommend that the Department of Education and Training: Response

4.    develop and implement a monitoring, evaluation and reporting framework that:

  • establishes requirements for data submission and key indicators with clearly specified program and the Department of Education and Training data sources, business rules and targets, that fairly represent program achievement
  • identifies baseline performance levels so it can assess improvement in program performance over time
  • enables fair assessment and benchmarking of individual service providers in meeting Navigator's objectives and outcomes
  • enables routine public reporting against the achievement of Navigator outcomes
  • allows for continuous improvement in the design and implementation of the program, including eligibility criteria. (see Section 3)

Accepted

Back to top

1. Audit context

Students with extremely low school attendance are at high risk of not completing their education. These students often experience many barriers to attending school and engaging with learning. They need help to overcome these barriers and stay in education.

DET provides resources and programs to schools to provide support so that disengaged students can stay at school. DET offers Navigator as a program for students when they need intensive and tailored support and have very low attendance.

This chapter provides essential background information about:

1.1 The Navigator program

Navigator is a program to reduce disengagement for students aged 12–17 years whose attendance is below 30 per cent in the previous school term. It is a program of last resort where earlier intervention for a student’s disengagement has not succeeded.

A Navigator pilot program commenced in 2016 and was rolled out statewide in 2021. Navigator is open to all students in government and non-government schools in Victoria.

Intended outcomes

Navigator aims to achieve the following outcomes:

  • Re-engaging young people with education.
  • Developing students with greater social and emotional capabilities.
  • Supporting schools to be better equipped to engage all young people.

Outcome targets for Navigator students are defined by DET in October 2021’s Navigator Operating Guidelines (the Guidelines). Of the students who receive services:

  • seventy per cent increase their attendance rate or newly enrol in an education setting, and of these 50 per cent achieve educational re-engagement 
  • seventy per cent report an increase in social and emotional capabilities, resilience and personal skills after receiving tailored program support.

Re-engagement is defined as 70 per cent or more attendance for two school terms or equivalent.

DET updated its Guidelines recently to state that some Navigator students may not meet the attendance target but still achieve a positive outcome of education re engagement. This may include flexible learning options and other education settings such as TAFEs or Virtual Schools Victoria where the approved curriculum is delivered. 

1.2 Who Navigator is for

The Navigator program works with the most severely disengaged students in Victoria. 

DET categorises students’ risk of disengagement from school according to absence patterns. Figure 1A shows how DET defines categories of student absence in its Student Attendance Guidance 2021.

FIGURE 1A: DET's categories of student absence

Category of student absence Days of school missed per year
Regular attendee Up to 10 days
At risk of chronic absence 10–19.5 days
Chronically absent 20–29.5 days
Severe chronic absence More than 30 days

Source: DET.

Severe chronic absence (more than 30 days per year) averages to an absence of eight days or more per term. Based on an average term length of 50 days, this means the student has been absent for at least 16 per cent of the term.

Students become eligible for Navigator when they have been absent for at least 70 per cent of the previous school term. This is 27 more days in a term over and above DET's criteria for severe chronic absence.

The result of applying the absence criteria is that students eligible for Navigator are severely disengaged learners with complex barriers to education engagement. They are highly likely to need significant mental health support, disability assessment and family support services. Some Navigator students need support for trauma, family violence, sexual abuse and alcohol and other drugs.

1.3 How Navigator is delivered

DET delivers Navigator through a partnership with contracted service providers, schools and other education providers. 

DET has a central team that manages Navigator plus Navigator coordinators who work in each of DET's 17 geographic areas. See Appendix D for a map of DET areas. The Navigator coordinator is the primary contact for service providers and manages the relationship between service providers, schools, student wellbeing area teams and other key services.

Navigator contracts specialist youth services that have expertise and resources not available within schools. There are 10 service providers delivering Navigator across Victoria, selected by DET through a competitive tender process.

Funding

DET funds Navigator based on a fixed and equal amount per area. This means that there is base funding for service delivery and a Navigator coordinator in each area. There is a central DET program management team.

DET adopted a differentiated funding model to distribute additional Navigator funding it received from government for 2021 and 2022. This funding was provided in response to COVID-19 impacts on students. DET used this to increase the number of students Navigator could support, and for extra mental health support and loadings for certain student cohorts.

DET distributed this additional funding based on average service demand for the previous two quarters and on characteristics associated with increased risk of chronic absence, such as Aboriginal and Torres Strait Islander status, youth justice involvement and the number of disengaged learners in the area. 

Augmented Navigator is an additional $460,000 over two years to support disengaged young people from culturally and linguistically diverse backgrounds in southern and western Melbourne and is focused on young people at risk of offending. Because of this additional funding, these two areas receive slightly less differentiated funding.

Figure 1B shows the base and differentiated funding components, their amounts and areas for the calendar years 2021 and 2022.

FIGURE 1B: Navigator funding components per calendar year: 2021 and 2022

Funding component Funding amount ($) Distribution
Base funding 666,793 Per area service provider (17 areas)
2021 and 2022 funding boost (capacity) 112,378 Areas with established service provision (14 areas)
53,530 Newly expanded areas in 2021 (3 areas)
2021 and 2022 funding boost (mental health support) 45,500 Areas with established service provision (14 areas)
21,000 Newly expanded areas in 2021 (3 areas)
2021 and 2022 funding boost (cohort) 100,000  Western Melbourne
Southern Melbourne
116,500  Brimbank Melton
Bayside Peninsula
Loddon Campaspe, Barwon, Goulburn
North Eastern Melbourne
34,625 Hume Moreland
Augmented Navigator 215,250 Western Melbourne
Southern Melbourne
TOTAL funding per calendar year 14,235,738  

Source: DET. Difference between category subtotal and total funding is due to rounding.

Referral to Navigator

Navigator has an open referral system. This means that anyone concerned by a student's low attendance can refer them to Navigator. In practice, most referrals to Navigator come from schools.

Navigator coordinators assess the referrals and ensure that students meet the program's eligibility criteria. They also consider whether the student is ready to receive services. Students do not have to participate in Navigator; case managers must have consent from the student's parent or carer—or the student if they are a mature minor—before providing services.

Sometimes, students are in a crisis situation when they are referred to Navigator. This may affect whether they are ready and willing to respond to Navigator's support. When this happens, Navigator coordinators put in place follow-up plans so they reassess students later. Navigator coordinators confirm these deferred referrals with area-based managers.

Case management and re-engagement

Service providers use their specialist staff to connect with young people and their families where schools are unlikely to have the capacity or skills to do so.

Service providers develop a re-engagement plan for each Navigator student. This is based on the student’s learning goals and describes the school and service provider support they need to achieve those goals.

When a student is about to exit Navigator, service providers and Navigator coordinators agree on an exit plan. This plan describes the support the student has received through Navigator and the kind of school-based support they will continue receiving. This is so the student will have ongoing support to maintain their connection to education.

When a service provider receives a referral from DET, they establish contact with the student and their family or caregivers, obtain consent to participate in the program and assess the student’s needs. DET expects that service providers will be assertive in making this contact.

The core services provided by Navigator are assertive outreach and intensive case management 

  • Assertive outreach means that the service provider seeks out the Navigator student using its expertise in youth and family engagement to establish contact with the young person and build a relationship with them. This contact is persistent and may take different forms (phone calls, messages, home visits) until they have established contact. If multiple attempts at contact (between 4–6 occasions) are unsuccessful, the service provider advises the Navigator coordinator of this.
  • Service providers deliver intensive case management services based on a young person’s needs and goals. This means that a case manager assesses and supports each Navigator student. The case manager refers students to additional services according to their needs and works closely with students to identify their education goals. Case managers advocate for students and work with schools to implement each student’s re-engagement plan.

Students may be supported with active hold services when the service provider is at case management capacity. This means that the student receives initial assessment and ongoing contact with the service provider. The provider may refer the student to other services before bringing them into their intensive case management service when capacity allows.

DET requires service providers to review students on active hold regularly. It does not specify a time limit for active hold, but Navigator coordinators work closely with service providers to monitor this.

Some service providers also offer a brief intervention service for students on active hold. This means they offer tailored and specific support to the student including counselling. This support is usually time-limited and it is not the same scope of support available through intensive case management.

1.4 DET support for student wellbeing and engagement

DET expects that schools will provide students at risk of disengaging from education with support. This means that DET requires schools to identify students with decreasing attendance and provide individualised support. This support may change over time if earlier support and interventions are not effective.

DET uses a three-tiered system of support (TSS) to categorise its student health, wellbeing and inclusion programs and interventions. At each tier, DET's area-based student wellbeing and engagement teams offer advice and resources to schools.

Navigator is a tier-three program because it provides participants with specialist and one-to-one support from community service organisations. DET expects that students referred to Navigator have previously received individualised school-based support under its tier-one and tier-two TSS services.

The tiers are:

Tier …

Is …

With these characteristics …

One
Universal support 

applicable to all students and based on preventative and health promoting activities

  • School-based programs and supports drawing on DET's policies, guidance and resources.
  • All students experience or may access these, such as Respectful Relationships or school-wide Positive Behaviour Support programs.

Two
Targeted support 

cohort-specific, with supports and interventions including some individualised support

  • Support delivered in schools, with specialised DET resources and some external resources.
  • Some students may access programs designed for students with similar needs. This includes Koorie students, those with disabilities or students in need of additional support services.

Three
Individual support 

highly targeted

  • Highly specialised support and interventions, drawing on specialised DET and community resources.
  • Some students are referred to specialist programs for one-to-one support, including Navigator.

Back to top

2. Accessing Navigator

Conclusion

DET does not have a process to monitor demand and adjust services based on factors such as varying need or differing student population in each of its areas.

Eligible students do not have equitable and timely access to Navigator. The majority of eligible students are not referred to Navigator, and some students wait much longer than others for support, depending on where they live at the time they are referred. Around three-quarters of students referred to Navigator have not received other specialised wellbeing support available through DET. 

Navigator is likely to be more effective when it is delivered consistently across the state so that referrals and access are timelier and more equitable.

2.1 Navigator program design and funding model

DET did not design Navigator on the assumption that every eligible student would access the program.

DET expects that some eligible students may refuse to participate. This is because some students are not ready for the support due to trauma or crisis situations. Others will have their individual needs met by their school or education provider through other tier-two and tier-three support. Older students otherwise eligible for Navigator may instead choose another education or training pathway.

DET analyses its attendance data to understand absenteeism. This shows that some areas have much higher numbers of eligible students than others—because some areas have a higher student population than others, and some areas have a higher prevalence of risk factors associated with chronic absence. These risk factors include low socio-economic status, low levels of parental education, language background other than English, and lower levels of student achievement.

But DET does not use its data to determine likely demand for Navigator, or to adjust substantially how many students Navigator can support in each area. 

As discussed in section 1.3, all DET areas receive the same fixed and equal amount of funding, although DET did adopt a differentiated funding model to distribute the additional Navigator funding it received from government for 2021 and 2022. 

2.2 Timely referral to Navigator

Not all students eligible for Navigator receive a timely referral to the program from their school or other person even though they meet Navigator’s age and attendance eligibility criteria.

We found significant variation in referral rates across Victoria, and this is likely due to inconsistent school practices. We examined Navigator referrals in term two of 2021:

  • Statewide, schools referred only 21 per cent of eligible students to Navigator at the end of this term.
  • Referral rates across DET areas in 2021 ranged from 10.6 per cent in North Eastern Melbourne to 41.3 per cent in Ovens Murray. 

This indicates inconsistent practice in identifying eligible students and making a timely referral to Navigator. 

DET does not communicate to schools whether it expects them to refer all eligible students to Navigator. DET does not monitor referral rates to help it understand this variation.

We surveyed Navigator coordinators and asked them whether they thought schools in their area were aware of Navigator:

  • Only 10 per cent thought that all schools were aware of Navigator.
  • 50 per cent thought most schools were aware of Navigator.
  • One third thought only a few or some schools were aware of Navigator.

Based on DET school attendance data for term two of 2021, we identified for each area the proportion of students aged 12–17 whose number of absence days (more than 35 days) made them eligible for Navigator in that term. We then calculated the proportion of these eligible students who went on to receive a referral to Navigator.

Figure 2A shows the proportion of students in each area that met Navigator’s age and attendance eligibility criteria and the proportion of those students referred to Navigator. For example, in Barwon, 2.3 per cent of students aged 12–17 were eligible for Navigator and, of these eligible students, only 19.2 per cent were referred to Navigator.

Statewide at the end of term two of 2021, 3,210 students or 1.3 per cent of the student population aged 12–17 met Navigator’s attendance criteria. Of these students, 667 (20.8 per cent of eligible students) received a referral.

FIGURE 2A: Proportion of eligible students referred to Navigator, term two 2021

Area Proportion of students eligible for Navigator (per cent) Proportion of eligible students referred to Navigator (per cent)
Barwon 2.3 19.2
Bayside Peninsula 0.9 20.1
Brimbank Melton 0.9 17.7
Central Highlands 1.8 23.4
Goulburn 2.4 26.3
Hume Moreland 1.5 15.8
Inner Eastern Melbourne 0.8 24.3
Inner Gippsland 1.8 18.3
Loddon 2.3 24.8
Mallee 2.3 13.9
North Eastern Melbourne 1.2 10.6
Outer Eastern Melbourne 1.0 29.4
Outer Gippsland 1.6 35.5
Ovens Murray 1.2 41.3
Southern Melbourne 1.0 27.6
Western Melbourne 1.2 15.6
Wimmera South West 1.9 17.6
Statewide 1.3
(3,210 students)
20.8
(667 of 3,210 students)

Source: DET attendance data.

2.3 Inconsistent use of early engagement support

Service providers and Navigator coordinators observed that not all schools support students in line with DET’s TSS for student engagement. Some schools provide effective, individualised engagement support and others do not. Our survey of school-based contacts also showed this variation in practice. We did not directly assess schools’ engagement and wellbeing practices.

We analysed DET data and found that around a quarter (24.5 per cent) of students referred to Navigator received individualised support from DET’s area-based Student Support Services prior to referral.

This indicates that schools may not have a consistent understanding of or use the full range of support available for disengaged students before they become eligible for Navigator.

School practices are not always consistent with DET guidance

DET provides schools with guidance and resources for student engagement. This includes advice on applying its TSS for individual students and accessing services from DET’s health, wellbeing and inclusion teams. DET has area-based teams that can support schools to develop and implement whole-of-school and individualised strategies for student engagement.

If schools do not refer students appropriately for additional support when issues first appear, tier-two intervention may not be as effective and may increase demand for tier-three interventions, including Navigator. It may also mean that schools do not refer students for tier-three intervention early enough for intervention to be as effective as possible.

Navigator coordinators observed that schools varied considerably in their implementation of tier-one and tier-two support despite the availability of programs to provide effective engagement structures. The coordinators also observed that tier two was where students ‘get lost’ in the system of support. This means that students with chronic absenteeism are not receiving individualised support before they become extremely disengaged.

We examined whether Navigator participants had a history of individualised support from area-based student support services prior to program referral. Given the complexity of the Navigator cohort and evidence of entrenched absence patterns and poor achievement, it is reasonable to expect many Navigator students to have a history of prior support from these specialised services.

We examined rates of referral to DET’s health, wellbeing and inclusion teams for students who were referred to Navigator in 2019. Schools make referrals to these services and this is recorded on DET’s Student Online Cases System. 

We found that only 24.5 per cent of the students in the Navigator Disengaged Students Register, an historical database of Navigator students, had received a school referral to these services before a referral to Navigator (340 of 1,389 students). Such a low proportion of the Navigator cohort receiving this formal support shows that available DET support programs and resources are not fully used.

2.4 Timely access to service once referred to Navigator

Not all students referred to Navigator receive timely access to Navigator services. Demand for Navigator exceeds program capacity across Victoria and we found that some students wait longer for services depending on where they live.

We tested likely demand for Navigator and service provider capacity to respond to this demand. We did this for two metropolitan and two regional areas. 

We defined capacity as the maximum total number of participants set out in each service provider’s funding agreement with DET.

We tested demand in two ways:

  • The number of students meeting Navigator eligibility criteria as at the end of term two of 2021, even if they did not receive a referral.
  • The number of students actually referred at the end of term two of 2021.

We understand that these are tests for point-in-time demand. They do not assess the flow of students through the program over time. We were not able to assess demand compared to real-time service provider capacity.

Figure 2B shows the results of the first test, comparing total eligible students with program capacity.

FIGURE 2B: Comparison of proposed point-in-time maximum case loads in 2021 with demand in four areas

Area Active hold: maximum capacity Case management: maximum capacity Maximum total participants Area demand in term two of 2021 Demand in excess of capacity
A 8 136 144 122 −22
B 15.2 60.3 75.5 169 +93.5
C 50 50 100 130 +30
D 25 110 135 284 +149

Source: DET 2021 service agreements and DET attendance data for term 2 2021 (students aged 12–17).
Note: DET area names were anonymised to avoid identifying service providers.

For both tests, demand significantly exceeded program capacity for three of the four areas. For two metropolitan areas, demand exceeded program capacity by more than 100 per cent.

Demand in excess of program capacity means that students may wait a very long time before they receive Navigator’s intensive case management services. For some students, this wait is more than six months.

When areas have a high number of referrals, students wait longer for that referral to go to the service provider. We found that it may take up to six weeks for a referral to be finalised. Once the service provider receives a referral, a student in an area of high demand can be on active hold between four and six months. 

DET did not have sufficient data to test whether there was an association between lower rates of referral and longer waiting times. Some comments from school-based contacts suggested that longer waiting times may affect their decision to refer a student to Navigator.

Back to top

3. Navigator outcomes

Conclusion

DET cannot demonstrate Navigator’s effectiveness. Navigator is not achieving its intended outcome of most students re-engaging in mainstream education with sustained attendance above 70 per cent.

Navigator students receive tailored support that should help them stay connected to education and achieve other positive outcomes. However, DET does not have sufficient data on Navigator’s full impact on students to understand the range of positive program outcomes. 

DET can use data better to monitor Navigator outcomes and improve program delivery.

3.1 Outcomes for Navigator students

Very few students achieve the program target of returning to mainstream education with high sustained attendance. However, many Navigator students do re-engage with education with attendance rates lower than the program target and achieve a range of other positive outcomes.

DET’s data is insufficient to identify for whom Navigator is most effective and does not capture the full range of program outcomes for Navigator students. Recent changes to DET’s data collection and management means that DET will have a better understanding of students’ progress and outcomes in the future.

Identifying Navigator outcomes

We assessed a range of evidence to understand the effectiveness of Navigator and whether it was achieving its stated outcomes.

We analysed Navigator data from 2016–19. Our analysis was limited by changes to data collection and outcomes reporting during that period.

We analysed Navigator data for students referred in 2019 and linked some of this data to DET’s records of student achievement and attendance. We interviewed service providers and analysed their case management data.

We note that disruptions to student engagement happened during periods of remote learning due to COVID-19 restrictions over the time period this audit examined. These restrictions also disrupted Navigator service delivery. Feedback from service providers suggested remote learning was beneficial for some Navigator students but not for others. It was not possible to account for these differences in conducting our analysis of outcomes.

Outcomes identified from Navigator data

DET provided aggregate data for 2,781 students in the program from 2016–19:

  • 372 or 13.3 per cent of students who received case management successfully completed the program, achieving 70 per cent attendance for 26 weeks.
  • 139 or 5 per cent maintained 70 per cent attendance for six months after a successful exit.

We aggregated the 2019 Navigator program activity data to better understand the quantitative return to education and exit outcomes for students referred to Navigator in that year. We first noted a difference in the recorded numbers of students referred to Navigator in 2019 that DET has in different sources: 

  • There were 1,389 referrals recorded in the Disengaged Students Register (the Navigator database prior to 2020).
  • There were 1,644 in quarterly reports from service providers. We used data from these quarterly reports to identify Navigator outcomes.

DET introduced the NDMS online data reporting system in 2020 but migrated limited data from 2016–19. This meant we could not identify the outcomes for the students who continued to receive services after 2019.

Figure 3A shows the progression of these students in 2019 according to reports from service providers. Students who did not exit the program in 2019 (n=993) continued to receive services. Changes to DET’s data collection system mean that we could not identify outcomes for these students and they are not included in the results below.

It is likely that some of these students exited the program successfully or achieved other positive outcomes. This would mean that more than 38 students from the 2019 cohort successfully completed but we were unable to identify them.

For those students who received Navigator services in 2019, we found that about half returned to education. This means they increased their attendance compared to their attendance before Navigator.

Only a few (38 of 1,337, or 2.8 per cent) had a successful completion in the same year. This result is not unexpected given that most Navigator students require more than two terms of support before they are able to achieve two terms of sustained attendance above 70 per cent. A successful completion in the same year was not possible for any student referred to Navigator after the beginning of term three in 2019. This is because successful completion requires at least two terms of high sustained attendance.

DET's Guidelines for Navigator do not place a time limit for receiving Navigator services. They require case managers to conduct case reviews with Navigator Coordinators after six months of support, and then every three months.

FIGURE 3A: Progression through Navigator in 2019 for students referred in the same year

Navigator exit Number of students %
Referrals received by service providers 1,644  
Less refused consent or unable to contact −307  
Received Navigator services in 2019 1,337  
Returned to education (increased attendance) 661 49.4
Successful exit 
(sustained 70% attendance for 26 weeks)
38 2.8
Exit before completion 306 22.9
Continuing students after 2019 993 74.3

Source: VAGO, using DET quarterly reporting data. Shows students referred to Navigator in 2019 and their progression in the same calendar year. Students who could not be contacted may have been contacted in a later quarter.

Figure 3B shows, in descending order, the reasons for exits before completion.

FIGURE 3B: Reasons for exiting Navigator before completion

Reason for exiting before completion Number of students %
Revoked consent or completely disengaged from contact 117 38.2
Alternative planned positive pathways* 91 29.7
Moving to an area where Navigator is not available 32 10.5
Request by service provider or student to cease service as re-engaged or achieved a positive outcome 29 9.5
Transferred to another area or provider 20 6.5
Other reason 17 5.6
Total 306  

Note: * DET defines an alternative positive pathway as a training, non-school education or employment pathway.
Source: VAGO, using DET quarterly reporting data. Shows students referred to Navigator in 2019 who exited the program before completion in 2019 (n=306). 

DET uses student entry and exit surveys to monitor progress towards the intended qualitative outcomes of increased learning confidence, motivation, resilience and school connectedness.

The response rate in 2019 for students who were referred, received services and exited in 2019, was very low (16.6 per cent) and these conclusions may not be representative of the whole cohort. Of the 57 students who completed both surveys, between 47 and 50 students reported improvement in each of these domains.

Outcomes identified from Navigator data and linked student data

We used 2019 program data to identify whether Navigator was more effective for some students than others. We linked this data to DET’s data on student enrolment and attendance to measure the rate at which Navigator students maintained their school enrolment. 

We found that Navigator was effective at helping highly vulnerable students in the 2019 Navigator cohort to remain connected to education. A slightly higher proportion of disadvantaged, Koorie, out-of-home care and students in the Program for Students with Disability who received Navigator services maintained their school enrolment than Navigator students who were not in these groups.

We also performed a retrospective cohort analysis of Victorian students who met Navigator eligibility criteria using DET school attendance data. Our analysis tested the proportion of students in this cohort who increased their attendance over a two-year period from term two of 2019. 

Our analysis showed no statistically significant difference in attendance between areas that have Navigator and areas that do not. DET advised that it does not intend that Navigator affect attendance rates at a population level. The lack of a statistical difference may be explained by differences in student population across areas, as well as by very low numbers of students participating in Navigator in 2019, rather than the presence of the program in an area.

Attendance is a proxy measure of success that can be applied to Navigator and non Navigator students. Data limitations meant that we could not compare outcomes for Navigator and non-Navigator participants in each area. This is because we were unable to link Navigator data to DET’s attendance data. Without this link, we could not track individual Navigator participants and compare their results with individual non-Navigator participants. 

Our study demonstrates the type of analysis DET could do routinely if it linked Navigator and attendance and achievement data, which it currently does not. This would be possible when a student’s Victorian Student Number was recorded in the NDMS.

Data linkage would make it possible to:

  • monitor long-term trends in program performance
  • identify factors that influence program effectiveness
  • use the information to improve the program.

Positive outcomes observed by Navigator service providers

DET expects that Navigator students will have improved wellbeing after receiving intensive case management services. It defines this program outcome as increases in:

  • social and emotional capabilities
  • resilience
  • personal skills.

DET’s entry and exit surveys collect information about these outcomes. Manual data collection methods, low response rates, and changes to data collection over time make it impossible to reliably track students’ outcomes in these areas.

We asked service providers about positive outcomes they observe in Navigator students. They identified a range of positive impacts and confirmed that they monitor student progress and record positive outcomes in their case management systems. They do this through case notes and tracking student responses to wellbeing questionnaires. Case managers do not upload their notes to the NDMS, to protect student privacy.

Most of the service providers reported monitoring wellbeing outcomes based on their experience in youth social work and aligned this with their practice models. For example, one service provider we examined monitors participant progress across domains including:

  • connection to family and friends
  • view of the future
  • strength of social and recreation networks
  • engagement with education or learning
  • vocational pathways.

Service providers may make observations about student progress in the NDMS notes function. DET now requires service providers to record students’ responses to entry and exit surveys in the NDMS. This means that DET can better track program outcomes in wellbeing.

Service providers gave examples of the type of improvements they have observed in student wellbeing. Service providers often described students at the beginning of the program as completely withdrawn from school and with poor or no connection to friends and family. Some participants are unable or unwilling to engage with support services or identify learning goals for themselves.

After receiving these services, students felt connected with their families and schools. They achieved education milestones such as increasing their school attendance and transitioning from one year level to another. Older students were able to obtain employment or apprenticeships.

3.2 Factors that contribute to Navigator's effectiveness

Research literature is clear that intensive and individualised case management support is likely to help students return to education. It suggests that students receive this support as soon as schools identify severe chronic absenteeism. The literature defines this as missing more than 20 days of school a year. Students are eligible for Navigator when they miss 35 days of school in a term (an average term is 50 days).

The literature does not suggest that access to this type of support should be determined by the student’s age.

Younger students may need Navigator for less time than older students

Evidence from service providers indicates that the longer a student is disengaged from education the longer it takes to establish and implement re-engagement goals.

Interviews with service providers indicate that case complexity, combined with entrenched non-attendance, makes it more likely that students referred at an older age require Navigator support for longer.

This suggests that prompt referral of a student to Navigator is more likely to help students re-engage with school sooner. This means that schools refer students as soon as they meet age and absence eligibility criteria. Our study of referral rates (see 2.2) indicates that prompt referral does not happen for most eligible students.

We tested whether earlier age at referral increases Navigator’s effectiveness. We did this by estimating duration of service for students in Navigator in April 2019 (Figure 3C). The 2019 Navigator census found that:

  • the 12–13 age group had the lowest proportion receiving Navigator services for at least a year
  • the 15–18 age groups had a higher proportion receiving the service for at least a year. 

We expect duration of service to decline for participants aged 16 and older. This is because students exit the program at 17 years old.

FIGURE 3C: Estimate of duration of service for Navigator students in April 2019, by age

Age Proportion of students supported for 13 months or longer (%) Proportion of all students in Navigator at 30 April 2019 (%)
12–13 5.8 17.0
14 21.1 23.0
15 34.6 28.3
16 25.6 24.3
17–18 13.5 7.5

Source: DET 2019 Navigator census report. Age is as of 30 April 2019. Results may not sum due to rounding.

We examined return to education data to see if there were any differences based on student age.

Navigator data from June 2021 shows most students returning to education have done so from case management. However, students who returned from active hold are younger (median 14 years) with a narrower spread of ages than those from case management (median 15 years).

Figure 3D shows the ages in June 2021 of students who have returned to education but not yet exited the program. Diamonds in the chart are outliers. This suggests that intervention at an earlier age may mean that students require less intensive support to return to education in school settings.

FIGURE 3D: Age in June 2021 of students with a return-to-education status

Figure 3D: Age in June 2021 of students with a return-to-education status

Source: VAGO, using DET NDMS data (n=53 for return to education from active hold, n=431 for return to education from case management). Excludes one student in return to education from active hold and seven students in return to education from case management who were entered into the NDMS with a year of birth of 2020. Age is as of June 2021, not return to education date. 

Older students may be more likely to refuse Navigator services

We found that entrenched severe chronic absence and older age at referral is associated with service refusal.

Our analysis of the 2019 Navigator cohort shows that students referred to Navigator in 2019 who refused or did not receive services had a higher average number of school days missed in 2017 and 2018. Students who did not receive services missed an average of 133.8 school days compared to 125.1 days for students who did receive services.

In general, the proportion of students in the cohort who did not receive services at the time of referral increased with year level. See Figure 3E.

FIGURE 3E: Percentage of students who did not receive Navigator, by year level

Year level Did not receive Navigator Total students Proportion who did not receive Navigator
5 0 2 0%
6 4 25 16%
7 11 124 9%
8 15 157 10%
9 25 227 11%
10 28 190 15%
11 14 57 25%
12 1 1 100%
Ungraded 2 23 9%

Source: VAGO, using DET data of students referred to Navigator in 2019 according to the Disengaged Student Register. Includes only students who could be matched to a Victorian Student Number. 

3.3 Using data for Navigator improvement

DET does not analyse program performance using its data on student achievement and attendance. This means it does not have a good understanding of long-term student outcomes. It also means that it is not able to understand the factors that help students achieve Navigator program outcomes.

Historically, DET has a poor understanding of program performance due to changes in its reporting and data capture systems, data limitations and the absence of a link between Navigator and DET’s records on student attendance and achievement.

This means that it was not possible for DET to understand for whom Navigator was successful or reliably compare service delivery.

DET has recently updated its NDMS and service reporting. This should help it better understand:

  • referrals to Navigator, including referral rates by school and area and the school support given before referral
  • students’ progress, including the type of service they receive and incremental changes to their attendance rates
  • student outcomes, including re-engagement in mainstream education settings as well as alternative positive outcomes.

DET should be able to monitor and compare outcomes for Navigator students based on a range of characteristics such as age at referral, date of referral, area, service provider, and cohort such as students with disability, Koorie students, students in out-of-home care, and low socio-economic status.

DET’s changes to the NDMS mean that it should have reliable data to compare student progress and outcomes across service providers. Data should be sufficiently reliable to allow DET to identify service delivery benchmarks.

These recent changes do not include recording a student’s Victorian Student Number. This will make it difficult for DET to link Navigator data to its student attendance and achievement data.

Back to top

Appendix A. Submissions and comments

Click the link below to download a PDF copy of Appendix A. Submissions and comments.

 

Download PDF

Click here to download Appendix A. Submissions and comments

Back to top

Appendix B. Acronyms, abbreviations and glossary

Acronyms  
DET Department of Education and Training
NDMS Navigator data management system
TSS Tiered system of support
VAGO Victorian Auditor-General’s Office

Back to top

Appendix C. Scope of this audit

Who we audited What the audit cost
DET, Jesuit Social Services, Berry Street, Mission Australia, Northern Mallee Local Learning and Employment Network The cost of this audit was $685,000

What we assessed

The audit followed the following lines of inquiry and criteria:

Line of inquiry Criteria
DET’s planning, management and oversight supports service providers to deliver Navigator to eligible students 1. DET's program design and funding model is based on a sufficient understanding of demand for Navigator and the needs of its target cohort so that eligible students have timely access to the program.
2. DET's service agreements have clear funding guidelines, deliverables, performance measures and targets to achieve Navigator's intended outcomes. 
3. DET's program and service agreement monitoring, evaluation and reporting enables assessment of the achievement of Navigator’s intended outcomes and supports continuous improvement.
DET and service providers’ delivery of Navigator results in students re-engaging with education or training 1. Service providers deliver intensive case management tailored to the needs of each Navigator student and identify and access additional services where needed.
2. DET and service providers involve schools in preparing student re-engagement plans and schools implement these as intended.
3. DET and service providers provide schools with guidance and support them to improve whole-of-school practices that reduce student disengagement and sustain re-engagement.
4. Navigator is achieving its stated outcomes.

Audit scope

This was a follow-the-dollar performance audit, which means we audited DET and agencies that provide services to Victorians on behalf of DET. The audit examined DET's implementation of the Navigator program and the extent to which it is achieving its intended outcomes.

In addition to DET, we selected four of ten service providers who receive funding from DET to deliver Navigator. We chose these four services providers, listed in the table above, based on their geographic location, service provider types and the likely factors contributing to student disengagement in their area.

Our methods

As part of the audit we:

  • analysed data from DET and four service providers to assess Navigator’s effectiveness
  • reviewed DET’s policies, procedures and plans and assessed whether they were effectively supporting Navigator delivery
  • surveyed school staff with experience of Navigator
  • surveyed DET’s Navigator Coordinators
  • interviewed key staff in the department and service providers.

Compliance

We conducted our audit in accordance with the Audit Act 1994 and ASAE 3500 Performance Engagements. We complied with the independence and other relevant ethical requirements related to assurance engagements.

Unless otherwise indicated, any persons named in this report are not the subject of adverse comment or opinion.

We also provided a copy of the report to the Department of Premier and Cabinet.

Back to top

Appendix D. Map of DET areas

FIGURE D1: Map of DET areas

This image shows a map of Victoria broken down into DET's areas.

Source: DET Region Map.

Back to top

Appendix E. Survey of DET Navigator coordinators and school-based contacts

We conducted two surveys for this audit. 

  • We surveyed DET Navigator coordinators in June and July 2021.
    We received responses from all 19 coordinators, representing all of the 17 DET areas.
    The survey included 23 questions asking about their experience and observations delivering the Navigator program in their area.
  • We surveyed school-based Navigator representatives in June and July 2021.
    We received 100 responses, representing 15 of the 17 DET areas.
    The survey included 35 questions asking about their experience and observations of the Navigator program from the school perspective. 

Back to top