Opinion: Supporting success in the TEF with engagement analytics
Janice Kay, Special Advisor to the Vice-Chancellor of the University of Exeter and to Kortext, discusses the importance of using digital platforms and engagement data to enhance educational gain and student outcomes.
As a panel member of the Teaching Excellence Framework (TEF) exercise, I’ve spent a good part of the past year reviewing the state of learning and teaching in England’s higher education providers with my colleagues. It’s a great story.
One thing I’ve taken away from the exercise is that providers who have been successful in the TEF have a strong narrative about how well they understand their students and work with them, leading directly to a better student experience, student outcomes and educational gains.
Photo by Brooke Cagle
Perhaps the most fundamental thing for an institution to get right is student engagement. Students can be self-starters, learning early on in their school careers how to support their learning style and knowledge acquisition to get the best out of their education. But for many, engagement in learning at a higher level is neither immediate nor straightforward, and for a variety of complex reasons. This means that a good transition to learning is vital. If a good transition is not achieved, or if engagement is left to chance, students are much more likely to drop out. Withdrawal rates for institutions are published as part of TEF data indicators and benchmarked through continuation and completion data, and poor withdrawal rates can result in considerable loss of income. So, institutions must understand what student engagement looks like across their student cohorts before this happens.
Higher education providers have thought long and hard over the years about how they engage and support learners. The TEF also examines the nature of academic support and learning environment via data and what the provider says about itself. A majority of support across institutions is good and thought through. However, whether institutions fully understand the quantitative pattern of learner engagement, and which students may be likely to withdraw is a genuinely open question. It can only be answered fully through access to analytic data and digital-first solutions. There are some very good examples of institutions that are using analytic data to pinpoint those who are only marginally, partially or fully engaged in their learning (Teesside University and Anglia Ruskin University, for example, are both Gold providers overall in TEF). This can be assessed analytically through engagement with learning materials or engagement in terms of attendance of in-person and online classes, submission of work and use of VLE and other tools. Kortext and StREAM are good examples of digital learning technologies that can do this.
Building engagement
Identifying the pattern of engagement across a student cohort and how to evaluate and demonstrate improvements is an important part of understanding educational outcomes. Building engagement through digital-first solutions creates key issues yet to be fully investigated about the nature of student support structures that follow. While some institutions focus on the tried and tested Academic Tutor System, others are developing specialist professional support, through student advisors and specialist teams. Some of these operate at the department level, some at the faculty or college level and some at the institutional level. We need to understand which systems work best and for which students in ways that are cost-efficient and effective. A good individual example of how this is being addressed is at Teesside University who have brought in a Student Success Programme with specialist student success tutors, and this is reflected in their TEF Provider Submission.
Failure to engage in learning can be precipitated by mental welfare issues. There is a question of whether digital systems of student engagement can appropriately interface with student welfare services, and this will require considerable thought and evaluation. But we know that student mental health issues have increased and are likely to continue to increase, because of the pandemic, cost of living and cost of debt. This is not just a national problem. A recent post by Stanfield and Tay (2024) describes a 2022 study using data at the height of the Covid-19 restrictions which revealed that three out of four students at Singapore’s flagship institution, the National University of Singapore (NUS), were at risk of depression, and over 83% cited high levels of stress. Building engagement and more tailored student learning and welfare support are only going to increase in need.
Educational gains
TEF2023 for the first time looked at how higher education providers capture the education gains of their students. Future iterations will likely ask providers to report on their approach to understanding and tracking educational gains, particularly around demonstration and evaluation. While educational gains can be defined in lots of ways – through degree and progression outcomes relative to entry profile; work readiness and employability; graduate attributes; skills and competencies – student engagement is fundamental to understanding the processes by which students develop and how their learning is enhanced.
The best explanations of educational gain that we saw in TEF submissions demonstrated a real understanding of students – and a tangible and evidenced commitment to keeping them on track. The use of technology was critical in generating opportunities for consistent engagement and in tracking and evidencing impact. Learning engagement can be used to best effect to demonstrate and evaluate educational gains. But we also saw several examples where providers had adopted digital technologies such as engagement analytics – but were much less able to explain how those analytics supported the core learning engagement agenda. Digital-first doesn’t mean technology-led, it means digitally enabled engagement in learning.
If TEF2027 goes ahead, and it looks as if it may, educational gains beyond employability skills and graduate attributes are going to be front and centre. This obliges universities to be clear about the educational benefits for their students of studying at their university. Universities must work hand in hand with their students to co-create a very clear ‘offer’. This is necessary but not sufficient. Universities must also be able to evaluate what their students achieve, as part of demonstrating gains. This will require predictive digital technologies.
A technologically challenged sector
With fragmented and dysfunctional legacy systems sometimes chosen for cost rather than for transformative learning and teaching, we are certainly a technologically challenged sector. Since the pandemic, as financial pressures on higher education have become even more intense, we’re witnessing a creep away from digital-first solutions – it has often been much easier to substitute hiring more people for effective digital systems. But despite the challenges, now is the time for boldness. Digital technology makes it much more possible to drive learning forward in cost-effective ways and to build outstanding student learning experiences for much better engagement.
Technology also allows us to build a picture in increasingly fine-grained detail of what successful education processes look like – as students interact with their lecturers, campus services, each other, and learning resources online. The practice of tracking learning engagement to support retention, well-being, and success is nowhere near reaching its potential to deepen our understanding of what learning processes look like in real-time. This needs to change. Predictive analytics are currently being developed that may ultimately allow us to differentiate why a student is less engaged and at higher risk of withdrawing: because of academic difficulties, for example, or emerging mental welfare issues or a combination of both.
The sector has a talent and training challenge. Investment can help staff to use technologies optimally and providers to bring in new roles. There is also a talent challenge at all levels from senior executives to academic and professional staff who do not yet understand the potential and applicability of new digital technologies. It isn’t good enough to rest on those staff who are ‘early adopters’. The mass of university staff need to be supported to develop better digital technology skills. This, combined with investment challenges, can mean that decision-making about new platforms can be siloed, involving different stakeholders with different agendas, with an end result of systems that do not integrate and which are not interoperable. Students want seamless and simple.
We don’t know exactly how AI will reshape our collective concept of learning, knowledge, and cognition over the coming decades, but we can be sure that generative AI tools will soon saturate education at every level, reshaping students’ expectations from school to entry to university to employers’ expectations of graduate capabilities.
From a fragmented approach to adopting digital
As the sector’s practice evolves, TEF can be seen as offering a framework to enhance learning and teaching by asking providers to articulate the links between learner engagement, student outcomes and educational gains.
But ultimately, success in the TEF should be seen as a side benefit of effective learning and teaching that demonstrably engages students. All of this means that the sector needs to move on from a fragmented approach to adopting digital technologies and prepare to invest in platforms and technologies, training and talent that are consistently applied across the whole institution, getting the best out of learner engagement, and truly creating opportunities for all.
In related news:
Girls’ cyber security skills recognised at CyberFirst awards
80% of students support use of learning engagement analytics
Leave a Reply