UK government user research is leaning too heavily on digital tools and it’s at the expense of those who rely on services most, says Jon Rimmer, CXO at Mercator Digital, who warns an online-only approach risks skewing insights and contributing to IT project failures.
As a CXO with a team working across government and business digital projects for many years, we know all too well just how important it is to look beyond what research tells you on paper.
Photo by Mimi Thian / Unsplash
In fact, I’ve seen first-hand how some of the most valuable insights – the ones that shift our understanding and shape better services – have only surfaced from being out in the real world.
One example comes from a project where one of our researchers visited a woman who had been told to file a self-assessment tax return. She wasn’t confident online, had tried calling the helpline but was eventually directed to complete the process digitally. In trying to find help online to support her, she ended up on a scam site and paid someone posing as a legitimate service. When our researcher visited her, she checked their passport before letting them in, her front door plastered with ‘no cold callers’ signs. She clearly knew how to spot and avoid scams on the doorstep… but online? Not so much. It was a crucial insight we wouldn’t have gained remotely, as she was deliberately staying offline.
In another case, we were redesigning the mobile check-in experience for click-and-collect grocery orders. Remote research gave us the basics but it wasn’t until we visited the stores that we noticed something important: Uber drivers were regularly skipping the queue, rushing staff and creating tension. That simple on-the-ground observation led to a better-informed design and helped avoid unnecessary investment in a system that wouldn’t have solved the real issue.
Then there was the project looking at fare evasion on a city tram network. Remote interviews gave us surface-level views but by riding the trams, talking to passengers and watching how they interacted with machines and staff, we uncovered patterns and motivations that never would have surfaced otherwise.
The digital divide in government research
The point is, these kinds of insights come only from being physically present. And yet increasingly, especially in the UK government post-pandemic, user research has moved almost exclusively online, with digital tools becoming the norm.
On one level, this is a good thing – online research removes barriers like geography, cost and time, increasing access to participation. But this convenience also comes at a cost, introducing all manner of problems.
First, many are being left out of the conversation, particularly some of the most vulnerable people who rely on government services: older adults, people with disabilities, low-income families and those without reliable internet access. Those without the flexibility to join a session during a researcher’s typical working hours are also at risk of being left out, for example those working in the construction industry.
If we fail to adjust research methods to accommodate these hard-to-reach groups, our sample becomes narrower, less diverse and more homogenised – skewing insights and increasing the risk of designing services around the easiest to reach, rather than those who need them most.
On top of this, we’re also seeing a rise in ‘imposter participants’ – people taking part dishonestly, sometimes multiple times or just for the financial incentive. In some circles, research has become a legitimate side hustle, which can undermine the authenticity and quality of feedback.
Stepping beyond the screen – what needs to change
With the above in mind, there is a lot that can and should change in the way senior decision-makers approach research in digital government projects, for example:
- Challenge the false consensus effect: Potentially one of the biggest errors in decision-making is the false consensus effect – the assumption that others share your beliefs, values and behaviours. To avoid this, shop-floor experiences that create empathy and understanding of living and walking in others’ shoes are important.
- Make research a shared responsibility, not a handover: Accountability to research outcomes should sit with the whole team, not just the researcher. There needs to be a genuine commitment to using insight to shape user experience and service design, not just gathering it to tick a box.
- Co-design with experts by experience: Where possible, involve experts by experience – people with lived experience of the service – in the design activities. Their input grounds decisions in reality and helps avoid solutions that miss the mark.
- Increase trust in research literacy: Building trust in research means improving research literacy across teams and leadership. Everyone should understand how research is conducted, what it reveals, and how it feeds into better outcomes. As the Government Digital Services mantra encourages, ‘User research is a team sport.’
- Rethink recruitment – one method won’t reach everyone: If budgets and timelines are tight, it’s even more important to be creative in how you recruit participants. Relying on a single recruitment method risks excluding entire user groups. To engage hard-to-reach users, explore multiple methods and be willing to meet people where they are and at times convenient for them. This might be outside of the normal Monday to Friday 9 to 5.
- Use personas to build shared empathy: Personas are not always used effectively (or even designed appropriately). Too often, a solitary exercise is carried out by a researcher for the purpose of saying it’s been done. Instead, they should be developed early and, most importantly, collaboratively with the whole team to create a shared understanding and empathy of who the users of a service are, where they are and what challenges they face.
For me, true user research means stepping beyond the screen and into the real world to understand people’s challenges first-hand. After all, a person’s behaviour is always an interplay of the person, their environment and the contexts that bind the two. And context is important. Not only is contextual analysis a fundamental attribute of qualitative research but in-person context builds empathy and understanding of the wider context of the user’s life outside of government digital services.
Across the government, it’s vital to understand the user’s context of using our services. Often when working extremely closely on projects we can forget that users have complex lives and other interactions within government.
Without a clear commitment to in-person engagement, policymakers and digital teams risk making decisions based on incomplete, biased data, which is ultimately when we see digital projects fail.
In related news:
Interview: Breathing beyond Earth – fighting for clean air in space exploration
Leave a Reply