Can ChatGPT help us identify deepfake content?

As governments scramble to regulate AI and stop the spread of misinformation, can more AI help? 

That’s the question a team at the University of Buffalo are asking. Researchers have used large language models (LLMs) such as OpenAI’s ChatGPT and Google Gemini to identify deepfakes of human faces. 

Presenting their results at last week’s IEEE/CVF Conference On Computer Vision & Pattern Recognition, the team found that currently LLMs cannot match dedicated, state-of-the-art deepfake detection algorithms. However, their innate ability to process language means they could become powerful tools for identifying fake content and improving human skills in this area.

‘What sets LLMs apart from existing detection methods is the ability to explain their findings in a way that’s comprehensible to humans, like identifying an incorrect shadow or a mismatched pair of earrings,’ said the study’s lead author, Siwei Lyu, PhD, SUNY Empire Innovation Professor in the Department of Computer Science at the University of Buffalo.

‘LLMs were not designed or trained for deepfake detection, but their semantic knowledge makes them well suited for it, so we expect to see more efforts toward this application,’ Lyu added. 

LLMs look at statistical patterns and relationships between words to generate automated responses. The most up-to-date versions are also able to analyse images, calling on databases of captioned photos to understand links between words and pictures. 

In testing, ChatGPT was found to identify synthetic artifacts in images 79.5% of the time, and 77.2% on StyleGAN-generated images. The programme was also able to explain how it came to its decisions on the authenticity in plain language. This can help users understand and learn themselves.

Examples given in the study report include LLMs pointing out ‘the hair on the left side of the image slightly blurs’ and ‘the transition between the person and the background is a bit abrupt and lacks depth’. 

‘Existing deepfake detection models will tell us the probability of an image being real or fake, but they will very rarely tell us why they came to this conclusion. And even if we look into the model’s underlying mechanisms, there will be features that we simply can’t understand,’ said Lyu. ‘Meanwhile, everything ChatGPT outputs is understandable to humans.’

According to Lyu and the team, ChatGPT and other LLMs have a ‘common sense understanding of reality’, such as the symmetry of a human face and the look of a real photograph, and use semantic knowledge to understand what they are looking at. In comparison, deepfake detection technology relies on large datasets which have been labelled real or deepfake. 

However, some drawbacks have been identified. The latest deepfake detection algorithms have accuracy rates well into the mid-high 90s because of their ability to catch signal-level differences invisible to the human eye. LLMs cannot match this.

And by focusing entirely on semantic ‘abnormalities’ in imagery, the software could wind up making decisions based on prejudice and stereotyping. Meanwhile, some LLMs, like Gemini, were less effective at explaining decision making than ChatGPT, and many commands were refused. When asked to analyse some images, software gave response like ‘Sorry, I can’t assist with that request.’

‘The model is programmed not to answer when it doesn’t reach a certain confidence level,’ Lyu says. ‘We know that ChatGPT has information relevant to deepfake detection, but, again, a human operator is needed to excite that part of its knowledge base. Prompt engineering is effective, but not very efficient, so the next step is going one level down and actually fine tuning LLMs for this task specifically.’