New research shows that even some of the world’s most advanced AI systems struggle to read clocks or the dates on calendars
Artificial intelligence is increasingly in the news. Proponents are keen to tell us how well AI can perform complex tasks such as generating art or essays from basic prompts. Yet new research by team from the University of Edinburgh suggests that such systems struggle to master certain skills that humans carry out with ease.
Robot clock, photo by Eric Schweichler / Openverse
The full findings will be shared in a peer-reviewed paper to be presented on April 28, 2025 as part of the Reasoning and Planning for Large Language Models workshop, held at The Thirteenth International Conference on Learning Representations (ICLR) in Singapore. But the team have shared some initial ‘headline’ findings.
They tested a series of so-called multimodal large language models (MLLMs), or AI systems that process text and images. The test involved presenting each MLLM with a clock or a calendar and then asking it a series of questions to evaluate its understanding. Various clock designs were used, including ones with Roman numerals, those with and without second hands and ones with dials of different colours.
The study found that even start-of-the-art AI models could not reliably interpret the position of hands on a clock face or correctly answer questions about dates on calendars.
At best, the AI systems tested could correctly identify clock-hand positions less than 25%. They were much worse when given clocks with Roman numerals or stylised clock hands. Even when there was no second-hand on the clockface, the AI systems did not perform any better.
The researchers also presented the AI systems with calendars and asked them questions such as to identify holidays or to work out past and future dates. Again, even the best-performing AI model got date calculations wrong on one-fifth of occasions.
Such understanding of analogue clocks and calendars requires more than simply recognising shapes: it involves a combination of spatial awareness, context and basic mathematics. The issues with clockfaces suggest a fundamental issue with hand detection and angle interpretation.
The researchers say that addressing these issues could enable AI systems to power time-sensitive applications such as scheduling assistants, autonomous robots and tools for people with visual impairments. But there is clearly still work to be done for such ‘intelligent’ systems to master basic skills.
The study was led by Rohit Saxena from the School of Informatics at the University of Edinburgh. He says: ‘Most people can tell the time and use calendars from an early age. Our findings highlight a significant gap in the ability of AI to carry out what are quite basic skills for people. These shortfalls must be addressed if AI systems are to be successfully integrated into time-sensitive, real-world applications, such as scheduling, automation and assistive technologies.’
Aryo Gema, also from the School of Informatics, adds: ‘AI research today often emphasises complex reasoning tasks, but ironically, many systems still struggle when it comes to simpler, everyday tasks. Our findings suggest it’s high time we addressed these fundamental gaps. Otherwise, integrating AI into real-world, time-sensitive applications might remain stuck at the eleventh hour.’
In related news:
Civil service braces for job losses impact as AI overhaul continues
£1m for West Midlands 5G manufacturing boost and social care transformation
Pioneering data skills training for mums in Blackburn with Darwen
Leave a Reply