The research, which will be presented at the annual meeting of the Association for Computational Linguistics in Bangkok, concludes that large language models such as ChatGPT show no evidence of the development of sophisticated or planned thinking. Instead, these models only demonstrate the ability to respond to simple instructions. This disproves the widespread assumption that AI models could exhibit increasingly intelligent behaviour through scaling.
Background to the study
The study, led by Prof Dr Iryna Gurevych from TU Darmstadt and Dr Harish Tayyar Madabushi from the University of Bath, investigated so-called “emergent capabilities” of AI models. These capabilities, which were observed with increasing model size and data volume, raised both hopes and fears regarding the future development of artificial intelligence. However, current research shows that these abilities are due to simple learning effects rather than complex, independent thinking.
Implications for the future of AI use
The results of the study have significant implications for the use of AI systems. Prof Gurevych warns against relying on the models to solve complex tasks without human support. She emphasises that users should provide explicit instructions and examples in order to use the systems effectively. In addition, the tendency of models to produce plausible-sounding but incorrect results remains a challenge.
Focus on future risks
Although the study shows that LLMs do not develop complex thinking skills, this does not mean that AI does not pose a threat. Prof Gurevych calls for research to focus on the real risks of AI models, particularly their potential to generate fake news. The findings of the study provide a valuable basis for better understanding the actual capabilities of AI models and making their use safer in practice.
