AI’s Roadblock: Transformers Aren’t Good at Generalizing
Researchers from Google have revealed that transformers, a technology powering AI tools like ChatGPT, struggle with generalizing and performing tasks beyond their training data. This poses a challenge for achieving artificial general intelligence (AGI), as AI currently lacks the capacity to transfer skills across domains like humans. The study has prompted a reevaluation of the capabilities of large language models (LLMs) and the need to temper expectations of imminent AGI. Despite the limitations, some experts remain optimistic about the future role of AI models.