Large language models claim to have developed analogical reasoning, but new mechanistic interpretability research suggests they're solving analogies in fundamentally different ways than humans do, and often failing where children succeed. Scientists at a leading cognitive science lab have conducted behavioral and mechanistic interpretability studies to investigate whether analogical reasoning has truly emerged in these AI systems, and their findings reveal surprising gaps in how AI models approach reasoning tasks that come naturally to young learners.

What Is Analogical Reasoning and Why Does It Matter?

Analogical reasoning is the ability to use what you know about one thing to infer knowledge about something new and related. When a child learns that a dog has four legs and then encounters a cat, they can reason by analogy that a cat probably also has four legs. This type of thinking is fundamental to human learning and problem-solving, allowing us to transfer knowledge across different domains and situations .

For artificial intelligence systems, analogical reasoning represents a critical capability. If large language models (LLMs), which are AI systems trained on vast amounts of text data, can truly perform analogical reasoning, it would suggest they're developing more human-like cognitive abilities. However, the latest research challenges this assumption by examining not just whether models can solve analogy problems, but how they're actually solving them internally.

How Are Researchers Studying AI Reasoning Abilities?

Mechanistic interpretability is an emerging field that looks inside AI models to understand how they actually work. Rather than treating models as black boxes that produce outputs, researchers examine the internal circuits and mechanisms that drive decision-making. This approach allows scientists to see whether AI systems are using reasoning strategies similar to humans or following entirely different computational paths .

The research team conducted a series of behavioral studies alongside mechanistic interpretability analyses to compare how children and large language models learn to solve analogies. By examining both what models produce and how they produce it, researchers can identify whether similarities in performance reflect similarities in underlying reasoning processes.

What Do the Findings Reveal About AI vs. Human Reasoning?

The research provides evidence of both similarities and critical differences in how children and LLMs develop analogical reasoning abilities. While both show some parallel patterns in learning to solve analogies, the developmental trajectories diverge in important ways. Children appear to develop analogical reasoning through a process of abstraction and generalization that differs from how large language models approach the same tasks .

The mechanistic analysis reveals that AI models may be relying on different internal mechanisms than humans use. For instance, research on transformer architectures, which power most modern LLMs, has shown that these models have theoretical limitations in certain types of reasoning tasks. Even after extensive pretraining on massive datasets, these architectural constraints can persist, affecting how models handle specific reasoning challenges .

Steps to Understanding How AI Models Could Improve Analogical Reasoning

• Developmental Insights: Researchers propose using insights from how children learn analogical reasoning to inform AI development. By understanding the cognitive processes that make human analogical reasoning effective, scientists can design better training approaches and architectures for AI systems.
• Mechanistic Analysis: Continued investigation into the internal circuits and mechanisms of large language models can reveal which components support analogical reasoning and which create bottlenecks. This allows targeted improvements rather than broad model scaling.
• Theory-Informed Design: Developing theoretical frameworks that explain how analogical reasoning should work computationally can guide the creation of AI systems that achieve more human-like reasoning abilities rather than superficially mimicking human performance.

The research team concluded with a discussion of developmental insights that could help AI models achieve human-like analogical reasoning. This suggests that the path forward isn't simply making models larger or training them on more data, but rather understanding the fundamental cognitive processes that underlie analogical thinking and building those processes into AI systems from the ground up .

Why Does This Matter for AI Development?

The gap between how AI models and humans perform analogical reasoning has practical implications for AI safety and capability development. If models are solving reasoning tasks through mechanisms that don't generalize the way human reasoning does, they may fail in unexpected ways when deployed in real-world applications. Understanding these differences helps researchers build more reliable and predictable AI systems.

Additionally, this research highlights a broader challenge in AI development: performance metrics alone don't tell the full story. A model might score well on an analogy benchmark while using entirely different reasoning processes than humans. This distinction matters because human-like reasoning processes tend to be more robust, generalizable, and interpretable than alternative computational strategies.

The work being conducted at the University of Amsterdam's Institute for Logic, Language, and Computation represents a growing recognition that mechanistic interpretability and cognitive science must work together to advance AI capabilities. By studying not just what AI models can do, but how they do it, researchers are building a more complete picture of artificial intelligence and its limitations .