When we marvel at artificial intelligence models running on thousands of cutting-edge chips, it is easy to assume they represent the pinnacle of learning. Yet a 1-year-old child accomplishes something these systems still cannot: making sense of the world with astonishing efficiency, using minimal data and energy.
Babies may not write code or solve calculus, but they recognize new objects after just one or two encounters. They learn through brief observation and physical interaction, while today's AI models require vast oceans of training data and consume as much energy as a small nation. This stark contrast has led researchers to ask whether the architecture of the infant brain could inspire a fundamentally different approach to machine learning.
The EgoBabyVLM Challenge
To investigate this question, researchers at Meta, Stanford University, the University of Tokyo, and France's École Normale Supérieure created a benchmark called the EgoBabyVLM Challenge. The test evaluates vision language models—systems that learn from both text and imagery—by asking them to describe the world as a baby experiences it.
The models must process approximately a thousand hours of video recorded from cameras mounted on the heads of infants and toddlers. When fed this unstructured, messy footage, even the most advanced models performed poorly. The results suggest that something fundamental about the baby brain's design enables it to learn rapidly from limited, chaotic input.
Babies do not learn from neatly curated datasets. Instead, they navigate a kaleidoscopic stream of experiences: parents discussing objects that have disappeared from view, pointing with gestures or gaze, and talking about past or future events rather than only what is happening in the moment. According to Michael Frank, a cognitive scientist at Stanford University who helped develop EgoBabyVLM, babies learn not just from language but from rich multimodal and tactile experiences. The test makes clear, Frank says, that language alone is insufficient for building truly intelligent systems.
Lessons from BabyLM
EgoBabyVLM builds on a growing effort to use AI as a window into human cognition. In 2023, a challenge called BabyLM asked AI models to learn language syntax using roughly the same volume of data a 10-year-old encounters—tens of millions of words, rather than the trillions fed to conventional AI systems. Transformer-based models, which analyze relationships between words across sentences, performed surprisingly well on this task. That finding challenges the long-standing ideas of Noam Chomsky, who argued that syntax is hardwired into the human brain.
Ryan Cotterell, a linguist at ETH Zurich who created BabyLM, notes that the situation changes dramatically when the goal shifts from language to understanding the physical world. He points out that there is no large corpus of human interactions comparable to the internet's text archives.
Joshua Tenenbaum, a cognitive scientist at MIT, observes that BabyLM revealed models do not develop common sense about physical environments, social dynamics, or theory of mind. Transformers excel at finding patterns in data, Tenenbaum says, but pure pattern-learning systems appear unable to take the kind of input a child receives and learn everything a child learns.