The Upcoming Age of Spatial Intelligence: Our Take on Fei-Fei Li and Justin Johnson's a16z Appearance.
The role of advanced maths in spatial intelligence.
Recently, Fei-Fei Li and Justin Johnson, co-founders of the newly established AI company World Labs, which has successfully raised $230 million in funding, featured in an insightful interview on the a16z podcast, hosted by partner Martin Casado. During the discussion, they revealed World Labs to be a deep-tech enterprise focused on developing spatial general artificial intelligence (AGI), with the ambitious aim of enabling machines to reason and operate within the physical world. The founders emphasised the profound significance of visual-spatial intelligence, a domain of AI that extends far beyond the capabilities of language comprehension alone. At its core, this level of intelligence encompasses a machine's ability to perceive, reason, and act within four dimensions—three spatial and one temporal.
To us, as mathematicians familiar with some historical interplay between mathematics and physics, this moment resonates with Einstein's key realisation that, in order to formulate his theory of general relativity, he needed to transcend the confines of Euclidean geometry. In his 1922 Kyoto address, Einstein reflected on this profound insight, stating:
"If all systems are equivalent, then Euclidean geometry cannot hold in all of them. To throw out geometry and keep laws is equivalent to describing thoughts without words. We must search for words before we can express thoughts. What must we search for at this point? This problem remained insoluble to me until 1912, when I suddenly realized that Gauss's theory of surfaces holds the key for unlocking this mystery. I realized that Gauss's surface coordinates had a profound significance. However, I did not know at that time that Riemann had studied the foundations of geometry in an even more profound way. I suddenly remembered that Gauss's theory was contained in the geometry course given by Geiser when I was a student...
I realized that the foundations of geometry have physical significance. My dear friend the mathematician Grossmann was there when I returned from Prague to Zurich. From him I learned for the first time about Ricci and later about Riemann. So I asked my friend whether my problem could be solved by Riemann's theory, namely, whether the invariants of the line element could completely determine the quantities I had been looking for".
It was Einstein's close friend, Marcel Grossman, who introduced him to Bernhard Riemann's pioneering work in differential geometry, which ultimately provided the mathematical framework for expressing how mass warps spacetime. Just as differential geometry revolutionised our understanding of the universe, we believe it holds immense potential for breakthroughs in spatial AGI, as described by Fei-Fei Li and Justin Johnson. AI researchers aiming to unlock such intelligence could greatly benefit from the matured structures of differential geometry, differential topology, and algebraic topology. Interestingly, category theory offers a powerful mechanism for connecting these mathematical fields.
Just as Einstein recognised the need to explore new mathematical tools, today’s AI researchers must similarly adopt advanced mathematics to enhance the development of systems that could potentially bring us closer to AGI—capable of reasoning and acting in 3D and 4D space-time. While deep learning has transformed fields such as computer vision and natural language processing, the next frontier could indeed be spatial intelligence as described by Fei-Fei and Justin. However, only by harnessing sophisticated mathematical frameworks can machines truly navigate and comprehend the complexities of the real world in its full dimensionality.
As spatial intelligence becomes more central to the development efforts of AGI, the application of advanced branches of mathematics will be vital in guiding this research. The algorithms that will power such systems must draw on ideas that are not only computationally feasible but also geometrically rich, capable of interpreting and interacting with the fabric of reality itself. Just as differential geometry once bridged gaps in physics, it now holds the key to advancing AI toward spatial reasoning in both virtual and real-world contexts.
For further reflection on the role of mathematics in physics and its potential impact on spatial intelligence, Quanta Magazine recently published "Why We Need Mathematicians to Understand Space-Time." The insights are pertinent not only to physics but also to frontier AI research, particularly in the context of spatial AGI. Below is an excerpt from the article, which we believe is highly relevant to today’s frontier AI research:
As scientists venture to understand our world, mathematics often serves as both their language and their guide. Physicists rely on math not just to describe what they see in their laboratories but also to predict and explore phenomena that their tools cannot otherwise touch, like the insides of black holes or the moments just after the universe began. Some of the biggest breakthroughs in physics were only made possible by mathematical advances. Isaac Newton’s laws of motion, for instance — which allow us to model how a planet will orbit its sun or how fast an object will fall — first required the invention of calculus.
But math isn’t just useful for introducing or developing a new physical theory. Long after physicists have established a theory, mathematicians comb over it with the rigor that their field demands, intent on placing it on more solid logical footing. This cleanup work can take decades, but it’s necessary to establish deeper trust in physical ideas.
There’s an entire field devoted to the mathematical study of physics-inspired problems, aptly called mathematical physics. One of its core aims is to work out the precise consequences of the complicated mathematics at the heart of general relativity. In 1915, Albert Einstein showed that the shape of our four-dimensional universe — composed of three spatial dimensions plus one dimension of time — is determined by the matter that lives within it. That shape in turn gives rise to what we experience as gravity. From the mathematics of general relativity, bizarre aberrations like black holes emerge. Even a century later, many of these phenomena remain mysterious. And so mathematicians continue to pore over Einstein’s equations, using them as a conceptual laboratory in which to test out new hypotheses, gain novel insights and prove ideas that physicists might take for granted.
What's New and Noteworthy
Long after physicists have accepted something as true, it’s often left to mathematicians to give it a rigorous foundation and to build a complete, coherent framework around it.
Einstein’s general theory of relativity predicts how the matter that fills space-time, like [MOU1] stars and galaxies, warps and curves its shape. But the equations that describe this are notoriously hard to work with. For example, physicists have long accepted that Einstein’s equations imply that less matter means less warping — that is, flatter space. But it wasn’t until last year that mathematicians definitively proved it. Similarly, a common assumption in physics holds that, unlike the other shapes that space-time might take, a negatively curved universe is deeply unstable: Any matter placed within it will eventually collapse into a black hole. Yet mathematicians were only recently able to verify this.
Mathematical proofs don’t always go the way physicists expect. In August, for instance, Quanta reported on how a pair of mathematicians felled the great Stephen Hawking’s “third law” of black hole thermodynamics. Fifty years ago, Hawking and two other physicists conjectured that “extremal” black holes, which pack so much electric charge or spin that they behave in incredibly counterintuitive ways, are mathematically impossible. But the new proof demonstrates that they can exist — at least in theory.
Of course, mathematicians do more than clean up physicists’ mess. They can also provide new and important insights. By rigorously redefining long-standing models of how fundamental particles interact, mathematicians have been able to offer a better understanding of how quantum gravity might work. They’ve also been able to explore black holes more deeply. Even though physicists can now observe black holes in the real world, they still can’t tell you whether any given patch of matter-filled space will eventually turn into one. Mathematicians can — and they’re getting really good at it. It was a mathematician, Roy Kerr, who realized in 1963 that black holes could rotate, and another who, two years ago, proved that such a black hole is stable.
Mathematicians’ emphasis on abstraction also lets them take black holes into weird worlds physicists might not even imagine. It turns out that in a five-dimensional universe, for instance, black holes wouldn’t have to be spherical anymore. They can instead come in all sorts of exotic forms.
Whether they’re providing theoretical scaffolding or exploring concepts in the nth dimension, mathematicians have been instrumental in propelling physics forward. And with gravity and quantum mechanics still at odds, space-time is one area where mathematical ideas tend to lead rather than follow.
Thank you for reading our newsletter. I wish you a fantastic week ahead!
Bambordé
{LinkedIn} {Twitter} {Substack}
Free Mathematical Proof Writing Course
We’re excited to announce a special two-lecture crash course on proof writing. It will begin this coming Tuesday, September 24, at 5 PM BST.
Registration:https://forms.gle/mH2xuV7eWNUEy5MN7.
 | A guest post by
|