Computer Vision for the Real-World Metaverse

In this episode, we talk to Marc Pollefeys, Professor of Computer Science at ETH Zurich and Director of the Microsoft Mixed Reality and AI Lab in Zurich, about Computer Vision for the Real-World Metaverse. We explore what the Metaverse means for him and how it can enable new ways of interacting with the physical and virtual world. We also discuss how his group is working on Visual Positioning Systems (VPS) and Azure Spatial Anchors, which are technologies that allow users to anchor digital content to specific locations in the real world. We learn about the challenges and opportunities of creating a large-scale 3D reconstruction of the world that can be accessed by mixed reality devices like HoloLens. Finally, we address some of the computer vision tasks that his group is working on for HoloLens, such as spatial understanding, object recognition, hand tracking, and eye gaze estimation. Tune in to find out more about this fascinating research that can bring us closer to the Real-World Metaverse.