Pushing the Boundaries of Structure-from-Motion with Machine Learning

AI has conquered the internet by storm with LLMs like ChatGPT. The interaction of AI Agents with the real world by means of robotics or augmented reality is generally seen to be the next frontier. To do this, however, AI Agent needs to be able to properly localize itself and the objects in its environment. These tasks are tackled by visual relocalisation and pose estimation, which allow the machine to visually perceive both its place in the world and its environment.

We are excited to welcome Eric Brachmann, Senior Staff Scientist at Niantic Spatial and a well-established researcher in visual relocalisation and pose estimation, to our joint Heidelberg.ai / NCT Data Science Seminar series. In this online seminar, Eric Brachmann will talk about his current work on camera relocalization where he merges machine learning with more traditional computer vision approaches. His work has been integrated into Niantic's Visual Positioning System (VPS) which powers game features in Ingress and Pokemon Go.

We look forward to your participation, as this seminar will equip us with the knowledge how AI will be able to perceive the world around it and interact with it in a meaningful way, which is crucial for the future of robotics and augmented reality.

Abstract

In 3D computer vision, we are currently witnessing a remarkable renaissance of interest in structure-from-motion (SfM), i.e. estimating camera poses and 3D geometry from a collection of images. Of course, SfM was never gone. Rather, solutions based on feature-matching and traditional multi-view geometry matured to a state about 10 years ago that turned them into reliable off-the-shelf components for various 3D vision tasks. Still, traditional SfM approaches are most reliable when certain conditions are met. For example, reconstructing very few or a huge amount of images can be challenging. The talk will investigate how learning-based formulations of SfM can address these challenges. We will focus on scene coordinate regression, an implicit scene representation, that naturally avoids the explosion of complexity inherent to image-to-image matching when the number of images is large. The talk culminates in the presentation of ACEZero, a self-supervised scene coordinate regression pipeline, that is able to reconstruct 10.000 images in reasonable time.

Biography

Eric Brachmann is a senior staff scientist at Niantic Spatial with extensive experience at the intersection of machine learning and computer vision, particularly in 3D vision. His work focuses on building and scaling the Visual Positioning System (VPS), with research interests spanning visual relocalization, pose estimation, robust optimization, end-to-end learning, and feature matching. He regularly publishes in top-tier computer vision conferences and is an active member of the community, serving as area chair and reviewer—receiving multiple outstanding reviewer recognitions. He has also co-organized several tutorials and workshops on visual relocalization and object pose estimation. Prior to his current role, he contributed to both academic and industry-led efforts in spatial computing and 3D vision research.

Event Info

Please help us plan ahead by registrating for the event via Meetup:
Event Registration .

What?	Pushing the Boundaries of Structure-from-Motion with Machine Learning
Who?	Eric Brachmann, Senior Staff Scientist at Niantic Spatial
When?	July 16th 2024 @ 4pm
Where?	Zoom
Registration	meetup event-site

---