Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization

Imagine you're trying to find a specific object in a city using images from different perspectives, like from a drone or a satellite. Traditionally, people would match the 2D appearances of these images, but this method often fails because it relies on limited datasets and doesn't account for the actual geometry of the scene. This is what's called the limitation of 2D appearance matching, which can lead to inaccuracies when the views differ significantly or when the dataset lacks diversity in prompts and imagery.

To tackle these issues, this paper introduces a new dataset that includes over 220,000 pairs of images from ground, drone, and satellite views, complete with multi-modal prompts and camera poses. This allows for a more flexible and accurate way to refer to targets in different views. The authors also propose a new framework that integrates visual features and referring prompts in a single model, which can predict bounding boxes, segmentation masks, and camera poses all at once. This approach is designed to overcome the limitations of previous methods by using a contrastive loss that aligns different views without needing extensive training data.

The result is a system that not only performs better than existing methods but also generalizes well to new scenes and setups. For anyone building applications that require accurate geo-localization across different perspectives, this work provides a robust solution that leverages both a rich dataset and an innovative modeling approach.

The introduction of a large-scale dataset and a novel framework for geo-localization represents a significant advancement in the field.

The claims are supported by extensive experiments and comparisons to state-of-the-art methods, though some details on baselines could be clearer.