how does it work?
no language model, no web search. the photo becomes an embedding, a classifier turns that into a prior over a world grid, two retrieval searches propose ~75 candidate locations, and a learned ranker picks one.
the pipeline
- encode. two frozen vision transformers, streetclip (vit-l/14) and siglip 2 (so400m), each embed the photo. the unit-normalized embeddings are concatenated and pca-projected to one 768-d vector.
- prior. a small mlp trunk feeds 7 classifier heads over a nested quadtree of the earth (osv-5m's
cells, country-sized down to ~town-sized), giving
log p(cell)for every leaf cell. - photo search. cosine similarity against ~2M training-photo embeddings, plus each hit's
log p(cell). exact search, no approximate index. the top 32 hits become candidates. - map search. a two-tower model: one tower maps the photo embedding, the other maps a location's google alphaearth satellite embedding (64-d, 10 m pixels, pooled at 80–320 m) into a shared space. searching land-grid points finds places nobody photographed. another 32 candidates; ~39% of final picks come from here.
- rerank. the 64 hits plus the top 8 cells and 3 point estimates (75 total) get 37 features each (nearby hit mass, cell probability, gallery density, agreement between searches). a lightgbm lambdarank model scores them, and the pick is the candidate with the highest expected geoscore under the softmaxed scores (minimum bayes risk).
how good is it?
about level with the best published model: slightly ahead at region and country scale, behind at street level. still a work in progress; newer versions aren't benchmarked yet.
| geoscore | ≤25 km | ≤200 | ≤750 | ≤2500 | |
|---|---|---|---|---|---|
| this model† | 4125 | 29.6 | 67.7 | 84.7 | 93.5 |
| pinpoint | 4114 | 35.6 | 67.5 | 83.7 | 93.2 |
| pinpoint, retrieval only | 4035 | 32.1 | 65.6 | 82.8 | 92.8 |
| hierloc | 3963 | – | – | – | – |
| rfm s² | 3767 | – | – | – | – |
| osv-5m baseline | 3361 | – | – | – | – |
| geoclip | – | 21.5 | 52.1 | 72.1 | – |
| locdiff | – | 11.0 | 46.3 | 77.0 | 88.2 |
| gre | – | 9.7 | 35.6 | 72.5 | 91.1 |
osv-5m test set. geoscore: 0–5000 per photo, averaged. km columns: % of guesses within that
distance. – = not reported. published results only, via pinpoint.
† 90% of the test set, run once; the other 10% was used for tuning.
limits
made for outdoor street photos; indoor shots and screenshots are guesswork. uploads aren't stored.
data: osv-5m (mapillary, cc-by-sa), alphaearth (google), streetclip (cc-by-nc), siglip 2. tiles: openstreetmap.