GELATTO: Geospatial Relation-Aware Soft Target Learning for Multimodal Region Representation

1. Environment Setup

conda create -n gelatto python=3.10 -y
conda activate gelatto
pip install torch torchvision transformers scikit-learn tqdm pillow numpy pandas matplotlib
pip install geopandas shapely

Set paths before running:

export INPUT_DATA=/path/to/CityLens-data
export HF_HOME=/path/to/.cache/huggingface
export TORCH_HOME=/path/to/.cache/torch

---

2. Data Folder Structure

Expected dataset layout:

CityLens-data/
├── satellite_image/
│   ├── Tokyo/
│   │   ├── 12345_67890.png
│   │   └── ...
│   └── ...
├── POI_sentence_vocab/
│   ├── Tokyo_poi_processed.csv
│   └── ...
├── validated/
│   ├── Dataset/
│   │   ├── all_global_pop_task_all.json
│   │   ├── all_global_gdp_task_all.json
│   │   └── ...
│   └── Benchmark/
│       ├── all_global_pop_task.json
│       └── ...
└── patchify_smart/
    ├── satellite/
    └── poi_sentence/

The patchify_smart/ directory is generated by the patch generation step below.

---

3. Patch Generation

Prepare OSM divider GeoJSON files:

export OSM_CACHE=/path/to/osm_divider_geojson_cache

Run patch generation:

bash run_patch.sh

This creates:

$INPUT_DATA/patchify_smart/
├── satellite/
│   └── Tokyo/
│       └── <tile>_p0.png
└── poi_sentence/
    └── Tokyo/
        └── Tokyo_poi_sentence.csv

---

4. Pretraining + Probing

Run pretraining and downstream probing:

bash run.sh

Outputs are saved under:

results/GELATTO/
├── seed0.pt
├── seed0_loss.csv
├── loss_plots/
└── probe_results.txt

---

5. Probe Only

Evaluate a pretrained checkpoint:

bash probe.sh /path/to/checkpoint.pt

Example:

bash probe.sh results/GELATTO/seed0.pt gelatto_probe results/gelatto_probe.txt

This evaluates the checkpoint on all downstream probing tasks.

---

Dataset

Download the dataset from:

• Pretrain & downstream label
• OSM divider