VS-Sim: A Synthetic Dataset for Viewpoint Shift Robustness

Riccardo Turra

Verizon Connect

Matteo Simoncini

Verizon Connect

Henrique Piñeiro Monteagudo

Verizon Connect

University of Bologna

Aurel Pjetri

Verizon Connect

University of Florence

Leonardo Taccari

Verizon Connect

Samuele Salti

University of Bologna

ICIAP 2025

Abstract

We release VS-Sim, a synthetic dataset of road scene images that can be used to study the robustness of several Computer Vision tasks to viewpoint shifts. Our dataset includes images from a frontal camera in different positions, and annotations for tasks both in frontal view (depth, semantic segmentation) and Bird’s Eye View (semantic segmentation). The paper also includes an analysis of the robustness of a state-of-the-art Bird’s Eye View method to viewpoint shifts. Experiments indicate that viewpoint shift significantly degrades the performance of a model trained on data from a single viewpoint, and suggest that using multiple viewpoints during training helps to mitigate the performance drop across different scenarios.

Overview

In this work, with the aim of raising awareness on the problem and speeding up the scientific research toward viewpoint robustness, we introduce VS-Sim, a dataset of synthetic driving scenes acquired from different viewpoints. We collect data using three types of virtual sensors: the RGB camera, acting as a road-facing dashcam to collect frontal view images; the depth camera, that generates depth masks corresponding to the RGB images; the semantic segmentation (SS) camera, used to generate SS ground truth both for frontal views and for the Bird’s Eye View. We provide an example of a RGB frame and corresponding annotations from the other virtual sensors in the figure.

Frontal RGB image and pixel-level ground truth for depth, semantic segmentation, BEV semantic segmentation.
Frontal RGB image and pixel-level ground truth for depth, semantic segmentation, BEV semantic segmentation.

Acknowledgements

Henrique Piñeiro Monteagudo is supported by the SMARTHEP project, funded by the European Union’s Horizon 2020 research and innovation programme, call H2020-MSCA-ITN-2020, under Grant Agreement n. 956086