News

• February, 2024: Paper accepted to The Visual Computer (see Downloads).
• February, 2024: Web launched.

Abstract

Advances in virtual and augmented reality have increased the demand for immersive and engaging 3D experiences. To create such experiences, it is crucial to understand visual attention in 3D environments, which is typically modeled by means of saliency maps. While attention in 2D images and traditional media has been widely studied, there is still much to explore in 3D settings. In this work, we propose a deep learning-based model for predicting saliency when viewing 3D objects, which is a first step toward understanding and predicting attention in 3D environments. Previous approaches rely solely on low-level geometric cues or unnatural conditions, however, our model is trained on a dataset of real viewing data that we have manually captured, which indeed reflects actual human viewing behavior. Our approach outperforms existing state-of-the-art methods and closely approximates the ground-truth data. Our results demonstrate the effectiveness of our approach in predicting attention in 3D objects, which can pave the way for creating more immersive and engaging 3D experiences.

Downloads

• Paper (PDF)
• Paper (Springer)
• Dataset

Code

We will make code available soon.

Bibtex

Related Work

• 2020: Panoramic convolutions for 360º single-image saliency prediction
@inproceedings{martin20saliency, author={Martin, Daniel and Serrano, Ana and Masia, Belen}, title={Panoramic convolutions for $360^{\circ}$ single-image saliency prediction}, booktitle={CVPR Workshop on Computer Vision for Augmented and Virtual Reality}, year={2020} }
• 2022: Scangan360: A generative model of realistic scanpaths for 360 images
@article{martin2022scangan360, title={Scangan360: A generative model of realistic scanpaths for 360 images}, author={Martin, Daniel and Serrano, Ana and Bergman, Alexander W and Wetzstein, Gordon and Masia, Belen}, journal={IEEE Transactions on Visualization and Computer Graphics}, volume={28}, number={5}, pages={2003--2013}, year={2022}, publisher={IEEE} }
• 2023: D-SAV360: A Dataset of Gaze Scanpaths on 360° Ambisonic Videos
@article{bernal2023d, title={D-SAV360: A Dataset of Gaze Scanpaths on 360° Ambisonic Videos}, author={Bernal-Berdun, Edurne and Martin, Daniel and Malpica, Sandra and Perez, Pedro J and Gutierrez, Diego and Masia, Belen and Serrano, Ana}, journal={IEEE Transactions on Visualization and Computer Graphics}, year={2023}, publisher={IEEE} }