Understanding human visual behavior within virtual reality environments is crucial to fully leverage their potential. While previous research has provided rich visual data from human observers, existing gaze datasets often suffer from the absence of multimodal stimuli. Moreover, no dataset has yet gathered eye gaze trajectories (i.e., scanpaths) for dynamic content with directional ambisonic sound, which is a critical aspect of sound perception by humans. To address this gap, we introduce D-SAV360, a dataset of 4,609 head and eye scanpaths for 360º videos with first-order ambisonics. This dataset enables a more comprehensive study of multimodal interaction on visual behavior in VR environments. We analyze our collected scanpaths from a total of 87 participants viewing 85 different videos and show that various factors such as viewing mode, content type, and gender significantly impact eye movement statistics. We demonstrate the potential of D-SAV360 as a benchmarking resource for state-of-the-art attention prediction models and discuss its possible applications in further research. By providing a comprehensive dataset of eye movement data for dynamic, multimodal virtual environments, our work can facilitate future investigations of visual behavior and attention in virtual reality.

Video Presentation


Paper: (Authors version) PDF
Slides: PowerPoint


D-SAV360 is composed of 50 stereoscopic and 35 monoscopic videos with ambisonic sounds. We provide the collected gaze data from 87 participants, the computed saliency maps, the estimated optical flow (obtained using RAFT), the estimated depth, and the computed Audio Energy Maps. Additionally, we provide our capture and visualization system for Unity, which can be used to capture new gaze data. The dataset and documentation is available for download in the following links:

Alternative download link: https://zenodo.org/records/10043919


@article{Bernal-Berdun2023dsav360, author={Bernal-Berdun, Edurne and Martin, Daniel and Malpica, Sandra and Perez, Pedro J. and Gutierrez, Diego and Masia, Belen and Serrano, Ana}, journal={IEEE Transactions on Visualization and Computer Graphics}, title={D-SAV360: A Dataset of Gaze Scanpaths on $360^{\circ}$ Ambisonic Videos}, year={2023}, volume={}, number={}, pages={1-11}, doi={10.1109/TVCG.2023.3320237}}

Related Work