Research / Game audio

Full paper · 2026

MAV-C: A Framework for the Joint Objective Estimation of Audio-Visual Complexity in Immersive Virtual Environments

Resti, L., Gully, A., McLoughlin, M., Kearney, G., & Denisova, A.

ConferenceProceedings of the AES 6th International Conference on Audio for Virtual and Augmented Reality and Immersive Games (AVARIG 2026), 2026

Teaser figure: MAV-C audio-visual complexity framework. Audio features (temporal, spectral and spatial entropy) and video features (edge gradients, colour uniqueness, optical-flow motion) are compressed and weighted into sub-scores C_A(t) and C_V(t), then fused into a single complexity score C_AV(t) between 0 and 1. Validated on synthetic stimuli, synthetic sweeps and 60 gameplay clips.
Open full-size image

In short

MAV-C estimates how complex an immersive virtual environment is to see and hear. It combines audio features (temporal, spectral and spatial) with visual features (edges, colour and motion) into a single score that changes over time, without needing a reference recording.

Key findings

  • On a controlled synthetic dataset, the audio and visual sub-scores each responded only to their own modality, and changing one feature at a time changed the score consistently.
  • On 60 gameplay clips, visual complexity differed between types of content while audio complexity did not, and the two were essentially uncorrelated (r = 0.03).
  • Calibrating the score against human ratings is the next step.

Read and cite

PDF (accepted version)

APA citation

Resti, L., Gully, A., McLoughlin, M., Kearney, G., & Denisova, A. (2026). MAV-C: A Framework for the Joint Objective Estimation of Audio-Visual Complexity in Immersive Virtual Environments. In Proceedings of the AES 6th International Conference on Audio for Virtual and Augmented Reality and Immersive Games (AVARIG 2026).

More on this topic