Does additional audio really work? A study on users' cognitive behavior with audio-visual dual-channel in panoramic digital museum

•Eye-tracking, WP scale, Personal Meaning Map, and knowledge tests were jointly used to investigate the impact of the inclusion of the audio channel on user cognition in the panoramic digital museums.•The adoption of dual-channel audio-visual interaction increases the overall cognitive load on users.•Adding an audio channel has no positive effect on learning outcomes.•Suggestions are proposed to museum practitioners and platform developers. Multi-channel human-computer interaction offers a new perspective for optimization research in the digital museum, though there is no empirical evidence on how these factors affect cognition. This paper conducted a controlled experiment using the audio guide as a breakthrough point, aiming to determine the cognitive changes before and after the audio-visual dual-channel interaction. The results demonstrate that adding the auditory channel brought more cognitive load but not better cognitive outcomes. This paper evaluated the reasons behind it and noted that designing channels based solely on technical feasibility is undesirable, with suggestions proposed for museum practitioners and platform developers.