Research / The placebo effect

Full paper · 2025

Exploring the (Placebo) Effect of an AI-Powered Dialogue System on Player Experience

Sridharan, S., Villa, S., Denisova, A., & Pirker, J.

ConferenceIEEE Conference on Games (CoG'25), 2025

Teaser figure: The (placebo) effect of AI dialogue. A screenshot of the murder-mystery RPG built for the study, where players type free-text questions to villagers. In a 2 by 2 study (N = 52), players were told the dialogue was AI-generated or hand-written, and played either version. Players told it was hand-written reported more positive player experience (p = .012) and higher immersion (p = .040), whichever version they played; the real AI dialogue raised autonomy (p = .001) but did not change overall player experience.
Open full-size image

In short

Does AI-generated dialogue change how players experience a game, or is it what they believe about it? In a murder-mystery game, we varied both the dialogue players actually got (AI-generated or human-written) and how the game was described to them (labelled as AI or as human-written), and measured overall player experience, immersion and perceived autonomy.

How the study worked

What players were told

Participants were told the study aimed to investigate the effects of GPT-generated dialogue in a game and received a brief explanation of the two game versions. They were not informed that they might be playing a different version than the one described to them.

Game
A 2D murder mystery RPG: as a detective, the player finds the murderer only through dialogue with the victim’s three neighbours
Players
52
Design
2 × 2 between-subjects: the version played (GPT-4o or hand-written dialogue) × the version described (AI or hand-written). For half of the players the two did not match.
Dialogue
GPT version: free text input. Hand-written version: choosing from several responses.
Play time
Up to 20 minutes, followed by a short interview
Measures
Expectations (revised iExpect), overall player experience (miniPXI), immersion (PXI), autonomy satisfaction and frustration (BANGS), emotional challenge (CORGIS)

The two versions of the game

The murder mystery game with a free-text input box for talking to a villager.
AI version: free-text input, with GPT-generated NPC responses. © Sarita Sridharan
The murder mystery game with a list of predefined questions to choose from.
Human-written version: predefined choices in a dialogue tree. © Sarita Sridharan

Key findings

  • Player expectations, shaped by the game description, affected overall player experience more than the dialogue players actually received.
  • Players who believed the narrative was human-written reported more positive player experience, higher immersion and higher perceived autonomy, whichever version they played.
  • Interacting with the AI-generated dialogue increased perceived autonomy, suggesting AI dialogue can enhance a sense of freedom without necessarily improving the overall experience.

The takeaway

From my CoG 2025 talk on this paper

  • Technology alone does not guarantee better play. What matters is how it is framed, and how well it supports the story at the heart of the game.

    Summary

Read and cite

PDF (accepted version) Publisher's version (DOI)

APA citation

Sridharan, S., Villa, S., Denisova, A., & Pirker, J. (2025). Exploring the (Placebo) Effect of an AI-Powered Dialogue System on Player Experience. In IEEE Conference on Games (CoG'25) (pp. 1-8).

Line of work

  1. Builds on · 2015The Placebo Effect in Digital Games: Phantom Perception of Adaptive Artificial Intelligence
  2. This paper · 2025Exploring the (Placebo) Effect of an AI-Powered Dialogue System on Player Experience

More on this topic