Lost at the Edge of Uncertainty: Measuring Player Uncertainty in Digital Games

In short

Uncertainty makes games engaging, but it can feel good or bad, and there was no way to measure it. Using survey data from over 700 players, we developed a questionnaire of how players experience uncertainty and validated it with a commercial puzzle game.

In games

Where uncertainty comes from in games

In the paper

Alien: Isolation

2014

External uncertainty: you never know when the alien will search for you and find you.
In the paper

Mass Effect 3

2012

Uncertainty about outcomes: players still debate what the ending means.
In the paper

Tetris

1984

Design that reduces uncertainty: previewing the next piece tells players more about what is coming.
Our example

Outer Wilds

2019

Exploration: you progress only by discovering how the world works, so finding things out is the game.

Key findings

The scale has five factors: three forms of uncertainty within the player, external uncertainty (chance or hidden information), and exploration to resolve uncertainty.
The factors showed good reliability and construct validity.
The scale gives researchers a tool for studying uncertainty as part of player experience.

PUGS · 24 items · 5 subscales

The questionnaire

Player Uncertainty in Games Scale (PUGS). Measures the uncertainty players feel while playing: three kinds of internal uncertainty, external uncertainty, and exploration to resolve it. Please cite the paper when you use it.

Download CSV Download text Qualtrics import

What do these give me?

Copy items: just the statements, one per line, ready to paste into any survey tool.
Copy everything or Download text: the full questionnaire with instructions, scale, scoring and citation.
Download CSV: one row per item with its ID, subscale and whether it is reverse-scored. Opens in Excel, R or Python and helps with scoring.
Qualtrics import: a file in Qualtrics' Advanced Format. In Qualtrics, choose Tools → Import/Export → Import survey. The items arrive as one matrix question; turn on statement randomisation if you want random order.

How to use it

Give the questionnaire after a play session. In the validation study it followed a typical play session of a game of the player's choice; in the experiment, it followed a set puzzle.
Present the 24 items in random order.
Use a 5-point agreement scale, from Strongly disagree (1) to Strongly agree (5), as in the development and validation studies.

Instructions for players

Thinking about the game you have just played, please indicate how much you agree or disagree with each of the following statements.

The paper does not give a set wording for these instructions; this is a neutral suggestion.

Response scale

1Strongly disagree
2Disagree
3Neither agree nor disagree
4Agree
5Strongly agree

Items

Decision-making 5 items

Internal uncertainty about how actions and choices shape the outcome.

1My actions were not influencing the outcome of the game.
2I could not choose which actions were better.
3I could not say if the game had more than one outcome.
4I did not know how my performance influenced the outcome.
5I did not know how the outcome(s) were connected to what I did.

Exploration 2 items

Gathering information about the game to resolve uncertainty.

6I needed to discover things to make progress.
7I needed to explore in order to know what to do next.

Taking action 7 items

Internal uncertainty about carrying out actions in the game.

8I felt I was stuck during the game.
9I found it difficult to keep track of all elements in the game.
10The game mechanics were overwhelming.
11I think what I was doing in the game was not right.
12I was not confident that I could perform some actions in the game.
13The actions I had to perform were too demanding for my skills.
14I struggled to do the right actions.

Problem solving 5 items

Internal uncertainty about how to achieve the game's goals.

15I knew how each goal could be achieved. Reverse-scored
16I understood the game mechanics. Reverse-scored
17I knew how to play the game when I started. Reverse-scored
18I often felt lost.
19I could find the solutions required for achieving the goals of the game. Reverse-scored

External 5 items

Uncertainty from chance and unpredictability, outside the player.

20The game was unfair.
21Unpredictable random elements were influencing my performance.
22I was relying on chance in the game.
23Random elements in the game were preventing me from achieving my goal.
24The outcome of my actions was mainly influenced by chance.

Scoring

Reverse-score items 15, 16, 17 and 19 (on a 5-point scale, 1 becomes 5, 2 becomes 4, and so on).
Then sum the responses within each subscale. Higher scores mean more uncertainty of that kind.

Things to know

Decision-making, taking action and problem solving are closely related and largely reflect a general sense of internal uncertainty. They are still worth scoring separately: in the validation experiment, only decision-making and problem solving dropped when players had more time with the puzzle.
Exploration and external uncertainty are distinct from the other three subscales and from each other.

Read and cite

PDF (accepted version) Publisher's version (DOI)

APA citation

Power, C., Cairns, P., Denisova, A., Papaioannou, T., & Gultom, R. (2019). Lost at the Edge of Uncertainty: Measuring Player Uncertainty in Digital Games. International Journal of Human–Computer Interaction, 35(12), 1033-1045.