Research / Indie developers and the games industry

Full paper · 2024

Towards Democratisation of Games User Research: Exploring Playtesting Challenges of Indie Video Game Developers

Denisova, A., Bromley, S., Mirza-Babaei, P., & Mekler, E. D.

JournalSJR Q1Proceedings of the ACM on Human-Computer Interaction (CHI PLAY 2024), 2024

Teaser figure: Democratising games user research. Interviews with 13 indie games professionals revealed five themes of playtesting challenges: early playtesting, recruiting players, choosing methods, analysing data and acting on feedback. These are hard because of lack of resources, lack of GUR expertise and wearing multiple hats.
Open full-size image

In short

Playtesting is a core games user research method, but recommended practices often assume resources and expertise that indie developers lack. We interviewed 13 indie games professionals about how they playtest and what gets in the way.

How the study worked

Participants
13 indie game developers, mostly in North America, Australia and Europe; none had formal training in user research
Studios
Team sizes ranged from 2 to 35 people
Recruitment
Social media, personal inquiries and developer communities, such as the IGDA Games Research and User Experience SIG Discord
Interviews
Semi-structured video interviews lasting between 45 minutes and 1 hour
Analysis
A codebook approach to thematic analysis

In developers’ words

  • Not having people playtesting something and giving feedback that is honest

    P1

  • I don’t have their email address, so making those connections with other people

    P8, on recruiting new players

  • Oh my God, somebody said this, we can simplify this and make it clear

    P2, on reacting to feedback

Key findings

  • Finding suitable playtest participants is a key challenge for indie developers.
  • So is handling the data that playtests generate.
  • We discuss how games user research practices could be adapted to make playtesting more accessible and useful for indie developers.

The challenges indie developers described

The five themes from the interviews, as named in the paper.

  1. Regular and early playtesting as reassurance: creating the right experiences for the right audience
  2. Recruiting enough representative and trusted players to gather objective and honest feedback
  3. Choosing appropriate research methods to gather valid and reliable data
  4. Interpreting and analysing collected data to extract valuable insights
  5. Dealing with feedback and prioritising actions

Suggestions from the paper

  • By tracking how many people sign up and participate, developers can determine the initial level of interest.

    Dual purpose of playtesting: enhancing game quality and validating market potential

  • Real-time feedback can offer valuable insights into players' moment-to-moment experiences.

    Exploring new tools and research methods for evaluating new and unique player experiences

  • Combining methods like data analytics and observations can be effective for pinpointing relevant information

    Handling large amounts of unstructured data to extract relevant insights

  • Creating and sharing research guidelines is another area that academic researchers can contribute.

    Democratising user research to empower indie developers to conduct research themselves

  • Educating and empowering others to be able to do research could be also seen as a researcher's main duty.

    Democratising user research to empower indie developers to conduct research themselves

Read and cite

PDF (accepted version) Publisher's version (DOI)

APA citation

Denisova, A., Bromley, S., Mirza-Babaei, P., & Mekler, E. D. (2024). Towards Democratisation of Games User Research: Exploring Playtesting Challenges of Indie Video Game Developers. Proceedings of the ACM on Human-Computer Interaction, 8(CHI PLAY), Article 343. https://doi.org/10.1145/3677108

More on this topic