The Trap of Evaluating Gameplay

Academia loves to create definitions for anything and everything. And, most of the time, that is extremely valuable and results in amazing progress for science and humanity.

But… (and you knew one was coming given the title) it can also be a waste of time. I spent 3 months reading pretty much everything out there on what ‘‘balanced’’ meant for different researchers and game designers. There were quite a few ideas floating around.

A system where everything is used equally. A game where there is no dominant strategy. An environment where all elements are valuable and none waste away. Challenges that are always passed successfully by skilled players but not by weak players. Challenges that are approachable by anyone, regardless of skill.

And you know what? They’re all wrong. Every single one of them. And you know what else? They’re all correct. Each one is true.

Do you think Shigeru Miyamoto (of Mario fame) agrees with Edmund McMillen (of Binding of Isaac fame) on what ‘‘balanced’’ means? I don’t know, but what I do know is that their games follow completely different design philosophies. And neither approach is as result of academic work.

I had focused on the wrong part of balance: defining it from my position of researcher. It took working in the industry for a brief while to understand what is valuable and what isn’t to game designers and developers.

My approach

Which leads us back to my 3 months of research. I realised I can’t do relevant work if I impose my own views on what ‘‘balanced’’ means for games. Being a gamer, having played so many different titles, from casual arcade games to extremely punishing RPGs, I had to take a step back and think what is valuable to a gamer or game designer, not to a researcher. Because, after all, research is done to push the boundary of human knowledge and help the relevant areas prosper and grow, not to be a small echo chamber of publications.

So, with the help of the aforementioned industry placement (thanks everyone at MindArk!), I changed my approach to the topic. My algorithms no longer assume anything about balance. Instead, I have designed ways for people to define their own goals as simply as possible. Game developers decide what ‘‘balanced’’ means for their game.

Game balancing is the process of systematically modifying parameters of game components and operational rules in order to determine satisfactory configurations regarding predefined goals.

That above is my non-definition definition.

Is the number of times a player had to restart the level something valuable? Are you yet uncertain as to what you want that value to be? Do you want it higher for some levels and lower for others? Do you want hard difficulty to be easy and easy to be hard? My code will not stand in the way of your creativity. You define it, I facilitate some balance suggestions. If you want to use those suggestions, great!

Results

Initially I believed this generalisation would result in harder work. However, the opposite is true. By allowing flexibility in designing requirements and by detaching the game from the algorithm, I was able to easily expand my work to any game (given a few hours of work on creating a bit of software to actually communicate between code and game). Some of the games I have worked with include Ms Pacman, StarCraft, TORCS, Teeworlds and ComPet.

Not only that, but the independence of game simulation from algorithm means I can attempt to balance virtually anything that can be represented as a list of numeric variables. Want a fun and wacky agent playing your game? Evolve a neural network and tell it to value wacky behaviour above all else (definition of ‘‘wacky’’ not yet available) (See this publication for some more details)!

So?

So, at the end of it all, I am a big supporter of the idea that some things, at least in games, should not be defined. Games are art and the creative vision of their creators should not be bogged down by science. Science should facilitate the creation of better games!