Emulating a Variable Reward Schedule In Trekking the World

Emulating a Variable Reward Schedule In Trekking the World

Nick Bentley Nick Bentley

Welcome to this week's behind-the-scenes post.

Today's is about a design principle we applied to make our new version of Trekking the World. I’ll start with the principle and then discuss how we applied it. Here it is:

The players' experience should emulate the feeling of a Variable Reward Schedule.

You can think of a game as a system for rewarding actions. The more motivating the rewards, the more compelling the game tends to be, all else equal.

Conveniently, behavioral scientists have experimented with many different reward patterns in task-learning experiments structured like the turns of a game.

An example: on every turn an animal performs a task well, it might get same size reward. Is that the most motivating pattern over turns?

Turns out no. One reward-system has emerged as the most motivating, and it's variable:

The Variable Reward Schedule

In a Variable Reward Schedule, the size of the reward varies from turn to turn, instead of staying the same.

This pattern tends to feel compelling and addictive, and therefore motivating. For this reason, Variable Reward Schedules are everywhere. 

For example, every social media platform builds them in. Below is a sample of my tweets. Each blue bar is one tweet, and the bar's height is the number of impressions it got:

Many people experience attention as a reward, so the pattern above emulates a Variable Reward Schedule, and thus makes folks more motivated to post on social media.

If you were going to build a Variable Reward Schedule into a strategy game, you'd want the players' actions to determine the rewards, depending on the amount of luck you wanted to build in.

It turns out a Variable Reward Schedule can be put into a game with any amount of luck. The standard Variable Reward Schedule is 100% luck. Tweeting has less luck. Then consider the legendary game Go, which has no luck: 

In the game, you can capture your opponents' pieces. the number of pieces you capture in a turn varies unpredictably across turns and games as in a Variable Reward Schedule:

  • On many turns you don’t capture anything.
  • Sometimes you capture a small number of pieces.
  • On rare occasions you capture a whole bunch.

The games we make at Underdog aren’t so serious or strategic, but the above illustrates a game can emulate variable rewards in systems with any amount of luck.

How we’re emulating a Variable Reward Schedule in our new version of Trekking the World

We've completely overhauled the mechanics of Trekking the World for it's 2nd Edition. One of our goals was to build stronger Variable Reward Emulation into it. 

Without going into details, here’s how:

Depending on your choices, there are two kinds of resources you can accumulate (among others): Itinerary Tokens, and Encounter Powers.

Importantly, you can save these tokens and powers up and then spend them together. When you do, they sometimes create effects greater than the sum of their parts. This creates the feeling of “pulling off a huge turn”, and thus replicates the “big moments” in a Variable Reward Schedule.

As an example, here are two encounter powers that enhance one another:

The card on the left can multiply the effect of the card on the right to create a major windfall.

If we've done our math right, the bigger a windfall is, the rarer it is and the harder it is to achieve.

Also, many of the game's rewards come to you in the form of physically acquired objects. Humans tend to feel more rewarded by physical objects than abstract quantities.

We've also tried to ensure big comebacks are possible, there aren't too many blowouts, and big turns don't slow the game.

Don't hesitate to share your opinions in a reply, positive or negative. I read all your replies (and reply when time permits) and it helps me understand what we're doing right and wrong.

Nick Bentley,

President, Underdog Games Studio

« Back to Blog