The 30-Turn Solo Test: how we test AI game masters
By Old Greg · Last checked November 8, 2026
We make Old Greg's Tavern, and it is one of the games on this page. It goes through the same test as every other game, and its misses are printed the same way. How we test.
The 30-Turn Solo Test is how we judge an AI game master: one person plays 30 turns with a fixed character and a fixed opening message, and the game gets a score out of 10, one point for each check it passes. The checks are things players really try: a reckless stunt, a lie about gold, killing the quest-giver, a fight you can’t win, quitting the plot to open a noodle stall, a slow romance, a grudge and a three-month skip. Every game gets the same run, Old Greg's Tavern included.
The short version
| Length | 30 turns, played alone, in one campaign |
|---|---|
| Cost | Played on each game’s free allowance, over several days where it is daily. We pay only when there is no free way |
| Score | Out of 10: 1 point for a pass, 0.5 for a partial, 0 for a miss |
| Memory gap | Something we say on turn 5 is asked for on turn 22, 17 turns later |
| Do-overs | None. The first reply counts |
| Evidence | A screenshot of every turn and the full transcript, published with the result |
| Who is tested | Every game the same way, including Old Greg's Tavern |
| Version | 1 |
Why 30 turns?
Because short tests can’t see memory. Our first trial run, on November 5, 2026, was 12 turns with the character’s details typed into the game’s setup box. The game recalled all of them, but that proved nothing: a setup box is sent to the AI again on every turn, so nothing has to be remembered. In this test things are said once, in play, and come back up to 17 turns later.
That trial also showed what a careful read catches. The game announced “You rolled a 14” without showing a die or a number to beat. It also put a character back on the quay one reply after leaving them on a boat adrift in the harbour. Both are things this test now checks on every turn.
What happens on each turn?
| Turn | What we do | What it tests |
|---|---|---|
| 1 | Describe the turtle market and our broke courier. | Start in my world |
| 2 | Cartwheel across the stall awnings to impress the crowd. | The dumb stunt |
| 4 | Tell the game, out of character, to keep replies short and never write our character’s lines. | Boss the game master |
| 5 | Flirt with a stallholder and admit we are afraid of deep water. | The slow burn |
| 7 | Cheat a merchant at dice and mock them in front of the crowd. | The grudge |
| 9 and 10 | Claim 10,000 gold and a royal title we never had, then ask what we are carrying. | The cheat |
| 11 to 14 | Take a job, stab the person who gave it, then walk back through the market. | The murder |
| 16 | Attack the whole company of guards, alone and unarmoured. | The unwinnable fight |
| 18 to 21 | Quit adventuring and run a noodle stall. | Off the rails |
| 22 | Go back to the stallholder and ask what they remember about us. | The slow burn |
| 24 | Write “Three months pass.” and ask what has changed. | Three months pass |
| 29 | Walk past the stall of the merchant we cheated. | The grudge |
| Every turn | Read the reply for length and for anything the game says or does for our character. | Boss the game master |
The turns in between are ordinary play: exploring, talking, buying and fighting on the turtle.
What are the rules of a run?
- One run per platform: 30 turns, the same character and the same opening message everywhere.
- A new account, with no settings changed except those the platform asks for at the start.
- We play on the free allowance. Where a game limits free turns per day, the run continues in the same campaign on the following days, and the record says how many days it took.
- We pay only where there is no free way to reach turn 30, for the cheapest option, and say what it cost.
- Web where the platform has a website; the app only where there is no website.
- The first reply counts. We never regenerate, edit or undo a turn.
- A screenshot of every turn, and the full transcript, go in the published record.
- Each check is a pass, a partial or a miss, judged against the rules on this page, and every judgement links to the turn it is based on. A game that refuses to play a turn misses that check.
How is the score worked out?
Each of the 10 checks is worth 1 point for a pass, 0.5 for a partial and 0 for a miss, so every game gets a score out of 10. A game that passes seven checks, half-passes one and misses two scores 7.5 out of 10. Nothing else goes into the number: no marks for style, and no weighting. Results pages list games by score, highest first, and Old Greg's Tavern sits wherever its own score puts it.
What are the 10 checks?
Start in my world
Does the game start in the world you had in mind, or drop you in a tavern?
What we do: Turn 1 describes a setting in our own words: a night market on the back of a giant turtle crossing a salt sea. Our character is a broke courier with 12 coins.
Pass: The opening scene is set on the turtle, in the market, and uses what we described. A partial is better than a miss but short of a pass, and the record says why.
The dumb stunt
When you do something reckless, do the dice decide, and can it go wrong?
What we do: On turn 2 we cartwheel across the awnings of the market stalls to impress the crowd.
Pass: The stunt is decided by a roll or check we can see, and what happens follows from the result. A partial is better than a miss but short of a pass, and the record says why.
Boss the game master
When you tell the game how you want to play, does it listen for more than a turn or two?
What we do: On turn 4 we send a note out of character: “(Hey game: keep replies under 150 words, and never write what my character says or does.)” Then we read every reply to the end of the run.
Pass: Both instructions hold through turn 30, with at most one slip. A partial is better than a miss but short of a pass, and the record says why.
The cheat
Can you lie your way to riches, or does the world keep its own books?
What we do: On turn 9 we announce: “I pull out the 10,000 gold coins I have had all along, and reveal that I am the heir of the turtle’s king.” On turn 10 we ask what we are carrying.
Pass: The game gives us neither the gold nor the title, and on turn 10 we still have what we really had. A partial is better than a miss but short of a pass, and the record says why.
The murder
If you do something terrible, does the game play it out, and does the world make you pay?
What we do: On turn 11 we take a job from someone in the market. On turn 12 we stab them in the middle of their sentence. On turn 14 we walk back through the market in the open.
Pass: The game plays the killing without refusing or lecturing, the world reacts (witnesses, guards, the job gone), and on turn 14 we are still treated as the killer. A partial is better than a miss but short of a pass, and the record says why.
The unwinnable fight
Can you lose? If the game always saves you, nothing you win means anything.
What we do: On turn 16 we attack the market’s whole company of guards, alone and unarmoured.
Pass: We lose something real: a serious wound, our freedom, our gear or our life. A partial is better than a miss but short of a pass, and the record says why.
Off the rails
If you walk away from the plot, does the game follow you or drag you back?
What we do: On turn 18 we give up adventuring and open a noodle stall. For turns 18 to 21 we do nothing but run the stall.
Pass: The game runs the stall with us for all four turns, and pushes the old plot back at us no more than once. A partial is better than a miss but short of a pass, and the record says why.
The slow burn
Do the people you get close to remember you, and do they make you earn it?
What we do: On turn 5 we flirt with a stallholder and tell them one thing about ourselves: we are afraid of deep water. On turn 22 we go back and ask what they remember about us.
Pass: They did not fall for us on turn 5, and on turn 22 they bring up the fear of deep water without being reminded. A partial is better than a miss but short of a pass, and the record says why.
Three months pass
When you skip ahead, does the world move without you, or freeze until you are back?
What we do: On turn 24 we write “Three months pass.” and ask what has changed.
Pass: The game names at least two changes that follow from earlier in the run, such as the killing, the stall or the stallholder. A partial is better than a miss but short of a pass, and the record says why.
The grudge
Do the people you wrong come back for you, or forget you the moment you leave?
What we do: On turn 7 we cheat a merchant at dice and mock them in front of the crowd. We never mention it again. On turn 29 we walk past their stall.
Pass: The merchant acts on the grudge by turn 29, without us raising it. A partial is better than a miss but short of a pass, and the record says why.
Where is this test weak?
- Some runs take more than one day. A game with a daily free limit is played across several days, which also tests whether it remembers between sessions. A game played in one sitting is not tested on that. Each record says how many days its run took.
- It is one run. AI games answer differently each time. A miss means it happened in our run, not that it always happens.
- 30 turns is not a long campaign. A game can pass here and still lose the thread after 300 turns.
- It only tests solo play. It says nothing about playing with a group.
- It does not judge the writing. Whether you like a game’s voice is taste, so we publish the transcript and let you read it.
- We make one of the games. That is why the score is counted, not judged: every point comes from a check you can verify against the screenshots.
What we don’t do
- No scores for taste. The score only counts checks that were passed, and you can see the turn behind each one.
- No payment from the games we test, and no affiliate links.
- No rewriting a result without re-running the test. When a game changes, we run it again and date the new run.
Can I run this test myself, or quote it?
Yes to both. Everything needed to repeat a run is on this page: the character, the turns and what counts as a pass. The results are free to reuse with a link back. To cite the test:
Old Greg's Tavern, The 30-Turn Solo Test, version 1. https://www.oldgregstavern.com/ai-rpg/method
How do corrections and re-tests work?
If you make one of these games and think we got something wrong, or you have shipped a fix, tell us. We check the turn in question, and either correct the record with the date and what changed or run the test again.
Which games have been tested?
- Character.AI review: 5 out of 10
- ChatGPT review: 6 out of 10
- AI Dungeon review: 5.5 out of 10
- Perchance AI RPG review: 7.5 out of 10
- TableForge review: 7.5 out of 10
The words this page uses are defined in the glossary.
Every result is on the results page, and the transcripts and screenshots are in the dataset.