How can I test a system I am creating?

Question

I'm currently trying to create a fairly basic system for running table top role playing games that is loosely based on my experience with Dungeons and Dragons 4th Edition and Pathfinder.

By system, I mean new character sheets, rules, ways of attacking (and defending) and GM rules - so in short, a new game.

It's fairly open (there aren't many specific rules other than what actions you can do and when), and ideally can be used over a multitude of settings (such as fantasy, sci-fi, modern day etc.)

Once I have completed a "first draft", what is the best way of testing my new game? And what questions should I be asking myself as it is tested? I'm interested in the entire testing process, from early days to late days.

Answer 1

You should test different aspects of the game in phases;

Phase 1: Rule mechanics / balance

This is something you can do on your own. You are proposing some game mechanics, formulas, dice, probabilities etc. Lay them out in front of you and calculate the end result probabilities and cross-interactions. What happens when character A attempts action B in situation C. Three possible outcomes of X, Y and Z. What are the probabilities? Is that what you want?

This phase can be a bit math-intensive but gives you a good big-picture of your game. Modify and tweak as needed, then move onto phase 2 when you got it approximately right. You will be revisiting these calculations later but for now, "about right" is "right"

Phase 2: Rules playtesting

Get yourself a few playtester players and run a few games. Your players will find and exploit a huge number of flaws and loopholes. Go back to phase 1 as needed to revise your rules.

Phase 3: Setting review

This only applies if you also have an established setting in your game. Have other people read your setting material and gather feedback. Clear out inconsistencies, fill gaps and ambiguities as needed.

Phase 4: Alpha testing

By now, you should have your rules and setting written out. Run games for playtesters but now, also join games as a player and observe from a different perspective. If possible, have others run and play games without you, and gather feedback. Revisit previous phases as needed. Move on when the game feels complete and solid.

Phase 5: Beta testing

…is similar to Alpha testing but you should have your book/pdf/website in your intended final format. Keep gathering feedback, but keep the focus on the output material. Build towards having a clear and understandable layout of your rules and setting. Fix your wording to clarify details.

At the end of it, you should have a working game. How much effort you put into any of these phases depends entirely on you and your game.

Answer 2

There's three questions you'll ask yourself repeatedly:

1. What are the rules supposed to do? (intent, spirit of the rules, etc.)
2. How are the rules doing that?
3. Is this communicated well enough people other than me?

Proof of Concept Testing

The first set of testing might be playing around with sections of the rules up to full sessions. Playing out a combat scene, screwing around with magic. If you've got friends who are generally into system nerdery, they're good for this.

Players should be testing with good intent - don't try to break or push the limits yet. You want to see if these ideas even work when people are trying to work with them, long before you push edge cases.

Everyone should ask questions while playing! "Hey, does this work like this, or like that?" Write down those questions, because it's going to show you what is important to communicate in your writing. If something is an edge case or not designed yet, just tell them and move on.

Things may not be fun at this point. Let everyone know that. Unconnected fights, playing around with subsystems - it can be very tempting to start fudging and changing everything as you play, but usually what happens is you'll end up pushing your game towards something you already know and not actually learn what these rules actually do.

This stage is pretty early and you may not need to do a lot of this, or you might need to do quite a bit. It depends on how many crunchy subsystems you have and if your core game mechanics push out in weird ways.

Stress Testing

Once you think the rules generally work, then you can start playing with people you don't know quite as well, and also have them push on the edges a bit. If that seems solid, you can ask players to start looking to deliberately push the edges of the system.

What happens if you min-max? What happens if you try to spend all of your power points? What's the most broken combo you can do? Is there a way that someone might get NO points? etc.

No matter what you do, players will find breaking points. The issue is not to create a game that cannot be broken (RPGs can always be broken), the issue is to figure out which of these cases you need to pay attention to.

Some of these problems are non-problems. If it requires playing with bad intent, or lots of work to break the rules, you don't have to worry about designing for that. You need to first look for the issues that are pitfalls people MIGHT accidentally fall into, and deal with those. ("Oh, I could see how a sub-genre of superhero stories does this thing, and this game doesn't do that. Is there a rule or advice I can put in about that?"). The second one is pitfalls that are easy to fall into that break the game that maybe you didn't think about. ("Yep, those two powers together are broken. Let me fix that.")

You should also be looking to make sure things CAN BE fun at this point. Be mindful of whether the game even fits what the players want - the guy who doesn't like these kinds of rpgs ("Too much politics, not enough action") aren't going to give you useful feedback other than to be clear on what kind of game it is. It doesn't have to be 100% fun, it just needs to be consistently hitting it more.

Pay attention to what best practices matter, what style of running helps it, what it does differently or how the GM or the players might have to think different about the game. Apocalypse World's Principles work well as an example.

Textual Review

Last part - having people playtest without you leading it, just going from the rules. This is testing your ability to communicate the rules, more than the rules itself. It's really easy to leave important things out - "Of course you play THIS way" isn't common knowledge to everyone.

If you can watch them play, or if they record a podcast or video of the play, it helps. Not everyone is good at self reporting and you can see where things go wrong or off the rails and consider if that's something you can fix in writing or if it's soemthing the group misread or didn't read at all. (This post-mortem video about D&D 5E shows them talking about nearly 50% of the feedback they got was simply people not reading the rules about an hour in.)

Pay attention to where people have questions. Where they say they had one kind of play experience that seems unusual or not to match up with what you expect or have seen in play. If someone's having fun in a way you didn't expect ("Dude! The skill rules change EVERYTHING!") ask some questions, see if they're playing it in a weird way or if they've stumbled upon some best practices you didn't know.

Playtest Pitfalls

Be really clear on the feedback you're looking for at any stage. Some people's feedback will be "Make your game into this other game I want instead" - that's not helpful.

Don't stress things that aren't finished yet. If you see something works/doesn't work, then you can end play early. Don't grind through endless unfun if you've already seen it doesn't work.

Your rules will go through 3 stages - notes you can understand, clear sparse directions for others, and finally a full text with extra handholding and "encouragment". Don't stress making it a fuller text than what you need at that point.

Answer 3

I am not that experienced with creating my own games, but I will try to answer your question from my experience with a game a friend of mine is creating with my help: (Also, from your question I'd guess that your system is playable in its current state)

What can you do?

Find a group of friends that could be interested in the system, and gm it. You'll get not only a direct look into your own system (as a gm, that is inevitable), but also some feedback from people other than you. I would suggest to play something like a one-shot and keep notes on the things people mention, like "Ohh, that sounds a bit hard to accomplish" but also on positive things and strengths your system may have. Tell your players from the beginning, that you want to know what to improve and that they should not hesitate to tell you if they think something is not done very well. But also encourage them to tell you what they like about the system.

What to ask yourself?

Most important for me would be: Does the system achieve what you want it to achieve? I guess you wanted to create your own setting and ruleset because of a concrete reason, instead of just using one of the many many existing systems. In the case of my friends, he wanted a mix of social interactions and making crafting actually valuable. So when we eventually play that setting, those are the things we will focus the first play tests on. But we'll still be fighting and do as many other things as possible, because we want to test them. But don't hesitate to "limit" a session/one-shot to a narrow part of your rules, this way you can test your rules in more detail. You will just have to find you own style (and the players to fit your style) for this.

Also, I would highly recommend to listen to your players too and maybe let someone else try to gm the game instead of you. You will then have the experience of being both a player and a gm in your setting, and you can see how other people play both roles. This might give you hints and even concrete problems you have to work on too.