An article for the tester AND test coordinator

By Brad Burton

http://people.ce.mediaone.net/bradbad/betatester.html

That new game you want will be out next year. Too bad you'll have to wait. You head over to their site to take a look at it and &endash; what's this! beta signup? Well, it looks like that game you want next year is accepting applications for their closed beta test coming in a few weeks. Maybe you won't have to wait after all. But most likely you will if you only want the beta to play the game.

A beta test is usually the second test (after alpha) of a product and is usually the first test by its actual customers. Beta testing a product can become very tedious for the designer to do all by himself. It's also not recommended because it's likely that you'll overlook several things. That's where us beta testers come in! You, the designer, send your product to a select few of the general public and HOPE we get back to you. Though by doing this you'll always catch those "unreliable testers" that just USE your program for their own benefit, you'll hopefully, thanks to the "reliable testers," find out where the bugs in your program are. Beta testers should do certain things when they're testing a product. The designer (or lead tester) should also do some things to prepare for testing and communicate this to the tester; they need to keep the tester active.

Filling out the beta form on the game's site can be tedious, buts it's extremely important. You need to know your computer settings and configuration. This alone can be difficult if you don't know a lot about your computer. You may need to answer questions relating to the game's genre and list previous games you've played. Most likely they'll ask how much experience you have as a beta tester and how much time you're willing to spend testing the game. These two are the most important. In order to be accepted for a closed beta, you need experience and a LOT of time. If what you fill out meets the requirements of the game and is impressive to the company, then they'll choose your application over others; thus you're accepted as a tester. You will be contacted and asked to sign an NDA (non-disclosure agreement) which asks you to keep everything about the game confidential until they say it's okay. Once this is done they'll send more information on how to get the beta (whether it's downloadable via ftp, or by Fed Ex to your home). Many times they'll send you a free copy of the game when it's released, or other rewards such as a T-shirt.

When I fill out the form I always put a LOT in the "comments" box. I list previous games I've tested and what I plan to do as a tester. I must be willing to devote hours of my time to help with testing. When I was accepted to my first closed beta, this helped me get into other closed betas, which in turn, helped me into other closed betas, until I can make a whole list of games I've tested for experience.

Beta testing is like proofreading a paper. Once you finish typing a paper there're tons of spelling, grammar, sentence structure errors, etc (for me at least). For some reason when you proofread it by yourself, you tend to pass up a lot. That's why I try to have a friend proofread it also because they'll have a fresh look. The designer is probably so used to his (or her) creation by now that he's VERY likely to overlook MANY, MANY things that a person with a fresh look on the project won't. When a product has finally gone through its design and alpha stages it may function, but it's not going to be used by the public (much less purchased) unless it's absolutely flawlessŠwell, for the most part at least. Sometimes it's impossible to get every little bug out of a program, but if one thing (even a little thing) doesn't seem right then the person that's using it will probably get the feeling that the entire product is flawed, especially if you haven't established yourself as a company. That's why it's important for you as a designer to get multiple testers on your project and for you as a tester to catch these bugs and report them.

Some people think they can't get into a competitive beta unless they have the newest model computer, best graphics card, etc. This is not at all true. The company wants to make sure that their product works well on EVERYONE'S computer. They want a DIVERSE group of testers with all kinds of different hardware, not just people that have the latest stuff.

Depending on the game and the number of applicants, a beta test can be HIGHLY selective, but your inclusion into these "HIGHLY selective" tests don't depend on your system specifications as much as it does on the kind of person you are. To get into some of these competitive betas you need to prove to the company that you're a dedicated tester; that you're willing devote lots of your time to TEST not just play games. The more enthusiastic you seem the more they're gonna want you.

The company is looking for a variety of testers, both amateur and hardcore. They are looking for someone that knows about games AND has no idea what they're doing. Think they wouldn't want you testing their RTS game if you don't know what an RTS game is? Not necessarily, you don't always need background on the genre to be an effective tester. Though I do believe the optimal tester knows what he's doing, Jay Adan from Cyberlore Studios makes a good point:

"You can get some VERY good feedback from people that don't normally play the style of game that you have createdŠI believe that these days many developers hope that they'll get some of the mainstream interested in [the] game as well. If you ONLY get feedback from hardcore gamers how will you possibly know how intuitive it is to people that don't normally play your style of game? This kind of feedback is important."

But stillŠdon't just blindly sign up for anything you see if you don't plan on actually testing and giving feedback. It's a waste of time for both you and the company and isn't a way to gain experience.

Alpha testing is quite a bit different than beta. The general public doesn't usually participate in this testing phase. Generally this is done in-house with the programmer and a few of his friends (employees in some cases), however there will be some chances for the public to get in on an alpha test. Alpha testing usually comes in after the basic design of the program has been completed. You will look over the program and make suggestions or give ideas to the designer to help improve it. You'll also report and give ideas to help get rid of or work around any MAJOR problems. Don't report the little things. There's bound to be a number of bugs after a program is created and the designer most likely knows about many of them. This isn't the main concern in alpha testing.

When beginning a testing phase it works best for testers to have a set guideline, a summary, or checklist of some sort. If you're starting a testing phase for your product, try to implement this and let the tester know beforehand that they'll be expected to go through a specific series of tests. This way they're more likely to spend their time TESTING the game instead of just playing it. If you're a tester and haven't been given any guidelines for testing, make up your own. It'll impress the designer if you write up a little report of the things you tested. It's also extremely important to COMMUNICATE with your testers. Even if there's not much to say just let them know that you haven't forgotten about them. Otherwise, most likely they'll forget about you. Sometimes those "unreliable testers" weren't really "unreliable" at all. It's just that they didn't think they were needed anymore. In many cases it's the designer's fault if a tester has become "inactive."

When I got the StarSiege beta CD in the mail, I read the letter that came with it first. It looks like I will get a free copy of the game when it's released! It told me where to report bugs, and where I could find a list of known bugs on the Internet. After looking at the list I will remember not to report these bugs when testing the game. For EverQuest, I won't get a free release copy of the game for testing, but it's still rewarding to know I've helped make a game better. For testing 7 Kingdoms II not only will I get a free final release copy for testing it, but if I do a good job I'll also get my name in the credits under a "Special Thanks" section! I don't always get a beta CD in the mail though. Sometimes it's required that you download the game off their ftp server. A LOT of free memory is needed on your computer to do this, and it's almost necessary to have a cable modem.

My goal as a beta tester is to give feedback on changes I'd make to the game, and to report bugs. I must act like I am the developer of the game. What would I change to make it better? To "draw in" the player? How could I make this game fun not only for beginning gamers, but also for experienced gamers that have been playing the game for months? What can I do to add to the features of the game? Are the graphics good enough? Does the sound add to the game? Does it make the game better, or does it get annoying? I must also try to find bugs in the game. I try to make it crash and make it do something wrong. But before this you have to install the beta. This is the first part of beta testing and sometimes can be very frustrating. It's called the installation test. It's quite simple; make sure the installation process works correctly. If it doesn't, it's extremely important for me to report it. If testing a multi-player game server, stress tests are very common. A stress test is used to put a game "beyond its limits". The company may have a set date and time of when to log on to the game so they can get as many people playing at once as possible. When testing I always have a notebook and pen handy because if the game freezes and I have to restart I'll need to write down any error messages I receive. I will also want to remember every step I took to encountering the bug. Then I restart the computer, open the game, and try to retrace the bug. I look at my notebook and do the same things I did before to find the bug. Hopefully the same thing happens again. I report the bug, restart the computer, and try to find another one. I always include in my bug report my system configuration, and any other programs I had running.

In EverQuest there were quite a lot of options and buttons to press. So I had a little "strategy" to testing it. I would first press every button to make sure they worked properly (Guess what! Pressing one of the buttons caused the game to crash). Then I pressed each one while fighting a monster to make sure they worked in combat. Then I killed the monster using every technique possible. I let it kill me. Then I earn experience, gain levels, and then play using the more advanced options. I walk along walls to make sure my character doesn't get stuck, I test out my skills and spells trying them where I normally wouldn't be able to use them (fire spell in water). These are known as function tests. I report all the bugs I find, and all my suggestions to make the game better. What do you know! In a few days they've come out with a new patch. They've fixed the bugs I reported. I make sure that the repaired bug or newly enhanced feature works correctly in the new patch. This can be called regression testing. Usually there's not a problem (but you never know!).

Testing can also be very frustrating. The whole point to beta testing a game is to find bugs. Trying to reproduce the bug is, by far, the most difficult part of testing. I list on the bug report the EXACT steps I took to encountering the bug. This is where testing can be tedious and often frustrating. When I was playing StarSiege I was in the middle of a battle with another mech when the game crashed. Well, I went back to try to reproduce it and found that this would be close to impossible. There were so many little things I did (and the littlest thing does matter) that no matter how hard I tried it probably won't happen again. I keep playing until I encounter another bug. Then I make an attempt to reproduce it, just like the one before that. It's very rewarding if I am able to reproduce it and may help out quite a lot if I report the bug properly. And yes, sometimes the NDA can be a frustration too. For a closed beta you'll need to agree not to give out any information of the game. So you'll have to wait to show your friends until the release of the game.

I've been through my share of beta tests, and it's a different experience each time. It ranges from very pleasant and rewarding to "They must have forgotten me!" It's a rewarding experience when the producers communicate with me, acknowledge any ideas and bugs I've reported or testing reports submitted, and notify me of when the product's been released. It's also really nice to see my name in the credits or get a free final copy of the product (hehe).

I always like being one of the first few people to see a game before the release. If I'm testing a game I don't have to worry about waiting for delays in the final product shipment, etc. As a beta news reporter I have insight to the game, will know the conditions of the beta and future betas, and with permission of the company may be able to write up a preview or post some screen shots. It's a nice feeling if you receive the final release of the product in the mail when it's all done. Then you can add the game to your beta-testing resume!

Copyright 2000 Brad Burton, used with permission.

Back to Fact and Fic

Back Home