Beta Testing in Wizardry 8
Author: ~Lockup~
Date: 03/07/2001

Beta Bites recently spoke with QA Lead Terry Brose about his role at Sirtech Canada, and how they ran the Beta test on their roleplaying game, Wizardry 8.

BB: ~Lockup~ Terry: Terry "Pokey" Brose

1. BB: Tell us a little about yourself and your role at Sirtech Canada as a QA Lead.

Terry : First just let me say, I am Canadian !! Buy our game or we will win the World Series again, we have Provinces here, and not States, and our prevalence to say "eh" and "aboot" is actually a subtle ploy to constantly proclaim our nationalistic pride to the World. I drink too much coffee, and have a strong aversion to marriage.

Define my role at Sirtech, eh... Well that would be to send any semblance of sanity remaining in the Wiz 8 team into oblivion by inundating them with as many bugs as I can find, reproduce, and validate. ;)

Of course, you might think my job is aboot actually preventing the game from ever being released, and torturing our fans as long as possible. Oooohhh, the power.. I'm drunk with it...

2. BB: How did you get the job as a QA (Quality Assurance) Lead?

Terry: You know I used to wonder how people got jobs at gaming companies. Often people in the industry would tell me it was just a stroke of luck, right place at the right time. Many of them had no previous experience and actually hold degrees in completely unrelated fields. I am now proud to join the fervent few.

One day an AC friend of mine (waves at his friend) sent me an e-mail telling me that Sirtech had posted a time limited Beta application on its Wizardry site. Now, she knew the Wizardry series was one of my favorite games, so I thought she was teasing me and replied to that effect. She finally convinced me it was no joke. I checked the site and filled out the application never dreaming I would be accepted. A few e-mails requesting more info flew back and forth... then nothing.

Maybe a week or two later, I received a phone message taken by a friend with only a phone number written down. When I returned the call, someone answered saying "Sirtech." I was trying to figure out which of my funny friends it was so I could respond with a suitable derogatory remark when Linda Currie identified herself. I did the same, and she explained that they were wondering if I would consider coming to work for them. Once I got my voice back, and found a chair my response was, "Sure !! When and where?"

YES, Virginia there is a Santa Claus, miracles do happen, Karma does exist. 8)

There are a myriad of factors involved. Think of a game as a symphony with the instruments making up the components of play, the whole, perfectly wrought, invokes a certain ambiance or emotional response. While one instrument out of key may not ruin the symphony, it can certainly detract from your experience. It's our job to find and tune those instruments.

QA testing is most definitely not all fun and games as some would have you believe. It can often be an extremely tedious and repetitive task. While we may be aware of a bug, the circumstances which cause it can be difficult to pinpoint. This may require constantly replaying a battle over and over, or redoing a quest adnauseam, all the while noting and eliminating variables each time until the bug becomes reproducible.

4. BB: Tell us a bit about the role-playing game, Wizardry 8.

Terry: Truly, you do not want to listen to me gush. ;) Check our site - www.wizardry8.com

Of course, I can tell you this: It's one of those games that will sneak up and grab all the extra time you thought you had. You will sit down to play, and suddenly realize it's the wee hours of the morning. You will once again rue the fact you have to participate in real life. Your significant other will attempt to sabotage your computer, and if you have children, they are likely to forget who you are. Start saving up that holiday time; you're going to need it.

Dominus is a wondrous place, full of adventure wrought with subtlety and nuance. Humor and terror abound in equal proportion. Explore at your leisure taking on the challenges of the world or pursue the climax of the story with single minded determination, but beware your actions have consequences...

5. BB: When Sirtech announced a beta test for Wizardry 8, how were the applicants selected? Did anyone lie, and tell you lame things to get into the beta? :) How many testers were finally selected?

Terry: A sign up period was opened for about a week on the Sirtech website. From those applicants, about 150 were chosen to participate in the next level of qualification. These "finalists" were sent qualifying questionnaires and, based on their answers, the final External testing team was chosen. The external portion of the test was limited in number with about 40 testers selected. The beta was sent out in phases with each phase adding new testers (and on occasion, dropping those who had not reported regularly). While we would have liked to included more people [~Lockup~ *sighs* for she was included.], the logistics involved in preparing the materials for each tester was not insignificant. Since it was limited, we had to be sure that the people we chose were dedicated to helping us TEST the game, and not just trying to get their hands on an early copy. We had a lot of great people apply, and we are very grateful to those people who volunteered so much of their time to helping us fine tune Wiz 8.

7. BB: How do you manage a Beta team? What/how are you looking for your testers to perform?

Terry: Actually, it boils down to recognizing the strengths and weaknesses of the Beta Team, the varying testing styles, and keeping an open mind and listening to their differing opinions. Different team members are better at different types of testing. It's important to be able to match the individual styles with suitable tasks whenever possible.

There are those who really like testing combat and are quick to pick up discrepancies in the AI, others thrive on character creation and the balances that must be maintained, some prefer breaking the quests or puzzles, others finding game exploits, and the list goes on.

While we do need testers to just play the game, they must also pay particular attention to minute details. It is very important that testers report to us step by step, in as much detail as possible, the exact sequence of events which lead to the bug they are reporting.

8. BB: How visible and accessible were you to helping out Beta testers with their problems or bugs? How were bugs submitted to Sirtech? Ex: Forum board, e-mail, bug reporting form?

Terry: Being a relative new addition to Sirtech I was actually very fortunate to have Linda screening all the external reports first. Linda received e-mails from testers daily, and when warranted spoke with them directly by phone. She prioritized and logged them for the Internal teams attention. This allowed us to concentrate on trying to resolve issues as quickly as possible so we could bring the game to our fans as soon as possible. :)

9. BB: Overall, how did the public selected beta testers perform in beta testing of Wizardry 8? [Would you tell me the truth if they all sucked? :)]

Terry: Our External Beta testers performed admirably, and contributed greatly to the stability and finishing touches on the game. Some of the suggestions we received were implemented in the final product.

Would I tell you if they sucked, eh... Undoubtedly, by now you would have heard about the nerf gun and foam bat attacks...

We use a detailed questionnaire to narrow down the applicants. If you are at all annoyed in filling out a lengthy questionnaire, then you should rethink your interest in joining a beta test team. ; ) Try to answer the questions such that you get across as much information as possible, as concisely as possible. What you hope to gain can often reveal as much as what you hope to contribute.

11. BB: Now that the Wizardry 8 public beta is ended, where does your role take you in finishing up the product?

Terry: While the external beta may have come to an end, the internal beta still continues as we finalize and polish the game for publication. Eventually, as with all children, the time will arrive for Wiz 8 to leave the nest. Lately, this one has been spreading its wings. ;)

12. BB: Wizardry 8 has some features that make testing really important. Sirtech moved the Wizardry 8 game into 3D acceleration with a new 3D game engine. How much of testing comes down to hardware compatibility problems or bugs with the game?

Terry: Hardware compatibility is definitely an issue in game development. We planned for a major compatibility cycle in our testing, and expected to encounter a number of 3D related problems (keep in mind this was Sirtech's first foray into 3D). In the end, it didn't turn out to be the nightmare we expected. For this, we can thank our Lead Engineer who did spend a fair chunk early on in the development working with different hardware setups.

The days of DOS and the complications it caused are pretty much gone now, but differing hardware configurations can still be a headache. The most important thing that we want to communicate to players is that they MUST make sure that they are running current drivers for their hardware and that the stock drivers that came with their video or sound card will likely be old and buggy. Visit your hardware manufacturer's website, and download the latest drivers! [gets off soapbox]....

Some of the quests in the game actually encompass traversing several areas to complete. However, you may encounter other NPC or RPC characters, and decide to undertake their quests while on your journey to complete the first. Some of these may be of a conflicting nature, and since we do not control where you go when in the game, we have no way to predict in what order you may encounter or choose to undertake these.

The AI problems we encountered during development were more of a result of the implementation of the 3D world, and of our high standards in what we wanted the creatures to do. If we were content to just let them stand around until you came into view, it would have been easy. ; ) But our goals went further. Creatures actually live and breathe in the world, and getting them to react accordingly in all the possible situations was a difficult task.

And then, too, is finding and recognizing the glitches that could be exploited to make for unfair battles. For instance, at one point in the development, you could lead creatures to certain types of terrain where they would stand there stupidly and allow themselves to be slaughtered. We had to determine what situations would set up this "turkey shoot" and make sure the creatures were smart enough to do something about it. We had to give them enough brains to let them react to certain conditions you could inflict on them. We are happy to report that creatures can and will retreat, flee, use appropriate type attacks, split up to surround you if possible, and, upon occasion, may even call others to their aid. ;)

14. BB: Most of us know that when a game is released both as a singleplayer and a multiplayer, the multiplayer generally (not always) is a bugfest. Did the problems with bugs in multiplayer games influence Sirtech' s decision to not make Wizardry 8 multiplayer?

Terry: Anticipation of bugs really was never a factor in deciding not to do multiplayer. Development resources and game design were. Including multiplayer in Wiz 8 would have been like making two completely separate games, and it became evident early on that it would be unfair to our multiplayer goals, or to the single player epic Wizardry to try to do both. Bringing the Wizardry of old into the modern era, and molding a design that had it's roots in abstract pen and paper concepts was a very large undertaking for just the single player portion. We knew that we would not do justice to muitiplayer to include it as well. That's not to say we don't have multiplayer designs, but there's a place for those designs, and it wasn't in Wizardry 8.

15. BB: Once Wizardry 8 is out at the stores, and people start purchasing it, more bugs or problems will surface. How do you handle a finished products bug problems and/or the general public's request for tech support?

Terry: Just WHAT exactly are you implying? That the beta team did not do its job.... hmmm??? [reaches for foam bat].

We've been very fortunate in the testing that we've been able to devote to this product. And while we endeavor to turn out as bug free an experience as possible, bugs do occasionally turn up in a finished game. We have a pretty good track record in supporting our products, and of course, will do the same with Wiz 8. Never forget that we welcome all types of feedback on our games.

I believe we are currently attempting to implement a forum on our site. This would give us a centralized location for feedback, as well as allow us to respond to our customer base when the situation warrants.

16. BB: We all have heard the "Ian Currie Wizardry 9 and 10 rumors." Are you planning on being a factor in testing these upcoming games?

Terry: Well, for now, let's just say that they managed to convince me to come out of semi-retirement, something a few others failed to coax me into, so if I'm needed they will have use of me.

17. BB: Anything you would like to add, brag about, or apologize for? Now is the time....:) YeeHaw!

Terry: Hiya Dad!...and you thought playing games was a waste of time. ;)

18. BB: How did you get the name Pokey?

Terry: While you seem to be inferring that I may be just that, it's actually the other way around. As we all learned from Einstein, time is relative, and as you approach the speed of light time actually slows. So, it may appear to others that I'm actually moving at a slower than normal speed. heheh

Beta Bites would like to thank Pokey, I mean Terry for spending time answering questions with us. :) Hope you all learned a bit more about Beta testing.

Be sure and check out the BB Wizardry 8 Preview for information on the game and lots of screenshots.