Fun Friday: QA Funnies Edition

Game Developer's Matthew "Wasteland" has taken the task of infusing some funny into their industry leading magazine. His back page article this month tackles QA bugs, but not the good kind. Here's my favorite:

1. Put the game in the disc tray and start playing it.
2. Walk 56 steps to the left and drink the green potion.
3. Explore the caves and proceed to the point where the wizard talks to you, then skip the dialog after about the third panel. I forgot what it said.
4. Hit the attack button repeatedly for five minutes.
5. Pause the game and leave it on the Pause menu for about as long as it takes to make and eat a microwave burrito, I think like 15 minutes.
6. Unpause the game, turn on all of the cheats, warp to the vampire's lair and shoot the panzerfaust at him over and over as fast as possible.
7. Notice that the game has frozen.

I'm sure the developers response to this bug was either, "We'll fix it in the sequel," "But LotCheck/Certification didn't find it," or " Do you think we can get this waived?"

Puzzle Quest Postmortem: Required Reading

Puzzle Quest developer, Infinite Interactive, posted a postmortem for their surprise hit in the September issue of Game Developer magazine. Of all the postmortems that I have read in this magazine, I was most impressed, and inspired, with the philosophy and wisdom the developers shared in the article. While many postmortems seem to spout the same "what went wrong" and "what went right" issues while creating a game, Infinite Interactive touched on many things that I have seen, and have been bothered by, during my short time in the industry. I have yet to play the game, but now that it is out on Xbox Live Arcade, that will be remedied soon.

This is the first postmortem that I can remember that stated that taking their time to finish the game was one of the things that went right. The game may have only been released on the DS and PSP, but polish is what seems to separate the quality titles of the current generation from those that are hoping to make a quick buck. I know that most publishers have quarterly goals to meet and that developers are usually working on tight deadlines and even tighter budgets. The problem is that it shows in the number of quality titles that are released throughout the year. Infinite Interactive spent nearly two and a half years making the game the best it could possibly be. They were rewarded by not being able to keep the game stocked on shelves.

Of the things that went wrong, "underestimation" is something that is not exclusive to Infinite. Every new developer we work with, and even ones that we've worked with a lot, seem to make the same mistakes. Something that I have suggested and pushed at work is echoed in the sentiment of Puzzle Quest's lead designer, Steve Fawkner.

My advice to any teams just getting into Xbox Live Arcade development is to read the TCRs over and over until you have them committed to memory. You may even want to run your design documents (especially the UI designs) past an Xbox Live Arcade Q/A team before implementing anything.

He talks about Live Arcade but I think that would be useful for anyone developing on the DS all the way up to PS3. It never fails that a game will come into test with only about two months till submission, but every Guideline fails. It then becomes a test of wills on whether or not to fix the issues or wait to see if they're found in submission. I can only imagine the time that would be saved if games were built around the guidelines for a particular system as opposed to throwing it together on one system and then attempting to shoehorn it onto another.

Lastly, he gives props to his "two excellent test teams." Being the lowest rung on the pole is not easy. At the best of times we can feel under appreciated and at the worst, seemingly blamed for everything. There is no glory to be had in Q/A and even the best teams are fallible as Fawkner points out that a heinous, if not serious, bug made it into the release version. It makes me happy to see developers chalk it up to an enormously complex game and not lazy testing.

Developers like Infinite Interactive seem to be few and far between. They have tremendous passion and the will to succeed, and it shows in their success.

It's Not Worth It

In preparation for the arrival of the the September issue, I flipped through the August issue of Game Developer Magazine. I reread Steve Theodore's latest Pixel Pusher article entitled Fight the Good Fight. He puts forth more strong arguments against the self inflicted practice of crunch time, many of which I strongly believe and attempt to convey to others around me, unfortunately, not to much avail. A couple of the things he stresses is pacing yourself and then learning how to manage your own time so you can foresee problems before they get out of control. However, the message that comes at the very end of the article hit closest to home, especially as I look to expand my own family.

There has never been a game that was worth a single divorce, but anybody with a few shipped titles on their resume could name you a title or two that cost that much.

That's why it's up to us to grapple the crunch monster. Growing out of our frat house work habits isn't just good advice - it's a deadly serious duty. The family wants you home from the trenches, safe and sound, not just another anonymous game industry hero in the small print of Mobygames. Don't disappoint them.

Graphs are just as good as pictures

In the most recent issue of game developer, Midway's QA Manager, Paul Sterngold, describes a situation in which a miscommunication between his teams and their outsourced QA partners led to an unexpectedly large bill at the end of the test cycle. He goes on to describe Midway’s solution, with the support of Amritt Ventures, which involves a series of graphs that quickly describe a project's status and performance statistics of the QA test teams.

Being in QA, I spent some time reading over the article and attempted to analyze the graphs, hoping to garner some information that may prove useful to my own company. After a while I noticed a trend that reinforces a one of my fundamental beliefs about this industry: more hours do not produce results. A quick comparison of the upper-right blue graph to the two yellow graphs indicates that as tester hours go up, the number and quality of the bugs go down. All the while, costs appear to be going up as indicated by the green graphs. Granted, I am deciphering these graphs with only the data and information I can glean from them. There may be more to the story, but as my title suggests, graphs are just as good as pictures.

Quality of life is very important to me, as it should be to all in the business. I have not been in the industry too long and I am fortunate to have worked myself into a position that does not require extensive overtime. I have done my share of late nights, but it has been my work ethic that has earned the respect of peers and advancement to my current position. I, like the people I respect in the industry, am willing to work as many hours as it takes to get the job done. If I feel I have given my all in a normal workday (by which I mean eight to nine hours), I see no need to stay any longer than necessary. And still, despite overwhelming evidence that supports it, people continue to believe that more is better when it comes to working a QA team. I, on the other hand, feel that a team of well-trained, motivated individuals will consistently out perform low-paid, under-trained, and over-worked testers any day of the week. Except on Saturday and Sunday, their job will be done within normal working hours.