Testing in the Open World: Crackdown

"Everyone knows that testing a game thoroughly is essential to its quality," is the debatable first sentence in a Gamasutra post from Microsoft's Gamefest concerning a forum on how they overcame testing the massive size of Crackdown. While the article is sparse in detail, it provides an overview of what steps were taken to ensure that the game was thoroughly hammered. Essentially, due to the overwhelming scope of the game world that was compounded by the fact that much of the game rose in the y-axis as well, much of the testing was automated. Environmental bugs, including the placement of items and graphical seam issues, numbered well above the 10,000 mark. I have not seen many multi-platform games get that high in total, let alone one bug type.