Right, so let's talk about that GTAIV score. For those who haven't seen or heard, here's the deal: there was a bug in our publishing system on Monday that caused the incorrect score for GTAIV, a 9.5, to appear for on some pages for a while. This happened hours before we posted the actual score with the review, a 10. This has all led to a lot of talk and speculation so I'm going to set things straight, short answer: it's not true. Longer answer requires me to run you through stuff which I'll be doing now.
Our review process is like so: Once our reviewer has finished the game, he writes his review, submits it to copyedit, and then goes back over the text to look over the changes copyedit made before submitting it to production who then produce the review. Once the review is produced as a Web page on our QA servers, a link goes out to the reviews team for the peer review process. During that time the review score is not final. I'll say that again for dramatic emphasis: During that time the review score is not final.
In the case of GTAIV, that non-final review had a score of 9.5 attached to it--a score which the peer review process is designed to fine-tune and then set in stone. As has been standard GameSpot operating procedure for years, the last step of the peer review process is a discussion about review content and score. The content discussion entails making sure that relevant gameplay features and whatnot are mentioned to ensure the review is accurate as possible--and that said score matches said review.
As Justin noted in his blog, there were enough people in the reviews group who felt GTAIV deserved a 10 that we all holed up in a room to settle the matter....as the review was in the QA process on our staging servers with a 9.5 score attached to it. Ironically, one of the staff poked their head in the room during the at-times heated discussion and mentioned that it looked like a bug on the staging site had caused the QA score to go live for a split second. The importance of this event didn't sink in at first, since were so embroiled in our talks. Why were we so focused? Because we take matters of review score seriously. I'll reiterate what Justin said in his blog: We don't hand out 10s lightly--we've given out just four in GameSpot's entire history.
I've hit up our tech folks to find out what the deal was. After some back and forth, they got me answers because I wanted to post about this and include some info. So here's the tech nitty gritty for those that care:
* a 9.5 score, which as I noted above was not final, was the original score placed on the GTA4 page and became the page that was cached while testing and reviewing the site for quality assurance.
* Due to a bug in our publishing system that has since been corrected, a cached page containing the score and review blurb, but not the review itself, appeared on the Production version of the site.
* The length of time this was up is not precisely known but logs show that changes were made from 4:45PM PST - 8:20PM PST, with the actual review going live at 8:20PM PST.
And there you have it. A publishing snafu + some tremendously awful timing = needless drama. I'm well aware there's a bunch of folks out there happy to add this to the conspiracy theory tapestry that has been woven about GameSpot. Adding to said conspiracy was a blog post from a moderator saying the 9.5 score was deliberately put up as a "red herring" to test reader reaction. The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1).
I'm disappointed that, end of the day, one of the best-written and best produced text and video reviews is being cynically written off by some. None of the naysayers know the effort that went into making sure our review of probably the biggest--and so far, the best--game of the year was as bulletproof as possible. The personal attacks on Justin are unnecessary. Apparently people don't realize that Justin's been at GameSpot for 9 years--starting with the original GameSpot UK--and has reviewed all sorts of games in that time.
But so it goes. Bugs and assorted glitches are a fact of life on the web just as misspellings and assorted printing issues can be with print magazines. We take our reviews seriously here and we don't post them until they're ready. This is why GTAIV went up when we wanted it to with the score we wanted it to have. For those that gave us the benefit of the doubt and that didn't jump to conclusions, you have my thanks. To the others? ...