Right, so let's talk about that GTAIV score. For those who haven't seen or heard, here's the deal: there was a bug in our publishing system on Monday that caused the incorrect score for GTAIV, a 9.5, to appear for on some pages for a while. This happened hours before we posted the actual score with the review, a 10. This has all led to a lot of talk and speculation so I'm going to set things straight, short answer: it's not true. Longer answer requires me to run you through stuff which I'll be doing now.
Our review process is like so: Once our reviewer has finished the game, he writes his review, submits it to copyedit, and then goes back over the text to look over the changes copyedit made before submitting it to production who then produce the review. Once the review is produced as a Web page on our QA servers, a link goes out to the reviews team for the peer review process. During that time the review score is not final. I'll say that again for dramatic emphasis: During that time the review score is not final.
In the case of GTAIV, that non-final review had a score of 9.5 attached to it--a score which the peer review process is designed to fine-tune and then set in stone. As has been standard GameSpot operating procedure for years, the last step of the peer review process is a discussion about review content and score. The content discussion entails making sure that relevant gameplay features and whatnot are mentioned to ensure the review is accurate as possible--and that said score matches said review.
As Justin noted in his blog, there were enough people in the reviews group who felt GTAIV deserved a 10 that we all holed up in a room to settle the matter....as the review was in the QA process on our staging servers with a 9.5 score attached to it. Ironically, one of the staff poked their head in the room during the at-times heated discussion and mentioned that it looked like a bug on the staging site had caused the QA score to go live for a split second. The importance of this event didn't sink in at first, since were so embroiled in our talks. Why were we so focused? Because we take matters of review score seriously. I'll reiterate what Justin said in his blog: We don't hand out 10s lightly--we've given out just four in GameSpot's entire history.
I've hit up our tech folks to find out what the deal was. After some back and forth, they got me answers because I wanted to post about this and include some info. So here's the tech nitty gritty for those that care:
* a 9.5 score, which as I noted above was not final, was the original score placed on the GTA4 page and became the page that was cached while testing and reviewing the site for quality assurance.
* Due to a bug in our publishing system that has since been corrected, a cached page containing the score and review blurb, but not the review itself, appeared on the Production version of the site.
* The length of time this was up is not precisely known but logs show that changes were made from 4:45PM PST - 8:20PM PST, with the actual review going live at 8:20PM PST.
And there you have it. A publishing snafu + some tremendously awful timing = needless drama. I'm well aware there's a bunch of folks out there happy to add this to the conspiracy theory tapestry that has been woven about GameSpot. Adding to said conspiracy was a blog post from a moderator saying the 9.5 score was deliberately put up as a "red herring" to test reader reaction. The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1).
I'm disappointed that, end of the day, one of the best-written and best produced text and video reviews is being cynically written off by some. None of the naysayers know the effort that went into making sure our review of probably the biggest--and so far, the best--game of the year was as bulletproof as possible. The personal attacks on Justin are unnecessary. Apparently people don't realize that Justin's been at GameSpot for 9 years--starting with the original GameSpot UK--and has reviewed all sorts of games in that time.
But so it goes. Bugs and assorted glitches are a fact of life on the web just as misspellings and assorted printing issues can be with print magazines. We take our reviews seriously here and we don't post them until they're ready. This is why GTAIV went up when we wanted it to with the score we wanted it to have. For those that gave us the benefit of the doubt and that didn't jump to conclusions, you have my thanks. To the others? ...
Comments
So basically, QA = rough draft.
but, i do have 1 question. How does tha game go from a 9.5 to a 10. Who can decide this. Is this decided by people who have read the review (but may not have necceseraly played tha game yet) and feel that the text reads more like a 10 or do the people who are involved in this decision have to have played the game. How does that work?
In all seriousness, the only problem should be that the score changed, not that it got a 10 instead of 9.5, but you've explained why it was changed, and why the 9.5 was not official, pretty well. The only thing I would add is how often review scores change before going live. I can only assume it happens semi-regularly. If this tweaking of the score happens very rarely, however, I think most people would want to know why. I got the game at the midnight launch (GTA IV, not Big Rigs) and am sure I will love it!
I don't doubt that this is exactly what happened... but you have to admit that GS hasn't had the most "scandal-free" reputation lately. So any little thing that goes wrong, or any major thing that GS does (like hand out a 10) is going to get drug through the mud because there are lots of people out there that just don't trust you folks anymore.
If I were you, and clearly I'm not, I'd find the CNET execs that botched the Gerstmann-gate thing, show them things like this ridiculous little drama, and make them understand how badly that donkey show is still hurting the site.
Yet today it was the voice of many telling one man that he was too critical of flaws, that it was he who over-evaluated their impact on the gameplay and fun. A true indicator of what business is today in group-led problem solving. Many people, one research, one solution.
Perhaps people should learn why a 10 is a ten and learn why either way still produced the same result. A game that cannot be reasonably improved at the time it came out.
Ultimately, it doesn't much matter, and this little ordeal simply highlights the pettiness of people on the internet who invest themselves entirely too much in spewing negativity. I'm not asking for everyone to sing kumbaya here, but damn people, give Gamespot a break would ya?
needless drama indeed. Im sure the infoil hat croud loved this episode
"The same moderator has since corrected himself, pointing out that in 2003, an accidental score of 0.0 was posted for one of the highest-review games of the year--Star Wars: Knights of the Old Republic (9.1)."
Heh, ever heard of 1984?
Bahamut, seeing things in terms of the old system isn't an accurate analogy. We don't give a score in the old system and then "round." There is no such thing as "rounding." The old system is gone; there is nothing to "round."
markwlin, your suggestion is ridiculous. Insane, fictional conspiracy theories have no intellectual merit and don't even closely resemble reality.
But what concerns me (and probably everyone else who was surprised to see the 10/10 score) is that when you see a 10/10; you think "wow, that game must be perfect", eventhough in the review and the video review Justin said -and you guys approved- that it's not perfect. Even if you guys keep saying that 10/10 is not perfect it just means its "prime', I find that very confusing.
If you really don't want to consider 10/10 perfect, then maybe you should have went with the more lenient 5 star scoring system (or the letter grading system). 5-Stars to me doesn't translate to perfect as strongly as 10/10 does. Besides, I bet that in the 5-star rating system, the whole disagreement you guys had about a 0.5 score difference woudn't have been an issue, because you would've all agreed that it deserves a 5-star rating anyways.
On second thoughts, im with you buddy. Clearly Ricardo and the rest of Gamespot have been infected with the same mind-mongering alien bacteria i read about on snopes. If you look at your net connections open ports you can actually see them transmitting gorgon-waves to your keyboard in an effort to infect you also. Close the doors, cut off the water supply and whatever you do dont buy GTAIV, no matter how much of a totally awesome, universally critically acclaimed game it is. THATS JUST WHAT THEY WANT YOU TO DO!!!
So, based on the explanation given here regarding GTA IV, that means the original score by the reviewer of KotOR was 0.0 until the other staff convinced him to raise it? Also, what are the chances that the other staff saw that nearly all other sites were giving the game a perfect score and were influenced by that since Gamespot was slower than many of them about getting the game reviewed? Would hate to draw the ire of millions of gamers, eh Gamespot, that would think you are out of line and unreliable? Hmmmmmmm...
Thanks for the post Ricardo, it feels ridiculous that you had to explain anything about this. Keep doing what you're doing.
Really, look it up. Some people claimed Halo was boring, or Gears of War repetitive when similar compaints appeared, but GTA? Nobody, anywhere, is claiming GTA is a bad game, or worse than any other game. As far as the people complaining know, 9.5 is no worse for the game than 10.
So, I guess my point is: who the heck cares?
Please, do get rid of numerical scores. It's an aberration of game criticism, music, book or movie critics don't bother with such stupidity. Take notice.
"ON CNET: Grand Theft Auto IV is PERFECT "
I can put up a screenshot if you want. And "PERFECT" is in all caps, I'm not just capitalizing it for dramatic effect or anything.
http://www.gamespot.com/news/6190520.html?action=convert&om_clk=latestnews&tag=latestnews;title;0
"...which is just the fifth game in GameSpot's history to score a perfect 10"
So you guys really consider 10 to be perfect then? What the h* is going on? You guys keep saying that the game is not perfect and the 10 doesn't mean it is, and then all over the website you post that it is PERFECT!
It now reads:
""...which is just the fifth game in GameSpot's history to score a 10"
Finally! Things were starting to get way too confusing (even for the GS news team it seems).
Thanks to news editor Brendan Sinclair for replying to my emails (quickly I might add) and fixing the issue.
JordanElek
And now we know.. the rest of the story. Thanks Ricardo.