The freelance reviewer who wrote a video game review that was pulled by GameSpot—and kept on Metacritic nonetheless—says he's sorry for the mistakes in his piece.
"I apologize to the readers and the fans for the factual errors in the Natural Selection 2 review and in any other reviews that I've done that had factual errors in them," writer Eric Neigher told me on the phone yesterday. "There really is a sacred trust between reviewers and their readers. You have to be confident that a publication and a writer are going to be giving you the facts as they exist, and are gonna be giving you an honest opinion. That's always what I strive to do. Unfortunately, I don't always get there."
"Understand that I'm just a human being and sometimes I miss things or get them wrong. It's not my intention to cut the legs out from any game or go after a game because it's not a big triple-A title—I don't do that. I just review games on their merits."
Neigher, an attorney who says he has been reviewing games for ten years, still stands by what he wrote about Natural Selection 2, "except for the stuff that was factually wrong... I feel like the score was definitely accurate. Again, everybody has different opinions. GameSpot reassigned the review to somebody else, and that guy had a different opinion of the game."
So why was it taken down? Neigher said he had a private conversation with GameSpot's reviews editor, Kevin VanOrd, who told him the Natural Selection 2 review would be pulled. Neigher wouldn't comment on what was discussed.
(When I reached out to VanOrd Wednesday as part of this story, he wouldn't comment beyond what he had written in his apology post on GameSpot.)
The main factual inaccuracy in the piece, Neigher told me, was the price, which he wrote was $30. Natural Selection 2 actually costs $25. "I kind of stupidly rounded the price up based on sales tax, as I usually do with store-bought games, not realizing that of course this is a downloadable game which doesnt have sale tax, so that was kind of dumb, but you know, that stuff happens."
"You have to be confident that a publication and a writer are going to be giving you the facts as they exist, and are gonna be giving you an honest opinion. That's always what I strive to do. Unfortunately, I don't always get there."
Neigher said there were two other common complaints: one omission—"I had omitted to say in the review that there were servers that were dedicated for new players, at the same time I criticized the game for being extremely hard on new players."—and one suggestion that he had exaggerated the game's loading times—"But that's all based on what kind of computer you're using and stuff, so load times are always kind of a subjective deal."
Any other factual errors? "Not to my knowledge," Neigher said. "You know, the load times thing I really don't consider to be a factual error, because you know, that was my experience. Other than those things, no."
When a GameSpot reader asked in the website's comments why VanOrd hadn't just fixed the errors and written a correction, VanOrd responded with an explanation.
"People scream about negative reviews all the time; but a well-argued review is an airtight review, meaning that its factual examples must support the argument," he wrote. "In this case there were multiple factual errors (regarding the price, regarding the load times, regarding the engine) as well as an overlooked but important detail (newcomer-only servers) that—while they could be fixed—cast a shadow over the review as a whole."
The people behind Natural Selection 2 told me that they never contacted GameSpot in any official capacity, although Hugh Jeremy, community manager for developer Unknown Worlds and a popular presence for the Natural Selection 2 team, did leave a comment on GameSpot's review to correct the price, among other things.
As I reported on Wednesday, this isn't the first time Neigher has run into complaints about inaccuracies. In a 2010 review of Elemental: War of Magic for 1up, Neigher talked about the game's multiplayer functionality—functionality that had not yet been out. The review was amended to reflect that.
"I think what happened in that review was I had meant to imply that the upcoming multiplayer would help with the problems and then, when it was edited, it was switched to a direct statement," Neigher told me. "But, yeah, so that was a whole big confusion. There was gonna be a multiplayer and then it wasn't released at the time of the version of the game I reviewed, so I think I put something like 'multiplayer will presumably fix these errors' and I think the 'presumably' got dropped or something like that, so it was just a big confusion."
In 2009, GameSpy pulled Neigher's review of Grand Theft Auto: Chinatown Wars, also for factual inaccuracies. "We have decided to remove our review of Grand Theft Auto: Chinatown Wars after finding some factual inaccuracies in the text," the staff wrote on their website as part of a round-up of reviews for the game.
When I asked Neigher why that review was pulled, he said he couldn't remember. When I asked him about what seems to have been a consistent trend over the past few years, he said I'd have to compare his record to other game reviewers before coming to that conclusion.
"You're drawing a conclusion that there's a pattern that's commonplace," he said, "Presumably that only has value if you're saying that I have a more common instance of people complaining about my reviews for whatever reason. Then you need to at least base that on—it's higher than the average rate. So if other reviewers are also getting the same kind of things at the same rate, then we wouldnt expect it to be a problem with me, it's just endemic to the industry...
"Neither of the two of us knows really the average industry amount of complaints, revisions, and pullings of reviews by a website and by a reviewer," he said. "So it's hard to say—and also, I can only offer anecdotal evidence as far as my own experiences go, but the vast majority of my reviews have not really had any issues at all, and certainly have not been pulled."
Game reviews on major websites are not often pulled—even Neigher admitted that he doesn't think this sort of thing is common—so to an outside observer, two pulled reviews would certainly seem like a trend. Kotaku, for example, has never had to pull a review.
On a WordPress blog that Neigher started this week, he wrote a more in-depth apology.
"The process of savaging reviews is a commonplace occurrence in our industry, and I've certainly had my share of that—sometimes justifiably," he wrote. "This is, unfortunately, one of those times, and I apologize to everybody for dropping the ball."
Nov 14, 2012
If you head on over to the Metacritic page for Natural Selection 2, you might notice something strange. Of the seven reviews listed, only one is scored below an 80.
Also, it doesn't actually exist.
Last week, GameSpot reviews editor Kevin VanOrd pulled his site's review of Natural Selection 2. The review, scored 60/100 and written by a freelancer named Eric Neigher, had been eviscerated by readers and commenters who pointed out a number of mistakes—for example, the review said the recently-released indie game was $30, when it actually costs $25. Other errors involved the game's engine and load times.
But the original review is still on Metacritic. A big yellow 60 sits right on Natural Selection 2's front page, warning readers that "disappointing execution holds it back." Clicking "read full review" will take you to a broken link.
So why hasn't the review been replaced? I asked Metacritic head Marc Doyle, who told me that they have a one-shot policy for all reviews and all gaming outlets. (Kotaku is not listed on Metacritic, as we do not use review scores.)
"Yes, the critics we track know—and I spoke to the GameSpot team about this this week - that we only accept the first review and first score published for a given game," Doyle told me in an e-mail. "I'm explicit about this policy with every new publication we agree to track. It's a critic-protection measure, instituted in 2003 after I found that many publications had been pressured to raise review scores (or de-publish reviews) to satisfy outside influences. Our policy acted as a disincentive for these outside forces to apply that type of inappropriate pressure."
(Note: GameSpot and Metacritic are both owned by the same parent company, CBS Interactive.)
"We only accept the first review and first score published for a given game... It's a critic-protection measure."
It's a policy that seems to be enforced with the best of intentions—just about every reviewer in the gaming industry has heard shady stories about publishers trying to get scores changed—but it doesn't account for situations like this, where a writer's mistakes could seriously impact the fate of a game. I asked GameSpot's VanOrd how he felt about this whole situation, but he wouldn't comment on the specifics. "The review had multiple factual inaccuracies that cast a shadow over the entire piece, so we chose to pull it and reassign, which is a rarity, of course," he told me.
"So what?" you might be asking. "Why does it matter what kind of review scores this game has on Metacritic? Aren't review scores just arbitrary anyway? You guys don't use them!"
For better or worse, Metacritic has become an important tool for people in the gaming industry; some publishers even use it to decide whether to give out bonuses, a practice I've criticized in the past. And studies have shown that a game's Metacritic score can have significant impact on its sales.
For an indie game like Natural Selection 2—a game made over the course of six years by a team of just seven people—one mediocre score could be catastrophic. (Although they seem to be doing okay, and Kotaku's Kate Cox had a ton of praise for the competitive game in our review.)
I asked Hugh Jeremy, community manager and jack-of-all-trades over at Natural Selection 2 developer Unknown Worlds, what the team thought of this whole situation.
"We fully respect Gamespot's journalistic processes and are very thankful that they took the time to review NS2. Many critics will not touch smaller indie games with a ten-foot barge pole," Jeremy told me. He said they've reached out to Metacritic for an explanation, but they haven't yet heard back. [Update: After the publication of this piece, Metacritic's Doyle sent me a note to say he had offered to get on the phone with Jeremy, but that Jeremy never got back to him.]
"In general, it is extremely important that score aggregators reflect accurate and timely information about the reviews they are aggregating," he said. "Their scores are used by players, publishers, investors, other critics and even developers themselves to judge the success or failure of a product. Those scores can make or break dreams."
While Eric Neigher, GameSpot's original reviewer, was certainly entitled to give the game whatever score he felt was appropriate, that score was pulled. GameSpot has re-reviewed the game. Should the most important game score aggregator on the planet not update their listings to reflect that?
This isn't the first time that Neigher's reviews have come under fire. In 2010, the developers behind Monday Night Combat publicly questioned if he had played the game before writing a review for 1up. (I reached out to Neigher earlier today to see if he'd like to give his thoughts for this story, but I haven't yet heard back.)
And a glance through some of Neigher's old work shows some rather bizarre choices.
"And if you're all butt-hurt about not having the same stuff as the kids across the Pacific, I hear you, but please believe me when I say that if you allow that butt-hurtedness to prevent you from buying and playing this game, you fail at life and will never have sex. Yeah, I said it. Because it's true. Don't say I didn't warn you," he wrote in his review of Yakuza 3.
We'll continue to follow and update this story as we hear more.
Nov 12, 2012
Natural Selection 2 is the first ever game I've enjoyed watching more than I've enjoyed playing.
If that sounds like an insult, it's certainly not intended as one. Rather, read it as perhaps one of the highest compliments I could give. Natural Selection 2 is good stuff—but playing it is only half its purpose.
First, some background: Natural Selection 2 was and is the little big game that should have been impossible. The game is from Unknown Worlds, a tiny studio of just seven people who took on the task of creating a massive, fully functional, online multiplayer shooter and strategy game.
The gambit worked, and over 144,000 people picked up the game in its first week of being live. Though in this instance, "live" is a bit of a misnomer. The game came to life through extensive collaboration between the dev team and fans, many of whom paid for pre-order and beta access, then helped bring the game to life. It well and truly launched on Halloween, ten years to the day from the moment the original Natural Selection, a Half-Life mod, was published.
So, yes, the saga behind Natural Selection 2 is a good story—and yet as we've seen over and over, it takes more than just a fairy-tale narrative to make a good game. Happily, NS2 is more than just a story about a scrappy studio. It's a genuinely well-made, top-notch game. One that I am manifestly terrible at playing. But more about that in a moment.
WHY: Because it's a very good, well-paced, well-balanced competitive multiplayer experience that really relies on player skill
Developer: Unknown Worlds
What I played: About five hours worth of getting my butt kicked as both aliens and space marines, mixed in with match spectating
Natural Selection 2
Developer: Unknown Worlds
Release Date: October 31
Type of game: Online competitive PvP alien-shooter, marine-muncher, and RTS base-builder
What I played: About five hours worth of getting my butt kicked as both aliens and space marines, mixed in with match spectating
My Two Favorite Things
- The way the alien infestation physically takes over the base is either really cool or really scary, depending which side you're playing
- Matches can feel incredibly tense and tight, even while lasting well over an hour
My Two Least-Favorite Things
- The game has not so much a learning curve to travel as a learning sheer cliff face to climb.
- Voice chat is all but required to play, which I'm uncomfortable using & jerks are comfortable abusing.
Made-to-Order Back-of-Box Quotes
- "Play this one with your friends. If you don't have any friends, make friends, then play with them." - Kate Cox, Kotaku.com
- "No, alien, it's cool, it's not like I needed that face to live—oh, wait, actually, I did. *respawn*" - Kate Cox, Kotaku.com
NS2 is a multiplayer, online, competitive, team-based shooter, but with a couple of twists. Only one set of players in any given match is taking on the role of the ever-popular gun-toting space marine. The other team takes on the role of the aliens, trying to conquer the station by taking to its walls, ceilings, and access tunnels. Marines take their first-person view down the barrel of a rifle (or, in moments of desperation, behind an axe); aliens take theirs from inside a mouth. A big mouth. Full of sharp, pointy, nasty teeth.
What I am saying here is: the aliens are serious bastards. Play as them.
Despite the many differences between sharp teeth and rifles, both sides are in fact exceptionally well-balanced. Or at least, they can be. Team performance, as in any multiplayer game, depends entirely on the players taking part. NS2, though, doesn't just demand cooperation of its cannon fodder. Instead, it requires true leadership.
Each team has a commander. The role isn't pre-assigned; both the aliens and the marines have particular stations that a player can stand in. The player in control of the command station is the commander, and while everyone else in the game is playing a first-person shooter, the commander is playing a real-time strategy base-building game. The commander chooses which upgrades to research and which structures to build. Structures get built when marines or aliens go and construct them, and so the commander issues orders and delivers tactics that the team, ideally, then follows.
The commander, like any great general, really is the linchpin of the team. If he (and in every match I played, on either side, it was a he) knows what he's doing and can issue clear, direct, intelligent orders, then the team has a good chance at survival. If he doesn't know what he's doing, and dithers too long over constructing resource extractors or gives contradictory orders to his side, then the team is going to be a mess.
I had expected to run into some ego-driven tantrums over the role of commander, as I wove through servers and parties, but I never actually did. I am sure there must be some pissing matches going on, but it seems that players are keenly aware of the responsibilities of command (and the potential blame), and take the duties to heart. I did, however, encounter one match where command was bugged. The command center kept ejecting players serially—sometimes, in as little as 30 or 45 seconds. Unable to keep anyone issuing orders for more than a minute or two, the marines eventually ground to a halt and, despite having superior numbers and having made significant progress in the match so far, crumbled.
Between good shooting and smart, quick RTS elements, Natural Selection 2 is a genuinely good multiplayer experience. It's also, in all honesty, a game that I should probably not be playing. To all the strangers on the rookie-friendly servers who put up with my fumbling, thank you. I promise not to do it again.
Learning to play NS2 can be a challenge. I, personally, found it quite intimidating. I watched the beginning tutorial videos, as the game itself prompted me to do, but I simply don't learn very well from YouTube. Part of why I play games is because I tend to learn best by doing—all the informational videos in the world fly out of my head when confronted with an actual experience later down the line.
After the videos, I wandered a map in Explore Mode. This, too, was beneficial: I managed to gain familiarity with some of the landscape, and to learn what certain machines and nodes are, how things are built, how things are killed, and overall to make sure my grasp of the basic mechanics was sound. And yet even memorizing an empty map only does one so much good as compared to a full map full of skittering creatures swarming at you in three dimensions to eat your face.
All of which, then, brings me back to the joy of watching.
After one loss (which admittedly, I had only joined the server six minutes prior to), when thrown back into the lobby, I more or less hurled myself through the door marked "spectate" in despair. I needed a break, and neither the tutorials nor the hands-on experience had yet taught me what Natural Selection 2 was all about.
Some major gaming phenomena are just not on my natural list of likes. The rise of eSports, streaming gaming, and player-vs-player competition are all on that list. I've never understood why so many people might prefer to watch a livestream of some strangers play a game when they themselves could be playing instead.
Now, I understand. I had hoped that viewing a match as a spectator would give me a better grasp of how the game should ebb and flow—and it did. More than that, though, I began truly to enjoy watching games for their own sake.
Watching both sides duke it out, with access to the full game map, has the same sort of strange tenterhooks satisfaction as watching a horror film. I'd find myself thinking, "Aaaah! No! Don't go in there! Go the other way! THE OTHER WAY!" And when I watched one very, very good space marine dodge, weave, and survive his way through most of the base, only to walk face-first into a nest of aliens that had been left in waiting for him, I admit to shouting, "IT'S A TRAP!" aloud, to the confusion of all.
In the end, Natural Selection 2 is perhaps best described as a skeleton, or a scaffold. It's an excellent, stable frame on which to hang cooperation, teamwork, and competition. As a multiplayer game, the experience is exactly as good as a player's allies and enemies are. Strong teams, that join the game and work together to perfect and enhance their skills, could truly be a thing of beauty.
I know my limitations; as far as NS2 is concerned, I am not born to be a leader. It's a grunt's life for me, as I try my best to do what I'm told and find the correct room I've been ordered to go to without getting killed twice along the way. Lacking a team to play with as a unit, I'm probably best left on the sidelines... but I think I'll still be watching.
Natural Selection 2 came out last week, and the folks who made it have created this neat infographic to show off the performance of their new real-time-strategy/first-person-shooter hybrid.
Notably: between pre-orders and sales, more than 144,000 people now own the game. Not bad for an indie made by a studio of seven full-time staff.
Click the image above to expand the whole chart.