Why Are Game Developer Bonuses Based On Review Scores?Last night, Obsidian's Chris Avellone tweeted an interesting detail about his roleplaying game Fallout: New Vegas.

"[Fallout: New Vegas] was a straight payment, no royalties," he said in response to a fan question about the game's financial success. "Only a bonus if we got an 85+ on Metacritic, which we didn't."

Metacritic, an aggregation website that collects scores from selected review sites and compiles them as a weighted average, currently lists the Xbox 360 version of Fallout: New Vegas at 84 (out of 100). The PC version is also listed at 84. The PlayStation 3 version of the game is listed at 82.

In other words, Obsidian may have missed its bonus and lost out on a significant amount of money because of a single point.

We've reached out to New Vegas publisher Bethesda, the company that financed the game, to try to confirm Avellone's statement, but they would not comment. If the New Vegas designer's tweet is accurate, then Bethesda put a portion of Obsidian's financial fate in the hands of a select group of game reviewers.

Finances have been an issue for Obsidian—earlier this week, the independent studio had to let go of 30 staff because a game it had been developing for the next Xbox was cancelled. So a potential Metacritic bonus may have been no small matter.

I understand the logic used by publishers like Bethesda when they dole out bonuses based on Metacritic numbers. As an aggregation of critic review scores, a Metacritic average can be an important benchmark for the perceived quality of a game. And it certainly makes sense that a boss would want to reward its employees based on the quality of their work.

Except Metacritic scores are not objective measures of quality. The Xbox 360 Metacritic page for Fallout: New Vegas consists of 81 reviews. If Obsidian's bonuses were determined by this aggregator, they were not based on the game's quality—they were based on 81 peoples' opinions of the game's quality.

Metacritic scores are not objective measures of quality.

Look through Metacritic's list of critic reviews. The list of selected websites is comprised of both professional and volunteer reviewers. Some write for the web. Others write for print. Some scores are weighted more heavily than others (Metacritic does not publicly discuss the formula it uses to create its averages). Some scores are even treated differently than others—a 7 at Game Informer does not mean the same thing as a 7 at Edge, for example.

Many of the reviews attacked the game for its bugs and glitches, many of which were fixed in subsequent patches and downloadable content packs. While reviewers may have been justified in marking down scores for the buggy product, those scores may not have been relevant after a month, or even after a week. Most review outlets don't change their scores once patches have been released. Is that something Bethesda took into consideration?

There is no such thing as an objectively good game. Nor is there such thing as an objectively bad game. We all secretly hate some games that are beloved by the rest of the world, and everyone has their favorite black sheep. I've strongly disliked some highly-rated games, like Dragon Age 2, and fallen deeply in love with some poorly-rated games, like Suikoden V. Should my personal opinion really be condensed into a mathematical formula and used to decide somebody else's bonus?

At Kotaku, we don't use review scores. Metacritic doesn't count our reviews. What if that made the difference? What if an outlet's choice of reviewer changed everything? What if a developer's bonus was determined by a single person's arbitrary distinction between a 7.8 and a 7.9? What if a game studio faced financial trouble after it missed its bonus by a single point?

This isn't healthy for anybody involved. It's not healthy for a reviewer to have to worry whether his criticism will directly affect peoples' jobs. It's not healthy for developers to focus on pleasing reviewers, rather than pleasing consumers. It's not healthy for individual opinions to impact bonuses and salaries.

Publishers need a better tool for measuring a game's quality. I don't know what that tool is. I don't know that it exists. But using Metacritic to hand out bonuses is dangerous—for developers, reviewers, and, quite frankly, you.

(Disclosure: While working at Wired.com, I gave Fallout: New Vegas a 9/10. My review appears on the game's Metacritic page.)


An early story concept for the first game in the hugely popular Fallout series saw you zipping back and forth in time, traveling through space and battling sentient dinosaurs, creator Tim Cain has revealed.

Speaking at a post-mortem panel at GDC in San Francisco today, Cain explained that the game's story morphed a number of times before its eventual post-apocalyptic setting was settled upon.

At first, it was going to be a traditional Dungeons & Dragons fantasy game.

"A lot of people who came on board said we could do something that's better than D&D - let's put our own twist on that," he recalled.

"We quickly threw that out as there were so many other fantasy games being developed. This is the one choice we made that saved us from being canceled."

Then came something rather more ambitious.

"Our second idea was epic. You started in the modern world, you were thrown back in time, you killed the monkey that would evolve into modern humans, you went through space, you went to the future which was ruled by dinosaurs, you were then exiled to a fantasy planet where magic took you back through the timeline, and then you came back to the modern world to save your girlfriend.

"It's weird even hearing myself talking about it now, but we were really going to go with this. One of the other producers kind of slapped me and said 'there's no way you're ever going to get this story made. You can work on it for years and nobody is ever going to do it'.

Sure enough, Cain and his team scrapped the idea. However, they held on to the extra-terrestrial theme for their next pass. That concept saw aliens invade earth and conquer all but one its cities. The game's hero would then venture out of this safe zone to fight back.

"This is what morphed into Fallout - the idea of a vault that you left and went out into the wasteland," said Cain.

However, getting the game finished and onto shelves proved a very challenging process, with the title nearly axed on a number of separate occasions.

Its first brush with cancellation arose when publisher Interplay picked up the Forgotten Realms and Planescape D&D licenses. Some at the company thought that a new RPG IP might detract from sales of those titles. However, Cain "begged" boss Brian Fargo not to pull the plug and Interplay duly let it live.

It had another close call when Steve Jackson's GURP role playing brand, which Fallout was initially tied into, decided the game was too violent and didn't approve of the art style.

"It was too late to change anything," explained Cain. "I figured we were going to be canceled."

But management gave Cain a last minute reprieve.

"I was asked to write a new combat system. We had a week to design it and a week to code it. If we could do that we wouldn't be canceled. I'm not exactly sure how we did it. I know we drank a lot of soda, we were there all the time, I know we smelled bad too, but we did it."

There was one more shaky moment just before launch. European ratings boards refused to classify the game for release as it allowed the player to kill children.

"We allowed it. We just said it's in the game. If you shoot them it's a huge penalty to karma. You're really disliked, there are places that won't sell to you, there are people that will shoot you on sight. We thought people can decide what they want to do.

"But Europe said no. They wouldn't even sell the game. We didn't have time to redo the quests so we just deleted kids off the disc [for the European release]. The story references children but you never actually see any."

Cain also discussed the struggles the team had coming up with a name for the game. It was originally going to be titled Vault13 but Interplay's marketing team rejected it as it "didn't give any sense of what the game was about."

"They suggested things like Aftermath, Survivor and the wonderfully generic Post Nuclear Adventure," recalled Cain.

"What eventually happened was Brian Fargo took the game home and played it over the weekend. He came back and put the CD on my desk and said 'you should call it Fallout'. It was a brilliant name - it really captured the essence of the game."

And the rest is history. The game launched on PC in 1997 to huge critical and commercial success and a franchise was born.

Cain now works as a senior programmer at Obsidian - the developer behind last year's Fallout: New Vegas.


Fallout Could've Been About Time-Traveling, Dinosaurs, and Monkey MurderThe original Fallout was a gamble that paid off big-time; it set in place a tone, gameplay philosophy and fiction that is still going strong today. Fallout games are best known for their evocative, funny, dark and violent post-apocalyptic world. But it could have been another kind of game entirely.

In a Fallout post-mortem at the Game Developers Conference, Tim Cain, the producer, designer and lead programmer described an alternate version of the game's story that could have come to be a reality.

"You started in the modern world," Cain said. "You traveled back in time, you killed the monkey that would evolve into humans, you went through space travel, you went to the future, which was ruled by dinosaurs, you were exiled to a fantasy planet where magic took you back to the original timeline that you restored to full, and came back to the modern world to save your girlfriend."

Okay, so. As much as I love the idea of a fantasy planet that magically returns things to how they were, allowing you to save your girlfriend, I think my favorite part about this is that you "killed the monkey that evolved into humans." What?

"It's weird to hear me talk about it now," Cain said, "but we really were going to go with this. And I think one of the other producers kinda slapped me and said, 'There's no way you're going to get this storyline made, it's not going to get through, you could work on it for years and no one would ever do it.'

"I sometimes wonder what it would be like if we had done this game, and believe Scott Campbell may have it written down somewhere. I'd love to see it, to see what we thought was cool eighteen years ago."

Well dang, I would love to see that too. I'm glad that Fallout exists and everything, but I'd also like to see what the guys who made it would have done with a time-traveling Dinosaur story.

Some other notes from the talk:

  • Before they came up with the (great) name Fallout, the following alternate names were toyed with: Aftermath, Survivor, and the particularly terrible/on-the-nose Postnuclear Adventure.
  • The game initially failed certification for Windows 95, but for a very strange reason. Namely, Fallout failed Windows 95 cert because the game worked on Windows NT. To get certified on Windows 95, the game was supposed to "fail gracefully" on WindowsNT. Instead it worked. Cain said he called microsoft and said "It fails so gracefully that it doesn't fail at all." Which didn't fly, The solution? Go into the game and code it to detect Windows NT and just sort of… fail. Heh.
  • The team had a rule about references: If a pop cultural reference was going into the game, it had to be unnoticeable by someone who didn't get it. As an example, Cain said that the "Slayer" perk was because Chris [Jones] was a huge fan of Buffy the Vampire Slayer
  • Fallout got in trouble with the folks at Steve Jackson games (who owned the GURPS role-playing system the game was based on) because of the design, art and violence. For a while, it looked like the game would be cancelled as a result. In the end, GURPS was torn out, and they redesigned and coded the game in two weeks. I'm not exactly sure how we did it, my memory of that time is vague. But we did it. The systems behind all of Fallout, with the exception of perks, was done in two weeks.
  • For some reason, the game was submitted for a "T" rating, even though it contained drugs, prostitution, and child-murder. So, when the ESRB saw that, they of course rated it "M."
  • There wasn't that much drama around the child-killing in America, but in Europe, the game wouldn't have made it to shelves. There wasn't time to re-code the game, so they simply deleted all of the kids from the disk.
  • People refer to Fallout as a game from an isometric perspective, but Cain pointed out that the perspective is in fact "Cavalier Oblique."
  • Cain worked on the game by himself for a year, before getting two team members—a scripter and a coder, both of whom were named "Jason." People referred to them as "Tim and the Jasons."
  • They originally wanted the Inkblots' "I Don't Want to Set The World on Fire" to be the game's theme song, but couldn't get it due to licensing reasons. Instead, they went with "Maybe," and it would up working better for the tone of the game. And, much later down the road, "I Don't Want to Set The World on Fire" became the theme of Fallout 3.
  • Fallout's S.P.E.C.I.A.L. stats system was originally called A.C.E.L.I.P.S. I wasn't entirely sure whether or not Cain was kidding about this.
(Dinosaur photo: Dapper Dinos)

Oblivion, Fallout Get Combined Release in April—but Not on PS3For those whose first foray into Tamriel came this year with The Elder Scrolls V: Skyrim, the series' maker will offer the preceding entry, The Elder Scrolls IV: Oblivion combined with another outstanding role-playing game, Fallout 3. The combo will be $29.99 on Xbox 360, $19.99 on PC in March.

What about PS3? Bethesda Game Studios' Pete Hines told Joystiq that they aren't allowed to release the combo for that console yet. "We will continue to work to try to change that," he said. "but at present it is still not approved. We would like to release a PS3 version a well."

Fallout 3 and Oblivion unite in double pack on April 3 [Joystiq]

Fallout 3's Tunnel Snakes Rule, and So Does This Amazing Classic RemixWho could forget the Tunnel Snakes, the dorkiest, least menacing group of post-apocalyptic greaser wanna-be gangsters to ever grace a role-playing game? I sure couldn't. And who could forget this classic video, which though it's a few years old, still holds up remarkably well?

Not Jenn Frank, anyway, who was kind enough to remind me of this video by YouTuber ElevateYourLevel, which remixes... well... dude, am I really going to explain what the video is to you? No. Maybe you've already heard it. But hey, this is Kotaku Melodic, and we like to listen to stuff! So let's listen again.

Tunnel Snakes rule!
We're the Tunnel Snakes!
That's us!
And we Rule!

I think I speak for everyone when I say: Yeah you guys rule!

If you have a pulse, this track will make you want to put on a leather jacket (be sure to slide it under your Pip-Boy!), take to the vault hallways, and dance, dance, dance.

Tunnel Snakes Rule! [YouTube via Infinite Lives]


Did you forego picking up Fallout: New Vegas and its downloadable content in the hopes that Bethesda would release some sort of Ultimate Edition featuring everything rolled into one? Then February 7 (10 in Europe) is your day to reap your patience's reward.

As someone that only played about an hour of Fallout: New Vegas, I am looking forward to being able to run to the store and snag a DLC-complete copy for $49.99 ($39.99 PC) on February 7. No fuss, no muss; just one disc packed with post-apocalyptic goodness waiting for me to happen to it.

Did you wait?


Fallout Wiki Founder Banned From Wikia for Promoting Curse

Anyone who's anyone in the Fallout community knows Pawel "Ausir" Dembowski. Founder of The Vault wiki which contains over 15,000 pages of Fallout lore, Pawel is a human encyclopedia of gaming's favorite post apocalyptic franchise.

Let me put it this way. When Chris Avellone has a question about Fallout, he asks Pawel.

Moreover, if you've read a breaking or original story about the Fallout universe in the past five years, there's a damn good chance Pawel was the source.

So how did the man who helped to grow Wikia with one of the most popular gaming wikis on the planet over the last five years find himself globally banned from the same network that employed him only a few months ago? By leaving a comment.

Fallout Wiki Founder Banned From Wikia for Promoting Curse You see, Pawel is now employed by Curse. He moved The Vault to the Curse Network in November of last year and took a position as Curse's "Wiki Team Lead." That means he's now in charge of all the Curse wikis, including the one for Cobalt, a game published by Mojang. Mojang links to Curse's wiki as their official wiki, but Wikia was claiming that their wiki was the official Cobalt wiki. Hence Pawel's comment. Next thing you know he was globally banned from Wikia without warning. Not cool bro.

However, there's obviously a bit more to this story.

You see, Pawel founded The Vault in 2005, moved it to Wikia in 2007, joined the Wikia staff in April 2010 and then moved it to Curse in November 2011.

Wikia has since kept their version of The Vault's original content before the split (which they're allowed to do under creative commons) but have renamed it to the "Nukapedia" and now update its content with a new team. However, due to years of search engine optimization and record breaking traffic, Wikia's wiki is still the most commonly used resource for Fallout info, even if it's no longer the official home of The Vault. Go ahead and Google anything Fallout related, Nukapedia will come up.

So why on earth would Pawel move The Vault away from Wikia, where he had a comfortable job and loyal following, to the Curse network knowing all too well that The Vault would become the #2 Fallout wiki and it would take him and his team of moderators years to build it back up?

Well, he says it was a group decision. The Vault admins had generally been displeased with Wikia's move towards becoming more of a social network than a wiki. They wanted more freedom on deciding how the wiki was being run, and more of a focus on content instead of social features. They wanted to focus on gaming instead of Wikia's recent focus on lifestyles.

Curse also pays better.

Pawel says the split was amicable at first and he has tried to stay on good terms with Wikia. However, as Pawel and his team continue to produce more original content at The Vault and more and more people leave Wikia's Nukapedia to follow Pawel to The Vault's new home, things have started to become bitter.

Relationships between certain entities at the two networks have become more noticeably strained and Pawel has started to be viewed by many as a man who split the Fallout community in twain.

Fallout Wiki Founder Banned From Wikia for Promoting Curse

So, one day, as he was just leaving a simple comment in hopes of clarifying an issue between the two networks, he was globally banned. Worse yet, he knows who banned him, and says they were once friends.

Pawel knows he has a long road ahead in order to build The Vault back up into the #1 Fallout resource on the web. Hell, even when you google "The Vault" Nukapedia comes up.

However, he and his team are confident they can make it happen by the time the next Fallout game comes around by providing superior content on a more consistent basis. He says that he's never been bitter about his split with Wikia, and this whole situation has saddened him more than anything. In fact, many of the administrators at both The Vault and Nukapedia remain good friends and share fresh Fallout content despite the split.

All Pawel hopes is that for all your post apocalyptic gaming needs, you'll be sure to check out the new and improved version of The Vault

It's never easy having to deal with your former employers or employees, especially on the vast wasteland that is the internet. However, one thing is certain. War never changes.

Headline image by Chad Lakkis

You can contact Dave Oshry, the author of this post, at doshry@gmail.com. You can also find him on Twitter when he's not watching over RipTen where he acts as the Consigliere to Don Chady .

The Great Fallout Legal Battle Ends Without a Fallout MMO As reported last week, the legal battle between Bethesda and Interplay over the final fate of a massively multiplayer online Fallout game has ended in a settlement, one that leaves full control of Fallout intellectual properties in the hands of Bethesda.

For those of you just joining the ongoing legal battle, when The Elder Scrolls developers Bethesda Softworks originally acquired the license to the post-apocalyptic Fallout series in 2007, original owner Interplay was granted the rights to create a massively multiplayer online game based on the property with a pair of conditions: Interplay had to secure $30 million in financing for the project, and development had to be in full swing by April of 2009.

Interplay met neither goal, so Bethesda parent company Zenimax took steps to take the permission back, and now they finally have it.

Under the terms of the settlement, Interplay no longer has a license to develop the Fallout MMO. They are still able to sell copies of Fallout Tactics, Fallout and Fallout 2, but its permission to do so ends on December 31, 2013. Bethesda parent ZeniMax agreed to pay Interplay $2 million in consideration as part of the settlement, while both sides are responsible for paying their own legal fees.

A separate but related lawsuit between Bethesda and developer Masthead Studios was settled in late December. Masthead has been tapped by Interplay to develop portions of the Fallout MMO, despite that the original Bethesda agreement forbade the company to subcontract. Masthead agreed that it had no rights to develop games using the Fallout license.

In an official statement issued today, ZeniMax CEO and chairman Robert Altman expressed satisfaction with the settlements. "While we strongly believe in the merits of our suits, we are pleased to avoid the distraction and expense of litigation while completely resolving all claims to the Fallout IP. Fallout is an important property of ZeniMax and we are now able to develop future Fallout titles for our fans without third party involvement or the overhang of others' legal claims."

I guess this leaves it up to Bethesda to make their own Fallout MMO. Good thing they've got a studio established for such things.


The Great Fallout Legal Battle is OverThe legal stoush between Bethesda and Interplay over the rights to certain aspects of the Fallout universe, which has been dragging on for years, has finally been settled, according to a report on Fallout fansite Duck & Cover.

While the actual details of the settlement are yet to be released - they're expected to be made public sometime later this month - it's still good news for fans of the franchise, as it can hopefully put a distracting and messy peripheral issue to bed.

The battle centred around the rights to a Fallout MMO, which series creators Interplay claim they held. Current Fallout publishers Bethesda, citing a number of failed milestones as part of the deal, disagreed.

Confirmed: Bethesda v. Interplay Settlement Has Occurred [DAC]


Fallout: New Vegas Developer Releases a Personal ModSometimes games aren't to your exact liking. If you have computer skills, maybe you can create a mod. If you have computer skills and developed the game, well, you can do that, too.

J.E. Sawyer, who worked as project director on Fallout: New Vegas and the game's DLC, created a mod for his own playthroughs. As Shacknews (via website No Mutants Allowed) pointed out, the mod increases the number of weapons and armor and cuts the level cap, XP gain, health, and healing.

The mod is available via Sawyer. You will need all Fallout: New Vegas DLC installed as well as the pre-order bonus packs and Fallout Mod Manager.

So why did Sawyer release a mod instead of a patch? "The game's over," he wrote. "The ship has sailed. No one is working on it anymore. No testers, nothing. This mod is just me working in my free time. If I horribly botch something, you can just un-check the mod and go on your way." Sounds good to me.

Fallout: New Vegas was originally released in Oct. 2010.

J.E. Sawyer releases his own Fallout: New Vegas mod [No Mutants Allowed via Shacknews]