The Politics of DX12

Last week, Oxide gave press early access to their pre-beta version of Ashes of the Singularity. If you ve been hiding under a rock, here s why this is important. So far, the only DX12 benchmarks anyone has been able to run have been synthetic in nature—the 3DMark API Overhead test pounds the GPU with draw calls until the GPU hits its limit, and gives a score; the earlier Star Swarm benchmark (formerly of AMD Mantle fame) was sort of in the same situation, except Star Swarm was a lot closer to being an actual game. And that game will be Ashes of the Singularity.

Now, the first thing to get out of the way is that Ashes of the Singularity sports an AMD Gaming Evolved logo, meaning they re actively receiving help and promotion from AMD. This is nothing new, as we ve had plenty of Nvidia The Way It s Meant To Be Played (TWIMTBP) titles over the years, including Batman: Arkham Knight (and all the other Arkham games), The Witcher 3, Assassin s Creed: Unity, Far Cry 4, the Borderlands series, and the Metro series, to name a few. On the AMD side, we have plenty of options as well: Tomb Raider, Civilization: Beyond Earth, Hitman: Absolution and its upcoming sequel, the recent and upcoming Deus Ex titles, Dragon Age: Inquisition, and most of the DiRT series. We list these merely to show that there are many games that are promoted by AMD or Nvidia, but usually not both; you ll also note that we re pretty evenly split on the games we re currently benchmarking for GPU reviews. But the short summary is that titles with an Nvidia logo are often better optimized—particularly near launch—for Nvidia GPUs, and likewise AMD titles are often better optimized for AMD GPUs. Capiche?

This discussion of AMD backing also becomes pertinent when we get to looking at performance. Nvidia contacted the press after the Ashes of the Singularity benchmark went out to point out that anti-aliasing was running sub-optimally on the DX12 path with Nvidia GPUs, and they recommended we test with AA disabled. Developer Oxide responded with a blog post titled The Birth of a New API, saying that the DX11 and DX12 MSAA is essentially unchanged. And here s where things get a bit sticky. Potentially, AMD has some hardware features that would enable a developer writing DX12 code to have better MSAA performance compared to DX11 code; Nvidia GPUs may or may not be able to do the same thing. (Just search for "DX12 Async Compute" if you're curious.)

Getting even deeper into the fundamentals of DX11 vs. DX12 programming, under DX11 there were a lot of things that could be done in the GPU drivers to try to optimize performance. With DX12 being a low-level API, most of the driver tweaks are not possible; instead, it s up to the software developers to write optimized code to extract maximum performance from the various GPUs. In a sense, it s like giving game developers the ability to write assembly language for the GPU rather than programming in C++, though it should be noted that DX12 is still a higher-level language. Regardless, if a developer is going to extract maximum performance from a GPU, they ll need to optimize their code for that GPU—and code optimized for one GPU s architecture may not run optimally on a different architecture! In a worst-case scenario, a developer might need to have different code paths for AMD Fiji, Hawaii, Tonga, Tahiti, etc., and Nvidia Maxwell 2.0, Maxwell 1.0, Kepler, Fermi, etc., architectures.

All of this is further compounded by the fact that Ashes of the Singularity is currently pre-beta, though the official beta should be starting very soon. The beta stage is often where a lot of performance optimizations and fine tuning takes place, so looking at performance right now is, at best, a preview of what may or may not come to pass.

We can argue about whether Oxide and Nvidia are being fully transparent, but that s sort of beside the point. The reality is that DX12 is supposed to be a low-level API that will allow the game developers to extract more performance from the hardware, which means better graphics and hopefully better gameplay will be possible. Or put another way, at the very least, DX12 performance should never be lower than DX11 performance; if it is, something is wrong with the code and the developer should look to fix things. The rumor is that Nvidia put a lot of effort into their DX11 drivers for Ashes, and didn t do much to help with DX12 optimizations for their hardware, but that s mostly speculation. What we do know is that DX12 with MSAA enabled does in fact tend to run slower on Nvidia GPUs than the DX11 code, and that s a clear problem. Ultimately, we opted to run all testing without MSAA enabled; when the game officially launches, we can revisit the subject.

But who cares about all the political stuff going on behind the scenes?! We re still looking forward to DX12 games and we want to know as much as the next guy what DX12 can do for performance, graphics quality, etc. All the above caveats aside, how does the current pre-beta release of Ashes run on the various GPUs? That s what we attempted to find out, which entailed running the benchmark many, many times.

Let me tell you, there s no better way to make someone hate a game than to have them watch the same sequence over and over again! Over three minutes per test gets to be quite lengthy, and we tested no fewer than three resolutions, three quality settings, two CPU clock speeds, four thread settings, and two graphics cards, plus looking at DX11 and DX12. (If you want the math: 3 * 3 * 2 * 4 * 2 *2 = 288.) That s 14.4 hours (minimum!) of running the same three minute sequence. Thank goodness for scripting…. [If you want to view all of the raw data, including Normal, Medium, Heavy batch FPS along with the average FPS and 97 percentile results we're showing, you can view all of our results on Drive.]

Team Red: ASUS Strix R9 Fury

Team Green: EVGA GTX 980 Ti ACX 2.0

In the interest of keeping the number of charts to a minimum (inasmuch as sixteen charts is a minimum ), we re only showing the Low and High quality presets, again with AA disabled on the High preset. Medium quality, as you d expect, ends up falling between the two and is thus not really necessary, but if anyone wants those charts as well, let us know. We ve grouped the charts according to the test GPU, with differing numbers of threads, resolution, and DX11/DX12 on each GPU. At the time of testing, we were somewhat limited in terms of what GPUs we had available, so we tested with an EVGA GTX 980 Ti (factory overclocked) and the Asus Strix R9 Fury. Note that this isn t an AMD vs. Nvidia performance test, but rather a look at how each vendor scales—or doesn t scale!—with the various settings/features.

Heavy batch of draw calls incoming!

You Take the High Road…

Starting with the high-quality setting, we re looking at the full average FPS for the entire benchmark. Oxide actually breaks things up into Normal, Medium, and Heavy batches, as the number of draw calls for the test scenes can vary quite a bit, but if we wanted to report those figures we d need to do another 48 charts. And as much as we like charts, that s overkill, so no thanks. Anyway, the average FPS correlates pretty well with the Medium batch results, and that makes sense: Normal has fewer calls, Heavy has more, and the overall average is close to Medium. We ll also look at the 97 Percentile FPS, which is a good indication of whether a game stutters at times.

Our test system is the same as we normally use for GPU tests, except we ran it overclocked at 4.2GHz as well as underclocked at 2.1GHz. For the multi-threaded testing, we used a command-line parameter for Ashes rather than actually disabling/enabling cores in the motherboard BIOS; unfortunately, that only seems to have partially worked: the two-thread option still ran with four threads, so we need to physically disable the cores and Hyper-Threading to get a proper 2-thread result. Given the preliminary nature of the testing, we ll go with what we have for now.

Starting with the AMD results, right away we find some interesting stuff going on. Using DX12 with a 4.2GHz processor is enough to basically max out the R9 Fury. It doesn t matter if we use four, six, or 12 threads: performance is nearly identical. Alternately, having six or 12 threads with a 2.1GHz processor also delivers nearly the same level of performance. Here s where DX12 is going to do AMD a ton of favors, at least in the CPU/APU arena, as it looks like four physical cores at a moderate clock (3GHz) should make the GPU the primary bottleneck (though multi-GPU configurations might still want more CPU power). Performance on the 2.1GHz processor improves by over 25 percent going from four to 12 threads (granted, no one is likely to be running a 12-thread 2.1GHz CPU). Perhaps more telling is that at 2.1GHz, DX12 is able to improve AMD s performance by 35 65 percent over DX11; even at 4.2GHz, DX12 still boasts a 15 35 percent improvement.

Of course, part of the reason for the above improvements is due to the poor DX11 results. We ve heard Nvidia put a lot of effort into their DX11 performance, but by contrast it looks like AMD has made virtually no effort to deliver good DX11 performance with Ashes. Threads don t matter under DX11 either, as there s little difference in performance under DX11, regardless of the number of threads, even at 2.1GHz. The 4.2GHz processor shows at most a five percent increase going from Four threads to 12 threads, while the 2.1GHz processor shows at most a 10 percent improvement. The change in clock speeds does help, of course: the 4.2GHz CPU is up to 30 percent faster than the 2.1GHz CPU, though at higher resolutions the margin of victory narrows.

Flipping over to the Nvidia side of things, it s a completely different story. DX12 helps performance… sometimes; other times, it s worse than DX11. This is without MSAA, which apparently further exacerbates the situation. Remember what we said earlier about software optimizations vs. driver optimizations? It looks like Nvidia s 980 Ti is currently running DX12 code tuned for AMD hardware, which in many instances is unable to match Nvidia s highly tuned DX11 driver performance. We might even go so far as to say that Nvidia set the DX11 bar really high, and Oxide failed to clear it—at least right now.

Digging into the details, what s interesting is that unlike AMD, Nvidia shows clear performance scaling with more threads with the lower clocked CPU. DX11 performance improves by 20 30 percent (depending on resolution) going from four to 12 threads, and DX12 performance improves by up to 40 percent. However, 4K performance is actually lower under DX12 than under DX11. Crank up the CPU clocks to 4.2GHz and threads become less of a factor; at best we see a 10 percent increase at 1080p under DX11, but at higher resolutions the 980 Ti becomes the bottleneck.

If you re wondering why Nvidia may have a bone to pick with Oxide, at 4.2GHz their DX11 mode outperforms DX12 mode across all resolutions and thread counts. Oops. Again, since DX12 is a low-level API, it s up to the software developers to optimize their code for different hardware. Oxide notes in their blog post, Some optimizations that the drivers are doing in DX11 just aren t working in DX12 yet. Oxide believes it has identified some of the issues with MSAA and is working to implement workarounds on our code. In other words, Nvidia s optimized DX11 drivers are doing a better job at certain things right now than Oxide s DX12 code—but Oxide is working to fix that.

Looking at the 97 percentile results, the story there is much the same. Nvidia with the 4.2GHz CPU delivers similar minimum FPS, regardless of DX11/DX12 or the number of CPU threads. At 2.1GHz, however, DX12 does make a sometimes sizable difference—the 1080p results with 12 threads are nearly 50 percent higher than the DX11 results. For AMD, DX11 minimums are horrific: well under 20fps. There s no scaling with CPU threads on DX11, but DX12 in turn delivers a great showing: the 12-thread 1080p DX12 performance is up to 2.5X higher than the DX11 performance on a 2.1GHz CPU. Having a 4.2GHz CPU helps some, but DX12 still shows nearly a doubling of minimum FPS at 1080p, a 75 percent boost at 1440p, and a still-hefty 50 percent increase at 4K.

So far, we ve avoided making direct comparisons between the two GPUs, as they re not in the same price bracket. However, if we take it as a given that the EVGA GTX 980 Ti is roughly 20 percent faster than Asus R9 Fury, and Fury X is 10 percent faster than Fury (that s what our earlier testing showed), it looks like AMD s Fury X may hold a slight performance advantage over a stock 980 Ti in DX12 mode. But we need to balance that against how badly AMD does in DX11. The R9 Fury in DX11 mode is pretty clearly running into CPU bottlenecks, even at 4.2GHz, but these bottlenecks are far lower than on Nvidia s hardware. Ergo, AMD s DX11 drivers are not nearly as efficient as Nvidia s DX11 drivers—this is something many people have noticed over the past several generations of hardware.

Keep those draw calls in moderation, soldier!

I ll Take the Low Road

That takes care of performance at the High settings, but what if we drop the quality? We ll skip over most of the analysis, as the story doesn t change too much from the above—and most people owning a 980 Ti or R9 Fury aren t going to be running low-quality settings in the first place! Here s a repeat of the above charts, only now we ve dropped the rendering quality.

Not surprisingly, the reduction in graphics fidelity has made Ashes more CPU bottlenecked. The biggest change is that frame rates are higher, naturally, but even at low quality we still see a decent amount of scaling on AMD hardware going from DX11 to DX12. In fact, the improvement is even greater this time, with up to 90 percent improvements at 2.1GHz and 60 percent at 4.2GHz. Nvidia also shows better performance across all settings with the 2.1GHz CPU, but 4K with the 4.2GHz processor still shows a performance drop of up to 10 percent.

As for 97 percentile frame rates, again we have to look at Nvidia and AMD separately. For Nvidia, there appears to be a wall at around 30fps at 2.1GHz in DX11 mode, and DX12 helps to lift that bottleneck to more than 50fps. With the 4.2GHz CPU, the wall is at 45fps, and DX12 increases that to nearly 70fps. Interestingly, AMD shows similar results under DX12: 50fps with 2.1GHz, 60fps at 4.2GHz. But that darn DX11 performance; 16fps at 2.1GHz and 23fps at 4.2GHz is horrible; there s no other way to put it.

It's quiet... too quiet!

Ashes to Ashes, Dust to Dust

As the first of what will likely be many DX12-enabled titles coming sometime between now and 2016, Ashes of the Singularity is at best a taste of what s to come. And that taste is… perplexing. Anyone hoping DX12 will mean the end of the GPU vendor wars is sure to be disappointed; if anything, DX12 looks to make the rivalry even more brutal.

We ve seen a few people hailing AMD as the decisive winner of DX12 performance (for their DX12 Async Compute support), the problem being that we re looking at an AMD-promoted title. They should offer better performance than Nvidia on a title they re promoting, especially at a pre-launch stage...except they don't, at least not by any meaningful margin. What we have is, at best, a close match in DX12 performance, with poor DX11 performance from AMD. Whether this will reflect future DX12 titles remains to be seen. Unreal Engine, Unity, Frostbite, and a host of other engines will more likely than not differ from Ashes.

Frankly, this testing is really just the tip of the proverbial iceberg. We ran two GPUs at a whole bunch of settings to find out how they performed, and this is in a single game. Does Fury X claim the crown from 980 Ti in this one title? We could answer that question with some additional testing, but that's sort of missing the point. Right now, we can see that DX12 definitely makes a difference in performance, giving the game developers a lot more power. But with great power comes great responsibility, and some developers may not be able to handle DX12, at least not without more time and effort.

The next fight is shaping up to be Lionhead s Fable Legends, and that will perhaps be a more neutral battleground as it s neither an AMD nor an Nvidia title. In fact, it appears Microsoft (who owns Lionhead) is determined to put forth a message that DX12 is unified. Microsoft doesn t want DX12 to appear as a fractured landscape, one where AMD or Nvidia rules, a place where processor graphics gets left in the dust. In that sense, Fable should be the most likely vendor-agnostic approach to DX12 we re going to see in the near term. We re certainly looking forward to testing it, though it may be a few months.

Ultimately, no matter what AMD, Microsoft, or Nvidia might say, there s another important fact to consider. DX11 (and DX10/DX9) are not going away; the big developers have the resources to do low-level programming with DX12 to improve performance. Independent developers and smaller outfits are not going to be as enamored with putting in more work on the engine if it just takes time away from making a great game. And at the end of the day, that s what really matters. Games like StarCraft II, Fallout 3, and the Mass Effect series have all received rave reviews, with nary a DX11 piece of code in sight. And until DX11 is well and truly put to rest (maybe around the time Dream Machine 2020 rolls out?), things like drivers and CPU performance are still going to be important.

Let's end with some questions. What games are you most looking forward to for the coming year? And will DX12 support—or a lack thereof—affect your buying decisions? Let us know what other games you re most interested in seeing benchmarked!

Follow Jarred on Twitter.

Twitch Plays Dark Souls may have turned into an improvised, turn-based version of the game, but it's still  fascinating to watch: especially if you've ever wondered what kind of build is necessary under these (admittedly pretty unlikely) circumstances. The answer, at least where the Capra Demon is concerned, seems to be 'poise'. You just need a lot of poise.

The Taurus Demon went down last week, but now I have two boss battles for you: the Belfry Gargoyles and the Capra Demon. The latter is one of the first significant difficulty spikes for newcomers to the series, so it's amusing to see Twitch make such short work of it. (Hint: it's not actually as easy as it looks).

Thanks again to YouTuber PvtBooger for editing the pauses out of these encounters.

Warlords of Draenor didn't launch with flying mounts, and for a long time Blizzard didn't intend to implement them. After months of consternation among World of Warcraft devotees though, Blizzard announced in June that, yeah, they'd allow players to fly in Draenor after all. Now, with tomorrow's 6.2.2 patch, the functionality will finally roll out.

Not without conditions though: you'll need to put a few weeks into the game if you want to unlock mounts, though most players have probably already met the requirements. When I spoke to lead game designer Ion Hazzikostas in June he put it this way:

"The solution was to preserve our original design goal of ensuring players are exploring the world and experiencing it from the ground, and getting to appreciate its scope and its size and engaging the gameplay aspects of exploration and mystery. Then, once they ve done that, there s no reason not to allow them to fly and navigate in all dimensions throughout the world, and also unlock that ability for their alts."

The video embedded below (courtesy of Wowhead) does a good job of outlining all the work you'll need to do in order to unlock the Pathfinder achievement, which itself unlocks access to flying mounts.

We know that Warcraft is "pretty much a finished movie" according to its director, but it's not scheduled to release until June next year. We've seen some images from the film, but now we catch a glimpse of how it will look in action, thanks to an apparent leak.

You'll need to head over here to see it. It's a trailer full of grandiose war scenarios and orcs, so it's pretty unmistakably Warcraft. Where it came from, though, is another question. Gamespot published the leak but did not indicate where it comes from, though behind-closed-doors sessions happened at ComicCon last month, so it's likely something to do with that.

Whatever the case, the grainy footage is long-awaited: the film has been in discussions for over a decade now.

After Torchlight 2 and the departure of two of Runic's co-founders, Travis Baldree and Erich Schaefer, company president Marsh Lefler says the team stepped back and tried to come up with something they "really want to play." That's Hob, which drops us into an alien world with puzzles, combat, and no dialogue—the story is picked up through exploration and observation. Lefler says they were inspired by games like Ico and Shadow of the Colossus. 

I played a brief demo of Hob at PAX Prime, and I especially enjoyed the speed of the movement, which was so much fun it made it hard to slow down, take a breath and soak in the colorful vistas. The combat is tougher than the friendly look might imply, but definitely not a meat grinder—it requires some methodical timing, blocking, and stamina management, but health is pretty generous. It's a far gentler Lords of the Fallen.

I had a chat about all this with Lefler yesterday, which you can watch above. There's no release date for Hob just yet, as Lefler says there's still iterating to do.

We got our hands on Battleborn at this year's PAX, the newest FPS from Gearbox Software. With its cel-shaded art style and RPG-ish approach to character progression, the biggest question is how it will differentiate itself from Gearbox's already popular Borderlands series, let alone all the other shooters on the way. 

Tyler took control of Thorn, an archer character, while Evan used a knife throwing mushroom named Miko who also has healing abilities. You can watch the video embedded above to get their impressions, while the video below is a full 20 minutes of gameplay at sugar sweet 60 fps.

You can find all of our coverage from PAX Prime 2015 right here.

Claymation point-and-click adventure Armikrog, from the creators of Earthworm Jim and The Neverhood, lands on PC in September. We're giving away 10 Steam keys to lucky readers. That could mean you!

To enter the giveaway, simply click through to the giveaway page and write in your name and email address. 10 random winners will be picked from the pool of entrants.

Enter the giveaway here

The giveaway for Armikrog runs until Friday, September 4th at 12 pm PDT.

The game releases on September 8th. For more on Armikrog, check out our hands-on preview from E3 2015.

Nuketown is probably the most popular multiplayer map in the entire Call of Duty: Black Ops pantheon. It debuted in the original Blops, was updated as Nuketown 2025 for Black Ops 2, and will return again—this time as Nuk3town—as a preorder bonus for the upcoming Call of Duty: Black Ops 3.

"The Nuketown maps have a special place in Call of Duty multiplayer history and we can't wait to build upon that heritage in Black Ops 3," Treyarch boss Mark Lamia said. "We carefully adapted the design of the map to make sure that it fit with our new movement system, but to also make sure that we retained the spirit and the fun that made fans fall in love with it in the first place."

Activision said Nuk3town has been "redesigned from the ground up," this time with a focus on integrating the new "momentum-based chained-movement system." Anyone who preorders the game will receive a code to download the map for free, while those who don't will presumably have the option to pay for it. Such is the way of things in the space year 2015.

We don't generally endorse preordering around here, but as I've said previously, odds are you'd decided whether or not to buy Black Ops 3 before it was even announced. So if you're on the "yes" side of that particular fence, this probably represents a bit more good news for you. Call of Duty: Black Ops 3 comes out on November 6.

Blizzard announced today that the TeSPA collegiate Hearthstone championship, an event they ran last fall, will be back again this year with a new format and significantly larger prize pool. Similar to the Heroes of the Dorm tournament Blizzard hosted earlier this year preceding the launch of Heroes of the Storm, the teams will have a chance to win a portion of a $100K prize pool as scholarship money for their college education. 

The competition format, however, is new to Hearthstone. Teams will be comprised of three students playing a single deck together at the same time. The team will be able to discuss plays during games, and won't switch players or decks between games in match. This is a very different format from most Hearthstone tournaments, including the Archon Team League—another competition with teams of three, but where each player uses two decks of their own. 

The regular season will span over nine weeks, starting 28 September, with the championships being held in January. Registration for the tournament begins today, and you can see if you're eligible here.

The Witcher 3: Wild Hunt has certainly received its fair share of heaping praise. After all, this is one of the most expansive and beautifully realized worlds to grace our screens in quite some time. But The Witcher 3 also faced some criticism at launch: about its graphics, and for its lack of cultural diversity, especially with the notable absence of any characters of color.

Back in June, Polygon published a controversial opinion piece by Tauriq Moosa criticizing the game for failing to include not only notable characters of color, but anyone from a race that wasn't characterized as white. During PAX Prime, members of The Witcher 3's narrative design team held a panel to discuss the challenges of adapting the source material, but they also addressed the issue.

"There are 16 different language versions of the Witcher 3," said Jakub Szmalek, a senior writer for The Witcher 3. "And we wanted each of these versions to feel like it was aimed at this very [specific] market. We don't want you to feel like you are playing a lesser translated version of the game."

"If we achieve this goal, the problem arises that you forget that you are playing a translated version of the game."

Szmalek went on to address the controversy surrounding the lack of racial diversity in the game, saying that some people assumed that it was some form of omission or a statement that the studio was trying to make.

"It is definitely not," Szmalek said. "It's just that we tackled certain issues from a different perspective."

Travis Currit, who was one of the team responsible for translating The Witcher 3 to English, suggested that for those living in more racially diverse areas, the lack of representation feels far more pronounced. He went on to say that while Poland is relatively "homogeneous" in terms of race, CD Projekt Red drew inspiration from the tensions that existed between the various cultures coexisting in the country.

"It's fifty shades of white," Szmalek said.

"Every day you read the papers there's a new controversy," Currit said. "That is a huge aspect of the reality in which the games and the books were created, and we feel like we make very direct references to those in-game. We had become so absorbed with hitting these issues that are absolutely at the forefront of Polish consciousness, that we didn't have time to address the issues that are bigger in other countries."

"That's the issue of making a game that releases to so many peoples and cultures," Szmalek added.

Beyond the challenges of developing a game with such a laser focus on the politics of Poland, the team also spent the hour discussing the difficulties inherent in bringing the words on a page to life on a screen. A lot of these problems stemmed from finding ways to take aspects of Polish culture and present them in a manner that wasn't alienating to foreign audiences. The team also had to find a balance between adherence to the lore of the story and concessions made in pursuit of a more enjoyable game.

In the books, for example, witchers strictly refuse to use a crossbow, a weapon that comprises a valuable component of Geralt's arsenal in the third game. French fans were also surprised to see Geralt and Yennefer climb on top of a unicorn in The Witcher 3's infamous love scene because, due to a translation error made in the French version of the books, Geralt and Yennefer were described as making love atop a stuffed rhinoceros.

"There's more room on a rhino, so there are advantages to it," Szmalek joked.

"It kind of lacks some of the romance," Currit countered.

When asked if the team would approach adapting a book again, however, the answer was still an overwhelming yes.

"We cannot overestimate the importance of the lore that we have at our disposal," Szmalek said.

As the first part of the panel ended and fans took to asking the team questions, the topic inevitably drifted toward the potential for a return to the Northern Kingdoms. Though the team was quick to express their excitement for the upcoming expansion packs, which they reiterated would be roughly the size of The Witcher 2, they were reluctant to discuss any future plans for The Witcher.

"The next book we're going to be adapting," Currit said, "is the Cyberpunk rule book."