GDS 2018 will be taking place next week, running from Friday 7th to Saturday 8th. This year, like last year, we are silver sponsors of the event, which means you will see some Factorio branding around the event and in their official booklet. Part of the preparation on our side was to produce a nice graphical asset for their use, which you can see below:



The image is an aesthetic composition to showcase the design and theme of the game and its elements (while not necessarily making logical sense), and also contains the first public display of our new official Wube Software logo.

About half the office team here will be attending the event, so if you are also going you might bump into us.

We are tackling the fluid changes in 3 stages:

• Move all the fluid logic into a separate system.
• Merge straight sections of pipes into segments.
• Tweak the fluid flow logic, which will not be an optimisation, but a gameplay mechanics improvement (FFF-260).

Dominik has just finished stage 1, and it has been merged into 0.17, so let me present what was changed, and how it helps. The approach is similar to what we did with transport belts (FFF-176).

One of the main things slowing down the simulation is that every entity handling fluid (pipe, assembling machine with fluid connection, refinery, mining drill with fluid connection, pump, offshore pump) has to be kept as updatable and its update needs to be called every tick just to update the flowing of the internal pipes. So mining drills/assembling machines are being forced to do its logic instead of going to sleep.

From the perspective of optimisation, every extra piece of data that needs to get to the CPU cache is like an extra weight you need to carry around. It slows the whole simulation down.

The pipe is used just to call the inner update:



The other problem is, that the pipe memory is scattered around randomly. So we cut all the fluidboxes from the entities using them, and put them in a separate system (we call it the Fluid Manager). It is storing the fluidboxes like this:



The advantage of having data in continuous memory is mainly the fact that when CPU is copying the data from memory to cache, it is doing it in chunks, so when updating one thing, the next one is already in the cache. Modern CPUs are also smart, and they can detect continuous memory access, so they start prefetching the subsequent fluidboxes automatically in advance.

But it is not that simple, as the fluidbox always has neighbours (as the flow is from one fluidbox to others), and one of the neighbours can be on the other part of the continuous memory, out of cache, so it is still not perfect.

The next step was to divide the fluid boxes in something we call a Fluid system. A fluid system is basically all the Fluidboxes that are connected together. So, for example, in this refinery setup there are 5 fluid systems.



Each of the fluid system is now its own separate continuous memory of fluid boxes in it. The first advantage of this is that it increases the chance that the neighbours of a water pipe are close (memory wise) to the original pipe, the cached data from touching it for update could still be there when it is being touched as a neighbour.

But the second advantage is even greater, as the fluid systems are now independent and fluid flow doesn't interfere with anything else in the map, their update can be completely parallelized without any worries. So we just did that.

The final benchmark on a real fully producing map that uses pipes for production of materials and power (nuclear reactors), I was able to see a 6.5 times speedup of only the pipes update. The speedup wasn't so big on less beefy computers as it depends on the CPU, cache sizes, CPU core count etc. but was still around 3 times faster. I also expect the speedup to get bigger with bigger factories with more separate fluid systems and also with future CPU with more cores.

Dominik did benchmarked every step with a real save game on his reasonable i7-4790k, and recorded with the following overall performance gains.


(The graph shows some unexplained intermediary steps.)

Just a reminder that this is just a stage 1, before actually merging straight pipe sections into one fluid box which should improve things again. Even in systems with a lot of branching, as even 2 fluidboxes merged into 1 should help. In the example of refinery setup above, it seems that the fluidbox count could be reduced by factor of 4 or so. That should make the savings big enough to justify the planned 2 times slowdown of the future fluid flowing algorithm, as we will probably need 2 passes to make it work good enough. Dominik will have an update on the algorithm and conclusions from the previous discussions in a future FFF.

The implementation of several GUI redesigns is in progress, and it is being done by several team members. Suddenly we realized that coordinating more people brings new problems (what a surprise^^). We started to have mess in the styles, as everyone was inventing their own styles for the GUI to be same as the graphical mock-ups given by the graphics department.

After some discussions of how to solve this, we realized, that we mainly need some common language with the graphical department when it comes to the style hierarchy names. For that we need everyone to be able to quickly inspect which styles are used where and what is the hierarchy. For that reason, we made the GUI style inspector. By pressing a key (Control + F6 by default), every UI element will show a tooltip with information about the widget and its style, and will highlight of the widget as well. We wanted to use it only internally first, but we decided, that if it shows some info for the GUI created by scripts/mods, like name and type, it might be also useful for mod writers to be able to quickly inspect what is going on.



We even use colors to help us navigate:

• Red: The value was needlessly redefined (which was happening a lot).
• Green: The style value that is being used
• White: A style value that is not used as it is overwritten by a descendant style.

This tool (even when quite easy to add), immediately became very useful and it has already helped us to clean up some mess, and should improve the work efficiency when it comes to further GUI implementation. The main reaction when I showed this to the rest of the team was, "Why didn't we have this earlier?"... Well, better late than never.

This also means, that we are showing quite a lot about how the GUI style is organized to the players, so we need to be extra careful about making it tidy, to avoid bug reports about 'the weird mess' in the styles.

As always, let us know what you think on our forum.

Steam has opened up the nominations for this years Steam Awards. Last year Factorio was actually selected as a nominee for the 'Haunts My Dreams' award.



There is a category this year for 'Best Developer', and many in the community have wanted to nominate us for that category. Unfortunately to be eligible, we would need to have a developer page set up on Steam.

We had some discussions, and decided to wait until we have a final 'Wube Software' logo and theme finalized before setting up a developer page. This means you won't be able to vote for us as best developer this year...

This doesn't meant that you can't nominate Factorio for one of the categories, and there has already been some discussion on the subreddit about which games people are voting for.

The Substation was another of the entities we couldn't just render in double resolution, so it has taken quite some time and iterations to get to something we are happy with.



One of the key parts of the save/load process in Factorio is that it must be deterministic. This means that for a given save file (when no external factors change) saving, exiting, and loading the save shouldn't change any observable behavior.

There are a few reasons and benefits for this strict requirement:

• Without it: You couldn't join a running multiplayer game (and by proxy save, exit, and resume one).
• Without it: the replay system wouldn't work if you ever saved, exited, and resumed playing.
• With it: we can easily test that saving and loading produces no observable change letting us know we implemented save/load correctly.
• With it: you won't see things change randomly as a result of 'reloading' like you do in so many other games.

The premise sounds simple enough: make 'running game -> save -> exit -> load -> running game' produce the same results every time (when nothing else changes). However in practice this ends up being quite complex.

Factorio uses serialization and de-serialization (more), the exact format of the serialized save file changes from map version to map version so we always have to keep that in mind when loading.

As a slight aside, we took some measurement of what parts of the map data take what amount of space in a typical uncompressed save game:



Saving the game is fairly straight forward:

• Save the ID mapping (See 'Stable prototype IDs')
• Save the map data

Loading however becomes quite complicated due to the variety of things we need to account for:

• Map version migrations
• Mod additions/removals/changes
• Potentially invalid/corrupted save files

Over the years of Factorio development we've had a handful of strange bug reports about saves not loading which have been tracked to the loading logic previously not handling all of these special cases. A few notable examples:

• When items are changed due to base game/mod migrations, it can change what type an item is, resulting in what used to be a module now being something else but still occupying an entities module inventory.
• During loading the bounding box of an entity can change if the prototype data for that entity was changed (mod changed, mod added/removed). When the bounding box changes it can change which chunks the entity will overlap with. During loading it's expected that the number of chunks doesn't change. When the bounding box of an entity changes it can overlap with a chunk which didn't exist during saving.
• When prototype data changes it can mean that entity bounding boxes have changed and what electric networks an entity was in could have changed.
• When something is removed during loading (mod removal) we have to destroy them in a specific order because entities/items/electric networks can have references to eachother, and expect to be destroyed from a specific state in a specific order.

Overall the entire process has grown quite complex to the point where it's difficult for us to remember all of the special rules. With that in mind I spent a day earlier this week documenting the entire process for easier reference as to what the order of things is and why we do what we do. For those interested in an even more technical overview of the save/load process a copy of that document it can be found here.

Also on the topic of loading, we have decided to drop support for loading maps from version 0.13 and 0.14 in the latest version, which mean when you get your hands on 0.17, you will only be able to migrate a save game from 0.15. However as it worked before, you can still download 0.16 to migrate from 0.13, then migrate from 0.16 to 0.17.

It is approaching the festive season once again, and as such it has been nearly a whole year of selling our official Factorio T-shirts. Since moving to the new office, the merchandise operation has been given a whole double room to handle all the storage and shipping of the T-shirts.



Last year there were some cases where the orders didn't arrive in time for the Christmas tree, and a few people reported the packages taking over 1 month to arrive at their destination. We ship the orders from our office each Wednesday, so depending on when you order it can take almost a week for any shipping confirmation. Once it is shipped, a parcel will typically be delivered within a week anywhere in Europe, and take about another week or 2 to arrive in the USA and Worldwide.

Outside of shipping each week, we can't make any promises on when the orders will arrive, so the best way to prevent any heartache, is to order from our store as soon as possible.

As always, let us know what you think on our forum.

A lot of people have been asking recently, when can they expect a new release and when is the game going to be finished. The original plan was to finish everything, and release the final version of Factorio ideally before the end of 2018. This was the plan at the beginning of the year. We worked in our usual way of "it is done when it is done" for quite some time, but then it started taking a little bit too long, and we weren't even sure what is a realistic timeframe to finish it in.

To help this issue, we tried to become a little more organized in the past few weeks. We went through our list of all the development tasks, and tried to finalize it. We removed all the things that we decided to cut, and added all the missing things that we need to do before the game is finished. Then we tried to make some kind of time estimate for each task, to get a general idea of when everything will be finished. We started to be more conscious of who is working on what, and how much time each task is taking, to know how accurate the estimates are. The result was, that if it all goes well, we could be done in 6-9 months. This is probably not something you wanted to hear.

After a few rounds of discussions, we decided split the releases of 0.17 and 0.18 in the following way:

0.17 plan
It will contain all the things we have done up to this point, mainly:

• New render backend, which helps performance and solves a lot of issues (FFF-251)
• The graphical updates: walls, gates, turrets, belts, biters, spawners, electric poles (FFF-268, FFF-228, FFF-253)
• The GUI reskin (FFF-243)
• New map editor (FFF-252)
• Resource generation overhaul (FFF-258)
• Robot construction tools (FFF-255)
• Rich text (FFF-237, FFF-267)
• And more...

It will also include some things we know we can finish soon enough, mainly:

• Redoing some of the most important GUIs (Action bar, character screen, main game GUI, train GUI, play GUI, tooltips)
• Fluid optimisations
• And several smaller things, which depends on how it goes

We will release this during January 2019, we will announce it more precisely in advance.

0.18 plan
It will become the final 1.0 version once it is stable. It will contain mainly:

• New tutorial
• New campaign
• Final mini tutorials
• Revision of rest of the GUI
• All remaining high res graphics graphics and final polish

We obviously don't know exactly when is it going to be ready, but we hope it to be sooner than 9 months from now.

Over time we have reworked many graphics, sometimes 'just' bringing them to high resolution, sometimes changing their design, and sometimes even changing their perspective.Our 'camera angle' is 45 degrees, which in 'real projection' would result in rectangular tiles, but in Factorio this is contradicted by our tiles being square. This contradiction makes for a whole lot of challenges which we are addressing more and more over time. You might remember the old rails, concrete with grid, or trains which did not stretch.

Now we have found the solution for the last 'perspectively' incorrect entity - transport belts.



The basic idea is to give the belts some kind of a wall/structure to bring them off the ground, and create the illusion that they fit into the world correctly. The transport belts have two main limitations - the belt lanes of items are not to be changed, and the belt sprites already occupy a big portion of the tile so there is very little space to show anything extra.



It would be a shame to abandon the idea with the conclusion of "it’s impossible", when we do physically impossible things all the time, so we thought of exactly where we can go over the tile edge to get more space.



Because of how our sprite sorting works, going past the bottom edge of the tile should be safe, which gives the desired effect for horizontal belts and curves. Verticals need to be happy with just a few pixels for the structure, but on the right side we can add a shadow to show their height.



This is the smallest unit you can build in the game and demonstrate the concept. You can now see how belts reverse in the endings and see the reversed belt running underneath.



We have spent many hours with Albert trying to find the final shape. How big should the holes be to show the belt movement underneath, versus how massive should the holding structure be to demonstrate the shape. In the limited space every pixel matters, and the design you see is the one we arrived at after many iterations of experiments and tests.



Previously it didn’t matter how the endings worked, because they 'just disappeared' into the ground. Since it is now possible to see the full loop, the endings are now much more constraining for the artist, because they need to have an exact integer amount of belt pieces in order to fit into the animation.
A Normal belt tile has 4 pieces.For the ending, 3 pieces would be too long, and with only 1 piece you can’t bend the belt, so 2 pieces is the only feasible option. That means the ending is still much longer than before, which invites a bunch of glitches and problems too, but most of them had easy solutions, or our sprite sorting 'just handled it'.

This seems to work more or less until you see and realize that belt sprites are being flipped by the engine, which causes the shadows to break entirely, while the horizontal and curve lighting isn’t having a great time either. In the following picture you can see that each sprite has 1 correct, and 1 wrong use, but it’s the same sprite so you can’t fix the issue just by changing the sprite without breaking the other use.



Therefore we are finally getting rid of all flipping logic for belt sprites! Originally the flipping logic was there to save as much VRAM as possible because belts used to be a significant portion of all VRAM. Nowadays they are not, and it makes little sense to save memory at all costs on one of the most visible entities in the game.

I already found the flipping super weird when I was working on the high resolution belts (FFF-154), mainly the fact that 3 of the defined curves would almost make a circle, but the fourth would be mirrored for 'reasons'. Now each 'rotation' has its own unique sprites, which allows us (and mods) to be a lot more creative with custom belt sprites, and it is much easier to work with.

More problems were caused by exceeding the bottom and right edges of the tile. From some pieces drawing incorrectly over other pieces, through the shadows showing where they shouldn’t, to belt endings and underground belts breaking badly in multiple cases.

We were speaking about redesigning the underground belt structures with Albert multiple times already, and this was an ideal opportunity, but a redesign alone couldn’t fix everything...



The underground belt structures are always drawn above belts and icons - that means if they go below the tile border, they draw over belts in the front tile which should not happen. You can see neither of the previous problems are present in the following screenshot:



Mods can now define <tt>back_patch</tt> and <tt>front_patch</tt> layers to underground belt structures. The front patch prevents the structure from overlapping with the belts in front of it. The back patch prevents the structure from overlapping with the items inside the structure.



The new underground belt structure now fits better to the rounded shape of the endings and covers everything it needs to cover, but in some cases it also reveals more than before!...



As a bonus, the underground belt structure has a variant which shows a hole when you side-load into the underground belt.



It’s been a lot of work of iterating, fixing glitches, and solving problems, but you can see that the correct perspective makes the whole picture feel much more natural than before. We believe this will have a very positive impact on your experience in 0.17.

As always, let us know what you think on our forum.

Besides vegetable and plant stuff, biters are the main population on the surface of the Factorio planet. They are the locals, and somehow, from a twisted perspective, they can even be considered the bad guys. Not anymore. The magic of high resolution gives us the chance to move deeper into their conceptualisation and we've added a new ingredient to their formulation: cute...-ish.

For the last couple of weeks Ernestas and I have been working on the new version of the biters. Together we worked on developing the concept and ideas behind them, and Ernestas was doing the rest: modeling, texturing, shading, rigging, skinning, animating, rendering, post-processing, and being patient with me and my constant comments and changes.

Cute is how we like them, we want you to feel sorry about planning massive biter massacres. In fact we want you to feel pity towards them, especially when you are killing them and destroying their habitat at industrial scale. But also we want you to be disgusted by them, because they are alien to you, and they need to look the part, so it is quite a complicated equation.



Basically after the experience in-game with the classic version, we've learned what aspects of the biters are working well, and how to improve the parts which aren't. So we've decided to elongate their legs and accent their eyes in order to provide this more insectoid feeling. Also their new design is optimized for their attacks, they have 2 stronger front legs for providing destruction, 4 back legs to be able to run and stand during the attack, and stronger articulated mandibles to chew on your factories.



In these animations we can fully see the potential of disgust, the way they move now is more insect-like, similar to a cockroach (many people are disgusted by cockroaches), and also we've balanced the animation loop with their speed in the game, so they shouldn't slide around.

Keep in mind that this is still a work-in-progress and we have some more tweaks to do and extra animations to make, like their attack and death. We are also working on their sound design, and apart from that we might have some other surprises to make killing them extra gratifying to watch.

Redmew is a Multiplayer community that has been working on some really interesting scenarios. One which particularly caught our eye was the Diggy map.

In this scenario the player starts underground, with only a pickaxe, some walls, and a market to buy items. The walls hold up the ceiling of the map, and if you are not supporting the base as you dig-out more factory area, there is a risk of a cave-in.

https://youtu.be/4cRsx-wl_fk

Redmew also hosts other custom multiplayer servers, such as a PvP biter battle scenario, and servers with custom maps. If you are interested in knowing more, you can check out their Discord and Github. You can find the game servers by searching for "Redmew" in the matching server.

As always, let us know what you think on our forum.

The Entity info is the information about the currently selected entity that appears on the right side of the screen:



We had 2 problems with its current state:

• As it is on the side of the screen, and the entity you are inspecting is generally in the center, it feels cumbersome to move your eyes from the entity to the info and back all the time.
• As it is always appearing, it adds unnecessary clutter to the screen. It is always blinking there, while 99% of the time it is not really needed.

So we experimented with entity info as a tooltip next to the cursor when hovering the entity:



So we tested 3 different ways to activate it:

• It always appears at the cursor, which has the disadvantage of always being in your way in the middle of the screen.
• It always appears, but it has some delay.
• It only appears when a hotkey is pressed (we tested it with Shift), which has the disadvantage that you have to actually do something to see it.

We assigned each of the options to be tested by someone, with the hope to figure out which (if any) of them is better than the current one. Vaclav tested option 1, Twinsen tested option 2 and I tested option 3.

Unfortunately the result was that in the end everyone preferred his option the most, and we had no conclusion at all. Then we realized that the flaw of the test was that each of us picked the kind of option we already knew we would probably like.

After some discussions, we decided on the following:

• The current version of entity info will be the default.
• We add an option to set a custom delay for it, that is different than the normal tooltip delay (or never).
• We add an option to activate it with a key.
• We add an option to have it next to cursor or on the side.

Two years ago, I was under the impression, that we need to eradicate all the weird options, to make the game just work for everyone. Over time and after all experience and feedback we have gathered, I started to realize that different people have different expectations, and their brains are wired differently. Some option might be useless for 99% or players, but for the 1% of players, it might be the most annoying thing to be able to customize it.

The initial task, "Add support for robots to blow up cliffs", was easy. 1 cliff gets marked for deconstruction and 1 robot is sent to blow it up. The tricky part comes when you want to account for the fact that 1 cliff explosive can blow up more than a single cliff. When a robot is sent to blow up a cliff, the system has to do a little extra work:

• Figure out which cliffs within the radius of the cliff explosive are also marked for deconstruction that aren't already going to be blown up by another robot.
• Re-center the 'drop' position for the explosives on the gravity-center of the found cliffs.
• Figure out which cliffs are within that new area and record that 1 more robot is on its way to blow them all up.

This extra work isn't for free, however the game isn't likely to have a large number of robots actively bringing explosives to blow up a cliff, since as soon as a robot is dispatched it quickly finishes the job and the cliff is permanently gone. Using this system the game can track if a given cliff is going to be blown up, even if it isn't yet marked for deconstruction. This way you don't get weird overlaps in robots flying out to cliffs that will be blown up shortly by another robot.

https://cdn.factorio.com/assets/img/blog/fff-267-cliff-explosives.webm

After making this system it has made me think if I could apply some variant of it to robots building things, so a robot could theoretically grab 3 inserters and build all 3 quickly in series. It's an interesting thought for another time...

As we mentioned in FFF-237, we are going to be adding some fancy text effects to the game in 0.17. In that blog post we mentioned the possibility of having some interactive text elements, such as clickable icons in chat messages, blueprints, etc.

Well, we have an update on that, as we have now implemented several of these features. These tags are created in a somewhat general way, if you have the console/chat open and shift-click something, it will insert a chat tag.

Blueprint tags
Hovering a blueprint tag will show the full blueprint info, and clicking it will create a copy of the blueprint in your cursor.

https://cdn.factorio.com/assets/img/blog/fff-267-blueprint-tag.webm

Map pings
Shift-Clicking the map with the console open will create a map ping. Clicking a Map ping will open the map at the specified position.

https://cdn.factorio.com/assets/img/blog/fff-267-map-ping.webm

Recipe tags
Hovering a recipe tag will show the full recipe info.

https://cdn.factorio.com/assets/img/blog/fff-267-recipe-tag.webm

Armor tags
Hovering an armor tag will show you the armor and equipment layout for that specific armor.

https://cdn.factorio.com/assets/img/blog/fff-267-armor-tag.webm

As you can see above, each special tag has some [description] text associated with it, which lets you know you can interact with it. It will still be possible to insert plain icons that won't have any interactions.

For now, 'interactable' tags will only work in chat, but they might end up being allowed in mod GUIs too.

As always, let us know what you think on our forum.

Hello...
Part of the GUI rework for 0.17 is also tweaking the tooltips:

• They should be structured better.
• They should contain more useful information.
• They should be a better tool for new players to understand how things work.

We will cover more of the tooltip changes in a future FFF, but the necessary preparation for this is to rethink the way we explain some basic properties of machines to avoid as much bloat as possible. One of the good ways to do that is also to remove the need to show some of the mechanics by simplifying them, or completely removing them if we figure out that they are not really important for the game.

It has been quite a long time since the work on Factorio started, and we obviously couldn't see which mechanics/systems would be useful later on, and which would not. At that time, it was completely okay to just throw concepts into the game and see how it all works together.

But now, when we are finishing the game, it is the time for cleanup. Time to identify which mechanics are just adding barriers to understanding the game while not adding much to the game-design aspect, a good example of something we already removed a long time ago is the old furnace mechanics.

In the ancient versions of Factorio, the furnace mechanics were much more complicated. The furnace had to "warm up" before being operational, and if it wasn't used, the temperature went down again. This sounded like a nice mechanic, but it was soon discovered, that it adds little value in the scale of the factory, and it just bloats the games rules, so these complex mechanics have been removed from the the game for a very long time already.

In the ancient times, you first had to create a wooden pick, to create a stone pick, to mine basic resources, to make iron, so you can make an iron pick. Yes... this was clearly Minecraft affecting our ideas. We identified soon, that this prequel of manual mining has nothing to do with the core of the game, and is an unnecessary distraction. The fact that it was the first thing the player had to do in the game was gravely affecting what new players thought the game is about. So we kept only the iron/steel pick to streamline things.

Fast forward to these days and play-testing some of the tutorial tweaks. We noticed that players, when they start with Factorio, they often try to mine by taking the pickaxe into the cursor and doing the mining (as they might be used to doing from Minecraft or other similar games).

https://cdn.factorio.com/assets/img/blog/fff-266-mine-with-axe.webm
https://cdn.factorio.com/assets/img/blog/fff-266-mine-with-axe.mp4

So we were thinking how to improve the tutorial to avoid this mistake, but the next natural question was: "Why would we even need to have pickaxe in the game?". We realized that it is the item that you just craft in the beginning, and upgrade once in the middle for a steel pick, and that is it. The cost of it is zero compared to the factory output. It is just bloat. So the change for 0.17 is that we completely removed mining tools from the game. The mining speed at the game start is the same as with iron pickaxe, and the research that unlocked steel pickaxe just increases player mining speed accordingly and that is it.

We use 3 energy sources in the base game, burner, electricity, and heat. When an entity uses a burner energy source, it consumes the fuel, and the energy is used for it to function. Now lets say, that we want to answer this question:

How many boilers can be continuously fed from a single yellow belt of coal?

Lets look on the information the player can get:







So we will need to calculate how much energy is supplied by the value of the full belt of coal, which is 13.33/s * 8MJ = ~107MW.

Now, we should divide by the energy consumption, so 107 / 3.6 = ~29. Wait, what is this efficiency, and do we have to factor it in?

In the base game, this efficiency mechanic is almost completely useless, so we decided to remove it. To keep the previous balancing on the same level, all the fuel values have been halved, and the efficiency set to 100%. This just means, that the fuel value is the amount of energy the machines can actually extract from the fuel and calculations like this will give clear results. In this case, the functionality will still remain for mods to use.

Lets start with a small quiz. Based on these two tooltips, are you able to calculate how much iron ore an electric mining drill will produce each second?





The answer is 0.525/s. The calculation is simple...

• The hardness of the iron ore (0.9) is subtracted from the mining power of the mining drill (3) to get the adjusted mining power of 2.1.
• This is multiplied by the mining speed of the mining drill (0.5), to get the adjusted mining speed of 1.05.
• Since the mining time of the iron ore is 2, I divide 1.05 by 2 to get the 0.525 ores mined per second.

The hardness was supposed to be something like an "armor" of mining. It was meant to provide possibilities to define different tiers of mineable materials. However this feature was hardly ever utilized in the end. So the decision is that the whole hardness and mining power mechanics was removed. To make things even more straightforward, I made the mining time of stone and iron the same (stone was the only basic resource with different mining speed), so now, we can get the information needed as directly as this:





Calculating how many miners can fill a belt is now quite a straightforward task, and even with ores of non-standard mining speed, it is still much clearer, as the modifier should be understandable:



We have 8 different damage types in the game now: physical, impact, poison, explosion, fire, laser, acid and electric. Every other building has some kind of resistance, and it got so much in the way, that we don't even show resistances in tooltips anymore, only for enemy units and spawners.

The plan is to:

• Reduce the number of damage types to something like: generic, impact, heat, and acid.
• Keep the resistances mainly only on fight-related entities (walls, turrets, enemies) and remove them from the rest.
• Show the resistances for everything except some very specific cases (fire resistance for rails and poles to survive fires etc.).

This is also related to the way these things will be presented in the tooltip, but that too is a topic for a future FFF.

The idea behind this mechanic was that better assembling machines can use more complex recipes. But the reality is, that there is not really a clear connection between the number of ingredients and the complexity of the recipe. Since it was yet another thing that had to be explained somehow, we decided to just remove it. The only real downside is, that the achievement "lazy bastard" will be much less of a puzzle, but we still consider it to be worth it.

As always, let us know what you think on our forum.

Today I want to discuss some common problems that we see in video games.

Inconsistent Terminology

When I asked out loud "So what is an Intermediate Product anyway?", I got a similar reaction as when someone mentions The Berlin Interpretation at a rougelike convention. So what is an Intermediate Product? Well it is a product that is used only as an ingredient for something else. No, that's not right because Science Packs are not used in any recipe. So what then, Intermediate products are just things that you can use Productivity Modules on? Perhaps they are simply items that can be found in the Intermediate Crafting menu. Then are they not Intermediate Recipes?

To give another example, answer these questions:

• Name the action a player performs when they add an entity to the world?
• Name the action a player performs when they remove an entity from the world?
• Name the action a player performs when they add a ghost entity to the world?
• Name the action a robot performs when they add an entity to the world?
• Name the action a robot performs when they remove an entity from the world?

Here are a few situations where the game displays your possible answers:


A player builds.


A player mines entities.


Robots repair and build entities, but wait… the player places buildings and builds ghosts?


But here Robots are constructing machines.


Here the robots are deconstructing items!

This leads into a discussion about what is an item and what are entities, and that discussion leads us into the next point...

Internal nomenclature leaking out

During game development it is very common to use internal names to refer to mechanics, items, or characters. It does not feel like such a big deal, and many early access games simply ignore the problem completely. I'm not going to point any fingers, but if you look you will find some examples. Oh wait, here is some from your favourite early access game!


Internally, things that exist on a surface in the game are called entities.


All these items are capsules internally, but only 5 of them are actually labelled as capsules.

Really, these should be categorised by how players use them, and indeed there is an attempt to do so. Remotes are items used to trigger an effect, Grenades are things you throw... but why is the Poison Capsule not called a gas grenade?

There are more inconsistencies but to keep this article reasonably not-short, I will let you find the others yourself (and to save something for a future FFF about Tooltips).

Why change?

You might be thinking that this is not a big problem. Some others might be thinking that the problem is too pervasive to bother changing. There are a few reasons why it is important, the first, and most important of which is our quality mindset; everyone on the team here wants the game to be as great as possible.

Next we should see this increase the quality of the translations. A translation is only as good as its source, and having a consistent usage of words can go a long way to helping the translators do better work. The effect of this can be increased by providing a dictionary of important words to the translators so they can be sure to always use the same term in all places.

Since we are also working on a guided experience (Campaign), this would also help us give much clearer instructions to the player. An example of confusion here would be if one quest said "Place a chest" and another said "Place the item in the chest". The player needs to read the entire quest caption (probably twice), and can never build up a mental map of our language. This leads to the player spending more mental energy (cognitive load) while playing the game. Changing this to "Build a chest" and being consistent, allows the player to create mental shortcuts, meaning the quest tasks require less effort to understand.

Finally, consistency in terminology will help new players, and I don't just mean sub-1 hour playtime players. Factorio is a 'Big Game' and players are encountering new items, entities, concepts, and text for a long time. How many hours did you play before you discovered this helpful trick, or this one?

How to change?

We could make the vocabulary consistent with what the current player base uses. This option sounds pretty good until I started asking people questions similar to those I asked you at the beginning of the article. Here are another two as a refresher:

• Where do biters come from?
• I come in 7 colors, what am I?

The only wrong answer is if you said there was only a single right answer.

Prepare your rotten tomatoes, Ben is about to say something unpopular. The influx of players that are to be expected from 1.0 give us an interesting option. We could theoretically change the vocabulary of the game to be more consistent, reasonable, and generally more helpful to players. Then, as new players join the community, this new language will slowly replace the old.

This would help ease communication between all players; veterans and new addicts alike. Consistency will also help polish the experience to the level that players expect from the game.

Who should change it?

Before Rseding jumps in with some awesome news, I would ask you to have your say in this Google form. It will be fun to see what you come up with, and I will publish the results in a few weeks.

As many of you might know if you've tried hosting a stand-alone multiplayer game (Factorio or otherwise) from a home internet connection, it's not a very reliable experience. This can be due to multiple different things (bad home routers, company/school firewalls and so on).

I arrived in Prague earlier this week and was given the task of improving that experience through Steam Networking support. The way Steam Networking works is: when you start a multiplayer game, the game can tell Steam that you want to accept connections from other Steam users. The benefit of doing this is if someone can connect to Steam, they can connect to you. Even if your local connection isn't setup to let the multiplayer game work, Steam can route the network data through it's existing connection and get the packets where they need to go.

After a few days of working on it and testing it seems to be working and working reliably. You can just click "start multiplayer game", and anyone on your friends list can click join. You can mix non-steam and steam players on the same server and it "just works". As a bonus while working on this I discovered some issues with the non-steam networking logic that should improve the "Could not establish network communication with server" error that keeps coming up.

As always, let us know what you think on our forum.

Hello, it is me, posila, with another technical article. Sorry.

In 0.16, we added graphics option mysteriously called "Low VRAM mode", it enables a basic implementation of texture streaming, but I didn't want to call it that way, because I feared its poor performance would give texture streaming a bad name. How it worked? Every sprite has specified some priority, and the "Video memory usage" option - despite its name - controls which priorities of sprites are included in the sprite atlases. What happens to sprites that don't go to atlas? They are loaded as individual sprites. Normally, these sprites are allocated into texture objects. The reasoning behind this is that graphics driver has chance to decide how it wants to layout these small textures in memory, instead of it being forced to work with huge atlases.



What part of a sprite atlas looks like.

When you enable "Low VRAM mode", non-atlas sprites are loaded only to RAM, and texture is allocated for them only when used during rendering. We called class that handled this BitmapCache and there was maximum limit of how many mega-bytes of texture data the bitmap cache could use at once. When this limit was reached, it failed to convert memory bitmap to video texture, and Allegro's drawing system would fallback to software rendering, which absolutely tanked FPS, but this didn't happen... most of the time.

So apart from the obvious problem of falling back to software renderer (which we don't have any more after the graphics rewrite, so the game would crash or skip a sprite if this happened), there are other performance issues. Most sprites have a unique size, so we can't reuse textures for different sprites. Instead, when a sprite needs to be converted to a texture, a new texture is allocated, and when the sprite is evicted from the cache, its texture is destroyed. Creating and destroying textures considerably slows down rendering. The way we do it also fragments memory, so all of the sudden it may fail allocate new texture because there is no large enough consecutive block of memory left. Also, since our sprites are not in an atlas, sprite batching doesn't work and we get another performance hit from issuing thousands of draw calls instead of just hundreds.

I considered it to be an experiment, and actually was kind of surprised that its performance was not as bad as I expected. Sure, it might cause FPS drop to single digits for a moment from time to time, but overall the game was playable (I have a long history of console gaming, so my standards as player might not be very high :)).

Can we make it good enough, so it wouldn't be an experimental option any more, but something that could be enabled by default? Let's see. The problem is texture allocations, so let's allocate one texture for the entire bitmap cache - it would be a sprite atlas that we would dynamically update. That would also improve sprite batching, but when I started to think how to implement it, I quickly ran into a problem dealing with the fragmentation of space. I considered doing "defragmentation" from time to time, but it started to seem like an overwhelming problem, with a very uncertain result.

As I mentioned in FFF-251, it is very important for our rendering performance to batch sprite draw commands. If multiple consecutive draw commands use the same texture, we can batch them into a single draw call. That's why we build large sprite atlases. Virtual texture mapping - a texture streaming technique popularized by id Software as Mega Textures, seems like a perfect fit for us. All sprites are put into a single virtual atlas, the size of which is not restricted by hardware limits. You still have to be able to store the atlas somewhere, but it doesn't have to be a consecutive chunk of memory. The idea behind it is the same as in virtual memory - memory allocations assign a virtual address that maps to some physical location that can change under the hood (RAM, page file, etc.), sprites are assigned virtual texture coordinates that are mapped to some physical location.

The virtual atlas is divided into tiles or pages (in our case 128x128 pixels), and when rendering we will figure out which tiles are needed, and upload them to a physical texture of much smaller dimensions than the virtual one. In the pixel shader, we then transform virtual texture coordinates to physical ones. To do that, we need an indirection table that says where to find the tiles from the virtual texture in the physical one. It is quite a challenge for 3D engines to figure out which virtual texture pages are needed, but since we go through the game state to determine which sprites should be rendered, we already have this information readily available.

That solves the problem of frequent allocations - we have one texture and just update it. Also, since all the sprites share the same texture coordinate space, we can batch draw calls that use them. Great!

However we could still run out of space in the physical texture. This is more likely if player zooms out a lot, as lot more different sprites can be visible at once. Well, if you zoom out, sprites are scaled down, and we don't need to render sprites in their full resolution. To exploit this, the virtual atlas has a couple levels of details (mipmaps), which are the same texture scaled down to 0.5 size, 0.25 size, etc. and we can stream-in only the mipmap levels that are needed for the current zoom level. We can use lower mipmap levels also if you are zoomed in and there are just too many sprites on the screen. We can also utilize the lower details to limit how much time is spent for streaming per frame to prevent stalls in rendering when a big update is required.

The Virtual atlas technique is big improvement over the old "Low VRAM mode" option, but it is still not good enough. In the ideal case, I would like it to work so well, we could remove low and very-low sprite quality options, and everyone would be able to play the game on normal. What prevents that from happening is that the entire virtual atlas needs to be in RAM. Streaming from HDD has very high latency, and we are not sure yet if it will be feasible for us to do without introducing bad sprite pop-ins, etc.

If you'd like to learn how virtual texture mapping works in more detail, you can read the paper Advanced Virtual Texture Topics, or possibly even more in-depth Software Virtual Textures.

The main motivation behind texture streaming, is to make sure the game is able to run with limited resources, without having to reduce visual quality too much. According to the Steam hardware survey, almost 60% of our players (who have dedicated GPU), have at least 4GB of VRAM and this number grows as people upgrade their computers:



We have received quite a lot of bug reports about rendering performance issues from people with decent GPUs, especially since we started adding high-resolution sprites. Our assumption was that the problems were caused by the game wanting to use more video memory than available (the game is not the only application that wants to use video memory) and the graphics driver has to spend a lot of time to optimize accesses to the textures.

During the graphics rewrite, we learned a lot about how contemporary GPUs work (and are still learning), and we were able to utilize the new APIs to measure how much time rendering takes on a GPU.

To simply draw a 1920x1080 image to a render target of the same size, it takes:

• ~0.1ms ~0.07ms on GeForce GTX 1060.
• ~0.15ms ~0.04 on Radeon Vega 64.
• ~0.2ms on GeForce GTX 750Ti or Radeon R7 360.
• ~0.75ms on GeForce GT 330M.
• ~1ms on Intel HD Graphics 5500.
• ~2ms on Radeon HD 6450.

This seems to scale linearly with the number of pixels written, so it would take ~0.4ms for the GTX 1060 to render the same thing in 4K.

Update: Several people wondered how come Vega 64 ended up slower than GTX 1060. I originally ran the tests with 60 FPS cap, so I re-ran the tests without the cap and got ~0.04ms on Vega, and ~0.07ms on GTX 1060. So the cards were probably operating in some kind of low-power mode, since they were idle for huge part of the frame. You should still take my measurements with big grain of salt, I didn't attempt to be scientific about it, I just wanted to illustrate huge performance difference between different GPUs people might want to use to play the game.

That's pretty fast, but our sprites have a lot of semi-transparent pixels. We also utilize transparency in other ways - from drawing ghosts and applying color masks, to drawing visualizations like logistic area or turret ranges. This results in large amount of overdraw - pixels being written to multiple times. We knew overdraw was something to avoid, but we didn't have any good data on how much it happens in Factorio, until we added the Overdraw visualisation:



The game scene being rendered.


Overdraw visualisation (cyan = 1 draw, green = 2, red >= 10).


Overdraw visualisation when we discard transparent pixels.

Interestingly, discarding completely transparent pixels didn't seem to make any performance difference on modern GPUs, including the Intel HD. Drawing sprites with a lot of completely transparent pixels is faster than an opaque sprite without having to explicitly discard transparent pixels with shaders. However, it did make difference on Radeon HD 6450 and GeForce GT 330M, so perhaps modern GPUs throw away pixels that wouldn't have any effect on the result automatically?

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms, 10ms on integrated GPUs, and more that a single frame worth of time (16.66ms) on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

As always, let us know what you think on our forum.

Building trains again and again might be a daunting task. Especially when you start making a lot of mining outposts, artillery/supply trains with filtered cargo wagon slots etc.

So I decided that we should extend blueprints to work with trains as well. The first condition was, that trains are only selected when you explicitly allow it in the checkbox, so they don't get in your way when building rail setups.



Checking the button allows the train that was there to be put into the blueprint (similar to the way tiles work). For the sake of simplicity, we decided that once there is any rail in the blueprint, the train in it will be always buildable (as a ghost obviously), even if there are not rails to support the train at the moment. The train ghost will simply stay there and won't be buildable until rails are placed under it in a way so it can be placed.



https://cdn.factorio.com/assets/img/blog/fff-263-train-rails-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-rails-blueprint.webm

If I remove the rails from the blueprint, I get a second type of rail blueprint. In this case, all the parts of need to have rails to support it, this is mainly needed as without rails, there is no rail grid forced, so we should make sure, the train ghost won't be created in some wrong position.



https://cdn.factorio.com/assets/img/blog/fff-263-train-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-blueprint.webm

The small touch here is, that the blueprint also contains the schedule. With little-bit of improvisation, I can optimize the mine building a lot in the late game. I create a blueprint of mine train station. The stop will be called Mine X.

Both of the trains in the blueprint will have the Mine X -> Smelting schedule setup. Once I build the blueprint, I just rename the Mine X to whatever I want (Mine 12 for example), and the train schedules are updated as well, so I'm almost ready to go.

https://cdn.factorio.com/assets/img/blog/fff-263-train-station-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-station-blueprint.webm

The last tweak I'm considering is to allow blueprints to contain the fuel insertion info similar to how they contain the module insertion info for assembling machines now.

When we showed it, the upgrade planner was Prototype ready, but playtesting uncovered various kind of missing tweaks, and as always the real usability is in the details. Especially since the mod already solves most of these issues, we have to at least be as good as the mod.

The first problem was, with underground belts. With the naive approach of just upgrading entities that are selected, it happened quite a lot, that I upgraded some belt area, but I didn't notice, that some of the underground belts have only one side marked for upgrade, so the belts get disconnected when the upgrade is executed:



This was be usually discovered half an hour later when I was investigating the reason of some part of factory not producing. When it happened few times in a row, it was quite annoying... So the fix was, to make sure that the upgrade planner always upgrades connected underground belts in pairs if possible:



Part of the tweak is, that the robot upgrades both part of the underground belt as one action, so the items that are "flowing" underground don't have to be touched.

The next thing we wanted was to not only upgrade entities, but their also contents, so you can now specify module upgrades in the upgrade planner as well. (Please note, that the upgrade planner/blueprint UI is a work in progress, and this particular screen was not yet addressed by our GUI polishing process).



Another feature is to be able to upgrade blueprints with the upgrade planner. The upgrade planner mod could do it as well, but we will be able to integrate it better with our GUI better.




You can immediately see, that another tweak might be to also upgrade the blueprint icons as well.

My thanks goes to galibert, who created the original pull request of this feature (giving source access to stranger sometimes helps us) and Rseding91, who fixed the technical problems and added the tweaks mentioned earlier.

It has been quite a long time since we described our latency hiding system in FFF-83. Since then, we have had to make a tough choice whether to incorporate a new interaction feature into the latency hiding or not as we developed it. With undo, it was kind of implied, that it needs to be incorporated in the latency state system, since you need accurate instant feedback of what you undo-ed, especially when you are undoing few things in a row, and you don't want to do more steps by accident because of the multiplayer delay. Since there are more and more things in the latency state, and they need to interact with each other in a reasonable way, the amount of possible cases starts to grow, so we have to make sure that the corner cases are covered by tests more than in other areas.


Let me present you a quite simple case of what you see compared to what is happening in latency state under the hood, this is a simple thing:

Start

Furnace built

Marked for deconstruction

Undone


But when you play in multiplayer, it might easily happen, that after the last undo step, not even the first command to build the actual furnace is in the game yet as the multiplayer latency might be too big. But as the local simulated state must act as it all was done instantly, it needs to solve different kind of situations

What you see (latency state) / Actual game state
Start / Start

Fake furnace created / Nothing yet

Fake furnace fake marked for deconstruction / Nothing yet

Fake furnaces fake mark for deconstruction fake undone / Nothing yet

Now real furnace fake marked for deconstruction / Furnace built in game state

Real furnaces fake mark for deconstruction undone / Furnace marked for deconstruction in game state

What you see is what you get / Furnaces mark for deconstruction undone in game state


Doing all of these kind of cases right might make a big difference between Prototype ready and Release ready. With the undo feature itself, I added 42 different tests and I'm not completely finished.

As always, let us know what you think on our forum.

In FFF-241 we discussed how the game delivers information to the player in a number of confused ways; Blinking arrows and circles, chat messages on the bottom left corner of the screen, objectives in the top left, orange modal boxes bubbles on top of the player, and so on. These problems are exacerbated on high resolution monitors, where the information becomes even further spread apart.

We have tried a few ways to unify this information, but much of it was required to be in the world space, or needed to have a link between the screen space and the world space. The common solution to this is to have the GUI 'point' to an entity in the game world, but we wanted something more interesting.

The idea for a robot sidekick in a video game is nothing new, but the solution fits to Factorio really well. When I arrived at the office and mentioned the concept, Albert immediately asked “Have you ever notice the forum FactorioBot?" and proceeded to show me some concept renders for the robot. Alberts idea had been discussed several times before as a solution to this very problem, just waiting for the right time. Well, Compilatron your time is now!


Albert's latest render of Compilatron, our companion character.

The Compilatron character allows us to couple text and positional data together, which creates a much tighter connection between the subject of the information and the information itself. Previously, the orange modal boxes and blinking arrows could be used in this way, as they could point to entities in the world. Compilatron can do this without breaking the immersion and game flow.


We don’t want to reveal too much, but you can see that this is already better than the orange modal windows.

Compilatron also gives us a hand with screen space information. If we can successfully style some of the user interface elements (such as the new as-of-yet-unshown objective window) to match the Compilatron style, I believe we can create an even stronger association between text in the screen space and that in the world space.

Another way the Compilatron helps us keep the immersion intact is as a plot agent. Currently campaign plot events just 'happen' when objectives are completed, or when characters internal monologue decides the next move. Now we can have Compilatron act as a driving force to the campaign, it gives the objectives to the player, interacts with entities during cutscenes, and overall makes things less awkward than they are now.

These are two of the major benefits of having Compilatron, there are also some more interesting things we can do with it in the future, The specifics of how Compilatron will work and interact with the player are still in development, so expect some changes from what you see here.

So far I feel like the solution is working out, but not without some issues, the largest of which was working with the current Unit AI...

I have been working on the technical side of Compilatron, which should be of interest to the modders among you. Compilatron is entirely scripted through Lua, it is not a custom C++ entity at all, which meant we needed to add a few things to our scripting API to make it possible.

First off, The Compilatron is a Unit, which means it is controlled in the same basic manner as the biters. As Units have never been used in the base game for anything but biters, there are quite a few unusual behaviours baked into the Unit control code, that didn't really make sense outside the context of biters. To fix this without breaking the base game, I've been adding prototype flags to units which disable these special behaviours by default, and enabling them only on biters (e.g., trying to return to a spawner every now and again, which can mess up your current command execution). These are being fixed on an ad-hoc basis whenever we run into them, so if you're a modder who's had some pain with this kind of thing, please let us know what you'd like to change.

The most significant change I've made is probably opening up the pathfinder to Lua script. This means that you can issue asynchronous pathfinding requests that are not associated with an actual command, useful for e.g., checking if you can path from one place to another. Later on, we will also include the ability to issue a move order and pass in the computed pathfinding data, which would let you modify paths before executing them, or even write your own totally custom pathfinding in Lua.
As well as what's mentioned above, there's been a whole host of miscellaneous changes and additions, including:

• Script event on AI command completion, so you know when a Unit has completed its current command.
• Allowing units to pathfind through friendly buildings (i.e. walk over to it, destroy it, then continue over the place it used to stand). Only when explicitly requested, of course.
• Allowing units to path through and open friendly gates.
• Added a whole bunch of flags to go_to_location commands, exposing pathfinder internals such a low priority pathfind requests, and prioritising straight paths.
• Units will now avoid pathing through all friendly entities, as opposed to the old behaviour of only avoiding same-force entities.
• Added new AI command stop, and improved the wander command significantly.
• Added a new HighlightBox entity, that will draw a selection box around a specified entity or at a specific position.
• Added a speech bubble entity.
• Allow accessing Lua scripts by __MOD-NAME__/script.lua style path, so you can reference the same Lua scripts from multiple scenarios in one mod.

Klonan has already made a mod to help test and debug all the new modding commands, scripting, and API features:

https://cdn.factorio.com/assets/img/blog/fff-262-mod-example.mp4
https://cdn.factorio.com/assets/img/blog/fff-262-mod-example.webm

We're very interested in the benefits that these changes will bring modders "for free", so if you're an interested modder with some opinions/requests about this topic, let us know, we've started a specific forum thread for the topic here.

As always, let us know what you think on our forum.