Hello...
Part of the GUI rework for 0.17 is also tweaking the tooltips:

• They should be structured better.
• They should contain more useful information.
• They should be a better tool for new players to understand how things work.

We will cover more of the tooltip changes in a future FFF, but the necessary preparation for this is to rethink the way we explain some basic properties of machines to avoid as much bloat as possible. One of the good ways to do that is also to remove the need to show some of the mechanics by simplifying them, or completely removing them if we figure out that they are not really important for the game.

It has been quite a long time since the work on Factorio started, and we obviously couldn't see which mechanics/systems would be useful later on, and which would not. At that time, it was completely okay to just throw concepts into the game and see how it all works together.

But now, when we are finishing the game, it is the time for cleanup. Time to identify which mechanics are just adding barriers to understanding the game while not adding much to the game-design aspect, a good example of something we already removed a long time ago is the old furnace mechanics.

In the ancient versions of Factorio, the furnace mechanics were much more complicated. The furnace had to "warm up" before being operational, and if it wasn't used, the temperature went down again. This sounded like a nice mechanic, but it was soon discovered, that it adds little value in the scale of the factory, and it just bloats the games rules, so these complex mechanics have been removed from the the game for a very long time already.

In the ancient times, you first had to create a wooden pick, to create a stone pick, to mine basic resources, to make iron, so you can make an iron pick. Yes... this was clearly Minecraft affecting our ideas. We identified soon, that this prequel of manual mining has nothing to do with the core of the game, and is an unnecessary distraction. The fact that it was the first thing the player had to do in the game was gravely affecting what new players thought the game is about. So we kept only the iron/steel pick to streamline things.

Fast forward to these days and play-testing some of the tutorial tweaks. We noticed that players, when they start with Factorio, they often try to mine by taking the pickaxe into the cursor and doing the mining (as they might be used to doing from Minecraft or other similar games).

https://cdn.factorio.com/assets/img/blog/fff-266-mine-with-axe.webm
https://cdn.factorio.com/assets/img/blog/fff-266-mine-with-axe.mp4

So we were thinking how to improve the tutorial to avoid this mistake, but the next natural question was: "Why would we even need to have pickaxe in the game?". We realized that it is the item that you just craft in the beginning, and upgrade once in the middle for a steel pick, and that is it. The cost of it is zero compared to the factory output. It is just bloat. So the change for 0.17 is that we completely removed mining tools from the game. The mining speed at the game start is the same as with iron pickaxe, and the research that unlocked steel pickaxe just increases player mining speed accordingly and that is it.

We use 3 energy sources in the base game, burner, electricity, and heat. When an entity uses a burner energy source, it consumes the fuel, and the energy is used for it to function. Now lets say, that we want to answer this question:

How many boilers can be continuously fed from a single yellow belt of coal?

Lets look on the information the player can get:







So we will need to calculate how much energy is supplied by the value of the full belt of coal, which is 13.33/s * 8MJ = ~107MW.

Now, we should divide by the energy consumption, so 107 / 3.6 = ~29. Wait, what is this efficiency, and do we have to factor it in?

In the base game, this efficiency mechanic is almost completely useless, so we decided to remove it. To keep the previous balancing on the same level, all the fuel values have been halved, and the efficiency set to 100%. This just means, that the fuel value is the amount of energy the machines can actually extract from the fuel and calculations like this will give clear results. In this case, the functionality will still remain for mods to use.

Lets start with a small quiz. Based on these two tooltips, are you able to calculate how much iron ore an electric mining drill will produce each second?





The answer is 0.525/s. The calculation is simple...

• The hardness of the iron ore (0.9) is subtracted from the mining power of the mining drill (3) to get the adjusted mining power of 2.1.
• This is multiplied by the mining speed of the mining drill (0.5), to get the adjusted mining speed of 1.05.
• Since the mining time of the iron ore is 2, I divide 1.05 by 2 to get the 0.525 ores mined per second.

The hardness was supposed to be something like an "armor" of mining. It was meant to provide possibilities to define different tiers of mineable materials. However this feature was hardly ever utilized in the end. So the decision is that the whole hardness and mining power mechanics was removed. To make things even more straightforward, I made the mining time of stone and iron the same (stone was the only basic resource with different mining speed), so now, we can get the information needed as directly as this:





Calculating how many miners can fill a belt is now quite a straightforward task, and even with ores of non-standard mining speed, it is still much clearer, as the modifier should be understandable:



We have 8 different damage types in the game now: physical, impact, poison, explosion, fire, laser, acid and electric. Every other building has some kind of resistance, and it got so much in the way, that we don't even show resistances in tooltips anymore, only for enemy units and spawners.

The plan is to:

• Reduce the number of damage types to something like: generic, impact, heat, and acid.
• Keep the resistances mainly only on fight-related entities (walls, turrets, enemies) and remove them from the rest.
• Show the resistances for everything except some very specific cases (fire resistance for rails and poles to survive fires etc.).

This is also related to the way these things will be presented in the tooltip, but that too is a topic for a future FFF.

The idea behind this mechanic was that better assembling machines can use more complex recipes. But the reality is, that there is not really a clear connection between the number of ingredients and the complexity of the recipe. Since it was yet another thing that had to be explained somehow, we decided to just remove it. The only real downside is, that the achievement "lazy bastard" will be much less of a puzzle, but we still consider it to be worth it.

As always, let us know what you think on our forum.

Today I want to discuss some common problems that we see in video games.

Inconsistent Terminology

When I asked out loud "So what is an Intermediate Product anyway?", I got a similar reaction as when someone mentions The Berlin Interpretation at a rougelike convention. So what is an Intermediate Product? Well it is a product that is used only as an ingredient for something else. No, that's not right because Science Packs are not used in any recipe. So what then, Intermediate products are just things that you can use Productivity Modules on? Perhaps they are simply items that can be found in the Intermediate Crafting menu. Then are they not Intermediate Recipes?

To give another example, answer these questions:

• Name the action a player performs when they add an entity to the world?
• Name the action a player performs when they remove an entity from the world?
• Name the action a player performs when they add a ghost entity to the world?
• Name the action a robot performs when they add an entity to the world?
• Name the action a robot performs when they remove an entity from the world?

Here are a few situations where the game displays your possible answers:


A player builds.


A player mines entities.


Robots repair and build entities, but wait… the player places buildings and builds ghosts?


But here Robots are constructing machines.


Here the robots are deconstructing items!

This leads into a discussion about what is an item and what are entities, and that discussion leads us into the next point...

Internal nomenclature leaking out

During game development it is very common to use internal names to refer to mechanics, items, or characters. It does not feel like such a big deal, and many early access games simply ignore the problem completely. I'm not going to point any fingers, but if you look you will find some examples. Oh wait, here is some from your favourite early access game!


Internally, things that exist on a surface in the game are called entities.


All these items are capsules internally, but only 5 of them are actually labelled as capsules.

Really, these should be categorised by how players use them, and indeed there is an attempt to do so. Remotes are items used to trigger an effect, Grenades are things you throw... but why is the Poison Capsule not called a gas grenade?

There are more inconsistencies but to keep this article reasonably not-short, I will let you find the others yourself (and to save something for a future FFF about Tooltips).

Why change?

You might be thinking that this is not a big problem. Some others might be thinking that the problem is too pervasive to bother changing. There are a few reasons why it is important, the first, and most important of which is our quality mindset; everyone on the team here wants the game to be as great as possible.

Next we should see this increase the quality of the translations. A translation is only as good as its source, and having a consistent usage of words can go a long way to helping the translators do better work. The effect of this can be increased by providing a dictionary of important words to the translators so they can be sure to always use the same term in all places.

Since we are also working on a guided experience (Campaign), this would also help us give much clearer instructions to the player. An example of confusion here would be if one quest said "Place a chest" and another said "Place the item in the chest". The player needs to read the entire quest caption (probably twice), and can never build up a mental map of our language. This leads to the player spending more mental energy (cognitive load) while playing the game. Changing this to "Build a chest" and being consistent, allows the player to create mental shortcuts, meaning the quest tasks require less effort to understand.

Finally, consistency in terminology will help new players, and I don't just mean sub-1 hour playtime players. Factorio is a 'Big Game' and players are encountering new items, entities, concepts, and text for a long time. How many hours did you play before you discovered this helpful trick, or this one?

How to change?

We could make the vocabulary consistent with what the current player base uses. This option sounds pretty good until I started asking people questions similar to those I asked you at the beginning of the article. Here are another two as a refresher:

• Where do biters come from?
• I come in 7 colors, what am I?

The only wrong answer is if you said there was only a single right answer.

Prepare your rotten tomatoes, Ben is about to say something unpopular. The influx of players that are to be expected from 1.0 give us an interesting option. We could theoretically change the vocabulary of the game to be more consistent, reasonable, and generally more helpful to players. Then, as new players join the community, this new language will slowly replace the old.

This would help ease communication between all players; veterans and new addicts alike. Consistency will also help polish the experience to the level that players expect from the game.

Who should change it?

Before Rseding jumps in with some awesome news, I would ask you to have your say in this Google form. It will be fun to see what you come up with, and I will publish the results in a few weeks.

As many of you might know if you've tried hosting a stand-alone multiplayer game (Factorio or otherwise) from a home internet connection, it's not a very reliable experience. This can be due to multiple different things (bad home routers, company/school firewalls and so on).

I arrived in Prague earlier this week and was given the task of improving that experience through Steam Networking support. The way Steam Networking works is: when you start a multiplayer game, the game can tell Steam that you want to accept connections from other Steam users. The benefit of doing this is if someone can connect to Steam, they can connect to you. Even if your local connection isn't setup to let the multiplayer game work, Steam can route the network data through it's existing connection and get the packets where they need to go.

After a few days of working on it and testing it seems to be working and working reliably. You can just click "start multiplayer game", and anyone on your friends list can click join. You can mix non-steam and steam players on the same server and it "just works". As a bonus while working on this I discovered some issues with the non-steam networking logic that should improve the "Could not establish network communication with server" error that keeps coming up.

As always, let us know what you think on our forum.

Hello, it is me, posila, with another technical article. Sorry.

In 0.16, we added graphics option mysteriously called "Low VRAM mode", it enables a basic implementation of texture streaming, but I didn't want to call it that way, because I feared its poor performance would give texture streaming a bad name. How it worked? Every sprite has specified some priority, and the "Video memory usage" option - despite its name - controls which priorities of sprites are included in the sprite atlases. What happens to sprites that don't go to atlas? They are loaded as individual sprites. Normally, these sprites are allocated into texture objects. The reasoning behind this is that graphics driver has chance to decide how it wants to layout these small textures in memory, instead of it being forced to work with huge atlases.



What part of a sprite atlas looks like.

When you enable "Low VRAM mode", non-atlas sprites are loaded only to RAM, and texture is allocated for them only when used during rendering. We called class that handled this BitmapCache and there was maximum limit of how many mega-bytes of texture data the bitmap cache could use at once. When this limit was reached, it failed to convert memory bitmap to video texture, and Allegro's drawing system would fallback to software rendering, which absolutely tanked FPS, but this didn't happen... most of the time.

So apart from the obvious problem of falling back to software renderer (which we don't have any more after the graphics rewrite, so the game would crash or skip a sprite if this happened), there are other performance issues. Most sprites have a unique size, so we can't reuse textures for different sprites. Instead, when a sprite needs to be converted to a texture, a new texture is allocated, and when the sprite is evicted from the cache, its texture is destroyed. Creating and destroying textures considerably slows down rendering. The way we do it also fragments memory, so all of the sudden it may fail allocate new texture because there is no large enough consecutive block of memory left. Also, since our sprites are not in an atlas, sprite batching doesn't work and we get another performance hit from issuing thousands of draw calls instead of just hundreds.

I considered it to be an experiment, and actually was kind of surprised that its performance was not as bad as I expected. Sure, it might cause FPS drop to single digits for a moment from time to time, but overall the game was playable (I have a long history of console gaming, so my standards as player might not be very high :)).

Can we make it good enough, so it wouldn't be an experimental option any more, but something that could be enabled by default? Let's see. The problem is texture allocations, so let's allocate one texture for the entire bitmap cache - it would be a sprite atlas that we would dynamically update. That would also improve sprite batching, but when I started to think how to implement it, I quickly ran into a problem dealing with the fragmentation of space. I considered doing "defragmentation" from time to time, but it started to seem like an overwhelming problem, with a very uncertain result.

As I mentioned in FFF-251, it is very important for our rendering performance to batch sprite draw commands. If multiple consecutive draw commands use the same texture, we can batch them into a single draw call. That's why we build large sprite atlases. Virtual texture mapping - a texture streaming technique popularized by id Software as Mega Textures, seems like a perfect fit for us. All sprites are put into a single virtual atlas, the size of which is not restricted by hardware limits. You still have to be able to store the atlas somewhere, but it doesn't have to be a consecutive chunk of memory. The idea behind it is the same as in virtual memory - memory allocations assign a virtual address that maps to some physical location that can change under the hood (RAM, page file, etc.), sprites are assigned virtual texture coordinates that are mapped to some physical location.

The virtual atlas is divided into tiles or pages (in our case 128x128 pixels), and when rendering we will figure out which tiles are needed, and upload them to a physical texture of much smaller dimensions than the virtual one. In the pixel shader, we then transform virtual texture coordinates to physical ones. To do that, we need an indirection table that says where to find the tiles from the virtual texture in the physical one. It is quite a challenge for 3D engines to figure out which virtual texture pages are needed, but since we go through the game state to determine which sprites should be rendered, we already have this information readily available.

That solves the problem of frequent allocations - we have one texture and just update it. Also, since all the sprites share the same texture coordinate space, we can batch draw calls that use them. Great!

However we could still run out of space in the physical texture. This is more likely if player zooms out a lot, as lot more different sprites can be visible at once. Well, if you zoom out, sprites are scaled down, and we don't need to render sprites in their full resolution. To exploit this, the virtual atlas has a couple levels of details (mipmaps), which are the same texture scaled down to 0.5 size, 0.25 size, etc. and we can stream-in only the mipmap levels that are needed for the current zoom level. We can use lower mipmap levels also if you are zoomed in and there are just too many sprites on the screen. We can also utilize the lower details to limit how much time is spent for streaming per frame to prevent stalls in rendering when a big update is required.

The Virtual atlas technique is big improvement over the old "Low VRAM mode" option, but it is still not good enough. In the ideal case, I would like it to work so well, we could remove low and very-low sprite quality options, and everyone would be able to play the game on normal. What prevents that from happening is that the entire virtual atlas needs to be in RAM. Streaming from HDD has very high latency, and we are not sure yet if it will be feasible for us to do without introducing bad sprite pop-ins, etc.

If you'd like to learn how virtual texture mapping works in more detail, you can read the paper Advanced Virtual Texture Topics, or possibly even more in-depth Software Virtual Textures.

The main motivation behind texture streaming, is to make sure the game is able to run with limited resources, without having to reduce visual quality too much. According to the Steam hardware survey, almost 60% of our players (who have dedicated GPU), have at least 4GB of VRAM and this number grows as people upgrade their computers:



We have received quite a lot of bug reports about rendering performance issues from people with decent GPUs, especially since we started adding high-resolution sprites. Our assumption was that the problems were caused by the game wanting to use more video memory than available (the game is not the only application that wants to use video memory) and the graphics driver has to spend a lot of time to optimize accesses to the textures.

During the graphics rewrite, we learned a lot about how contemporary GPUs work (and are still learning), and we were able to utilize the new APIs to measure how much time rendering takes on a GPU.

To simply draw a 1920x1080 image to a render target of the same size, it takes:

• ~0.1ms ~0.07ms on GeForce GTX 1060.
• ~0.15ms ~0.04 on Radeon Vega 64.
• ~0.2ms on GeForce GTX 750Ti or Radeon R7 360.
• ~0.75ms on GeForce GT 330M.
• ~1ms on Intel HD Graphics 5500.
• ~2ms on Radeon HD 6450.

This seems to scale linearly with the number of pixels written, so it would take ~0.4ms for the GTX 1060 to render the same thing in 4K.

Update: Several people wondered how come Vega 64 ended up slower than GTX 1060. I originally ran the tests with 60 FPS cap, so I re-ran the tests without the cap and got ~0.04ms on Vega, and ~0.07ms on GTX 1060. So the cards were probably operating in some kind of low-power mode, since they were idle for huge part of the frame. You should still take my measurements with big grain of salt, I didn't attempt to be scientific about it, I just wanted to illustrate huge performance difference between different GPUs people might want to use to play the game.

That's pretty fast, but our sprites have a lot of semi-transparent pixels. We also utilize transparency in other ways - from drawing ghosts and applying color masks, to drawing visualizations like logistic area or turret ranges. This results in large amount of overdraw - pixels being written to multiple times. We knew overdraw was something to avoid, but we didn't have any good data on how much it happens in Factorio, until we added the Overdraw visualisation:



The game scene being rendered.


Overdraw visualisation (cyan = 1 draw, green = 2, red >= 10).


Overdraw visualisation when we discard transparent pixels.

Interestingly, discarding completely transparent pixels didn't seem to make any performance difference on modern GPUs, including the Intel HD. Drawing sprites with a lot of completely transparent pixels is faster than an opaque sprite without having to explicitly discard transparent pixels with shaders. However, it did make difference on Radeon HD 6450 and GeForce GT 330M, so perhaps modern GPUs throw away pixels that wouldn't have any effect on the result automatically?

Anyway, a GTX 1060 renders a game scene like this in 1080p in 1ms. That's fast, but it means in 4K it would take 4ms, 10ms on integrated GPUs, and more that a single frame worth of time (16.66ms) on old, non-gaming GPUs. No wonder, scenes heavy on smoke or trees can tank FPS, especially in 4K. Maybe we should do something about that...

As always, let us know what you think on our forum.

Building trains again and again might be a daunting task. Especially when you start making a lot of mining outposts, artillery/supply trains with filtered cargo wagon slots etc.

So I decided that we should extend blueprints to work with trains as well. The first condition was, that trains are only selected when you explicitly allow it in the checkbox, so they don't get in your way when building rail setups.



Checking the button allows the train that was there to be put into the blueprint (similar to the way tiles work). For the sake of simplicity, we decided that once there is any rail in the blueprint, the train in it will be always buildable (as a ghost obviously), even if there are not rails to support the train at the moment. The train ghost will simply stay there and won't be buildable until rails are placed under it in a way so it can be placed.



https://cdn.factorio.com/assets/img/blog/fff-263-train-rails-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-rails-blueprint.webm

If I remove the rails from the blueprint, I get a second type of rail blueprint. In this case, all the parts of need to have rails to support it, this is mainly needed as without rails, there is no rail grid forced, so we should make sure, the train ghost won't be created in some wrong position.



https://cdn.factorio.com/assets/img/blog/fff-263-train-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-blueprint.webm

The small touch here is, that the blueprint also contains the schedule. With little-bit of improvisation, I can optimize the mine building a lot in the late game. I create a blueprint of mine train station. The stop will be called Mine X.

Both of the trains in the blueprint will have the Mine X -> Smelting schedule setup. Once I build the blueprint, I just rename the Mine X to whatever I want (Mine 12 for example), and the train schedules are updated as well, so I'm almost ready to go.

https://cdn.factorio.com/assets/img/blog/fff-263-train-station-blueprint.mp4
https://cdn.factorio.com/assets/img/blog/fff-263-train-station-blueprint.webm

The last tweak I'm considering is to allow blueprints to contain the fuel insertion info similar to how they contain the module insertion info for assembling machines now.

When we showed it, the upgrade planner was Prototype ready, but playtesting uncovered various kind of missing tweaks, and as always the real usability is in the details. Especially since the mod already solves most of these issues, we have to at least be as good as the mod.

The first problem was, with underground belts. With the naive approach of just upgrading entities that are selected, it happened quite a lot, that I upgraded some belt area, but I didn't notice, that some of the underground belts have only one side marked for upgrade, so the belts get disconnected when the upgrade is executed:



This was be usually discovered half an hour later when I was investigating the reason of some part of factory not producing. When it happened few times in a row, it was quite annoying... So the fix was, to make sure that the upgrade planner always upgrades connected underground belts in pairs if possible:



Part of the tweak is, that the robot upgrades both part of the underground belt as one action, so the items that are "flowing" underground don't have to be touched.

The next thing we wanted was to not only upgrade entities, but their also contents, so you can now specify module upgrades in the upgrade planner as well. (Please note, that the upgrade planner/blueprint UI is a work in progress, and this particular screen was not yet addressed by our GUI polishing process).



Another feature is to be able to upgrade blueprints with the upgrade planner. The upgrade planner mod could do it as well, but we will be able to integrate it better with our GUI better.




You can immediately see, that another tweak might be to also upgrade the blueprint icons as well.

My thanks goes to galibert, who created the original pull request of this feature (giving source access to stranger sometimes helps us) and Rseding91, who fixed the technical problems and added the tweaks mentioned earlier.

It has been quite a long time since we described our latency hiding system in FFF-83. Since then, we have had to make a tough choice whether to incorporate a new interaction feature into the latency hiding or not as we developed it. With undo, it was kind of implied, that it needs to be incorporated in the latency state system, since you need accurate instant feedback of what you undo-ed, especially when you are undoing few things in a row, and you don't want to do more steps by accident because of the multiplayer delay. Since there are more and more things in the latency state, and they need to interact with each other in a reasonable way, the amount of possible cases starts to grow, so we have to make sure that the corner cases are covered by tests more than in other areas.


Let me present you a quite simple case of what you see compared to what is happening in latency state under the hood, this is a simple thing:

Start

Furnace built

Marked for deconstruction

Undone


But when you play in multiplayer, it might easily happen, that after the last undo step, not even the first command to build the actual furnace is in the game yet as the multiplayer latency might be too big. But as the local simulated state must act as it all was done instantly, it needs to solve different kind of situations

What you see (latency state) / Actual game state
Start / Start

Fake furnace created / Nothing yet

Fake furnace fake marked for deconstruction / Nothing yet

Fake furnaces fake mark for deconstruction fake undone / Nothing yet

Now real furnace fake marked for deconstruction / Furnace built in game state

Real furnaces fake mark for deconstruction undone / Furnace marked for deconstruction in game state

What you see is what you get / Furnaces mark for deconstruction undone in game state


Doing all of these kind of cases right might make a big difference between Prototype ready and Release ready. With the undo feature itself, I added 42 different tests and I'm not completely finished.

As always, let us know what you think on our forum.

In FFF-241 we discussed how the game delivers information to the player in a number of confused ways; Blinking arrows and circles, chat messages on the bottom left corner of the screen, objectives in the top left, orange modal boxes bubbles on top of the player, and so on. These problems are exacerbated on high resolution monitors, where the information becomes even further spread apart.

We have tried a few ways to unify this information, but much of it was required to be in the world space, or needed to have a link between the screen space and the world space. The common solution to this is to have the GUI 'point' to an entity in the game world, but we wanted something more interesting.

The idea for a robot sidekick in a video game is nothing new, but the solution fits to Factorio really well. When I arrived at the office and mentioned the concept, Albert immediately asked “Have you ever notice the forum FactorioBot?" and proceeded to show me some concept renders for the robot. Alberts idea had been discussed several times before as a solution to this very problem, just waiting for the right time. Well, Compilatron your time is now!


Albert's latest render of Compilatron, our companion character.

The Compilatron character allows us to couple text and positional data together, which creates a much tighter connection between the subject of the information and the information itself. Previously, the orange modal boxes and blinking arrows could be used in this way, as they could point to entities in the world. Compilatron can do this without breaking the immersion and game flow.


We don’t want to reveal too much, but you can see that this is already better than the orange modal windows.

Compilatron also gives us a hand with screen space information. If we can successfully style some of the user interface elements (such as the new as-of-yet-unshown objective window) to match the Compilatron style, I believe we can create an even stronger association between text in the screen space and that in the world space.

Another way the Compilatron helps us keep the immersion intact is as a plot agent. Currently campaign plot events just 'happen' when objectives are completed, or when characters internal monologue decides the next move. Now we can have Compilatron act as a driving force to the campaign, it gives the objectives to the player, interacts with entities during cutscenes, and overall makes things less awkward than they are now.

These are two of the major benefits of having Compilatron, there are also some more interesting things we can do with it in the future, The specifics of how Compilatron will work and interact with the player are still in development, so expect some changes from what you see here.

So far I feel like the solution is working out, but not without some issues, the largest of which was working with the current Unit AI...

I have been working on the technical side of Compilatron, which should be of interest to the modders among you. Compilatron is entirely scripted through Lua, it is not a custom C++ entity at all, which meant we needed to add a few things to our scripting API to make it possible.

First off, The Compilatron is a Unit, which means it is controlled in the same basic manner as the biters. As Units have never been used in the base game for anything but biters, there are quite a few unusual behaviours baked into the Unit control code, that didn't really make sense outside the context of biters. To fix this without breaking the base game, I've been adding prototype flags to units which disable these special behaviours by default, and enabling them only on biters (e.g., trying to return to a spawner every now and again, which can mess up your current command execution). These are being fixed on an ad-hoc basis whenever we run into them, so if you're a modder who's had some pain with this kind of thing, please let us know what you'd like to change.

The most significant change I've made is probably opening up the pathfinder to Lua script. This means that you can issue asynchronous pathfinding requests that are not associated with an actual command, useful for e.g., checking if you can path from one place to another. Later on, we will also include the ability to issue a move order and pass in the computed pathfinding data, which would let you modify paths before executing them, or even write your own totally custom pathfinding in Lua.
As well as what's mentioned above, there's been a whole host of miscellaneous changes and additions, including:

• Script event on AI command completion, so you know when a Unit has completed its current command.
• Allowing units to pathfind through friendly buildings (i.e. walk over to it, destroy it, then continue over the place it used to stand). Only when explicitly requested, of course.
• Allowing units to path through and open friendly gates.
• Added a whole bunch of flags to go_to_location commands, exposing pathfinder internals such a low priority pathfind requests, and prioritising straight paths.
• Units will now avoid pathing through all friendly entities, as opposed to the old behaviour of only avoiding same-force entities.
• Added new AI command stop, and improved the wander command significantly.
• Added a new HighlightBox entity, that will draw a selection box around a specified entity or at a specific position.
• Added a speech bubble entity.
• Allow accessing Lua scripts by __MOD-NAME__/script.lua style path, so you can reference the same Lua scripts from multiple scenarios in one mod.

Klonan has already made a mod to help test and debug all the new modding commands, scripting, and API features:

https://cdn.factorio.com/assets/img/blog/fff-262-mod-example.mp4
https://cdn.factorio.com/assets/img/blog/fff-262-mod-example.webm

We're very interested in the benefits that these changes will bring modders "for free", so if you're an interested modder with some opinions/requests about this topic, let us know, we've started a specific forum thread for the topic here.

As always, let us know what you think on our forum.

We have already pointed out, that we are trying to make a new campaign (FFF-245), and part of it is the core beginning, the NPE/tutorial.

The tutorial is one of the very critical parts of the game, as if the first 15 minutes of a game feels shitty, there is big chance, that the player will not play any further. I had this experience in many games myself.

So the challenge could be articulated like this: "The current tutorial is okay, but can we make it great?"
The approach in the current tutorial is to feed the player with the basic knowledge of how to control the basics of the game (the first mission and the start of the second mission) in the fastest way possible.



The player is even given descriptive info like this, to diminish the chance of not understanding how the basic entities work.



After few steps in the 2nd level, the player can start exploring his first self-feeding loop (make iron to make more iron).



The tools used to this is mainly:

• The message dialog that stops the game and explains the player various things.
• Minimization of the amount of things the player can interact with to absolute minimum, so he can't start doing other things before the basics are clear.

The possible drawbacks:

• The constant interruptions can get really annoying (typically around 22 message dialogs before the player starts to play relatively freely in the 2nd level).
• The possibility, that the player will mindlessly follow the step-by step tasks without understanding it, so he can become really lost later on, and the tutorial basically wouldn't help him to understand things.

So the question is: Can we make a tutorial that makes these problems go away?, and alternatively, How big are these problems actually?

The current direction of the new tutorial attempt is to never use the message dialogs, so the player can learn the game more fluently, and to leave way more things to explore, as learning things yourself has a better chance of success than force-feeding generally.

We made a few tests of the new approach with a few people, the main takeaway, is that nothing is for free, and this approach created new drawbacks.

• If the player doesn't figure out something basic, it can be very frustrating for him to figure out what is going on when not moving forward for a long time.
• It might be possible, that some things are just not fun to learn by exploration, and it is more comfortable if they are force fed to you. I would compare this to a friend explaining you how to play a game for 5 minutes compared to 2 hours of trial and error.

There are more possible outcomes from this, and we shall see how different tweaks of both strategies work in the testings. It might be interesting if you mentioned your experience with the tutorial in the comment section as well.

Regardless of the final tutorial approach we choose, we made some generic improvements that should help to avoid some of the pitfalls.
Interaction error messages

People sometimes struggled (in the beginning), to figure out why they can't interact with an object (open/mine/build, connect wires etc.) when it is too far away. We still keep the Beep Boop error sound and the different (yellowish) color, but we added a short flying text explaining the problem.

https://cdn.factorio.com/assets/img/blog/fff-261-not-reach.mp4
https://cdn.factorio.com/assets/img/blog/fff-261-not-reach.webm

https://cdn.factorio.com/assets/img/blog/fff-261-not-wire.mp4
https://cdn.factorio.com/assets/img/blog/fff-261-not-wire.webm

This also includes GUI related stuff like this:

https://cdn.factorio.com/assets/img/blog/fff-261-not-ingredients.mp4
https://cdn.factorio.com/assets/img/blog/fff-261-not-ingredients.webm

https://cdn.factorio.com/assets/img/blog/fff-261-not-hand.mp4
https://cdn.factorio.com/assets/img/blog/fff-261-not-hand.webm

https://cdn.factorio.com/assets/img/blog/fff-261-not-machine.mp4
https://cdn.factorio.com/assets/img/blog/fff-261-not-machine.webm

Some of these error had description messages, but these were in the bottom-left corner in the console, but we observed, that since it might quite far from the mouse and focus of the player, it can be easily missed. Even if it is not missed, it is not that comfortable to look somewhere "far away" for an error.

Highlighting of inserter/miner related entities

When building/hovering an inserter, miner or any entity that inserter/miner can interact with, the corresponding entities are highlighted. This should help to understand the connection, and even for me, it is useful sometimes to see instantly, whether the inserter is one tile off or not when building it.



Sometimes, I wondered why is the performance in Factorio is very unstable. The update runs fine (lets say 5ms) most of the time, and from time to time, it takes much longer (16+), so the frame is skipped, so the game feels choppy. For a long time, I expected, that this is caused by some Factorio tasks that can peak once in a while (train path finding), but the weird realisation came, when I played multiplayer, and I noticed, that other people didn't have the problem, so I started to investigate.
Browser

I realized this long time ago when doing performance tests. Running any browser with modern (dynamic) page, or and browser related applications (slack for example) take more performance than you think. I could measure non-trivial increases in performance when I closed all these.

Windows - performance options
There is a Power options panel in windows (Control Panel → Hardware and Sound → Power options), you can select different power options. It mainly controls things like how long it takes before it goes to sleep, or when are the Hard disks stopped, but it contains more, notably the minimum processor state, which by default (in the default plan, is 5%):

With this settings, I am seeing performance similar to this on my test save
Values are avarage/minimum/maximum in the context of the last 10 seconds.


While the default for high-performance is 100%:

With this, I am getting this result:


You can see, that the average time spent on update is roughly the same, but with the balanced mode, ther are quite big peaks of slowdowns that are happening regularly. You can test this yourself.

The minimum processor state basically allows the system to somehow slow down or turn off some parts of the processor. Factorio works in a way, that it does a lot of work in the update steps (the 5ms time), and then, the update thread has to wait until the time of the next update.
My theory is that when the thread is waiting for the next update, the system thinks that it doesn't need that much CPU power suddenly, so it slows down, but it doesn't switch back to full power fast enough when it is needed again in the next update.

I would be careful with this, as I can imagine that having this option all the time might not be that a good idea, mainly for laptops, but if your game starts to be choppy because of performance, it is worth a try to set this option for the time when you play Factorio at least.

The new forum theme caused quite some debate on the forum and in the office this week, specifically about the limited content width, and the result is that we added a slightly modified theme to scale the forum to the whole screen width. I would like to know your opinion, whether you prefer a limited content width, or prefer the content to fill your whole browser window, there will be a poll to vote in the forum post for this FFF.

As is standard, you can leave us your thoughts and feedback on our forum.

Hi Factorians,
This is Dominik, and my first FFF post ever! I will use this opportunity to talk to you about the exciting subject of pipes. Yeah, I know, right?

Spring came and with it Twinsen, saying "Pipes suck. Two people already tried to fix it and failed, who wants to be next?", and I’m like "Hey, that’s just pipes, you just make a simple simulation, simple AF. I’m in."
The conditions were even quite lenient:

• Fluids get where they should.
• They should act in a predictable manner, with reasonable splitting/joining in junctions.
• Fluids can travel instantly, if need be.
• Respect the pipe throughput limitations.
• Flow can be viewed on the pipes.
• Don’t do f**** up stuff like running in a circle indefinitely, sloshing back and forth endlessly etc.
• Should be faster/more UPS efficient.

I am mostly working on the new GUIs, but still, the fact that summer is over and pipes are not done, kinda shows how simple fixing them is. Very naive I was.

There are couple mishaps in the current pipe system. Good thing is that they work - the fluids do get from A to B, in most cases anyway. It follows a simple elevation model, fluidboxes will try to equalise with their neighbours, which is quite fast to evaluate and solves the simple cases, but it does not address much outside of running through a straight pipe.


(Green column represents volume, Blue represents flow)

The first of three main issues is that in junctions it behaves in a very random fashion. As a result, you might get recipients that are not getting any fluid where they obviously should be.



The second issue players voice is the limited behavior of the elevation functions. Only a fraction of the fluid is moved to its neighbour, so you rarely have a tank that is entirely full or entirely empty, which is a problem when you try to mix that with the integer based circuit network.

The third issue is performance. Even though the formulas are simple, they are calculated for every connection in every fluidbox, which adds up. As a result, nuclear power is unfeasible for megabases. There are other problems too, such as throughput loss over distance, fluids moving faster or slower depending on the entity update order, etc.


(Fluid flows much quicker in the bottom pipe)

The game is not at all practical for testing and debugging pipes, so I built a simple command line pipe simulator to develop the new system in. As I was attacking what I thought to be a simple problem, I started simple, and then had to refactor it several times whenever I realized that the problem is harder than what my simulator can support. I will shortly describe how it works, before going to the model itself.

The pipe layouts are loaded from a text file such as “5 1 \n s p p d 0”, which is 2 dimensions of the system, followed by a 2d layout, in this case just one row source-pipe-pipe-drain-empty. The simulator loads the layout, connects pipes and then updates the system tick by tick on request. I get a very rough overview, as picture below, but most of the time I have to inspect the running code to see what is going on under the hood.



Though now with the new map editor and the ability to step through single ticks, it will be much easier to test in-game too.

Before going to the current, simulation based model, I will shortly discuss other approaches that we have rejected.

Optimizing the current system
This is something Harkonnen tried a long time ago, to reduce indirection in the update of the fluidboxes. Essentially, instead of each entity updating their own fluidbox, all fluidboxes in a segment would be kept in a singular part of memory, and then the simulation could be updated much faster. Initial experiments showed a performance increase of 30-50% updating all fluidboxes. However this would not address any of the other issues, and would add a significant amount of complexity to the currently quite simple handling of the fluidboxes, which we decided isn't worth the price.

Flow model
Since we are doing flow here, flow algorithms look like a candidate. The most naive Ford–Fulkerson method, although theoretically infinite, could work super fast in our case. Problem is that it only finds the maximum flow - the top limit of what we can push through. We could then divide this max fluid flow between the consumers and kinda get a working results. But the behaviour on the way would be ridiculous with full flow through one pipe and 0 through next, 0 to dead ends etc. In other words, the junction behavior would be entirely broken. Better balanced flow algorithms exist but these also don’t do exactly what we need and the complexity quickly jumps to astronomical realms.

Electric network model
Another proposal was modelling like an electric network. Fluid flow is a popular analogy to their workings, and they do have a lot in common. The great thing about them is that they precisely model the flow branching and it could work out of the box. What it does not allow though is to limit the flow - one wire can, theoretically, run any current, but not so for the pipes, and we don’t want one pipe to be enough to supply whole factory. The limitations could be added, but that would, again, kill it with complexity.

A simplified version of this is what we consider the 'nuclear option'. In short, fluid network and pipes would work like the current electric network, instant transmission from production to consumption. This would increase performance by orders of magnitude, and remove the unintuitive flow of the current system, all consumers would get an equal split of the production, and storage tanks would act like accumulators.

However we have decided not to pursue this, for a few reasons.

• There would be no visualisation or indication of the flow of fluid.
• There would be unlimited throughput, one water pipe could supply all boiler and reactor setups.
• It abstracts away part of the realism and charm of the game. (While this is subjective at best, it does mean something to us.)

Merging fluidboxes
This would mean merging all the similar pipes in a segment into only a single fluidbox that needs updating. This would solve the performance issues, and the throughput loss over distance, however on its own, it would not solve the issues with update order, unintuitive flow direction/splitting etc.

Physics
When it comes to the basic physics model, I ended up with something that is not at all realistic, but works well in practice. At the beginning, I tried to do almost realistic physics - as a proof of concept, with momentums and all that. But quickly I became cluttered with complicated formulas and it did not work at all. The system is so heavily discrete that physics are not really applicable.

In Factorio, a full content of a ~meter long pipe moves into the next pipe in one tick and instead of mixing single molecules in a junction all in real time (infinitely little steps) we have to mix/split huge blocks in one shot. It’s more like moving Lego blocks than running fluids. For a long time I was playing with pressure but I dropped even that in favour of just two variables - volume and speed, where the speed kinda models the pressure as well. Volume is the amount of fluid in the segment, speed affects how much of it wants to move on in a tick - as speed x volume. Just this is enough to provide a pretty decent simulation.

Junction model
Most difficulties come from modelling the junctions. The general problem is that when anything does not behave entirely correctly, there exists a situation where it creates a total breakdown, such as no flow into some pipe.

There are colliding forces in play. We can only evaluate one pipe connection (one inlet/outlet) at a time, but the behavior needs to be symmetrical and fair towards pipes that are to be evaluated later (currently it is not). What is the right order of evaluation and how to make it symmetrical without super complex look-ahead?
Well, another big consideration is accuracy vs. complexity.

Over many iterations I kept developing models, followed by counterexamples that killed them.

The evaluation model I have arrived to works with two passes through the pipes in one tick. The formulas are more complex than the current one so it should be slower, but there will be one improvement to counterbalance it. The rough algorithm in one tick is as follows (I omit many necessary but boring details):

• (Done at the end of previous step) Every pipe states how much fluid it intends to send to neighbours (called flow reservation) using a simple heuristic.
• Perform topological sorting by direction of reservations from 0.
• Evaluate pipes in the opposite direction, i.e. from end to start, against the expected flow.
• Update a pipe in two stages:
  • Move fluid in it to neighbours proportionally to the space in them, space allocation they gave to the current pipe (providing they were already evaluated), and previous tick flow.
  • Allocate the resulting free space to neighbours that are yet to be evaluated to ensure they get fair share of it.
• Go through the pipes again and do clean-up and calculate reservations for the next tick.

So in this algorithm, we go from the last pipe, always moving fluid and making space for the next one. The reservations/allocations system ensures good behavior in the junctions. Essentially at every junction, the fluid will try to be split evenly between all the possible connections, which makes things very intuitive.

It works nicely, but unfortunately this system is even more computationally demanding. Here comes in the key improvement which is taking every straight pipe (every segment that has max two connections) and connect it into one piece. No matter how long it is, it will be evaluated in one calculation. Naturally, this loses some realism, but it is insignificant as it is the junctions that matter and those will remain unaffected. So as long as you don’t do crazy stuff like making pipe grids and keep your pipes straight, they will have zero effect on UPS.

To put it more simply, updates will only be run on junctions and segments. At each junction, fluid will be split evenly among the neighbouring segments and junctions, any excess from one neighbour will spill evenly to the others. A segment is just a section of pipe without any splits, and fluid will transfer instantly through segments, but with a limited throughput.



So in the setup above, we would go from updating 32 individual pipes with the 'old' system, to updating 3 junctions (blue) and 8 segments. Since a segment can be any length, overall we expect a big performance increase. It could also lead to more UPS efficient designs, trying to minimize the number of splits in your pipe network, we know some players really enjoy trying to optimize for different metrics.

A last convenience improvement are some rounding mechanisms to fill or move away those 0.0001 fluids, draining the last drops from the pipe system if the sources are disconnected, and filling all the way up if production is greater than consumption. Another point left to consider is how to solve accidentally connecting pipes and tainting your whole oil system with water.

All this is nice and already working, but still in the stage of the simulator.
What I still need to do to get it to 0.17:

• Figure out good model for storage tanks, and how they fit in.
• Perhaps refactor it from floats to integers, which would make it work more cleanly, but would also make some calculations more complicated. TBD.
• Finish the algorithm for correctly drawing the flow direction in the connected straight pipes (not that simple).
• Move it all into the game code.
• Write tests for it (I am probably overreacting but I feel that this will take as long as all the work up to this point).

So to sum up, we have had this on our minds for a long time now, and performance was not the only issue we have considered. The new system will hopefully address all the issues we mentioned at the start.

As always, let us know what you think on our forum.

Actually, speaking of the forum, you might notice that it underwent some changes this week. Sanqui has updated our forum from phpBB version 3.0 to 3.2, which is a bigger jump than it sounds. It brings us more in line modern web features, the forums are now usable on phones, it heightens security, and paves the way for future extensions. Some style changes are final, but if you have any particular gripes, please say so and we will try to accommodate everybody.

While migrating from Allegro to SDL, HanziQ and Jiri replaced the keystroke handling from using key-codes to scan-codes. Before you start jumping with joy, you’ll probably wonder: What is that and why should I care?

Well, funny you should ask. You probably won’t care, unless you live outside of USA and/or you use a non-US keyboard layout. In short, key-code is a key identifier dependant on the symbol the key will output when pressed, scan-code is a key identifier based on the physical location of a key. So for example players with French keyboard layout (AZERTY) have to jump to the control options after launching the game for the first time, and remap movement from WASD to ZQSD, in order to be able to move their character without hurting their hand.



In 0.17, the controls will map to the correct keys by default, regardless of your layout, and stay mapped to those physical keys even if you for some reason change your keyboard layout while the game is running. The disadvantage is, most of the non-US layouts didn’t end up with completely broken controls, so people kept playing with them and got used to them. So they’ll need to get used to the layout the game was originally designed with, or manually configure controls back to what they are used to.

But wait, there is a catch...

A few weeks ago we have announced new construction tools, which are by default bound to quite universal shortcuts (Ctrl+C for copy, Ctrl+X for cut and Ctrl+Z for undo). Bilka pointed out that the German keyboard has Z swapped with Y (as does the Czech one, but developers often don't use it) and undo incorrectly defaults to Ctrl+Y instead. To fix these kind of shortcuts we determine the appropriate default scan-codes at start-up, so that undo is always Ctrl+Z, regardless of your layout, but the action will stay bound to those keys if you change keyboard layout at runtime, which is hopefully a reasonable compromise.

We might do it for other controls too (it feels natural for M to always be the default key to open the map, and T to open the technology screen), but there is another catch. It is completely reasonable for player to walk north, and Ctrl+click some entities. Remember AZERTY keyboard? Player keeps Z pressed down to walk north and presses Ctrl to start control clicking. Well, I tested this and it doesn’t trigger undo, but still stops player from walking. So it is not completely destructive, rather annoying. I am not sure how or if we’ll solve this, perhaps people with these layouts that create these kind of collisions will need resolve them by changing controls options manually.

The new Map Editor is finished, with one of the last things completed being importing surfaces from other save files. Importing surfaces from one save into another turned out to be quite complex, but not for the reasons you might think. The copying of a surface to a new surface is relatively easy, the main problem came from how we store map data in a save file.

Factorio internally makes lists of every entity, item, decorative and so on at startup and generates a mapping of the text name to an integer ID. This is internally called an ID mapping, and provides several benefits both runtime and when it comes to saving/loading the game. The main benefit being: we (and modders) don't need to track and manually assign ids to everything added to the game - the engine does it all automatically. Anyone familiar with modded Minecraft before version 1.7 can attest to how much of a pain manually handling IDs can be and how easily it can break.

Because things can be added and removed due to our internal changes or by adding/removing/changing mods, the ID mapping often changes. Because it can change we have to include it in the save file to know what it was when the save was created. When Factorio saves the map one of the first things that's written to the save file is that ID mapping.

The way it worked before was: at load time - the game would attempt to restore whatever the ID mapping was for the save it's loading. Anything it can't find means that thing doesn't exist any more, and anything new is stuff added. This lead to saves having different ID mappings for the same set of mods depending on the steps taken to make that save file. This worked for the most part but had a few small problems:

• Any time something was removed it was signalled by setting the ID at that location in the mapping to 0.
• The removed IDs couldn't be re-used until the save was fully loaded, saved, and loaded again (leading to gaps in the IDs).
• Different save files could end up using different IDs even when using the same set of mods.

With the new Map Editor wanting to take any save file and import it into your running game the fact that the ID mapping could be different was a problem. To add to the complexity we also have a migration system in place that lets you tell the game "I want to change all entities/items/... named *A* into *B*". This migration system also works when you remove the source otherwise it wouldn't be that useful.

After several false starts (a common theme when it comes to these complex reworkings), I came up with the following simple requirements:

• IDs will get assigned once after loading mods - after that they are never allowed to change for the lifetime of the program
• When a map is loaded instead of restoring the ID mapping it was using it will create a migration from the old ID to the new ID (if it still exists)
• The tracking of what was removed is done through a different system instead of treating a '0' ID as a removed ID.

During this rework I discovered we were doing a bunch of extra work (some of it even causing bugs) just to restore the save file ID mapping, and I was able to simply delete all of it now that the IDs simply migrate to the correct values any time a save file is loaded. I even inadvertently fixed a bug someone reported related to crashing when loading a specific scenario-using save file due to this rework.

Overall the system is simpler now and doesn't have any of the 'quirks' it used to. The new Map Editor can import any save file the game is capable of loading and it all 'just works'.

As you might know already from an earlier post, we are working with Ernestas on the high-res version of the enemies. This is a great excuse for also refining the concept of them. This week we want to show the new look of the worms.



The intention is to keep what we had in the previous version, but to emphasize its personality and behaviour. This new version tries to be more aggressive, powerful and disgusting, while also acting more nervous and agitated. The "fingers" on its head are for digging tunnels, something that wasn't really explained before, and which will also help to express the character of the worm.

Now we are working on the animations, trying to emphasize these concepts, basically the worm is a creature of pure hate and killer instinct. Together with the sound effects, I hope it is going to be a pleasure killing them.

As always, let us know what you think on our forum.

One of the things in the large TODO list for 0.17 is giving a final polish to the map generator.

There are quite a few obvious problems now in 0.16, and some less obvious ones. Here are some of the fixes and improvements (some work in progress):

• All combinations of settings should no longer create strange maps such as circles of cliffs.
• Much more predictable starting area resources that don't overlap each-other and are not covered by water.
• The resource generation settings now have a much more dramatic effect (previously they had little to no effect).
• Increased the number of steps (small, medium, big, etc) for each setting from 5 to 9 for even more customization.
• The starting area will always contain water, most often a lake close to the spawn position.
• Since the algorithm for generating ores was pretty much completely rewritten, there are many small improvements.

Now for the less obvious problem: unpredictability. I saw quite a few people complain with vague comments like "the map generator sucks". So I often asked them what the problem is in detail. Some were complaining about the above problems, some did not understand what the settings do, and some had problems finding a "good map". I saw quite a few players click "regenerate" like crazy until they got a map with fat patches in the starting area, big oil patch and also uranium, complaining that it's too hard to find a "good map". Due to the randomness we seem to have set the expectation for "good map" a bit too high. Oil and uranium were never intended to be in the starting area, but due to the randomness of the generator they sometimes were there. Also sometimes maps were so wild that you would start off either swimming in resources or desperately looking for another iron patch.

It would be simple to just say "that's just RNG, deal with it", but blaming poor game experience on RNG is just bad design.
So what we did is:

• The starting area contains only iron, copper, coal and stone, in very predictable amounts. Uranium and oil are explicitly excluded from the starting area.
• Starting area resources are usually in one ore patch each (depending on settings).
• The starting area patches are usually close together.
• The starting area size setting no longer affects resource placement, it just has a fixed size.

Outside the starting area, the regular algorithm "kicks in" so you can still get quite wild results, but they are far enough that it averages out. I believe this is a good balance where you can still have different experiences depending on your luck, but your starting experience is much more predictable and does not leave you with the feeling that you got screwed over by the map generator. We definitely don't want the map generator to be extremely flat and predictable. Opinions on the subject are quite wild too, with people having different expectations of what a good map should look like, so we try to only make changes based on actual problems.

This might seem a bit controversial so we can add an option that disables this whole starting area logic, for purists.

We plan some small tweaks coming to biters also (a tiny bit more biters close to the starting area), small tweaks to terrain, cliffs, water generation and possibly some new features to make the generated trees and decoratives look better.

Most of these problems including the obvious and apparently simple ones were not that easy to solve. It's hard to make random generators do what you want, so TOGoS will explain what it took to actually get it done.

Good day procedural map generation enthusiasts!

The terrain generation in Factorio works by calculating probability and richness for every autoplaceable tile, entity, and decorative at every point on the map. To oversimplify slightly, the thing with the highest probability "wins" and then gets placed (if it's a tile) or has that probability of being placed (for entities and decoratives).

As some of you may recall, one of the features we added in 0.16 was a new terrain generation systemdriven by functional expressions built in Lua code. Mods define a function (not a Lua function, but a data structure representing a function in the mathematical sense)to be applied at every point on the map to calculate those values. This gave us a lot more control over elevation, temperature, humidity, and a few other variables across the map. However, the probability and richness functions for specific objects (notably resources) were controlled by a separate system.

I had wanted to unify these two systems since I started working on terrain generation last summer. Since releasing 0.16, our desire to improve resource placement, combined with my inability to come up with a good way to do it using the existing autoplace system, led me to finally bite the bullet and undertake 'the big autoplace refactoring'.

It was a lot of work.

The result is that existing AutoplaceSpecifications still work (because rewriting them all would have meant even more work), but under the hood they are compiled to expression trees, just like the ones for elevation, temperature, etc. As an alternative to the peak/dimension system, autoplace specifications can be defined in terms of a probability and richness expression directly, allowing a mod author to use the full potential of the programmable noise system.

An advantage of this approach is that we can now add new types of noise expressions without the need to reconcile them with all of the existing autoplace specifications or cram them into the ever-mode-complex monolithic AutoplaceSpecification object.

Specifically to make generation of resource patches more controllable, I added a new noise expression type called "spot noise". The way it works is that the map is divided into regions (large areas whose size is configurable per spot noise expression) and for each region:

• A list of random points is generated.
• Density, quantity, radius, and favorability are calculated for each point, based on noise expressions provided as parameters.
• The total desired quantity for the region is calculated by averaging the density from all points and multiplying by the region's area.
• Points are sorted according to their favorability, highest-to-lowest.
• Points are chosen from the front of that sorted list until the target quantity for the region is reached.

Having generated a list of spots with position, quantity, and radius, the output of the spot noise function is high near the spot centers and zero at a distance equal to their radius, such that the total value in the spot equals the spot's quantity. This value can then be used (for example) as the richness for a resource (such as iron-ore). By itself, this gives us 'conical' spots (if you think of resource patches as being mountains):



This result can have some noise added to it to make the resulting spots non-circular:



I had to be somewhat careful when applying this noise; since there is more area outside the spot where richness can be raised above zero than there is inside to be lowered, any randomization will add a positive bias to the overall quantity. I have been compensating for this by subtracting some constant fraction of the amplitude of the noise, though it's been on my mind that the problem could also be resolved by using differently shaped spots.

This system opens up a lot of possibilities:

• We can use the maximum of two different spot noise expressions to place starting area ores using completely different settings than we do for the rest of the map.
• We can vary quantity per spot and frequency of spots independently, which will allow the sliders on the new map screen to have more predictable effects.
• Spot quantity can depend on the suitability of the location. For example, we could set suitability to correspond to elevation so that spots are not placed underwater. The system would then continue through the list of candidate spots, placing more spots at locations above water to compensate. In the base game we're planning to do this for starting area resources, but not for resources outside of the starting area.
• In general, spot noise allows us to mess around with the placement of resources while keeping overall quantities constant.

Here are some map previews of the same seed, to illustrate spot-placed resource patches being moved to avoid water in the starting area as the water level is raised:





Putting everything together, here's what a typical starting area and surrounding region generated by the new system looks like.We may make a few more tweaks before 0.17 but this is probably pretty close:



All that said, I was perfectly happy when ore placement was unpredictable and sometimes there was no copper in the starting area and really long belts (and walls to defend them) were in order. So if I have my way there will be a "no special starting area resource placement" option.

As always, let us know what you think on our forum.

With most of the team away for Gamescom or vacation, I have the pleasure of writing a Friday Facts for you this week.

As you probably know, I've been working on the New Player Experience (NPE, as we call the new demo + tutorial) and the new Campaign. However we don't want to talk about it too in-depth or be too specific about either of these subjects, purely to not spoil it for the FFF readers, but there is some related news to share.

The NPE needs a lot of scripting and design work to make sure the new player does not get stuck anywhere because it is a tutorial, and to make sure that it gives a taste of what the game is about because it is also a demo. Since the NPE was the first thing we started, we already have a working version and we have been testing it with people at the office, and also people who have never played Factorio, to get the necessary feedback to polish it further. From what we gather, it seems that the NPE is in a decent state, the main design seems to work very well and we are just improving details and making the map look nice.

The main campaign design has been completely finished for a while now, we have made a 'geographic' map layout and now we are 'just' implementing it in cooperation with Rseding, who is improving/adding tools for the map editor which makes our work a lot easier and in many cases just makes it possible (love letters, non-genital-shaped candy and other gifts to his address please), just like for any future map creators. It's still going to take quite some time to finish all of this, but the progress is steady and we can't wait to see people play it in 0.17.0.

One of the things we could really use in the campaign was a terrain that the player can't build on, but can walk on, so we are adding shallow water as a new terrain type. So far it's a cheap solution from our side - it's just a new tileset that is combined with decorative entities to make it look like walkable. Currently we don't plan to use this in freeplay, but if we can find its gameplay purpose and make the terrain generator use it properly, it's not impossible, but for now not a priority.



As I already wrote in an earlier FFF, that when working on the design of the NPE and the campaign, it's not rare that we come across something in the game which could work much better just by changing some recipes or technologies.

As I also wrote in that FFF, all science packs will have their own unlocking technology in 0.17 - but I did not show or tell how does this apply to Space science pack since you never unlock it as a recipe - you only need satellite and rocket silo for it.



At the same time when designing the campaign, we feel that the player should get more an idea of what they're progressing to reasonably early in the game and build towards it. On top of that I really don't like the dynamic in freeplay that you need as soon as you research rocket silo, you immediately have to build all the production lines for the rocket parts at the same time.
This results in:

• You no longer win the game by launching a satellite, there is a new "Rocket escape pod" that you launch in a rocket to finish the game. Currently the escape pod has no other use.
• Satellite is unlocked by its own "Space science pack" technology that has rocket silo as a prerequisite.
• Rocket silo technology only unlocks the Rocket silo and the Rocket part item the silo crafts internally.
• Rocket fuel is unlocked by researching its own technology which has Rocketry and Engine as a prerequisite. This means you can get rocket fuel for your vehicles way before the rocket silo.
• Nuclear power is split to Uranium processing and Nuclear power, where Kovarex enrichment process only requires Uranium processing and Rocket fuel so you can get Nuclear fuel for your vehicles without the rocket silo.
• Rocket control unit is unlocked by its own technology which has Production and High tech science pack technologies as a prerequisite. Additionally, Rocket control units are in the recipe for Atomic bomb instead of Processing units.
• Low density structure is unlocked by researching its own technology which has Science pack 3 and Advanced material processing as prerequisites. The Low density structure is also used in multiple advanced personal equipment recipes (mk2 things, Portable fusion reactor, Personal laser defense) instead of steel.

We hope that these tweaks and changes will help to smooth out the experience for new players and veterans alike. If you have any suggestions, feedback or comment, we are always happy to hear it over on our forum.