We are aware that we don't add so much content to Factorio these days. The reason we always state is that we are focusing on polishing and finishing the game. There are a lot of little things and details (some call it quality of life improvements), that don't look that awesome when reviewed one by one, but once they accumulate together they should make a big difference in the game experience. I'm going to talk about some of them now, but don't be afraid that we won't add anything new in the 0.16, as the work on the artillery train has already started and it is probably going to be epic :).
Blueprint connection
One of the problems with blueprints and ghosts was when you create blueprint of this:
Entities don't connect in the blueprint itself, so it looks as ugly as this:
Once you build the blueprint, it looks like this until it is actually constructed:
This bugged me, but the solution wasn't so straightforward. The logic of connecting entities is based on the them being actually placed somewhere on the map, but in the blueprint, the entities kind of exist "in the air". So I had to make special piece code for connecting entities in blueprints. I only cared about the visuals of belts, walls and pipes, so the code wasn't that complex after all. The looks like this in 0.16:
Next step was to make special code to connect ghosts entities together (and also normal versus ghost). This was the most tricky part, as we needed to make sure to not connect the internal transport lines of belts when doing so. Otherwise, the ghost belts would just work as normal belt :). We also needed to avoid ghosts of connected gates to not interfere with opening/closing logic of existing gates etc, but it works now and this is the result of the built blueprint now:
You can see, pipes are still not connecting. They are more delicate, as it would be harder to make the connections without affecting the performance of the pipe transition logic. We might either just not do it for pipes, use some special trick later, or wait until the fluid flow gets optimized. The optimizations would probably mean that the fluid control logic would be handled externally like belts, which might simplify things regarding the fake connections.
Blueprint and mods
We described the problem in fff-201. Basically, blueprints with modded content would lose all their modded content whenever you load a game with the related mod deactivated. There is no going back, once you activate the mod again, the items are still lost.
The solution of this was one of the first task to Tom, one of our new programmers. Now when the blueprint is being loaded and it encounters entities that are no longer available, the original binary source of the blueprint and some additional meta data is saved along the blueprint, so once the mod is activated again, it can be fully revived.
Blueprint with modded content.
How it looks in the blueprint library when the mod is removed.
Preview of the blueprint that indicates where are entities that are not available now.
As always, let us know your thoughts, ideas and feedback on our forum.
We are aware that we don't add so much content to Factorio these days. The reason we always state is that we are focusing on polishing and finishing the game. There are a lot of little things and details (some call it quality of life improvements), that don't look that awesome when reviewed one by one, but once they accumulate together they should make a big difference in the game experience. I'm going to talk about some of them now, but don't be afraid that we won't add anything new in the 0.16, as the work on the artillery train has already started and it is probably going to be epic :).
Blueprint connection
One of the problems with blueprints and ghosts was when you create blueprint of this:
Entities don't connect in the blueprint itself, so it looks as ugly as this:
Once you build the blueprint, it looks like this until it is actually constructed:
This bugged me, but the solution wasn't so straightforward. The logic of connecting entities is based on the them being actually placed somewhere on the map, but in the blueprint, the entities kind of exist "in the air". So I had to make special piece code for connecting entities in blueprints. I only cared about the visuals of belts, walls and pipes, so the code wasn't that complex after all. The looks like this in 0.16:
Next step was to make special code to connect ghosts entities together (and also normal versus ghost). This was the most tricky part, as we needed to make sure to not connect the internal transport lines of belts when doing so. Otherwise, the ghost belts would just work as normal belt :). We also needed to avoid ghosts of connected gates to not interfere with opening/closing logic of existing gates etc, but it works now and this is the result of the built blueprint now:
You can see, pipes are still not connecting. They are more delicate, as it would be harder to make the connections without affecting the performance of the pipe transition logic. We might either just not do it for pipes, use some special trick later, or wait until the fluid flow gets optimized. The optimizations would probably mean that the fluid control logic would be handled externally like belts, which might simplify things regarding the fake connections.
Blueprint and mods
We described the problem in fff-201. Basically, blueprints with modded content would lose all their modded content whenever you load a game with the related mod deactivated. There is no going back, once you activate the mod again, the items are still lost.
The solution of this was one of the first task to Tom, one of our new programmers. Now when the blueprint is being loaded and it encounters entities that are no longer available, the original binary source of the blueprint and some additional meta data is saved along the blueprint, so once the mod is activated again, it can be fully revived.
Blueprint with modded content.
How it looks in the blueprint library when the mod is removed.
Preview of the blueprint that indicates where are entities that are not available now.
As always, let us know your thoughts, ideas and feedback on our forum.
It’s been several weeks since we showed you the graphics for new high resolution circuit connector modules (FFF-202). However now is finally the time when we have them in the game. In this article I will briefly show you what was done both in the graphics and code, and what new benefits are there for you as players and modders.
I find the 0.15 version of the circuit connector module has following “problems”:
The wire connectors are different from the combinators.
Wires sometimes completely overlap, making only one of them properly visible.
Modularity - you can somewhat tell what is happening based on the LED states, but it could be much nicer.
Connecting a belt always looks weird, while the yellow structure which holds the connector box could be made more specific.
Some of the rotations are utterly useless.
The Lua definitions are spread over every single entity, so revisiting them all is a big pain.
1. The wire connectors are different from the combinators
We have been experimenting with the design of how the little pieces which connect to wires should look like. Most of them don’t really make sense in terms of physics, but visually the combinator ones seem to work pretty nicely in my opinion. That, and the circuit connector should be somewhat consistent with the combinators in this regard, so we just used the combinator design from when Albert made the combinators back in 2015 (FFF-88).
2. Wires sometimes completely overlap
As we already tried to hint in FFF-202 (but we didn’t have a proper picture for it yet because the hr circuit connector wasn’t in the game yet), when you build entities vertically, they would very often only show one colour of wire.
To prevent this from happening, I took a pixel grid in Blender and I tried to always have the connectors far away from each other to prevent this from happening. The same issue happens with combinators, but let’s see if we ever have time to change that.
3. Modularity
The system of drawing has slightly changed - now when you connect an entity, you can clearly recognize what it is doing and what is it's state. The rule is: Red/Green LED means write mode - generally stopping/starting the entity. Blue LED means reading mode - generally reading chest contents, reading signal colours, and so on.
If you only connect to the logistic network, even the wire connection points are going to disappear as you can see on the pump above.
4. Transport belt connectors
Previously we were using the universal connector on belts, and to support it we made a special layer with a frame to hold it. This just uses another layer and generally looks rather weird.
We found it better to just make specific sprites for belts, which should integrate nicer and use less layers.
5. Useless rotations
If you ever looked at how the circuit connector module spritesheet actually looks, you would have noticed that there are 32 rotations of it. For the first row, which is just flat on the ground, all of the 8 rotations are useful, but when the box starts tilting, the last three are looking away from the camera and are basically impossible to use.
At the same time, on some entities and in some cases it would be really cool to be able to just put the wire connection points at the opposite side of the box.
I put the two things together and made the impossible-to-use rotations just be a flipped version of the useful ones. The picture below is a mockup, the actual spritesheets have separate layers to allow the modularity mentioned above.
The game does not actually utilize all of the 32 rotations, but it’s easier to have them all for future new entities - importantly also new entities added by mods.
6. Lua definitions made at each entity
When the graphics were finished, I was looking at how could I put the connector in the game. We had good experience with generating the Lua code in combinators, and I wanted to make use of some similar system again, because there is just a stupid amount of shiftings and definitions to be made.
To get all the shifting values, we are using a very similar system to what was described in FFF-202, special pictures from After Effects processed by python scripts. The only real difference is that now After Effects also has to import spritesheets of our existing entities and align them with the shifting values. Luckily, After Effects supports javascript based expressions which makes this work simpler.
var shiftingX = 16;
var shiftingY = 8;
var finalX = transform.position[0]+ (shiftingX * 2);
var finalY = transform.position[1]+ (shiftingY * 2);
[finalX, finalY]
This simple expression just lets me center the sprite and copy the X and Y shifting values from Lua, instead of having to worry about making calculations all the time, and it’s more readable when re-visiting and checking.
With the shifting values ready, the next step is to tell the game to accept them somehow. I wouldn’t dare to try overwriting all of the entries in entities.lua and similar files, simply because it would probably mean massive amount of errors, both by hand or somehow automatically.
So instead Michal (Posila) wrote a python script which generates a new circuit connector file where all of the specific definitions are kept, and refactored the entities so that each entity connectible to the circuit network just grabs the values from the master circuit connector file. Hopefully this makes it more maintainable for the future...
High resolution lamp
As one of the often used entities in the circuit network, the lamp is getting a high resolution version. The graphics are a work in progress so there will likely be some changes.
As always let us know what you think, just like you have been in the last 4 years (FFF-1) on our forum. We would like to thank you for all the attention and feedback through all this time, it’s really nice to be able to talk about our work, and sometimes even add something we forgot and you mentioned.
It’s been several weeks since we showed you the graphics for new high resolution circuit connector modules (FFF-202). However now is finally the time when we have them in the game. In this article I will briefly show you what was done both in the graphics and code, and what new benefits are there for you as players and modders.
I find the 0.15 version of the circuit connector module has following “problems”:
The wire connectors are different from the combinators.
Wires sometimes completely overlap, making only one of them properly visible.
Modularity - you can somewhat tell what is happening based on the LED states, but it could be much nicer.
Connecting a belt always looks weird, while the yellow structure which holds the connector box could be made more specific.
Some of the rotations are utterly useless.
The Lua definitions are spread over every single entity, so revisiting them all is a big pain.
1. The wire connectors are different from the combinators
We have been experimenting with the design of how the little pieces which connect to wires should look like. Most of them don’t really make sense in terms of physics, but visually the combinator ones seem to work pretty nicely in my opinion. That, and the circuit connector should be somewhat consistent with the combinators in this regard, so we just used the combinator design from when Albert made the combinators back in 2015 (FFF-88).
2. Wires sometimes completely overlap
As we already tried to hint in FFF-202 (but we didn’t have a proper picture for it yet because the hr circuit connector wasn’t in the game yet), when you build entities vertically, they would very often only show one colour of wire.
To prevent this from happening, I took a pixel grid in Blender and I tried to always have the connectors far away from each other to prevent this from happening. The same issue happens with combinators, but let’s see if we ever have time to change that.
3. Modularity
The system of drawing has slightly changed - now when you connect an entity, you can clearly recognize what it is doing and what is it's state. The rule is: Red/Green LED means write mode - generally stopping/starting the entity. Blue LED means reading mode - generally reading chest contents, reading signal colours, and so on.
If you only connect to the logistic network, even the wire connection points are going to disappear as you can see on the pump above.
4. Transport belt connectors
Previously we were using the universal connector on belts, and to support it we made a special layer with a frame to hold it. This just uses another layer and generally looks rather weird.
We found it better to just make specific sprites for belts, which should integrate nicer and use less layers.
5. Useless rotations
If you ever looked at how the circuit connector module spritesheet actually looks, you would have noticed that there are 32 rotations of it. For the first row, which is just flat on the ground, all of the 8 rotations are useful, but when the box starts tilting, the last three are looking away from the camera and are basically impossible to use.
At the same time, on some entities and in some cases it would be really cool to be able to just put the wire connection points at the opposite side of the box.
I put the two things together and made the impossible-to-use rotations just be a flipped version of the useful ones. The picture below is a mockup, the actual spritesheets have separate layers to allow the modularity mentioned above.
The game does not actually utilize all of the 32 rotations, but it’s easier to have them all for future new entities - importantly also new entities added by mods.
6. Lua definitions made at each entity
When the graphics were finished, I was looking at how could I put the connector in the game. We had good experience with generating the Lua code in combinators, and I wanted to make use of some similar system again, because there is just a stupid amount of shiftings and definitions to be made.
To get all the shifting values, we are using a very similar system to what was described in FFF-202, special pictures from After Effects processed by python scripts. The only real difference is that now After Effects also has to import spritesheets of our existing entities and align them with the shifting values. Luckily, After Effects supports javascript based expressions which makes this work simpler.
var shiftingX = 16;
var shiftingY = 8;
var finalX = transform.position[0]+ (shiftingX * 2);
var finalY = transform.position[1]+ (shiftingY * 2);
[finalX, finalY]
This simple expression just lets me center the sprite and copy the X and Y shifting values from Lua, instead of having to worry about making calculations all the time, and it’s more readable when re-visiting and checking.
With the shifting values ready, the next step is to tell the game to accept them somehow. I wouldn’t dare to try overwriting all of the entries in entities.lua and similar files, simply because it would probably mean massive amount of errors, both by hand or somehow automatically.
So instead Michal (Posila) wrote a python script which generates a new circuit connector file where all of the specific definitions are kept, and refactored the entities so that each entity connectible to the circuit network just grabs the values from the master circuit connector file. Hopefully this makes it more maintainable for the future...
High resolution lamp
As one of the often used entities in the circuit network, the lamp is getting a high resolution version. The graphics are a work in progress so there will likely be some changes.
As always let us know what you think, just like you have been in the last 4 years (FFF-1) on our forum. We would like to thank you for all the attention and feedback through all this time, it’s really nice to be able to talk about our work, and sometimes even add something we forgot and you mentioned.
I'm addicted. Addicted to optimisation. I earned some money by the game ... you know .. I could just .. I don't know .. play games and ... have some leisure time, but do you know what would be the problem? I would have to think about some Factorio optimisations I wanted to do anyway.
When I go to bed in the evening, I think about cache locality and data structures. I dream about ways to reorganize structures to minimize the amount of data needed to be read to perform Factorio logic. When I walk in the park, i visualize the tricks that could be done to decrease data and logic dependencies. This usually only stops when I see how many bugs need to be fixed after release when lot of changes are done.
Electric network optimisation
The electric network optimisation for 0.16 was simple in principle. We currently have one continuous buffer of electric connector data stored in the electric network. It needs to be iterated twice (first for calculation, second for power distribution).
The change was to categorize energy connector data by the prototype:
It adds another layer of indirection when something needs to be added/removed from the network, but that happens quite rarely. The advantage is, that all the prototype specific values, like inputFlowLimit, outputFlowLimit etc. can be accessed once at the beginning of the processing of one category, which (together with other size squishing), results in a much smaller memory layout, which results in faster iteration of the algorithm.
The conclusion is that electric network transfers are more than twice as fast compared to 0.15.
Smoke optimisation
Smoke already had the first iteration of optimisation 125 weeks ago. But Factorio is many times faster since than, so smoke started to appear in our profiling again.
The problem was that even creating the smoke object and registering it on the map was slowing the game down. The plan was to make something we internally call TrivialSmoke. The Trivial smoke data size is 32 bytes which is quite small compared to the previous "smoke as entity" size of 256 bytes. It contains only the most basic data squashed as much as possible, and it is stored in a continuous piece of memory on the chunk it was created.
As the updates happen in regular intervals (120 ticks), it is always guaranteed, that the smokes to be updated this tick are at the beginning of the buffer, I don't have to check the rest. The same buffer is used for update and for render, so these objects don't need any other references to it.
It is similar to the Chunk update planner mentioned in fff-161, just flattened. The result is, as expected, that the smoke creation/update/removal times were significantly reduced.
The logistic bots dilemma
The dilemma is, whether to use similar approach to the logistic bots as for the smoke. This would mean that logistic bots would be created/updated/removed very fast and their memory footprint would decrease a lot, but they wouldn't be regular entities any more. They couldn't be mined (which could be for the better actually), couldn't be attacked or damaged by anything (which again, would be welcomed by some players). I'm quite sure that it shouldn't apply for the construction robots, as they are more related to the fighting part of the game, but logistic robots aren't.
The dilemma is, whether changing the game rules like this just for optimisation isn't going too far.
Running total
I benchmarked a huge save of Vaclav (https://imgur.com/a/qDDs1), and it can already run 2.4 times faster in 0.16 compared to 0.15 so we met our "2 times faster release" quota, but I'm afraid that this won't stop the addiction.
Stone path update
Almost all of the terrain has been updated for 0.16, which includes the high-res update of it. We will certainly cover the terrain/trees/decals improvements in future FFF. Here I would like to present the new version of the stone path terrain made by Ernestas. It is inspired (as many things in Factorio) by the eastern style of pavements, and I'm starting to be a fan of his work.
As always, let us know your thoughts, ideas and feedback on our forum.
I'm addicted. Addicted to optimisation. I earned some money by the game ... you know .. I could just .. I don't know .. play games and ... have some leisure time, but do you know what would be the problem? I would have to think about some Factorio optimisations I wanted to do anyway.
When I go to bed in the evening, I think about cache locality and data structures. I dream about ways to reorganize structures to minimize the amount of data needed to be read to perform Factorio logic. When I walk in the park, i visualize the tricks that could be done to decrease data and logic dependencies. This usually only stops when I see how many bugs need to be fixed after release when lot of changes are done.
Electric network optimisation
The electric network optimisation for 0.16 was simple in principle. We currently have one continuous buffer of electric connector data stored in the electric network. It needs to be iterated twice (first for calculation, second for power distribution).
The change was to categorize energy connector data by the prototype:
It adds another layer of indirection when something needs to be added/removed from the network, but that happens quite rarely. The advantage is, that all the prototype specific values, like inputFlowLimit, outputFlowLimit etc. can be accessed once at the beginning of the processing of one category, which (together with other size squishing), results in a much smaller memory layout, which results in faster iteration of the algorithm.
The conclusion is that electric network transfers are more than twice as fast compared to 0.15.
Smoke optimisation
Smoke already had the first iteration of optimisation 125 weeks ago. But Factorio is many times faster since than, so smoke started to appear in our profiling again.
The problem was that even creating the smoke object and registering it on the map was slowing the game down. The plan was to make something we internally call TrivialSmoke. The Trivial smoke data size is 32 bytes which is quite small compared to the previous "smoke as entity" size of 256 bytes. It contains only the most basic data squashed as much as possible, and it is stored in a continuous piece of memory on the chunk it was created.
As the updates happen in regular intervals (120 ticks), it is always guaranteed, that the smokes to be updated this tick are at the beginning of the buffer, I don't have to check the rest. The same buffer is used for update and for render, so these objects don't need any other references to it.
It is similar to the Chunk update planner mentioned in fff-161, just flattened. The result is, as expected, that the smoke creation/update/removal times were significantly reduced.
The logistic bots dilemma
The dilemma is, whether to use similar approach to the logistic bots as for the smoke. This would mean that logistic bots would be created/updated/removed very fast and their memory footprint would decrease a lot, but they wouldn't be regular entities any more. They couldn't be mined (which could be for the better actually), couldn't be attacked or damaged by anything (which again, would be welcomed by some players). I'm quite sure that it shouldn't apply for the construction robots, as they are more related to the fighting part of the game, but logistic robots aren't.
The dilemma is, whether changing the game rules like this just for optimisation isn't going too far.
Running total
I benchmarked a huge save of Vaclav (https://imgur.com/a/qDDs1), and it can already run 2.4 times faster in 0.16 compared to 0.15 so we met our "2 times faster release" quota, but I'm afraid that this won't stop the addiction.
Stone path update
Almost all of the terrain has been updated for 0.16, which includes the high-res update of it. We will certainly cover the terrain/trees/decals improvements in future FFF. Here I would like to present the new version of the stone path terrain made by Ernestas. It is inspired (as many things in Factorio) by the eastern style of pavements, and I'm starting to be a fan of his work.
As always, let us know your thoughts, ideas and feedback on our forum.