Hello!

Once we started the matching server, we finally had to face the reality of the multiplayer over the internet with people around the world. We realised there are A LOT of problems rising to the surface, and that it needs to be worked on.
I left all the multiplayer logic to be done by cube and tomas until now, and I had only a very simplistic idea how it works. With tomas on holiday and cube busy with other tasks, I realized how big of a problem it is that no one else has a clue how it works under the hood, so I took this opportunity to dive into it personally. After a week of reading, discussions with cube and partial rewrites, I can present you with my findings and a roadmap of ongoing changes.

As some of you might know, the Factorio multiplayer was originally written to be always peer-to-peer. The motivation was to minimize the latency, as in the theoretical case of everyone having the same connection with everyone else in the game, the latency would actually be smaller compared to the server-client model.
The problem is, that there are many things that had to be paid as a price.

• Everyone needs to be actually sending packets to everyone else, which isn't that easy in the current world, where IPv6 isn't everywhere, and public IPv4 address is becoming quite a luxury. This can be solved by nat punching, but it also isn't 100% reliable.
• The logic of events, like joining, quitting, disconnecting, is very complex, as it always has to be discussed by the peers before anything can be done. And as we have the lock-step simulation, it always has to be ensured, that these actions are performed in perfect synchrony. Complexity means bugs, and in this case, some of them are hard to fix. On top of that, even if it was written perfectly, it wouldn't feel perfect.
• Everyone needs to have the same latency.
• No defence from lag spikes of individual players.
• One packet per player per tick sent and received by everyone, so the amount of packets sent is O(n^2)

So once we encountered the "real internet" network communication, these problems shown to be too serious. We could have anticipated this if we had only listened to the people warning us that peer to peer will lead to trouble when we were first writing about the implementation more than a year ago. But sometimes, you just have to learn from your own mistakes.

So we added an additional option to run in the Server mode, which became the only option later on.

But our server mode only solved the first problem, as it was just a patch that re-routed all the communication between peers to go through the server, but all of the peer discussion related complexities stayed.

The original peer to peer model:



The new server model:


In other words, we took the worst from both of the models and combined it.

The current state can't be solved by just small fixes and tweaks, fundamental changes in the internals of the multiplayer logic on almost all of the layers has to be done to take advantage of the possible simplifications implied from the fact that peer to peer isn't supported anymore. Let me present the most important changes that I'm working on:

One of the most obvious changes is, that instead of re-sending all the packets, the server is unpacking these and merging them. He first waits to get the actions of all players in a certain tick, and then sends it to all the clients as a single message. This not only reduces the number of packets sent (from O(n^2) to O(n)), but it also keeps the clients from having to deal with the synchronisation and shit. They just accept the package as it is and apply it. If they miss something due to the packet loss, the client just asks the server for the whole package to be resent, in other words, the clients don't communicate with each other at all.



As clients don't need to communicate, they don't even need to know about their existence in the game. This doesn't mean that you wouldn't know about other players in the game. When a player joins, clients receive an input action Player joined as part of the merged package, so the player is created on the map and in the player list, but this is not related to network logic, and it is a different layer that works like this already. The difference is, that the clients don't need to know what network entity is related to what player, they don't care. Ignorance is bliss!

The clients are also sending input actions, but only to the server, and it is up to the server to decide whether it should be included in the merged package or not. As the merged package is the only source of the actions to be applied, the server can safely omit a player from the package if he has a lag spike, so the lag spike is isolated from the rest of the game. This is not possible in the peer to peer model.

Currently, when player wants to join a game, first it had to be discussed to stop the game at a certain tick. This tick had to be at least one latency step in the future, as other players could already be ahead of us, and you can't go backwards in Factorio state (Yes entropy works the same way in Factorio world). This is the strange freeze that happens when someone is connecting, disconnecting etc. During this time, the new client is introduced to others so they know they have to count with him.

But as we decided that clients know nothing about other clients, this can be removed completely. Once the server agrees on the new player to join the game, it can just start sending his actions as part of the merged package without any interruption. The save-game still needs to be uploaded by the server, so there will still be waiting, but there shouldn't be any strange freezes inbetween the download progress bar and normal game anymore.

As all the logic is straightened, the internal code will get actually much more simple as well. Simplier code means less bugs. Also this should mean, that if we want someone else to tweak the internals of the multiplayer, it shouldn't take him 3 weeks of study to understand what is going on.

Once this is all implemented and working, which will take some weeks, we could use this architecture as a reliable base to make additional improvements. These are ideas that shouldn't be that hard to do, but can't be promised.

The latency is now a global parameter of the multiplayer game, and it is the delay between creation of the user action (Input action), and it's execution, the bigger the latency is, the more time to deliver the actions between players, so the game might be less laggy. Big latency is bad for gameplay, small latency is bad for distant players. But with the proper server-client architecture and the server being the only input action authority, everyone can have different latency. The guy in the same street as the server only needs 30ms to send the package to the server to be included in the next merged package and next 30ms to get his action back to see it on the screen. But someone from the other part of the world in the same game might need 500ms to send the message, so his actions will just be packed into the merged package with bigger delay.

The latency of individual players should be tweaked automatically during the game by the server, so it could make sure that it is as small as possible for a flawless game.

An implication of this would be, that the server would have 0 latency, this would be unfair in a competitive game, but in Factorio, there is no reason to drag everyone down just to make it fair.
This feature could also be added. When someone joins the game, the server needs to save it, and others have to wait for it, this can't be removed, but once it starts uploading the game, other players (including the server) could just continue playing, while the server is providing the map. On top of that, the server would save all the actions made by the players in the meantime. Once the map is uploaded, the server would send these actions and the client would fast-forward to catch up. The only limitation is, that the client has to be able to run the map in faster speed, so he can catch up.
Apart from the latency tweaking based on the network roundrip time of individual peers, the server could also measure slowdown related to CPU lag. CPU lag means, that some of the players computers are not able to simulate the Factorio map fast enough, so others have to slow down their simulation and wait for him. It should be possible to set an option to auto-kick players who drag down the game too much. A similar limitation could be applied to upload speed.

Scott and Mishka have been spending this week working through the crowdin, this is the website we use to allow the community to help us translate the game. To this date the community on crowdin has helped us translate the game into dozens of languages, along with the subtitles for the trailer and other related media.

We'd like to take an opportunity to thank all the translators, as their contribution really has a great impact on the game, and their perspective often helps us in understanding how terms and descriptions we write in the game are interpreted by the players.

We still have many untranslated languages on crowdin, so if you think you might be able to help out with the translation for your language, please checkout the factorio project.

As always, let us know what you think on our forums

Hello!

Once we started the matching server, we finally had to face the reality of the multiplayer over the internet with people around the world. We realised there are A LOT of problems rising to the surface, and that it needs to be worked on.
I left all the multiplayer logic to be done by cube and tomas until now, and I had only a very simplistic idea how it works. With tomas on holiday and cube busy with other tasks, I realized how big of a problem it is that no one else has a clue how it works under the hood, so I took this opportunity to dive into it personally. After a week of reading, discussions with cube and partial rewrites, I can present you with my findings and a roadmap of ongoing changes.

As some of you might know, the Factorio multiplayer was originally written to be always peer-to-peer. The motivation was to minimize the latency, as in the theoretical case of everyone having the same connection with everyone else in the game, the latency would actually be smaller compared to the server-client model.
The problem is, that there are many things that had to be paid as a price.

• Everyone needs to be actually sending packets to everyone else, which isn't that easy in the current world, where IPv6 isn't everywhere, and public IPv4 address is becoming quite a luxury. This can be solved by nat punching, but it also isn't 100% reliable.
• The logic of events, like joining, quitting, disconnecting, is very complex, as it always has to be discussed by the peers before anything can be done. And as we have the lock-step simulation, it always has to be ensured, that these actions are performed in perfect synchrony. Complexity means bugs, and in this case, some of them are hard to fix. On top of that, even if it was written perfectly, it wouldn't feel perfect.
• Everyone needs to have the same latency.
• No defence from lag spikes of individual players.
• One packet per player per tick sent and received by everyone, so the amount of packets sent is O(n^2)

So once we encountered the "real internet" network communication, these problems shown to be too serious. We could have anticipated this if we had only listened to the people warning us that peer to peer will lead to trouble when we were first writing about the implementation more than a year ago. But sometimes, you just have to learn from your own mistakes.

So we added an additional option to run in the Server mode, which became the only option later on.

But our server mode only solved the first problem, as it was just a patch that re-routed all the communication between peers to go through the server, but all of the peer discussion related complexities stayed.

The original peer to peer model:



The new server model:


In other words, we took the worst from both of the models and combined it.

The current state can't be solved by just small fixes and tweaks, fundamental changes in the internals of the multiplayer logic on almost all of the layers has to be done to take advantage of the possible simplifications implied from the fact that peer to peer isn't supported anymore. Let me present the most important changes that I'm working on:

One of the most obvious changes is, that instead of re-sending all the packets, the server is unpacking these and merging them. He first waits to get the actions of all players in a certain tick, and then sends it to all the clients as a single message. This not only reduces the number of packets sent (from O(n^2) to O(n)), but it also keeps the clients from having to deal with the synchronisation and shit. They just accept the package as it is and apply it. If they miss something due to the packet loss, the client just asks the server for the whole package to be resent, in other words, the clients don't communicate with each other at all.



As clients don't need to communicate, they don't even need to know about their existence in the game. This doesn't mean that you wouldn't know about other players in the game. When a player joins, clients receive an input action Player joined as part of the merged package, so the player is created on the map and in the player list, but this is not related to network logic, and it is a different layer that works like this already. The difference is, that the clients don't need to know what network entity is related to what player, they don't care. Ignorance is bliss!

The clients are also sending input actions, but only to the server, and it is up to the server to decide whether it should be included in the merged package or not. As the merged package is the only source of the actions to be applied, the server can safely omit a player from the package if he has a lag spike, so the lag spike is isolated from the rest of the game. This is not possible in the peer to peer model.

Currently, when player wants to join a game, first it had to be discussed to stop the game at a certain tick. This tick had to be at least one latency step in the future, as other players could already be ahead of us, and you can't go backwards in Factorio state (Yes entropy works the same way in Factorio world). This is the strange freeze that happens when someone is connecting, disconnecting etc. During this time, the new client is introduced to others so they know they have to count with him.

But as we decided that clients know nothing about other clients, this can be removed completely. Once the server agrees on the new player to join the game, it can just start sending his actions as part of the merged package without any interruption. The save-game still needs to be uploaded by the server, so there will still be waiting, but there shouldn't be any strange freezes inbetween the download progress bar and normal game anymore.

As all the logic is straightened, the internal code will get actually much more simple as well. Simplier code means less bugs. Also this should mean, that if we want someone else to tweak the internals of the multiplayer, it shouldn't take him 3 weeks of study to understand what is going on.

Once this is all implemented and working, which will take some weeks, we could use this architecture as a reliable base to make additional improvements. These are ideas that shouldn't be that hard to do, but can't be promised.

The latency is now a global parameter of the multiplayer game, and it is the delay between creation of the user action (Input action), and it's execution, the bigger the latency is, the more time to deliver the actions between players, so the game might be less laggy. Big latency is bad for gameplay, small latency is bad for distant players. But with the proper server-client architecture and the server being the only input action authority, everyone can have different latency. The guy in the same street as the server only needs 30ms to send the package to the server to be included in the next merged package and next 30ms to get his action back to see it on the screen. But someone from the other part of the world in the same game might need 500ms to send the message, so his actions will just be packed into the merged package with bigger delay.

The latency of individual players should be tweaked automatically during the game by the server, so it could make sure that it is as small as possible for a flawless game.

An implication of this would be, that the server would have 0 latency, this would be unfair in a competitive game, but in Factorio, there is no reason to drag everyone down just to make it fair.
This feature could also be added. When someone joins the game, the server needs to save it, and others have to wait for it, this can't be removed, but once it starts uploading the game, other players (including the server) could just continue playing, while the server is providing the map. On top of that, the server would save all the actions made by the players in the meantime. Once the map is uploaded, the server would send these actions and the client would fast-forward to catch up. The only limitation is, that the client has to be able to run the map in faster speed, so he can catch up.
Apart from the latency tweaking based on the network roundrip time of individual peers, the server could also measure slowdown related to CPU lag. CPU lag means, that some of the players computers are not able to simulate the Factorio map fast enough, so others have to slow down their simulation and wait for him. It should be possible to set an option to auto-kick players who drag down the game too much. A similar limitation could be applied to upload speed.

Scott and Mishka have been spending this week working through the crowdin, this is the website we use to allow the community to help us translate the game. To this date the community on crowdin has helped us translate the game into dozens of languages, along with the subtitles for the trailer and other related media.

We'd like to take an opportunity to thank all the translators, as their contribution really has a great impact on the game, and their perspective often helps us in understanding how terms and descriptions we write in the game are interpreted by the players.

We still have many untranslated languages on crowdin, so if you think you might be able to help out with the translation for your language, please checkout the factorio project.

As always, let us know what you think on our forums

• Bugfixes
  
  • Fixed craftable unit entities desyncing when held over belts. (https://forums.factorio.com/28730)
    
  • Fixed character inventory size was limited at 255 instead of the correct 65536. (https://forums.factorio.com/28717)
    
  • Fixed mouse wheel left/right bindings not working. Changed default blueprint book switching shortcut on OS X to Command + Mouse Wheel up/down
    
  • Fixed aliens getting stuck sometimes. (https://forums.factorio.com/28653)
    
  • Fixed inserters not putting fuel in modded burner assemblers in some situations (https://forums.factorio.com/28196)
    
  • Fixed headless server would crash when loading save containing sprite-button. (https://forums.factorio.com/28736)
    
  • Fixed maps loaded from versions < 0.13.7 not generating any more chunks (https://forums.factorio.com/28760)
    
  • Fixed First Steps campaign character death error. (https://forums.factorio.com/27926)
    
  
  
• Scripting
  
  • Reverted changes related to 0.13.7 fix for (https://forums.factorio.com/28564). If entity specified in sprite path doesn't have an icon, button will be created anyway, but no sprite will be drawn inside of it.
    
  • Added LuaGui::is_valid_sprite_path() function
    
  • Fixed that if a land mine got blown up and replaced by a robot, it wouldn't arm itself. (https://forums.factorio.com/28761)
    
  • Added LuaPlayer::admin read - if the player is an admin.
    

Use the automatic updater if you can (check experimental updates in other settings) or download full installation at http://www.factorio.com/download/experimental.

• Bugfixes
  
  • Fixed craftable unit entities desyncing when held over belts. (https://forums.factorio.com/28730)
    
  • Fixed character inventory size was limited at 255 instead of the correct 65536. (https://forums.factorio.com/28717)
    
  • Fixed mouse wheel left/right bindings not working. Changed default blueprint book switching shortcut on OS X to Command + Mouse Wheel up/down
    
  • Fixed aliens getting stuck sometimes. (https://forums.factorio.com/28653)
    
  • Fixed inserters not putting fuel in modded burner assemblers in some situations (https://forums.factorio.com/28196)
    
  • Fixed headless server would crash when loading save containing sprite-button. (https://forums.factorio.com/28736)
    
  • Fixed maps loaded from versions < 0.13.7 not generating any more chunks (https://forums.factorio.com/28760)
    
  • Fixed First Steps campaign character death error. (https://forums.factorio.com/27926)
    
  
  
• Scripting
  
  • Reverted changes related to 0.13.7 fix for (https://forums.factorio.com/28564). If entity specified in sprite path doesn't have an icon, button will be created anyway, but no sprite will be drawn inside of it.
    
  • Added LuaGui::is_valid_sprite_path() function
    
  • Fixed that if a land mine got blown up and replaced by a robot, it wouldn't arm itself. (https://forums.factorio.com/28761)
    
  • Added LuaPlayer::admin read - if the player is an admin.
    

Use the automatic updater if you can (check experimental updates in other settings) or download full installation at http://www.factorio.com/download/experimental.

• Minor Features
  
  • Holding the "drop item" key will keep dropping items.
    
  • Rocks can be mined while holding blueprints. Graphics;
    
  • Updated the stack inserter technology icon.
    
  
  
• Balancing
  
  • Medium and large worms spawn further from the starting area.
    
  
  
• Bugfixes
  
  • Connect game dialog remembers DNS the address as written rather than only IP more
    
  • Science packs and ammo are now recorded in the items consumed portion of production stats.
    
  • Fixed the mysterious crash when building rails. more
    
  • Unified the mass production 3 achievement to 20 M (it was 10M in game and 100M on steam). more
    
  • Fixed rendering layer of gate wall. more
    
  • Fixed fullscreen toggle in graphics options. more
    
  • Fixed mining drills using slightly too much energy per item mined.
    
  • Fixed that rail signals connectable to two rails at the same time were marked as buildable even when the buildability was valid only for one of the rails. more
    
  • Fixed burner inserters sometimes getting stuck. more
    
  • Fixed inserters with stack bonus waiting for stacks indefinitely when there's nothing to take from. more
    
  • Fixed blueprint previews with tiles and rails vs tiles with no rails. more
    
  • Fixed crash that could sometimes happen when a biter couldn't reach a target for a long time. more
    
  • Fixed on_player_placed_equipment was not called when quick transferring equipment into armor. more
    
  • Fixed map generator problems very far from the start.
    
  • Map size is now limited to 2000 km by 2000 km with a black bar rather than crashing when reaching this distance
    
  • Fixed clean-cursor with armor not putting it into filtered slots in your quickbar. more
    
  • Fixed that expansion chunk candidates values weren't updated properly. more
    
  • Fixed landfills could be included in blueprints. more
    
  • Fixed multiple instances of walls blocking movement when they shouldn't. more
    
  • Fixed that the solaris achievement unobtainable.
    
  • Fixed flamethrower turret would cause blueprint preview move up and down.
    
  • Fixed crash when the currently-playing folder can't be deleted when exiting game. more
    
  • Fixed crash when connection to the mod portal fails more
    
  • Fixed Update Mods button being sometimes disabled more
    
  • Fixed that LuaGameScript::write_file treated data as null-terminated byte string. more
    
  • Fixed entities marked with "not-repairable" still being repairable manually. more
    
  • Attempt to fix "Access is denied" error message during autosaves. more
    
  • Fixed desync caused by transport belt connected to circuit network reading in pulse mode.
    
  • Fixed save corruption when driving vehicles on transport belts in some instances. more
    
  • Fixed copy-paste not waking up inserters when copying filters between different inerter types. more
    
  • Fixed error when biters tried to expand while evolution factor was 0. more
    
  • Fixed building paths not refilling the cursor in some instances. more
    
  • Fixed unlocked achievements blinking too quickly. more
    
  • Fixed crash when setting the force of a logistic container in ghost form. more
    
  
  
• Scripting
  
  • Fixed label size issues when using different font sizes or resizing the game window. more
    
  • Fixed crash when trying to create sprite-button with valid SpritePath to sprite that doesn't exist. (https://forums.factorio.com/28564) LuaGuiElement::add won't create sprite-button and returns nil if invalid SpritePath is passed.
    
  • Added an option to LuaSurface::set_tiles() to disable the correction logic for total control of what the tiles end up as.
    
  • Added item-group, fluid, tile, virtual-signal and achievement icons to be accessible by the SpritePath used in the sprite-button element. Added SpriteButton to the scripting documentation.
    
  • Added "grid" to the on_player_placed_equipment event.
    
  • Added LuaRecipe::localised_name read.
    
  • Added LuaGuiElement type "scroll-pane".
    

You can get experimental releases by selecting the 'experimental' beta branch under Factorio's properties in Steam.

• Minor Features
  
  • Holding the "drop item" key will keep dropping items.
    
  • Rocks can be mined while holding blueprints. Graphics;
    
  • Updated the stack inserter technology icon.
    
  
  
• Balancing
  
  • Medium and large worms spawn further from the starting area.
    
  
  
• Bugfixes
  
  • Connect game dialog remembers DNS the address as written rather than only IP more
    
  • Science packs and ammo are now recorded in the items consumed portion of production stats.
    
  • Fixed the mysterious crash when building rails. more
    
  • Unified the mass production 3 achievement to 20 M (it was 10M in game and 100M on steam). more
    
  • Fixed rendering layer of gate wall. more
    
  • Fixed fullscreen toggle in graphics options. more
    
  • Fixed mining drills using slightly too much energy per item mined.
    
  • Fixed that rail signals connectable to two rails at the same time were marked as buildable even when the buildability was valid only for one of the rails. more
    
  • Fixed burner inserters sometimes getting stuck. more
    
  • Fixed inserters with stack bonus waiting for stacks indefinitely when there's nothing to take from. more
    
  • Fixed blueprint previews with tiles and rails vs tiles with no rails. more
    
  • Fixed crash that could sometimes happen when a biter couldn't reach a target for a long time. more
    
  • Fixed on_player_placed_equipment was not called when quick transferring equipment into armor. more
    
  • Fixed map generator problems very far from the start.
    
  • Map size is now limited to 2000 km by 2000 km with a black bar rather than crashing when reaching this distance
    
  • Fixed clean-cursor with armor not putting it into filtered slots in your quickbar. more
    
  • Fixed that expansion chunk candidates values weren't updated properly. more
    
  • Fixed landfills could be included in blueprints. more
    
  • Fixed multiple instances of walls blocking movement when they shouldn't. more
    
  • Fixed that the solaris achievement unobtainable.
    
  • Fixed flamethrower turret would cause blueprint preview move up and down.
    
  • Fixed crash when the currently-playing folder can't be deleted when exiting game. more
    
  • Fixed crash when connection to the mod portal fails more
    
  • Fixed Update Mods button being sometimes disabled more
    
  • Fixed that LuaGameScript::write_file treated data as null-terminated byte string. more
    
  • Fixed entities marked with "not-repairable" still being repairable manually. more
    
  • Attempt to fix "Access is denied" error message during autosaves. more
    
  • Fixed desync caused by transport belt connected to circuit network reading in pulse mode.
    
  • Fixed save corruption when driving vehicles on transport belts in some instances. more
    
  • Fixed copy-paste not waking up inserters when copying filters between different inerter types. more
    
  • Fixed error when biters tried to expand while evolution factor was 0. more
    
  • Fixed building paths not refilling the cursor in some instances. more
    
  • Fixed unlocked achievements blinking too quickly. more
    
  • Fixed crash when setting the force of a logistic container in ghost form. more
    
  
  
• Scripting
  
  • Fixed label size issues when using different font sizes or resizing the game window. more
    
  • Fixed crash when trying to create sprite-button with valid SpritePath to sprite that doesn't exist. (https://forums.factorio.com/28564) LuaGuiElement::add won't create sprite-button and returns nil if invalid SpritePath is passed.
    
  • Added an option to LuaSurface::set_tiles() to disable the correction logic for total control of what the tiles end up as.
    
  • Added item-group, fluid, tile, virtual-signal and achievement icons to be accessible by the SpritePath used in the sprite-button element. Added SpriteButton to the scripting documentation.
    
  • Added "grid" to the on_player_placed_equipment event.
    
  • Added LuaRecipe::localised_name read.
    
  • Added LuaGuiElement type "scroll-pane".
    

You can get experimental releases by selecting the 'experimental' beta branch under Factorio's properties in Steam.

Hello again Factorians!

The GFX department is back today for covering the process of making graphics for Factorio.
I'll get a bit technical, but not too much in order to don't be absolutely boring.

Next comes a resumed description of the actual process, so if you are a Factorio modder this might be interesting for you.

The Game designer (kovarex) realizes that Factorio will be better with the addition of a new entity, he speaks with the team about it, most of the time he makes a prototype with a placeholder graphic, he improves the behaviour of the new entity by observing it and playing with it. Once the entity is solid we have a meeting together. We speak about guidelines and I make my contributions concerning visual requirements, colours, animations, different layers, mood, etc.
Right now I'm in the middle of this process with the new pumps. See the starting of the sketching for a new entity:



Once the concept and the technical requirements are clear we jump into Blender for modeling.
Blender is a great software which allows us to have a very flexible workflow, taking care of all our needs for the Factorio engine.
Most of the time, a single entity in Factorio is pretty complex, rendered in multiple layers, and assembled again in the engine in order to work as we want. So for this we'll require multiple layers and outputs in Blender.

It is very common also that once the renders are finished and all the process is done, the entity is integrated into the game and some changes are necessary, so it's very convenient to have the blend file ready to changes with the minimum effort. In order to be able to re-render by clicking just one button, we need to use RenderLayers, and/or separate scenes, possibly with addition of linking Group instances.



Scenes are always split to MODEL scene(s), and RENDER scene(s). The RENDER scene then links to all necessary scene RenderLayers via compositor nodes.
It is necessary to split the scenes as compositor nodes are activated even when pressing F12, which would overwrite the render outputs accidentally, etc.

Another option is batch rendering through a .bat command, but since that doesn't support RenderLayers, it is not viable for us yet.



Almost every one of our sprites have went through Photoshop at some point, we paint hand-drawn masks to enforce contrast, make edges clearer, define shape of entities better and so on.
In Photoshop, the general approach is that we take the rendered image (preferably by Place Linked function), duplicate it twice, set the duplicates to Multiply and/or Screen blend modes and give them empty(black) masks.

Always is better to have the best render results directly from Blender, but reality is that many times we need a second round of tweaks.
Here an example of the importance of post-production:



But because projects often get very large, with many output sprites, we need to automate the rendering process, for which we use After Effects. We can combine both the hand-drawn post-production in Photoshop with After Effects by importing the PSD files and/or separate layers from it.


The biggest hurdle in adopting Ae workflow tends to be getting used to the layering system when putting compositions in other compositions.
A good comparison is using Groups in Photoshop. In After Effects this is called pre-composing.


Often you need to pre-compose / create multiple levels of compositions in order to reach desired results. Such a thing is likely to happen when:
You want to treat something like 1 unit - when moving, rotating, scaling, ...
You want to render some things separately.
You want to apply an effect to just some layers below, but not all.
You want to apply additive blending to some layer which should affect something else than just the entity (like fire on flamethrower turret).
You want to use multiple layers as a matte.
You want to render various time regions from the same composition (if you have multiple different outputs in one time sequence like cargo wagon, or when rendering shiftings).



As the last step of our process, we use a python script spritesheeter.py made by the Factorio coders to create spritesheets ready to the engine with relevant pieces of lua code in text files, alongside with gif previews.



Now we have everything ready to put the new entity into the game and see how it goes, but as I said before it is very probable that something is not exactly as we want and we have to come back to Blender in order to tweak it again. That's the reason of having a flexible system.

I need to say that this entire system is new for us, and now, since we are more artists in the department we had to standardize it. Also big thanks to Vaclav for his hard work on documenting the entire process, and also for his contribution to the system with his knowledge of After Effects which demonstrates that can make the workflow very effective once is setted up. I still working on this part :)

As usual comments and thoughts in the forums

Hello again Factorians!

The GFX department is back today for covering the process of making graphics for Factorio.
I'll get a bit technical, but not too much in order to don't be absolutely boring.

Next comes a resumed description of the actual process, so if you are a Factorio modder this might be interesting for you.

The Game designer (kovarex) realizes that Factorio will be better with the addition of a new entity, he speaks with the team about it, most of the time he makes a prototype with a placeholder graphic, he improves the behaviour of the new entity by observing it and playing with it. Once the entity is solid we have a meeting together. We speak about guidelines and I make my contributions concerning visual requirements, colours, animations, different layers, mood, etc.
Right now I'm in the middle of this process with the new pumps. See the starting of the sketching for a new entity:



Once the concept and the technical requirements are clear we jump into Blender for modeling.
Blender is a great software which allows us to have a very flexible workflow, taking care of all our needs for the Factorio engine.
Most of the time, a single entity in Factorio is pretty complex, rendered in multiple layers, and assembled again in the engine in order to work as we want. So for this we'll require multiple layers and outputs in Blender.

It is very common also that once the renders are finished and all the process is done, the entity is integrated into the game and some changes are necessary, so it's very convenient to have the blend file ready to changes with the minimum effort. In order to be able to re-render by clicking just one button, we need to use RenderLayers, and/or separate scenes, possibly with addition of linking Group instances.



Scenes are always split to MODEL scene(s), and RENDER scene(s). The RENDER scene then links to all necessary scene RenderLayers via compositor nodes.
It is necessary to split the scenes as compositor nodes are activated even when pressing F12, which would overwrite the render outputs accidentally, etc.

Another option is batch rendering through a .bat command, but since that doesn't support RenderLayers, it is not viable for us yet.



Almost every one of our sprites have went through Photoshop at some point, we paint hand-drawn masks to enforce contrast, make edges clearer, define shape of entities better and so on.
In Photoshop, the general approach is that we take the rendered image (preferably by Place Linked function), duplicate it twice, set the duplicates to Multiply and/or Screen blend modes and give them empty(black) masks.

Always is better to have the best render results directly from Blender, but reality is that many times we need a second round of tweaks.
Here an example of the importance of post-production:



But because projects often get very large, with many output sprites, we need to automate the rendering process, for which we use After Effects. We can combine both the hand-drawn post-production in Photoshop with After Effects by importing the PSD files and/or separate layers from it.


The biggest hurdle in adopting Ae workflow tends to be getting used to the layering system when putting compositions in other compositions.
A good comparison is using Groups in Photoshop. In After Effects this is called pre-composing.


Often you need to pre-compose / create multiple levels of compositions in order to reach desired results. Such a thing is likely to happen when:
You want to treat something like 1 unit - when moving, rotating, scaling, ...
You want to render some things separately.
You want to apply an effect to just some layers below, but not all.
You want to apply additive blending to some layer which should affect something else than just the entity (like fire on flamethrower turret).
You want to use multiple layers as a matte.
You want to render various time regions from the same composition (if you have multiple different outputs in one time sequence like cargo wagon, or when rendering shiftings).



As the last step of our process, we use a python script spritesheeter.py made by the Factorio coders to create spritesheets ready to the engine with relevant pieces of lua code in text files, alongside with gif previews.



Now we have everything ready to put the new entity into the game and see how it goes, but as I said before it is very probable that something is not exactly as we want and we have to come back to Blender in order to tweak it again. That's the reason of having a flexible system.

I need to say that this entire system is new for us, and now, since we are more artists in the department we had to standardize it. Also big thanks to Vaclav for his hard work on documenting the entire process, and also for his contribution to the system with his knowledge of After Effects which demonstrates that can make the workflow very effective once is setted up. I still working on this part :)

As usual comments and thoughts in the forums

• Changes
  
  • Stack inserters are unlocked by their own research that is dependent on advanced electronics and logistics 2. This also solves that stack inserters were available but not buildable in the campaign.
    
  • Deconstructing and cancel deconstruction can be toggled between by using the modifer key. more
    
  
  
• Bugfixes
  
  • Fixed desync when inserters would insert things directly onto splitters. more
    
  • Fixed that manipulating the fuel inventory of locomotives didn't count towards the inactivity condition. more
    
  • A train with a circuit condition will now always stay at the station if no wire is connected. more
    
  • Fixed that steamrolled achievement was obtainable by just killing the spawner, not only by killing it by impact. more
    
  • Fixed the player getting stuck in some instances when using landfills. more
    
  • Fixed crash when train stop that is in train schedule of some train was opened in the map editor. more
    
  • Fixed crash when the player would equip power armor they currently had open while in the same tick a robot delivered items to the player which would end up in the new slots added by the power armor. more
    
  • Fixed crash when mining tiles you're standing on that results in you being killed.
    
  • Fixed load game GUI size issues when trying to load invalid save files. more
    
  • Fixed fuel and water from pumps not being counted in the production stats consumed/produced items. more
    

You can get experimental releases by selecting the 'experimental' beta branch under Factorio's properties in Steam.

• Changes
  
  • Stack inserters are unlocked by their own research that is dependent on advanced electronics and logistics 2. This also solves that stack inserters were available but not buildable in the campaign.
    
  • Deconstructing and cancel deconstruction can be toggled between by using the modifer key. more
    
  
  
• Bugfixes
  
  • Fixed desync when inserters would insert things directly onto splitters. more
    
  • Fixed that manipulating the fuel inventory of locomotives didn't count towards the inactivity condition. more
    
  • A train with a circuit condition will now always stay at the station if no wire is connected. more
    
  • Fixed that steamrolled achievement was obtainable by just killing the spawner, not only by killing it by impact. more
    
  • Fixed the player getting stuck in some instances when using landfills. more
    
  • Fixed crash when train stop that is in train schedule of some train was opened in the map editor. more
    
  • Fixed crash when the player would equip power armor they currently had open while in the same tick a robot delivered items to the player which would end up in the new slots added by the power armor. more
    
  • Fixed crash when mining tiles you're standing on that results in you being killed.
    
  • Fixed load game GUI size issues when trying to load invalid save files. more
    
  • Fixed fuel and water from pumps not being counted in the production stats consumed/produced items. more
    

You can get experimental releases by selecting the 'experimental' beta branch under Factorio's properties in Steam.