the work on the 0.13 is slowly getting to the final phase and hence it feels like a good time to give an overview of actual multiplayer changes that have been implemented over past couple of weeks.
This is a recap and extension of a FFF post that mentioned the Multiplayer improvements while there were in progress.
Browsing games
Probably the biggest change is implementation of a service that holds available Multiplayer games. We call this service Multiplayer Matching Server even though the name doesn't precisely describe what it does.
This service greatly simplifies the game discovery and connectability. Player hosting the game (or a headless server) will simply publish the game to the Multiplayer Matching Server. This makes the game available to other players via the Browse Games Gui. This will make the clumsy connecting directly to the server pretty much obsolete (though the functionality stays).
As mentioned earlier there are quite a few little features to make the user experience smooth. Just to recap:
Games can be created as publicly listed / unlisted / LAN only
LAN supports automatic discovery without internet connection
Games can be protected by password.
Browse Games Gui supports searching in name of the game / description / tags.
Browse Games Gui shows some basic game information (game length, player names, etc.) and allows directly join the game.
User Verification
The usage (creating games as well as listing games) of our Multiplayer Matching Server will require user to be logged in with his Factorio username. If you have a Factorio account you can simply use those credentials in the in-game login dialog. If you are coming from Steam there will be an in game Factorio account creation dialog that creates a Factorio account for you the first time you need to use it.
The reasons to use strictly Factorio usernames are mostly technical. However also we have been from the beginning committed to provide all our services both to people using the Steam and those not using it as well. The same will hold for Mod Portal and other services coming in the future.
Another advantage of having user verification is that when creating the game it can be requested that all the users connecting to the game needs to be verified. Which basically means that they are connecting with their Factorio username and they don't pretend to be someone else.
Technical background for user verification
Just a little technical rant about how user verification works behind the scenes. This is to show that even a minor feature might get quite interesting. The diagrams show graphically what is happening.
When the server (Factorio game) is started it requests a server_padlock from the Auth Server. This is a hash of random server generated server_identifier and secret Auth Server value.
New players receive server_identifier when connecting to the game. With the server_identifier, its username and password (or token if he logged in previously) the player goes to the Auth Server and obtains the user_server_key. This is a hash of server_padlock (generated again on the Auth Server from server_identifier) + player's username + timestamp.
The User then sends its user_server_key to the Server together with its username and timestamp.
Server regenerates its own version of user_server_key from its padlock + given username + given timestamp
If the server calculated user_server_key and provided one match, the user is accepted to the game.
In this protocol the server_padlock is private information shared between server and Auth Server. The server_padlock allows to verify that the user coming with user_server_key was successfully verified by the Auth Server.All this hassle is to avoid too many requests to the auth server. This way there is one request when the game is created and one request for every new player connecting to the game while keeping a good level (in our opinion) of security.
Steam Friends
An often asked for convenience for the Multiplayer is the integration with Steam Friends. Thanks to Ondra this has been already implemented. In the Browse Games Gui you can filter games that have some of your Steam friends in them. Then in the game details you would see your friends highlighted.
But more importantly (and conveniently) you can use this feature directly via Steam. In your steam client you will see your friends that are currently playing Factorio and you can just click and join them in the game. This translates to our internal game connection mechanism which might still ask you for a password if the game requires one.
NAT punching
The final "cherry on the top" which is still in progress is NAT punching via the matching server. Basically this will allow the servers without public IP address (they are behind a NAT) to host the game. The idea is simple:
The Matching Server (or a standalone service) will provide a simple UDP API which can be pinged and it will respond with the IP address it sees the incoming packets to come from. This will be our little imitation of the STUN. The application (Factorio client hosting the game) will receive the reply with the public IP address it is visible to the Matching Server (Matching server has a public IP). This "punches" the hole to the NAT the Factorio client is behind.
The game is then published with the IP address returned by the STUN-like service and hence when players connect to the game, they are connecting to the public IP address on the outside of the NAT (if there is any). The packets are then forwarded to the actual Factorio client hosting the game via internal NAT mechanisms. As usual this might not work in 100% cases because of the special cases and oddities in NATs but should cover most of the scenarios.
Other
0.13 is a big big release. The biggest so far. So there are quite a few more "little" features that make the MP experience smoother.
Admin system will allow the player hosting the game to give admin commands, most importantly kicking out and banning (potentially annoying) players.
Console commands for the server (also via RCON connections).
Peer2Peer mode has been removed. This simplifies the guis as well as internal coding solutions. The player starting the game is always the server. We have thought about this quite a bit and the potential upsides of peer2peer (possibly faster on LAN and more "robust") are just not worth the hassle. For the future we might look into some peer 2 peer communication optimisations behind the scenes but there are more urgent things to be solved now.
Entity info shows which player has built given entity.
If you have some more ideas what how the MP User Experience could be improved please let us know at our forums. For 0.13 we are pretty much set, but we will gladly consider the ideas for the future. For instance the observer mode has been on our TODO list for a while and it is slowly getting to the top.
the work on the 0.13 is slowly getting to the final phase and hence it feels like a good time to give an overview of actual multiplayer changes that have been implemented over past couple of weeks.
This is a recap and extension of a FFF post that mentioned the Multiplayer improvements while there were in progress.
Browsing games
Probably the biggest change is implementation of a service that holds available Multiplayer games. We call this service Multiplayer Matching Server even though the name doesn't precisely describe what it does.
This service greatly simplifies the game discovery and connectability. Player hosting the game (or a headless server) will simply publish the game to the Multiplayer Matching Server. This makes the game available to other players via the Browse Games Gui. This will make the clumsy connecting directly to the server pretty much obsolete (though the functionality stays).
As mentioned earlier there are quite a few little features to make the user experience smooth. Just to recap:
Games can be created as publicly listed / unlisted / LAN only
LAN supports automatic discovery without internet connection
Games can be protected by password.
Browse Games Gui supports searching in name of the game / description / tags.
Browse Games Gui shows some basic game information (game length, player names, etc.) and allows directly join the game.
User Verification
The usage (creating games as well as listing games) of our Multiplayer Matching Server will require user to be logged in with his Factorio username. If you have a Factorio account you can simply use those credentials in the in-game login dialog. If you are coming from Steam there will be an in game Factorio account creation dialog that creates a Factorio account for you the first time you need to use it.
The reasons to use strictly Factorio usernames are mostly technical. However also we have been from the beginning committed to provide all our services both to people using the Steam and those not using it as well. The same will hold for Mod Portal and other services coming in the future.
Another advantage of having user verification is that when creating the game it can be requested that all the users connecting to the game needs to be verified. Which basically means that they are connecting with their Factorio username and they don't pretend to be someone else.
Technical background for user verification
Just a little technical rant about how user verification works behind the scenes. This is to show that even a minor feature might get quite interesting. The diagrams show graphically what is happening.
When the server (Factorio game) is started it requests a server_padlock from the Auth Server. This is a hash of random server generated server_identifier and secret Auth Server value.
New players receive server_identifier when connecting to the game. With the server_identifier, its username and password (or token if he logged in previously) the player goes to the Auth Server and obtains the user_server_key. This is a hash of server_padlock (generated again on the Auth Server from server_identifier) + player's username + timestamp.
The User then sends its user_server_key to the Server together with its username and timestamp.
Server regenerates its own version of user_server_key from its padlock + given username + given timestamp
If the server calculated user_server_key and provided one match, the user is accepted to the game.
In this protocol the server_padlock is private information shared between server and Auth Server. The server_padlock allows to verify that the user coming with user_server_key was successfully verified by the Auth Server.All this hassle is to avoid too many requests to the auth server. This way there is one request when the game is created and one request for every new player connecting to the game while keeping a good level (in our opinion) of security.
Steam Friends
An often asked for convenience for the Multiplayer is the integration with Steam Friends. Thanks to Ondra this has been already implemented. In the Browse Games Gui you can filter games that have some of your Steam friends in them. Then in the game details you would see your friends highlighted.
But more importantly (and conveniently) you can use this feature directly via Steam. In your steam client you will see your friends that are currently playing Factorio and you can just click and join them in the game. This translates to our internal game connection mechanism which might still ask you for a password if the game requires one.
NAT punching
The final "cherry on the top" which is still in progress is NAT punching via the matching server. Basically this will allow the servers without public IP address (they are behind a NAT) to host the game. The idea is simple:
The Matching Server (or a standalone service) will provide a simple UDP API which can be pinged and it will respond with the IP address it sees the incoming packets to come from. This will be our little imitation of the STUN. The application (Factorio client hosting the game) will receive the reply with the public IP address it is visible to the Matching Server (Matching server has a public IP). This "punches" the hole to the NAT the Factorio client is behind.
The game is then published with the IP address returned by the STUN-like service and hence when players connect to the game, they are connecting to the public IP address on the outside of the NAT (if there is any). The packets are then forwarded to the actual Factorio client hosting the game via internal NAT mechanisms. As usual this might not work in 100% cases because of the special cases and oddities in NATs but should cover most of the scenarios.
Other
0.13 is a big big release. The biggest so far. So there are quite a few more "little" features that make the MP experience smoother.
Admin system will allow the player hosting the game to give admin commands, most importantly kicking out and banning (potentially annoying) players.
Console commands for the server (also via RCON connections).
Peer2Peer mode has been removed. This simplifies the guis as well as internal coding solutions. The player starting the game is always the server. We have thought about this quite a bit and the potential upsides of peer2peer (possibly faster on LAN and more "robust") are just not worth the hassle. For the future we might look into some peer 2 peer communication optimisations behind the scenes but there are more urgent things to be solved now.
Entity info shows which player has built given entity.
If you have some more ideas what how the MP User Experience could be improved please let us know at our forums. For 0.13 we are pretty much set, but we will gladly consider the ideas for the future. For instance the observer mode has been on our TODO list for a while and it is slowly getting to the top.
0.13 release is getting close. The programming guys are trying to finish the features so we can start internal playtesting and bugfixing next week. The art guys are working hard on the new trains graphics.
In the meantime, here's some more information about the circuit network.
Circuit Network Connector Discarded
If you remember my last FFF I wrote about adding a new entity called Circuit Network Connector that would have been used for connecting normal entities to the circuit network. The community had some good examples of why it was a bad idea, so we went back to the way it worked before. This means that you can continue to connect entities to the circuit network directly.
Consistent GUI
One of the problems that had to be solved was GUI interaction. Sprinkling circuit network related GUI elements randomly in the entity GUI looked pretty bad and was hard to understand. The solution is 2 small buttons in the top-right of circuit connectable entities GUI, they will appear when you research the circuit network and the logistic network respectively. Clicking the circuit network icon will open a window that lets you set up how you want the entity to interact with the circuit network. The logistic network button works in a similar way.
The window is usually separated into 3 parts:
Connection information: if and what it's connected to.
Mode of operation: sets how you want this entity to interact with the circuit network. Tooltips briefly explain what each mode does.
Mode of operation settings: various settings depending on the selected mode of operation.
Everyone gets a logistic condition!
The good thing about this GUI separation is that it also allows finer control, so most entities that can be turned on or off by the circuit network can also be turned on or off by directly using a logistic condition. You can now turn off pumps, offshore pumps, inserters, transport belts, lamps and power switches directly using a logistic condition, no new entities required, just click the magic button after researching the logistic network.
Wire highlights
This is something for those who make huge combinator networks. It's really easy to get lost in wires when making complicated setups. Now hovering over entities that have red/green wires connected to them will highlight the wire network.
Accumulators and Transport belts
Probably one of the more interesting entities that can now be connected to the circuit network are Accumulators and Transport belts.
Accumulators send the current charge level, so you can finally easily turn off steam engines at night or activate backup power based on accumulator charge levels.
Transport belts can be turned on or off based on circuit or logistic condition. You can also read the contents of the belt in 2 ways: pulse mode and hold mode. Pulse mode sends a signal for 1 tick when an item enters the belt. Hold mode will send the number of items on the current belt as long as they sit there. Those familiar with combinators will probably figure out that pulse mode is very useful for calculating the throughput of the belt or calculating the total number of items that passed though that belt.
This logic works for one belt tile, not for the whole length of belt.
Performance and how it works [Technical]
Previously, the way the circuit network worked was that every entity that could be connected to the circuit network had all network connection information, circuit conditions, logistic conditions, etc inside of it. That meant that if you had thousands of inserters in your map that were not connected to the circuit or logistic network, some RAM and savegame space was wasted.
This was not really a problem, but if it would have been done the same to all the new entities, especially transport belts, it would have become a problem.
The solution is simple. The relevent data(network connection information, circuit conditions, logistic conditions) is packed into something I call 'control behaviors' (because they tell the entity how to behave when they are controlled by the circuit or logistic network). These behaviors are only created when it's required: when you connect a circuit wire or when you want to set a logistic condition.
This is relevant to modders. Modders have access to these behaviors. To get a behavior instance you call luaEntity.get_control_behavior() or luaEntity.get_or_create_control_behavior(). The second method creates the behavior instance if it does not exist.
TLDR version of all the circuit network changes implemented and coming to 0.13
Power switch, which can turn electricity on or off automatically based on circuit condition.
You can turn off pumps, offshore pumps, insertes, belts, lamps and power switches directly using a logistic condition.
All types of inserters can be controlled by the circuit and logistic network, once the respective network is researched.
All types of chests can be connected to the circuit network. Smart Chest was removed from the game.
Circuit network and logistic network conditions can now be accessed by icons in the top-right corner of the entity's GUI, for all entities.
Decider combinator "input count" option makes the combinator copy the count of the specified output signal from the input signals, instead of copying the count from the condition. This might break some setups. https://forums.factorio.com/13706
The plan is to release 0.13 soon. I'll try to get more things ready. The list of plans now is as follows(in no particular order)
Mining drill - read the amount of resources remaining, read the Pumpjack resources per minute
Gate - Detect if a player is nearby, close and open the gate
Train Station - read the contents of the stopped train and control trains by disabling/enabling the station
Train Signals - still in the process of designing it. Basically be able to stop trains or see if a train is about to arrive
Czech game of the year awards
Last week some of us went to some Czech game of the year awards ceremony. And good news, we managed to win not one, but two of the four awards. Factorio won Best Technical Award and Best Gameplay Award.
As usual, PM me or reply at the forums for suggestions, complaints and comments.
0.13 release is getting close. The programming guys are trying to finish the features so we can start internal playtesting and bugfixing next week. The art guys are working hard on the new trains graphics.
In the meantime, here's some more information about the circuit network.
Circuit Network Connector Discarded
If you remember my last FFF I wrote about adding a new entity called Circuit Network Connector that would have been used for connecting normal entities to the circuit network. The community had some good examples of why it was a bad idea, so we went back to the way it worked before. This means that you can continue to connect entities to the circuit network directly.
Consistent GUI
One of the problems that had to be solved was GUI interaction. Sprinkling circuit network related GUI elements randomly in the entity GUI looked pretty bad and was hard to understand. The solution is 2 small buttons in the top-right of circuit connectable entities GUI, they will appear when you research the circuit network and the logistic network respectively. Clicking the circuit network icon will open a window that lets you set up how you want the entity to interact with the circuit network. The logistic network button works in a similar way.
The window is usually separated into 3 parts:
Connection information: if and what it's connected to.
Mode of operation: sets how you want this entity to interact with the circuit network. Tooltips briefly explain what each mode does.
Mode of operation settings: various settings depending on the selected mode of operation.
Everyone gets a logistic condition!
The good thing about this GUI separation is that it also allows finer control, so most entities that can be turned on or off by the circuit network can also be turned on or off by directly using a logistic condition. You can now turn off pumps, offshore pumps, inserters, transport belts, lamps and power switches directly using a logistic condition, no new entities required, just click the magic button after researching the logistic network.
Wire highlights
This is something for those who make huge combinator networks. It's really easy to get lost in wires when making complicated setups. Now hovering over entities that have red/green wires connected to them will highlight the wire network.
Accumulators and Transport belts
Probably one of the more interesting entities that can now be connected to the circuit network are Accumulators and Transport belts.
Accumulators send the current charge level, so you can finally easily turn off steam engines at night or activate backup power based on accumulator charge levels.
Transport belts can be turned on or off based on circuit or logistic condition. You can also read the contents of the belt in 2 ways: pulse mode and hold mode. Pulse mode sends a signal for 1 tick when an item enters the belt. Hold mode will send the number of items on the current belt as long as they sit there. Those familiar with combinators will probably figure out that pulse mode is very useful for calculating the throughput of the belt or calculating the total number of items that passed though that belt.
This logic works for one belt tile, not for the whole length of belt.
Performance and how it works [Technical]
Previously, the way the circuit network worked was that every entity that could be connected to the circuit network had all network connection information, circuit conditions, logistic conditions, etc inside of it. That meant that if you had thousands of inserters in your map that were not connected to the circuit or logistic network, some RAM and savegame space was wasted.
This was not really a problem, but if it would have been done the same to all the new entities, especially transport belts, it would have become a problem.
The solution is simple. The relevent data(network connection information, circuit conditions, logistic conditions) is packed into something I call 'control behaviors' (because they tell the entity how to behave when they are controlled by the circuit or logistic network). These behaviors are only created when it's required: when you connect a circuit wire or when you want to set a logistic condition.
This is relevant to modders. Modders have access to these behaviors. To get a behavior instance you call luaEntity.get_control_behavior() or luaEntity.get_or_create_control_behavior(). The second method creates the behavior instance if it does not exist.
TLDR version of all the circuit network changes implemented and coming to 0.13
Power switch, which can turn electricity on or off automatically based on circuit condition.
You can turn off pumps, offshore pumps, insertes, belts, lamps and power switches directly using a logistic condition.
All types of inserters can be controlled by the circuit and logistic network, once the respective network is researched.
All types of chests can be connected to the circuit network. Smart Chest was removed from the game.
Circuit network and logistic network conditions can now be accessed by icons in the top-right corner of the entity's GUI, for all entities.
Decider combinator "input count" option makes the combinator copy the count of the specified output signal from the input signals, instead of copying the count from the condition. This might break some setups. https://forums.factorio.com/13706
The plan is to release 0.13 soon. I'll try to get more things ready. The list of plans now is as follows(in no particular order)
Mining drill - read the amount of resources remaining, read the Pumpjack resources per minute
Gate - Detect if a player is nearby, close and open the gate
Train Station - read the contents of the stopped train and control trains by disabling/enabling the station
Train Signals - still in the process of designing it. Basically be able to stop trains or see if a train is about to arrive
Czech game of the year awards
Last week some of us went to some Czech game of the year awards ceremony. And good news, we managed to win not one, but two of the four awards. Factorio won Best Technical Award and Best Gameplay Award.
As usual, PM me or reply at the forums for suggestions, complaints and comments.
the 1st of June, which is the goal of the 0.13 release is starting to feel uncomfortably close, especially if I want to start play-testing 2 weeks before the release date. There are a lot of bigger and smaller tasks appearing all over the place. It is now the time of moving some of them to the next release again. Many of them are moved like this for years already.
Czech game of the year
I have to finish the Friday facts soon today, as we are going on a trip to Třeboň, where the Czech game of the year ceremony is held. Factorio is nominated on 3 categories, so there is a chance to win something, but mainly, we will have the opportunity to talk and party with other game developers, which doesn't happen often.
Rapid inserter
The final decision has been made and instead of a loader, we will have not heavy, but rapid inserter. To make things straightforward, I decided to limit the inserter stack size bonus to the rapid inserter only. Some people certainly won't like it, as they won't be able to use long handed inserters or burner inserters to take advantage of this bonus, but the advantages outweigh it.
Streamlined definition of the inserter stack size bonus, now it increases rapid inserter capacity whatever is it doing.
The rapid inserter will have its specific usages. As the rapid inserter currently waits until its hand is filled with items, it is not clearly better than fast inserter. When the bonus is 5, and it has 4 items in hand, it can wait minutes (or forever) for the last item, which might be undesirable especially with expensive, low throughput items. This implies that rapid inserters will eventually become stuck on mixed belts as long as some explicit mechanism to prevent it won't be there.
To make it useful, we will increase the maximum rapid inserter capacity (instead of inserter stack size bonus) from 4 to 11, so it can hold up to 12 items of the same kind.
Four fast inserters can be easily matched by one fully upgraded rapid inserter:
Four fully upgraded rapid inserters can almost fully compress an express belt:
Train station visualizations
Another one of the ease of use additions for 0.13, is the train station visualization. Whenever you are building next to a rail that leads to a station, you can see the future train wagon positions. This simplifies the building of the stations a lot.
The visualization is also helpful when building the train stop. You can immediately see, how many wagons can fit in the platform. It is combined with the indicator of possible train station placements. These are limited by the direction in a similar way as rail signals are. With this you won't build the station on the other side of the rail by accident anymore.
Fanart
I encountered this cute picture by ironhand on our forums, and I just had to share it with you.
As always, let us know what you think on our forums.
the 1st of June, which is the goal of the 0.13 release is starting to feel uncomfortably close, especially if I want to start play-testing 2 weeks before the release date. There are a lot of bigger and smaller tasks appearing all over the place. It is now the time of moving some of them to the next release again. Many of them are moved like this for years already.
Czech game of the year
I have to finish the Friday facts soon today, as we are going on a trip to Třeboň, where the Czech game of the year ceremony is held. Factorio is nominated on 3 categories, so there is a chance to win something, but mainly, we will have the opportunity to talk and party with other game developers, which doesn't happen often.
Rapid inserter
The final decision has been made and instead of a loader, we will have not heavy, but rapid inserter. To make things straightforward, I decided to limit the inserter stack size bonus to the rapid inserter only. Some people certainly won't like it, as they won't be able to use long handed inserters or burner inserters to take advantage of this bonus, but the advantages outweigh it.
Streamlined definition of the inserter stack size bonus, now it increases rapid inserter capacity whatever is it doing.
The rapid inserter will have its specific usages. As the rapid inserter currently waits until its hand is filled with items, it is not clearly better than fast inserter. When the bonus is 5, and it has 4 items in hand, it can wait minutes (or forever) for the last item, which might be undesirable especially with expensive, low throughput items. This implies that rapid inserters will eventually become stuck on mixed belts as long as some explicit mechanism to prevent it won't be there.
To make it useful, we will increase the maximum rapid inserter capacity (instead of inserter stack size bonus) from 4 to 11, so it can hold up to 12 items of the same kind.
Four fast inserters can be easily matched by one fully upgraded rapid inserter:
Four fully upgraded rapid inserters can almost fully compress an express belt:
Train station visualizations
Another one of the ease of use additions for 0.13, is the train station visualization. Whenever you are building next to a rail that leads to a station, you can see the future train wagon positions. This simplifies the building of the stations a lot.
The visualization is also helpful when building the train stop. You can immediately see, how many wagons can fit in the platform. It is combined with the indicator of possible train station placements. These are limited by the direction in a similar way as rail signals are. With this you won't build the station on the other side of the rail by accident anymore.
Fanart
I encountered this cute picture by ironhand on our forums, and I just had to share it with you.
As always, let us know what you think on our forums.
Fixed loading of electric network of pre 0.12 saves.
Fixed blueprint building. Blueprints are migrated as long as you load the version 0.12.30 or earlier, existing bluepritns might be off if you resaved in 0.12.31 already.
Scripting
Lua interface to create blueprint now expects the entity positions to be relative to center exactly, so in rail-less blueprints, the position 0.0 translates to center of the tile when the blueprint is built.
Fixed loading of electric network of pre 0.12 saves.
Fixed blueprint building. Blueprints are migrated as long as you load the version 0.12.30 or earlier, existing bluepritns might be off if you resaved in 0.12.31 already.
Scripting
Lua interface to create blueprint now expects the entity positions to be relative to center exactly, so in rail-less blueprints, the position 0.0 translates to center of the tile when the blueprint is built.