Nov 8, 2019

Factorio - Klonan

Read this post on our website.

Color correction (Albert, V453000)

Factorio is in a state that even though is not yet finished, it is very close to it's 1.0 version. That means that most of the work is done and we are polishing the game in order to make it bright. That's what we've been doing for the past 2 weeks. Literally making it bright.

Since years I wanted to do this post-production work. But I didn't dare to do it until most of the graphics were finished. I was afraid of breaking the consistency of the look and our production pipeline.
Now it's different. There's only a couple of entities to re-design and some other stuff to do, but in general this missing details are not affecting the possibility of working in the post-production.

Factorio is a dark game. I mean conceptually. All these things about industrializing a planet, polluting an entire world just for the sake of the factory, and killing all its inhabitants are not precisely happy concepts full of light. This old article could explain better my thoughts regarding this concept. But the look of the game was dark, too dark. So we cleaned it up without betraying its spirit. Like restoring an old painting.

The difference can be subtle, but very effective. We added more light, and a little bit of color saturation. Adding these general changes to the entire sprites collection is not an easy task. Many sprites were badly affected by this general correction. V453000 was fixing individually the broken sprites and icons in order to keep the consistency with the new context.

We took the chance to work on the terrain a bit further. Not only this color correction was applied, but the contrast and integration with other terrains was also improved. Also experimenting with the color of the trees, trying to achieve a more colorful feeling with the excuse of an alien planet. I have to say the Alien Biomes mod was opening my mind - a little - to experiment with the color a bit further.

In order to break this general brown feeling, we added a more orange tonality to the sand biome. Here is were you can the difference more. Going further to too saturated colors is dangerous, after all, the terrain is a background that should provide a good and comfortable contrast with the entities and the icons.

Touching terrain colours means touching map colors also. We were very keen to keep the visibility of the map information and the similarity with the terrain. The result is a more vibrant look in the entire game.

We tweaked the night also. Thanks to posila and Wheybags, we can use LUTs (Look up tables) to dynamically modify the colors. Instead of playing with the alpha channel of a solid black layer on top of the game. Now we can gradually move to a different color palette for night with more control. So the colors are losing their saturation and becoming more blue and cold. This is important, because part of the annoying darkness of the game comes from this black layer.

We are still experimenting with this LUT, and the transitions of day/night cycles. I'm pretty sure also that I will have to touch the map colors for some missing details and fine tuning. Possibly there is some entity that is not in its best shape with these new color palettes, and maybe we keep tweaking the terrain. But I feel very confident with these additions and I'm very sure that these changes will improve the experience of playing Factorio. After playing with these colors, the feeling is good. I hope you see it the same way.

Updating mods (V453000)

By changing the colors of all game graphics, most mods are going to look out of place. We’ve used scripts to apply LUTs to most of the images, and one such script is ready for mod authors to clone or download here, together with the information of which LUT was used for which spritesheet in the game data, with the exception of terrain tiles, electric wires and combinators which were adjusted manually.

For technical support don’t hesitate to contact me directly, you’ll get quickest reply by messaging me (V453000#1894) on the Factorio Discord.

We still have some work to do on the LUTs, so they aren't going to be ready for release for another short while.

As always, let us know what you think on our forum.

Nov 1, 2019

Friday Facts #319 - New T-shirts & Lua event filtering

Factorio - Klonan

Read this post on our website.

Hello,
There is a bit of a cold/flu going around the office, but it isn't severe enough to dampen our spirits (I don't like the daylight savings though).

New T-shirts (Jitka, Albert, Aleš, Klonan)

Ever since we launched the classic Grey T-shirt back in November 2017, we've been asked when we're going to have more designs. Over the last few months Jitka, Albert, and Aleš, have been working on our new collection, which is now available.

Check out our store page for full product details.

We chose the Inserter and the Electronic circuit, as we feel that they are some of the items that best represent the game.

Once again, its the small details of which we are really proud. This time we are incorporating the new Wube logo into the print, as well as continuing our Factorio Dude printed label.

We have also updated our classic Factorio logo on grey with the new T-shirt design, which means that all the T-shirts are available in 3XL (Triple-XL), including the older design which previously only went up to 2XL. We have also printed a small batch of the Factorio logo T-shirts in a Ladies cut.

Several weeks ago, we mentioned that we bought a new rack for the server room (FFF-315). The secret we have been keeping is that we didn't just buy new racks for the server room, but also for the merchandise room. Now that we have the new T-shirt collection, the shelves are filled quite nicely.

The new T-shirt collection is available on our website from today. Please bear in mind that we are packaging and shipping all the T-shirts ourselves in the office, we aren't handling things through any 3rd party distributor. What this means is that shipping isn't super fast and we can't really guarantee any shipment dates. Generally the T-shirts reach European addresses within 1 week, and reach North America within 2.

With Christmas approaching, shipping times may be longer, so if you are looking to gift a Factorio T-shirt this upcoming holiday season, we would recommend ordering as soon as possible. Please note we still impose a 3 T-shirt limit on all orders.

Lua event filters (Rseding)

Every so often someone makes a mod interface request that grabs my attention - not because of the actual benefits the request will give (although that helps) but from a pure challenge perspective: I start thinking "that sounds difficult (and fun) - but I think I can do it - so how I can do that in an efficient/expandable way?...".

Several weeks ago someone requested a way to filter the different prototypes from the game on the C++ side instead of having to do it all Lua side:

Grab all of them.
Iterate through them one by one.
Store only the ones I care about.

Their reasons made sense to me and it was one of those challenging things that grabbed my attention. I finished it - and it has been out for several weeks.

Shortly after that someone asked for the same system for Lua events; mentioning that most mods don't actually want most events when they subscribe to one - but only a small subset of what would trigger a given event. It also made sense to me and it was even more of a challenge to take the already working filtering system I had and make it work for something I didn't intend. The end result though works quite nicely and can be expanded as we find different events needing different filters.

What this means overall, is that modders will be able to write cleaner and more efficient Lua scripts, and everybody will benefit from less UPS spent filtering the events on the script side.

/r/Factorio Extra Life charity stream (Klonan)

This weekend the moderators over at the Factorio subreddit are taking part in the Extra Life charity event. They will be streaming Factorio to help raise money for Children's hospitals in the US and Canada. There are some more details on the Reddit post.

As always, let us know what you think on our forum.

Oct 25, 2019

Friday Facts #318 - New Tooltips

Factorio - Klonan

Read this post on our website.

Hello,
we just released 0.17.73, with 0.17.74 coming very soon. This is just some bug fixes and further pathfinding improvements, and we hope to be able to mark the release as Stable next week.

The new tooltips (Twinsen)

As part of our big GUI update, I've been working on one particular part: the tooltips. We worked not only on updating the style, but also how the information is structured and sorted, added missing information, removed irrelevant information. This concerns entity, item and recipe tooltips, but almost all tooltips were touched one way or another. Many things were changed. I will go through some of the more important changes.

The new look

First thing to see is the new style Albert has worked on. They now have the same general style as the technology tooltips. We tried to keep them as compact as possible, as sometimes there is quite a lot of information to show.

For the screenshots in this blog post, I set the background to be non-transparent. Unfortunately they don't blend very well with our blog background, but in game you will notice that they are slightly transparent and the also have a blurring effect. Together with the shadows, they integrate nicely in the game.

Categorization

As you may have noticed already, some common properties like electricity consumption are grouped into categories. Most of the work was defining these categories and trying to figure out what makes sense. These categories help grouping the information but also gives more context to some entity properties. Properties that are directly related to the selected entity type are placed in the "root" category that has no name. This is to avoid having pointless categories like "Inserter" and "Transport belt".

New information
The place where the categorization really shines is in the tooltips of power generating entities. Now it's much clearer what each entity does and what the ratios are. Entities related to nuclear power were especially confusing. Creating an optimal nuclear setup was almost impossible without the help of the wiki. Now all the important information is there.

Entity tooltips and item tooltips generally show the same properties, but I tried to make the entity tooltips show state information when possible. For example, here's how the item tooltips above look when the entities are placed in the game. Categories are even more useful now, since properties like the fuel inside the machine or the state of the fluid output pipe can now be grouped inside the relevant categories.

Other entities have more properties added to them, such as inserter rotation speed, rolling stock weight, laser turret energy use per shot, flamethrower turret burning and slowdown effect, and many more.

Tooltip separation

The recipe tooltip was kind of a Frankenstein's monster of recipe information and item information mashed together. We also had the problem with what properties to show when a recipe has multiple output products. The solution was to split the tooltips and show a "multi-tooltip" when hovering a recipe,

Now, when hovering over a recipe in the crafting menu the recipe tooltip will be shown. An additional item tooltip will be shown for every product, as a separate tooltip, if the item tooltip has a description and/or properties to show. While this improves things quite a bit in vanilla, complex mods will benefit from it even more. Recipes can now have their own textual description and each separate product can be explained independently if necessary.

The same mechanism is used for the tooltip shown when hovering a logistic request in the character window.

This means that an item tooltip will look the same regardless if it's shown while hovering a recipe, an item in the player inventory or a logistic request. No more mixing of information.

Most of the implementation is done, just a few tweaks and bugfixes left to do, plus any changes based on your feedback. If all goes well, the new tooltips will be part of one of the next experimental feature releases, which we hope to release in the next couple of weeks. After that, more GUIs to come.

Construction robot tile batching (Rseding)

One of the things I've wanted to tinker with for some time is having a construction robot build multiple things at the same time. Construction robots spend the vast majority of their time just flying around doing very little work and can technically use the cargo capacity research but only ever use it for logistic related things.

https://cdn.factorio.com/assets/img/blog/fff-318-not-batched.mp4

One of the main things which stopped me from looking into this in the past was performance concerns: figuring out which thing(s) a robot can work on in a batch gets expensive very quickly and with robots existing in the 10s of thousands range I can't just make each one 5 times as expensive.

A few weeks ago I thought I finally figured out a way to at least make robots able to batch build tiles without loosing too much performance. The thing is: when tiles are built they are built in large square patterns so I can safely assume there will be other tiles to be built directly next to a given tile that hasn't already been assigned a robot to work on. After some experiments and then several re-works to optimize what was already quite fast I was satisfied with the result.

https://cdn.factorio.com/assets/img/blog/fff-318-batched.mp4

Of course, the next question people ask is: what about doing it for entities? I could, but it doesn't make as much sense for them because they aren't always 1x1 (performance drops off quickly as the size grows - tiles are always 1x1), aren't typically built in tightly packed squares like tiles are, and in the common case it would just make robots more expensive to run while rarely making them build faster. So, for now they just batch build tiles.

https://cdn.factorio.com/assets/img/blog/fff-318-batched-comparison.mp4
As a more direct comparison, Boskid made this nice setup using two forces with different cargo bonuses.

As with the tooltips, the tile job batching will be released as part of our next experimental feature release, which we're calling internally 'Stable 3' (0.17.69 is Stable 1, 0.17.74 will be Stable 2). As always, let us know what you think on our forum.

Oct 18, 2019

Friday Facts #317 - New pathfinding algorithm

Factorio - Klonan

Read this post on our website.

New pathfinding algorithm - Oxyd

Last week we mentioned the change to make biters not collide with each other, but that wasn’t the only biter-related update we released this past week. Somewhat coincidentally, this week’s updates have included something I’d been working on for a few weeks before – an upgrade to the enemy pathfinding system.

Pathfinding
When a unit wants to go somewhere, it first needs to figure out how to get there. That could be as simple as going straight to its goal, but there can be obstacles – such as cliffs, trees, spawners, player entities – in the way. To do that, it will tell the pathfinder its current position and the goal position, and the pathfinder will – possibly after many ticks – reply with a path, which is simply a series of waypoints for the unit to follow in order to get to its destination.

To do its job, the pathfinder uses an algorithm called A* (pronounced 'A star'). A simple example of A* pathfinding is shown in the video below: A biter wants to go around some cliffs. The pathfinder starts exploring the map around the biter (shown as white dots). First it tries to go straight toward the goal, but as soon as it reaches the cliffs, it 'spills' both ways, trying to find a position from which it can again go toward the goal.

https://cdn.factorio.com/assets/img/blog/fff-317-basic-pf.mp4
In this video, the algorithm is slowed down to better show how it works.

Each dot in the animation represents a node. Each node remembers its distance from the start of the search, and an estimate of the distance from that node toward the goal – this estimate is provided by what's called a heuristic function. Heuristic functions are what make A* work – it's what steers the algorithm in the correct direction.

A simple choice for this function is simply the straight-line distance from the node to the goal position – this is what we have been using in Factorio since forever, and it’s what makes the algorithm initially go straight. It’s not the only choice, however – if the heuristic function knew about some of the obstacles, it could steer the algorithm around them, which would result in a faster search, since it wouldn’t have to explore extra nodes. Obviously, the smarter the heuristic, the more difficult it is to implement.

The simple straight-line heuristic function is fine for pathfinding over relatively short distances. This was okay in past versions of Factorio – about the only long distance pathfinding was done by biters made angry by pollution, and that doesn’t happen very often, relatively speaking. These days, however, we have artillery. Artillery can easily shoot – and aggro – massive numbers of biters on the far end of a large lake, who will then all try to pathfind around the lake. The video below shows what it looks like when the simple A* algorithm we've been using until now tries to go around a lake.

https://cdn.factorio.com/assets/img/blog/fff-317-long-pf-before.mp4
This video shows how fast the algorithm works in reality; it hasn’t been slowed down.

Contracting Chunks
Pathfinding is an old problem, and so there are many techniques for improving pathfinding. Some of these techniques fall into the category of hierarchical pathfinding – where the map is first simplified, a path is found in the simplified map, and this is then used to help find the real path. Again, there are several techniques for how exactly to do this, but all of them require a simplification of the search space.

So how can we simplify a Factorio world? Our maps are randomly generated, and also constantly changing – placing and removing entities (such as assemblers, walls or turrets) is probably the most core mechanic of the entire game. We don’t want to recalculate the simplification each time an entity is added or removed. At the same time, resimplifying the map from scratch every time we want to find a path could quite easily negate any performance gains made.

It was one of our source access people, Allaizn, who came up with the idea that I ended up implementing. In retrospect, the idea is obvious.

The game is based around 32x32 chunks of tiles. The simplification process will replace each chunk with one or more abstract nodes. Since our goal is to improve pathfinding around lakes, we can ignore all entities and consider the tiles only – water is impassable, land is passable. We split each chunk into separate components – a component is an area of tiles where a unit can go from any tile within the component to any other within the same component. The image below shows a chunk split into two distinct components, red and green. Each of these components will become a single abstract node – basically, the entire chunk is reduced into two 'points'.

Allaizn’s key insight was that we don’t need to store the component for every tile on the map – it is enough to remember the components for the tiles on the perimeter of each chunk. This is because what we really care about is what other components (in neighbouring chunks) each component is connected to – that can only depend on the tiles that are on the very edge of the chunk.

Hierarchical Pathfinding
We have figured out how to simplify the map, so how do we use that for finding paths? As I said earlier, there are multiple techniques for hierarchical pathfinding. The most straightforward idea would be to simply find a path using abstract nodes from start to goal – that is, the path would be a list of chunk components that we have to visit – and then use a series plain old A* searches to figure out how exactly to go from one chunk's component to another.

The problem here is that we simplified the map a bit too much: What if it isn’t possible to go from one chunk to another because of some entities (such as cliffs) blocking the path? When contracting chunks we ignore all entities, so we merely know that the tiles between the chunks are somehow connected, but know nothing about whether it actually is possible to go from one to the other or not.

The solution is to use the simplification merely as a 'suggestion' for the real search. Specifically, we will use it to provide a smart version of the heuristic function for the search.

So what we end up with is this: We have two pathfinders, called the base pathfinder, which finds the actual path, and the abstract pathfinder, which provides the heuristic function for the base pathfinder. Whenever the base pathfinder creates a new base node, it calls the abstract pathfinder to get the estimate on the distance to the goal. The abstract pathfinder works backwards – it starts at the goal of the search, and works its way toward the start, jumping from one chunk’s component to another. Once the abstract search finds the chunk and the component in which the new base node is created, it uses the distance from the start of the abstract search (which, again, is the goal position of the overall search) to calculate the estimated distance from the new base node to the overall goal.

Running an entire pathfinder for every single base node would, however, be anything but fast, even if the abstract pathfinder leaps from one chunk to the next. Instead we use what’s called Reverse Resumable A*. Reverse means simply it goes from goal to start, as I already said. Resumable means that after it finds the chunk the base pathfinder is interested in, we keep all its nodes in memory. The next time the base pathfinder creates a new node and needs to know its distance estimate, we simply look at the abstract nodes we kept from the previous search, with a good chance the required abstract node will still be there (after all, one abstract node covers a large part of a chunk, often the entire chunk).

Even if the base pathfinder creates a node that is in a chunk not covered by any abstract node, we don’t need to do an entire abstract search all over again. A nice property of the A* algorithm is that even after it 'finishes' and finds a path, it can keep going, exploring nodes around the nodes it already explored. And that's exactly what we do if we need a distance estimate for a base node located in a chunk not yet covered by the abstract search: We resume the abstract search from the nodes we kept in memory, until it expands to the node that we need.

The video below shows the new pathfinding system in action. Blue circles are the abstract nodes; white dots are the base search. The pathfinder was slowed down significantly to make this video, to show how it works. At normal speed, the entire search takes only a few ticks. Notice how the base search, which is still using the same old algorithm we've always used, just 'knows' where to go around the lake, as if by magic.

https://cdn.factorio.com/assets/img/blog/fff-317-long-pf-after.mp4

Since the abstract pathfinder is only used to provide the heuristic distance estimates, the base search can quite easily digress from the path suggested by the abstract search. This means that even though our chunk contraction scheme ignores all entities, the base pathfinder can still go around them with little trouble. Ignoring entities in the map simplification process means we don't have to redo it every time an entity is placed or removed, and only need to cover tiles being changed (such as with landfill) – which happens way less often than entity placements.

It also means that we are still essentially using the same old pathfinder we've been using for years now, only with an upgraded heuristic function. That means this change shouldn’t affect too many things, other than the speed of the search.

Bye bye TOGoS - TOGoS

Hi everybody!
I'm going to be resigning from official Factorio staff after today to start a new job closer to home.

The main reason for this switch is that working entirely remotely turned out to be something that I wasn't able to handle all that well. But also, my primary contribution to the project, the programmable map generator, is complete, stable, and pretty well understood by at least one other person, so timing-wise this seemed a good point for me to move on. To working in a cube farm writing Android applications for treadmills, apparently.
We'll see how that goes!

It's been an honor to be part of this awesome project and to leave my imprint on it. And working on a large and pretty well-managed C++ codebase, full of things done just a bit different than I had ever thought to, has been mind-opening.

I also want to thank the community for your continual patience, honest feedback, and the occasional bit of pressure that you put on us to make the game as good as it can be.

I'll continue to read the Friday Facts every week, lurk on the forums, and probably update documentation here and there. Hopefully I'll find time to crank out some terrain-altering mods of my own. And I'm looking forward to seeing what else y'all do with it, too. (Which includes working through a backlog of mods -- I have yet to get to the space exploration part of Space Exploration!)

Peace out for now.

Community spotlight - 13x9 Micro Factory - Klonan

Over 100 Friday Facts ago we covered DaveMcW's 9x14 Micro Factory (FFF-197). While it may have taken over 2 years, he has improved his design even further, and the result is just as impressive as before.

https://youtu.be/9dzQge6pe2o

He offers some more explanation of his Micro Factory in this forum post.

As always, let us know what you think on our forum.

Oct 15, 2019

Version 0.17.70 released

Factorio - wheybags

Features

Added interface option to adjust the number of shortcut bar rows that are visible on the screen.
Added ability to shift click a research in the technology screen to start that research.
Allowed setting filters with the ghost cursor.

Changes

Biters and Spitters will no longer collide with other biters/spitters. more

Graphics

Added new remnants for several entities. They are still work-in-progress and subject to further change.

Bugfixes

Excluding runtime fluid mixing check from assembler migration that caused some crashes when loading a save with mixing in it.
Fixed landfill map color on old saves. more
Added migration to recalculate tile transitions when tile layer definition changes in tile prototype. more
Fixed that artillery remote failure sounds could be duplicated in some cases. more
Fixed that the game would still process mod dependencies for disabled mods. more
Fixed that underground pipes with different length could sometimes connect one tile further.
Fixed that the cursor would appear in the wrong position after textbox alignment was changed by scripting. more
Fixed placement of scrollbars around textboxes. more
Fixed a crash related to teleporting biters during the ai-completed events. more
Fixed that non-square crafting machines without fluid boxes didn't rotate correctly. more
Fixed that the game would freeze when trying to find automatic artillery targets with very high levels of artillery range. more
Fixed tile ghosts sometimes overlapped on edges which created visible lines. more
Fixed that the on_gui_switch_state_changed event would fire twice in some cases. more
Fixed wrong damage bonus values in tooltips of combat robots and units in a different force. more
Fixed that the research screen would show a technology as "available" when it was queued. more
Fixed that LuaEntity::last_user didn't support writing `nil`. more
Fixed that mod GUIs could show on top of the technology screen after loading a save. more
Fixed a crash when using the /screenshot command. more
Fixed that the --disable-migration-window command line option didn't always work. more
Fixed that pumps had a drain when using a non-electric energy source. more
Fixed a bug in fluid system splitting.
Changed order of pipe connections to avoid previous cases of fluid mixing. A.k.a Boskid's deferred pipes.
Offshore pumps now set fluid filter automatically based on produced fluid. more
Fixed crash when closing shortcut selection list while dragging. more
Fixed that biters could get overloaded by artillery and stop moving. more
Fixed that biters could get stuck during attacks. more
Fixed that biter pathfinding could cause unreasonably large save files. more
Fixed that biters would attack in a single file due to their colliding with each other. more
Fixed that personal roboports would function with literally no power.
Fixed migrating pre-0.17 maps with detached characters that had a grid armor in quick bar would corrupt game state. more

Modding

Added optional flying robot prototype property "max_speed".
Added light_renderer_search_distance_limit to utility constants.

Scripting

Changed LuaEntity::color to also work for cars.
Changed LuaStyle::padding, margin to also accept arrays of padding values.
Added optional "unit_number" to on_post_entity_died event.
Added support to teleport car entity types between surfaces.
Added LuaEntityPrototype::call_for_help_radius, max_count_of_owned_units, max_friends_around_to_spawn, spawning_radius and spawning_spacing read.
Added LuaForce::research_enabled read.