The multithreading patch is now available on the Beta branch! Please report bugs you find so we can fix them before pushing it on to the main branch.
Game
Added a planetary trading post that sells goods from a nearby planet
Added an event where a freighter is chased by pirates
The cultists in asteroid ring sectors now behave the way they're supposed to
Improved aiming of salvaging turrets
Fighters search for mothership only every ~5 seconds when it's gone
Disabled player events while in sector (0:0)
Multithreading
We improved the multithreading handling of the server and client. For those of you who don't know what this means: In short: The server and client will both run a lot smoother on machines with more CPU cores. Performance for machines with 2 or less cores should be unchanged.If you're interested in the details, I've published a more technical post about the multithreading rework here: http://steamcommunity.com/games/445220/announcements/detail/1434814734654746607But basically, the server is now capable of using all its cores on calculating large battles so that performance should go up by a lot.
Pirates, Xsotan and other enemies are generated asynchronously, reducing server lag when ships spawn
Server sector updates are now completely multithreaded
Client particle updates are now completely multithreaded
We're planning on doing client sector updates in parallel as well, but we need a little more time for this.
RCON Interface
Added an RCON interface to the server (default port: TCP 27015)
The RCON interface requires a password which can be configured in server.ini. Without password, RCON is disabled
The RCON interface can be used to remotely control the server and enter commands as if over command line
But you can use any RCON client you'd like
UI
Improved window handling in build menu
Added showing amount of goods on the current ship in trading overview
Added icons to transfer crew & goods ui
Scripting API
The amount of async script threads can be configured in the server.ini (default: 2)
Changed setFont parameter from string to FontType enum
Planet specs are now controlled from within sectorspecifics.lua
Added a toRandom() function to generate random Uuids
Added a execute() function to execute code directly within scripts
Warning Modders: This function should NOT be called with code that is sent from a client!
Added an AsyncPirateGenerator
Added an AsyncShipGenerator
PlanGenerator can now generate plans asynchronously
Renamed piratehunter.lua to pirateattackstarter.lua
Renamed events.lua to eventscheduler.lua
Plan damaging callbacks are now bundled and sent at the end of a frame
Added a damage type that can be applied to inflictDamage() or destroyBlocks() functions
Added callback functions to script API for ListBox
Added a new InputWindow class
Added a function to return indices of all present factions in a sector
Server
Added a /workers command for controlling the amount of worker threads
Sending performance stats to admins is now configurable in server.ini
Client
Added debris on block destruction
Improved rendering stability
Misc
Minor improvements to spelling in loading screen tips
Added better support for multiple localizations
Language config files are now .xml and can include more information, such as new fonts
Improved german translation
Improved weapon accuracy of ships in sectors without players
Copy-Pasting of text via Ctrl-V is now possible on Linux
Frequent updates are now gathered and sent at the end of a frame to save bandwidth
Improved performance of plan damaging
Status command prints amount of worker threads into filename
Improved script updates to avoid updates at the same time for less performance outliers
Improved logging
Bugfixes
"As usual, user bug reports are marked with [UBR]. Thanks everybody, keep it up!"
Fixed Fighters not being updated when a player is out of sector
Fixed wormhole guardian not aggroing alliances
[UBR] Fixed broken database conversion when fighters are on a ship
[UBR] Fixed TurretAI not switching targets when target gets out of range
[UBR] Fixed a crash when shutting down server with lots of players in the same group
[UBR] Fixed alliance ships being unable to be controlled in strategy mode
[UBR] Fixed AI health bar not working correctly
[UBR] Fixed a crash when trying to invite non-existant players to an alliance
[UBR] Trading overview sorts correctly in all languages, not just english
[UBR] Fixed huge "Small Xsotan Breeders"
[UBR] Fixed some wreckages not having resources displayed and not being recognized by salvaging AI ships
[UBR] Fixed AllianceRank not being registered in the scripting API
[UBR] Fixed setting and getting script values from alliances
[UBR] Fixed some components always being added, even when they've been removed from the descriptor
[UBR] Fixed lasers and glows not showing up in asteroid rings when activating the Xsotan portal
[UBR] Fixed mouse position being reset after loading screen
[UBR] Fixed an issue when multiple mines with the same name are founded
[UBR] Fixed alliance ships being deleted when entering (0, 0)
[UBR] Fixed wreckages not being where they're supposed to be
The multithreading patch is now available on the Beta branch! Please report bugs you find so we can fix them before pushing it on to the main branch.
Game
Added a planetary trading post that sells goods from a nearby planet
Added an event where a freighter is chased by pirates
The cultists in asteroid ring sectors now behave the way they're supposed to
Improved aiming of salvaging turrets
Fighters search for mothership only every ~5 seconds when it's gone
Disabled player events while in sector (0:0)
Multithreading
We improved the multithreading handling of the server and client. For those of you who don't know what this means: In short: The server and client will both run a lot smoother on machines with more CPU cores. Performance for machines with 2 or less cores should be unchanged.If you're interested in the details, I've published a more technical post about the multithreading rework here: http://steamcommunity.com/games/445220/announcements/detail/1434814734654746607But basically, the server is now capable of using all its cores on calculating large battles so that performance should go up by a lot.
Pirates, Xsotan and other enemies are generated asynchronously, reducing server lag when ships spawn
Server sector updates are now completely multithreaded
Client particle updates are now completely multithreaded
We're planning on doing client sector updates in parallel as well, but we need a little more time for this.
RCON Interface
Added an RCON interface to the server (default port: TCP 27015)
The RCON interface requires a password which can be configured in server.ini. Without password, RCON is disabled
The RCON interface can be used to remotely control the server and enter commands as if over command line
But you can use any RCON client you'd like
UI
Improved window handling in build menu
Added showing amount of goods on the current ship in trading overview
Added icons to transfer crew & goods ui
Scripting API
The amount of async script threads can be configured in the server.ini (default: 2)
Changed setFont parameter from string to FontType enum
Planet specs are now controlled from within sectorspecifics.lua
Added a toRandom() function to generate random Uuids
Added a execute() function to execute code directly within scripts
Warning Modders: This function should NOT be called with code that is sent from a client!
Added an AsyncPirateGenerator
Added an AsyncShipGenerator
PlanGenerator can now generate plans asynchronously
Renamed piratehunter.lua to pirateattackstarter.lua
Renamed events.lua to eventscheduler.lua
Plan damaging callbacks are now bundled and sent at the end of a frame
Added a damage type that can be applied to inflictDamage() or destroyBlocks() functions
Added callback functions to script API for ListBox
Added a new InputWindow class
Added a function to return indices of all present factions in a sector
Server
Added a /workers command for controlling the amount of worker threads
Sending performance stats to admins is now configurable in server.ini
Client
Added debris on block destruction
Improved rendering stability
Misc
Minor improvements to spelling in loading screen tips
Added better support for multiple localizations
Language config files are now .xml and can include more information, such as new fonts
Improved german translation
Improved weapon accuracy of ships in sectors without players
Copy-Pasting of text via Ctrl-V is now possible on Linux
Frequent updates are now gathered and sent at the end of a frame to save bandwidth
Improved performance of plan damaging
Status command prints amount of worker threads into filename
Improved script updates to avoid updates at the same time for less performance outliers
Improved logging
Bugfixes
"As usual, user bug reports are marked with [UBR]. Thanks everybody, keep it up!"
Fixed Fighters not being updated when a player is out of sector
Fixed wormhole guardian not aggroing alliances
[UBR] Fixed broken database conversion when fighters are on a ship
[UBR] Fixed TurretAI not switching targets when target gets out of range
[UBR] Fixed a crash when shutting down server with lots of players in the same group
[UBR] Fixed alliance ships being unable to be controlled in strategy mode
[UBR] Fixed AI health bar not working correctly
[UBR] Fixed a crash when trying to invite non-existant players to an alliance
[UBR] Trading overview sorts correctly in all languages, not just english
[UBR] Fixed huge "Small Xsotan Breeders"
[UBR] Fixed some wreckages not having resources displayed and not being recognized by salvaging AI ships
[UBR] Fixed AllianceRank not being registered in the scripting API
[UBR] Fixed setting and getting script values from alliances
[UBR] Fixed some components always being added, even when they've been removed from the descriptor
[UBR] Fixed lasers and glows not showing up in asteroid rings when activating the Xsotan portal
[UBR] Fixed mouse position being reset after loading screen
[UBR] Fixed an issue when multiple mines with the same name are founded
[UBR] Fixed alliance ships being deleted when entering (0, 0)
[UBR] Fixed wreckages not being where they're supposed to be
Hey guys, we've been working on several server improvements concerning multithreading! This post will get a little technical again, so brace yourselves!
The reason why the update is taking so long, even though we promised otherwise (we know, sorry for that!) is that multithreading is a very complex issue that can easily introduce bugs when not done correctly.
Multithreading
We've improved the multithreading handling of the server. With the upcoming update, the server will use all worker threads for a single update of a sector.
The Avorion server uses a thread pool for updating sectors, and then gives work packets to these threads.
Before this update, the server created one work packet per sector update, meaning sectors were updated in parallel, but each sector was only updated by a single thread. Due to technical reasons a server tick/frame can't be finished until all sectors have finished updating, and it often happened that the server had to wait for one thread that took very long to finish, leaving the remaining threads (and thus CPU cores) idle (doing nothing).
For example, take a server with 8 worker threads, 10 sectors in memory and there's a large battle going on in one of them, sector 10. Sectors with battles take longer to update, so let's say that the sector with the large battle takes 70ms on average to update (which is a very realistic number that we've seen plenty of times) and the others each takes 2ms. With 8 workers, sectors 1 to 9 would be finished after 4ms, and then these threads would be idling (sitting around doing nothing) since they didn't have any more work to do, while the remaining thread, that's updating sector 10, would still have to run for 66 more milliseconds. The server can't finish the frame and has to wait for the thread to finish the update, while the other 7 worker threads do nothing, which is an obvious waste of CPU power.
With this update, we've changed the way that this system works. Instead of creating a single work packet per sector, each sector is subdivided into many small work packets that can then all be updated in parallel. This way we can update all sectors in parallel AND have all worker threads work on a single sector.
This is a graph of what sector updates look like now (you should enlarge it or look at it in a separate tab):
This is a profiling image of a server with 7 worker threads and with 2 HUGE faction battles (30 ships vs 30 ships per sector, one near the barrier, both including carriers and about 150 fighters each) going on in 2 different sectors. The X axis is time, while each row represents a worker thread. The colored bars are the work packets that the threads have to work through. An update frame goes from the purple bars [1] to the grey boxes [3], which represent the end of the sector update step. I've included multiple frame updates on top of each other so you get a better idea of how different they can look, even though it's the same 2 sectors each time. Yay multithreading!
Same color means the same kind of work. A few examples: The pale red ones [5] ship AI updates, purple [4] are turrets aiming and turning and blue [6] in the 4th row are fighter AI updates. The green [7] at the end of the frame is update sending and the golden and black ones [8] are collision handling and detection. When there's multiple work packets with the same color above each other, that means that multiple threads are working on the same logical update, but distributed over the different threads.
When there are no bars at some time (the white gaps) then that means that the worker is idle, either because there is no work to do or because a thread from another program has been scheduled by the operating system to do some other work. This is solely influenced by the operating system and we have no control over this. But basically, the denser the colors are packed, the better. These gaps will get more the more different other programs you run on the same machine as the Avorion server.
So what does all of this mean? To make it short: Avorion will run a LOT better on machines with multiple cores! The updates you're seeing here are only about 5ms long - meaning that 2 huge battles in 2 sectors can be updated within, on average, 5 milliseconds! (CPU: Intel i7-6700K with 4 cores @ 4.00GHz & Hyperthreading)
These are the two battles that were going on at the time of the profiling:
Of course there are still outliers (for example when a ship is spawned or destroyed), but even those have been reduced to maybe 3 - 5 times the average update time (5ms means we get outliers from 15 ms to 45 ms every 15 frames or so).
Multithreading Part 2: Async Scripting
We've also introduced a new way of executing scripts (actually this is not true, it's been in since the Alliances update, but we haven't used it yet): Async calls. These are mostly used to asynchronously generate ships that spawn during play time.
While profiling we realized that generating the ship plans take up 80% - 98% of the time required to spawn ships, so in order to reduce server lag while spawning huge armadas ship plan generation is now done asynchronously and the server won't lag as much.
RCON Interface
With the new update we'll also introduce an RCON Interface to the server, with which you can control the server with typical RCON tools. There are plenty of tools at your disposal, so simply pick the one that suits you most.
When is it coming?
Soon! We're done with all the features we want and we'll do several more bugfixes, but then it's ready. We'll also post detailed patch notes once the update goes live.
Hey guys, we've been working on several server improvements concerning multithreading! This post will get a little technical again, so brace yourselves!
The reason why the update is taking so long, even though we promised otherwise (we know, sorry for that!) is that multithreading is a very complex issue that can easily introduce bugs when not done correctly.
Multithreading
We've improved the multithreading handling of the server. With the upcoming update, the server will use all worker threads for a single update of a sector.
The Avorion server uses a thread pool for updating sectors, and then gives work packets to these threads.
Before this update, the server created one work packet per sector update, meaning sectors were updated in parallel, but each sector was only updated by a single thread. Due to technical reasons a server tick/frame can't be finished until all sectors have finished updating, and it often happened that the server had to wait for one thread that took very long to finish, leaving the remaining threads (and thus CPU cores) idle (doing nothing).
For example, take a server with 8 worker threads, 10 sectors in memory and there's a large battle going on in one of them, sector 10. Sectors with battles take longer to update, so let's say that the sector with the large battle takes 70ms on average to update (which is a very realistic number that we've seen plenty of times) and the others each takes 2ms. With 8 workers, sectors 1 to 9 would be finished after 4ms, and then these threads would be idling (sitting around doing nothing) since they didn't have any more work to do, while the remaining thread, that's updating sector 10, would still have to run for 66 more milliseconds. The server can't finish the frame and has to wait for the thread to finish the update, while the other 7 worker threads do nothing, which is an obvious waste of CPU power.
With this update, we've changed the way that this system works. Instead of creating a single work packet per sector, each sector is subdivided into many small work packets that can then all be updated in parallel. This way we can update all sectors in parallel AND have all worker threads work on a single sector.
This is a graph of what sector updates look like now (you should enlarge it or look at it in a separate tab):
This is a profiling image of a server with 7 worker threads and with 2 HUGE faction battles (30 ships vs 30 ships per sector, one near the barrier, both including carriers and about 150 fighters each) going on in 2 different sectors. The X axis is time, while each row represents a worker thread. The colored bars are the work packets that the threads have to work through. An update frame goes from the purple bars [1] to the grey boxes [3], which represent the end of the sector update step. I've included multiple frame updates on top of each other so you get a better idea of how different they can look, even though it's the same 2 sectors each time. Yay multithreading!
Same color means the same kind of work. A few examples: The pale red ones [5] ship AI updates, purple [4] are turrets aiming and turning and blue [6] in the 4th row are fighter AI updates. The green [7] at the end of the frame is update sending and the golden and black ones [8] are collision handling and detection. When there's multiple work packets with the same color above each other, that means that multiple threads are working on the same logical update, but distributed over the different threads.
When there are no bars at some time (the white gaps) then that means that the worker is idle, either because there is no work to do or because a thread from another program has been scheduled by the operating system to do some other work. This is solely influenced by the operating system and we have no control over this. But basically, the denser the colors are packed, the better. These gaps will get more the more different other programs you run on the same machine as the Avorion server.
So what does all of this mean? To make it short: Avorion will run a LOT better on machines with multiple cores! The updates you're seeing here are only about 5ms long - meaning that 2 huge battles in 2 sectors can be updated within, on average, 5 milliseconds! (CPU: Intel i7-6700K with 4 cores @ 4.00GHz & Hyperthreading)
These are the two battles that were going on at the time of the profiling:
Of course there are still outliers (for example when a ship is spawned or destroyed), but even those have been reduced to maybe 3 - 5 times the average update time (5ms means we get outliers from 15 ms to 45 ms every 15 frames or so).
Multithreading Part 2: Async Scripting
We've also introduced a new way of executing scripts (actually this is not true, it's been in since the Alliances update, but we haven't used it yet): Async calls. These are mostly used to asynchronously generate ships that spawn during play time.
While profiling we realized that generating the ship plans take up 80% - 98% of the time required to spawn ships, so in order to reduce server lag while spawning huge armadas ship plan generation is now done asynchronously and the server won't lag as much.
RCON Interface
With the new update we'll also introduce an RCON Interface to the server, with which you can control the server with typical RCON tools. There are plenty of tools at your disposal, so simply pick the one that suits you most.
When is it coming?
Soon! We're done with all the features we want and we'll do several more bugfixes, but then it's ready. We'll also post detailed patch notes once the update goes live.
As promised, here's the official roadmap for Avorion, so you can see what's planned for the future, what's coming and what's not coming (when it's not on there, it doesn't mean it's never going to be in the game. It's just that the features we put on there have a higher priority and we know how we want to do them).
These are the features we still want in the game and that we believe we can add until the release. Since a question came up a lot over the last weeks, I'd like to state this: We will most likely not stop making updates for Avorion after the big release.
As promised, here's the official roadmap for Avorion, so you can see what's planned for the future, what's coming and what's not coming (when it's not on there, it doesn't mean it's never going to be in the game. It's just that the features we put on there have a higher priority and we know how we want to do them).
These are the features we still want in the game and that we believe we can add until the release. Since a question came up a lot over the last weeks, I'd like to state this: We will most likely not stop making updates for Avorion after the big release.
it's been six month since the game was released into Early Access and it's been an exciting and trying time. Thank you to everyone who's been with us since the beginning and welcome to all those who have joined us on the way. Avorion was originally scheduled to leave EA in the third quarter of this year. Unfortunately time flies, we have reached the third quarter and we can all agree that there is much more work to be done. The Alliance Update took longer than anticipated, the engine needs more work and there are balancing problems that need fixing - a big thanks to the community for pointing those out and sharing your insight.
Now, we could simply postpone the release date by a few months. However we want to be sure we can make the new date even if other unforeseen problems occur. We don't want to continue giving you an over optimistic estimation we'd have to adjust every time we hit a road block. That is why we have decided to move the full release of Avorion to Q3 2018. This will allow us to deliver you a high quality product containing all the features we initially planned and maybe some more.
We'll post the roadmap for upcoming updates in a few days, so stay tuned!
it's been six month since the game was released into Early Access and it's been an exciting and trying time. Thank you to everyone who's been with us since the beginning and welcome to all those who have joined us on the way. Avorion was originally scheduled to leave EA in the third quarter of this year. Unfortunately time flies, we have reached the third quarter and we can all agree that there is much more work to be done. The Alliance Update took longer than anticipated, the engine needs more work and there are balancing problems that need fixing - a big thanks to the community for pointing those out and sharing your insight.
Now, we could simply postpone the release date by a few months. However we want to be sure we can make the new date even if other unforeseen problems occur. We don't want to continue giving you an over optimistic estimation we'd have to adjust every time we hit a road block. That is why we have decided to move the full release of Avorion to Q3 2018. This will allow us to deliver you a high quality product containing all the features we initially planned and maybe some more.
We'll post the roadmap for upcoming updates in a few days, so stay tuned!