This development patch brings in a features that lay the groundwork for a followup patch to the stable branch.
We are releasing this to the dev branch now so that you can help us test it and provide feedback on the direction things are taking. Thus, this patch contains a lot of experimental features with moderate chance for breakage.
We're aware it's been a long wait since our last patch. Thank you for your patience! We assure you, we are still hard at work on Neverwinter Nights: Enhanced Edition.
Please note that this patch does not represent the full set of features we are working on. Instead, we have split away partial changes so that the community could test and evaluate them while we’re working on other features including the new graphics rendering engine, improved NWSync capabilities, and softening up more parts of the ruleset.
This patch introduces backwards-breaking changes to save games and character files (specifically, the Red Dragon Disciple). Please read the patch notes carefully and make backups of your saves and characters.
Known Issues/Caveats in this patch
Networking relays are still nonfunctional. We will have to address further-reaching changes to the network protocol for the upcoming console release. Any related fixes and features, such as better NAT traversal, have been postponed for now.
Rendering console text (the yellow debug console) sometimes renders white squares instead.
Old CampaignDB files ("CodeBase", "Bioware DB", "FoxPro", etc) remain unreadable (for now, see below).
Features
Both client and server are now 64bit binaries. We do not provide 32bit binaries anymore as of this patch. (The binary directories in the game install are now misnamed; we didn't change this yet but will in a future patch.)
Achievements have been added to the Steam release. These are the same as have been available on Android for a while. There is a Debug UI panel that allows you to clear previously-achieved achievements if you’d like to start fresh. Please note that they will remain flagged as Hidden until support for them hits the stable branch.
CodeBase (“Bioware Campaign Database”) has been replaced with sqlite3, as the old database code would not work when compiled with a 64bit compiler. Moving forward, we will be writing new, minimal code to import existing CodeBase entries; but for now, old data remains inaccessible. All new database files end up in the database/ directory in your user home. They have the extension .sqlite3.
A debug UI for advanced users to showcase experimental new features and UI scenes has been implemented (Ctrl+Shift+F12 to access).
A new configuration system has been put in place. Most client and server config is now stored in a file called settings.tml, instead of nwn and nwnplayer ini.
A lot of new configuration keys have been added, too numerous to list here.
Configuration will be imported from ini files. Once imported, the .ini will not be written anymore (but left in place in case you want to go back to a previous build).
All configuration keys can be overridden via ENV variables (useful for automated server setups), like so: NWN_CONFIG_SET_key, where key is the fully-qualified configuration key.
Some configuration still remains in .ini files; these will be migrated as we go along.
The .tml file is in TOML format and has a full schema embedded for better introspection and tool-based editing.
The debug UI offers access to the full new configuration system.
Networking/Crypto: Servers can now have fixed identities that persist between restarts. This identity is stored in a file named “cryptographic_secret” in the user home. Leaking this identity allows others to impersonate your server, much in the same way as leaking a private CD key. Favourites and History storage will now use server identities if available, so that floating IP/port servers can be found again easily.
The game now defaults to windowed mode (instead of exclusive fullscreen) on new installs.
The game will now remember the window position between restarts when in windowed mode.
A simple frame limiter has been added, which can be used to reduce CPU/GPU usage on battery-constrained devices. It is accessible through the Debug UI as explained below.
Walking/Running with shift held down is now an invertible config toggle. This is also accessible through the Debug UI.
NWSync now automatically purges outdated server manifests, removing stale and unneeded data. Manifests are considered outdated if they are coming from the same server URL and carry the same group_id (see NWSync documentation).
NWSync can now download offline modules from preconfigured servers. This is a experimental feature and only accessible via the Debug UI, and requires adding a specially-prepared repository.
The debug UI offers to purge unwanted nwsync manifests and all associated data. This will eventually be turned into a more user-friendly UI.
Music and ambient sounds are now loaded through ResMan and can thus be in hak and nwsync.
The game will now read plain mp3 files (the .bmu extension rename is still required, because this extension is used in all existing resources/haks).
The game user home (i.e. Documents/Neverwinter Nights) is now provisioned “development” directory, which detects changes and content is reloaded at runtime. This directory sits at the very top of the resources search path (unlike override/, which is sitting below ERF containers such as HAKs).
NB: This content reload only affects resource types that are not cached. For example, it can be used to inject scripts or GFF data, but not replace textures on the fly.
ResMan resource management and priorities has been rewritten. The debug UI can show this to you.
[Linux/Mac only] The game can now optionally mmap() all core game resources. This speeds up loading times drastically on some systems. This is turned off by default.
We have added a script command to JIT-compile and run a chunk of NWScript.
We have added functionality that allows attaching UUIDs to objects. These UUIDs are persisted to GFF and can be used by authors to identify items across module restarts, among other use cases.
Script calls that serialise/deserialise objects (StoreCampaignObject and CopyObject as of now) can now work with the following object types: Creatures, Items, Placeables, Waypoints, Stores, Doors, Triggers.
Fixes
C++: Thousands of cases of undefined behaviour have been addressed. A lot of code duplication has been removed in the process.
KTX texture loading order has been fixed.
The Load/Save dialogue now properly removes savegames from the resource manager when exiting the UI.
GL Render Culling was fixed (i.e. flickering trees and keyholing showing backfaces of geometry instead of punching through).
We fixed invalid user savegames being deleted on read errors; for example when erroneously storing savegames in subdirectories, or when AV denies access to reading the .sav.
Automatic tangent generation has been fixed.
Steam CD key retrieval should be more robust; and in the case of errors, will emit a useful support token that can be used to follow up with Beamdog.
A memleak in script evaluation was fixed.
SetPhenoType: The constraint condition was too narrow in clamping the phenotype; this has been fixed.
SetName() on areas now sends the configured language, instead of English.
The rather arbitrary builtin effect icon limit has been removed.
Object Visual Transforms: We now send VTs for newly-created doors and placeables. Some inefficiencies in network updates have been removed.
Material Shader Params: Some inefficiencies in network updates have been removed.
The server now waits 3 seconds before attempting to list with the master, allowing a module to load in non-interactive use.
Running a local multiplayer game now should never reject the primary player if the master is unreachable.
A case where arturo texture animations failed to animate has been addressed.
A bug in the creature serialisation code (for StoreCampaignObject) has been addressed where it would erroneously embed transient object IDs for inventory items, resulting in issues when loading this creature again in a persistent world setting.
A nullptr deref crash related to funny/zombie-walking a player was fixed.
A nullptr deref crash when clicking placeables without a valid player creature was fixed.
A thread race crash during sound system initialisation was fixed.
A nullptr deref crash when the server sends a weapon switch request for a creature where the model did not yet load was fixed.
A selection of nullptr deref crashes when the server sends a creature update for a non-existing creature were fixed.
A case where encounters containing creatures only of the same CR would fail to select the proper amount/difficulty was fixed.
Ruleset
classes.2da: New column StatGainTable. This column contains a table that describes ability and natural AC progression per level.
Palemaster Natural AC progression has been changed to use this mechanism.
Dragon Disciple Natural AC and Ability progression has been changed as well. As a consequence of this change, existing creatures will be stripped of their baked-in modifiers; thus changing characters/saves irreversibly and making them invalid with older game versions. Character changes are tracked in the character file itself: A new field “DataMigration” has been added to the GFF CreatureStats struct to accommodate this. In the interest of transparency, changes made to character files are published in the patch notes, below. If you want to return to a previous game version, make sure to create a backup first.
racialtypes.2da: New columns to configure extra skill points and feats at first level.
racialtypes.2da: New columns to configure the skill points modifier at first level.
racialtypes.2da: New columns to configure Ability Point Buy.
racialtypes.2da: New columns to configure normal feat progression.
ruleset.2da: New 2da added that contains a set of builtin ruleset defines. These defines were previously hardcoded in the game engine and are now overridable on a per-module basis. No explicit guarantees of functionality are given due to their scope and amount; however we will gladly listen for feedback on what works, what doesn’t work as expected, and what needs changing to accommodate further use cases. We reserve the right to change and remove entries with future patches.
You can find all of these files in the game installation directory, subdirectory ovr/.
Irreversible character data migrations in this patch
DataMigration < 1: Dragon Disciple Class:
if (level >= 10) STR -= 4;
if (level >= 4) STR -= 2;
if (level >= 2) STR -= 2;
if (level >= 7) CON -= 2;
if (level >= 9) INT -= 2;
if (level >= 10) CHA -= 2;
// Note: Natural AC was already calculated at runtime and not persisted to character data.
New Script Commands
// Execute a script chunk.
// The script chunk runs immediately, same as ExecuteScript().
// The script is jitted in place and currently not cached: Each invocation will recompile the script chunk.
// Note that the script chunk will run as if a separate script. This is not eval().
// By default, the script chunk is wrapped into void main() {}. Pass in bWrapIntoMain = FALSE to override.
// Returns "" on success, or the compilation error.
string ExecuteScriptChunk(string sScriptChunk, object oObject = OBJECT_SELF, int bWrapIntoMain = TRUE);
// Returns a UUID. This UUID will not be associated with any object.
// The generated UUID is currently a v4.
string GetRandomUUID();
// Returns the given objects' UUID. This UUID is persisted across save boundaries,
// like Save/RestoreCampaignObject and save games.
//
// Thus, reidentification is only guaranteed in scenarios where players cannot introduce
// new objects (i.e. servervault servers).
//
// UUIDs are guaranteed to be unique in any single running game.
//
// If a loaded object would collide with a UUID already present in the game, the
// object receives no UUID and a warning is emitted to the log. Requesting a UUID
// for the new object will generate a random one.
//
// This UUID is useful to, for example:
// - Safely identify servervault characters
// - Track serialisable objects (like items or creatures) as they are saved to the
// campaign DB - i.e. persistent storage chests or dropped items.
// - Track objects across multiple game instances (in trusted scenarios).
//
// Currently, the following objects can carry UUIDs:
// Items, Creatures, Placeables, Triggers, Doors, Waypoints, Stores,
// Encounters, Areas.
//
// Will return "" (empty string) when the given object cannot carry a UUID.
string GetObjectUUID(object oObject);
// Forces the given object to receive a new UUID, discarding the current value.
void ForceRefreshObjectUUID(object oObject);
// Looks up a object on the server by it's UUID.
// Returns OBJECT_INVALID if the UUID is not on the server.
object GetObjectByUUID(string sUUID);
Neverwinter Nights: Enhanced Edition - Julius Borisov
Hello, friends!
We’ve been substantially updating the Neverwinter Nights Toolset, and want you to test our newest build which adds tons of fixes, optimizations, & new features!
Grab the new Neverwinter Nights: Enhanced Edition Toolset build at this link, test it out, and share your feedback!
We've uploaded an out-of-band update for the Toolset so that content creators could test it without having to wait for a patch (which will take a while with the work on the renderer and 64-bit integration).
We're shipping a quick pre-holidays development patch for Neverwinter Nights: Enhanced Edition. We're at 8187 now!
Changes
We ported keyholing from Android. It is turned ON by default, but can be turned off with a console command (KeyholeToggle) or the setting in nwn.ini. A UI setting will be added before the next stable patch ships.
Shaders now support #include. Use with care: It's not a real preprocessor, just a verbatim string replace.
We're changing how networking is done in NWN:EE, with the following benefits: Cryptographic server and client identity authentication; full network encryption to prevent traffic snooping - and in the case of public servers via the master, connection hijacking; floating connections (you might be able to IP/port-hop and retain your connection); better use of compression; and no more host/port confusion when connecting multiple times from behind the same LAN.
NWSync now cleans up downloaded manifest mappings when getting updates from known servers. It will not remove unreferenced data yet (as desktop users might not want to lose it), but that will come next.
NWSync will automatically reconnect you to the server if you are still around after the download finishes (based on keyboard/mouse input detection).
Fixes
Performance and memory usage improvements at NWSync startup.
History/Favorites have been fixed to once again show what has been saved to the .ini.
We fixed the SetName and SetHiddenWhenEquipped script commands not reliably updating all clients.
We fixed some doors becoming unclickable after having been destroyed, despite having a scripted transition present.
We fixed the script compiler sometimes confusing functions where the full name of one is a prefix of the other ("Action" <-> "ActionTwo").
We fixed module content (hak, tlk, nwsync) not properly unloading when disconnecting from a server.
We fixed a bug where resources were indexed twice when loading a module, resulting in prolonged load/connect times.
We fixed an issue where sending visual transforms or material updates for creatures only seen on the party bar, but not yet in an area, crashed clients.
Known Issues
Relay network functionality is not yet functional for this new patch. This will not impact connectivity, unless both server and client are behind very restrictive NAT.
We'd like to wish you all a very happy new year! Level up responsibly!
Today’s the day for Android Neverwinter Nights fans! Neverwinter Nights: Enhanced Edition has just launched on Google Play and it's cross-play compatible with desktop versions!
NWN:EE on Google Play introduces true type font, new main menu art, achievements, bug fixes, features, and more which Steam and other desktop players will be seeing in future patches.
Today, we are shipping Development Build 8186 to the stable branch of Neverwinter Nights: Enhanced Edition as 1.78!
This is the very same build that was recently updated on the development branch and contains no changes since then.
The previous build (8181) is available as a Steam branch, so you can switch back if your favorite persistent world needs more time to update. To do so, select 'oldstable' under the beta dropdown list in the Neverwinter Nights: Enhanced Edition Steam properties.
New Feature: NWSync
This update contains a major new feature called NWSync, which will allow players to easily download required content data as they join servers, without having to manually manage HAK and TLK files. In this first iteration, this will mostly benefit persistent worlds, as it requires special setup on the server owner's part.
For details on this feature, please see the source repository that contains all the needed tooling and links to the most up-to-date documentation, here: https://github.com/Beamdog/nwsync.
Client Changes
Features:
We added a new console variable that enables a over-time FPS display. Type "fpstrace 1" into the game console to enable.
You can type `Gamma x` in the console, where x is a floating point value larger than 0. The default is 2.2. This value is persisted to nwn.ini (under a new configuration key). Gamma only affects the rendered ingame scenery, not menus or UI, as it is applied through a FBO shader, not GL/system-wide gamma correction.
The game now supports normal DDS textures without having to mangle the header.
The game now supports KTX texture containers.
The game now writes out all Aliases to nwn.ini even if the file already existed.
We bumped the default maximum texture memory to 256MB. Previous settings of 64MB are rewritten to this value. The new minimum is 96MB. The bigger-by-default texture memory should improve stability in some situations.
Fixes:
We fixed lighting uniforms not assigning properly on some GPU drivers. This should address some flickering and maybe improve stability.
Premium Modules don't show in "Other Modules" anymore.
Yet another crash for creature appearances with PERSPACE=0 has been addressed.
The multiplayer server browser now shows a progress window when downloading the server list. It also switched over to libcurl, which should prove to be much more reliable than the previous solution.
The chat window will not capture mouse clicks anymore, only scroll events. This means you can once again click through it to action the game world.
Server Changes
Fixes:
Greater Sanctuary interacting incorrectly with True Seeing has been fixed.
A rare server hang when a relayed player suddenly regains direct connectivity has been fixed.
A bug in RetrieveCampaignObject has been fixed that resulted in a double free issue, crashing the game. This was another 1.69 issue that was unearthed by us switching to VS2017 for the Windows build.
Same week, new patch! 8186 contains another update preparing for the next stable release.
We'd like to thank the online PW community for their help in testing NWSync. Your feedback has been incredibly helpful in making NWSync a reality!
With the Android beta now released, we're looking to keep feature parity by pushing a new release on both platforms; leading to a stable release for desktop that is cross-play compatible with Android.
Changes
You can now configure in nwn.ini the number of concurrent connections used to download NWSync data from server repositories. The default has been bumped to 20 (up from 5). This increases performance significantly.
The major version has been bumped to 1.78 in preparation for releasing this as stable. Unless major issues crop up, this is the next stable build.
NWSync documentation is being rewritten to be easier to read and include more needed detail.
New week, new patch. 8185 contains stabilization fixes for a upcoming stable release.
As always, we appreciate your support in testing development patches. With the upcoming Android beta, we are looking to keep feature parity by pushing a stable release soon.
Fixes
The chat window will not capture mouse clicks anymore, only scroll events. This means you can once again click through it to action the game world.
Greater Sanctuary interacting incorrectly with True Seeing has been fixed.
A rare server hang when a relayed player suddenly regains direct connectivity has been fixed.
A bug in RetrieveCampaignObject has been fixed that resulted in a double free issue, crashing the game. This was another 1.69 issue that was unearthed by us switching to VS2017 for the Windows build.
NWSync has been sped up by avoiding yet more memory copying.
Minor shader update to remove light specularity components, as they are handled by the light diffuse uniforms.
Upcoming/Errata
We have not yet been able to work on documentation and use-cases with HOWTOs. This has not been forgotten and will be done.
Greater Sanctuary may not sync properly to clients after they re/connect.
There is a known issue where static tile lighting sometimes does not update in the toolset, and even rarer, in the game. We are looking into this as we work on the renderer.
The NWSync client part is still lacking a way to remove manifest data or expire old manifests in a more fine-grained fashion.
This patch refactors the NWSync client-side storage concept in a major way. Data is now sharded out to multiple databases, to support Android file systems on MicroSD cards, where files cannot grow larger than 2GB, give or take.
As part of this refactor, the game will both save and load faster as de/compression, reading and writing can be done in parallel (on multi-core systems with fast SSDs at least). Additionally, this change will make it much easier on the system to vacuum deleted data in the background, and free up disk space.
Unfortunately, part of this change means that previously-downloaded data will be left sitting in your userhome/nwsync/ directory, as it is not migrated automatically. You can delete the file "nwsync.sqlite3" if you still have it from testing previous dev patches. To backfill this data, a re-download will be required.
Features
NWSync client-side storage is sharded out to many databases in a round-robin fashion, as described above.
We bumped the default maximum texture memory to 256MB. Previous settings of 64MB are rewritten to this value. The new minimum is 96MB. The bigger-by-default texture memory should improve stability in some situations.
Fixes
We fixed a regression we introduced (when porting platform-specific code to C++14) where CodeBase database files ("Bioware Campaign Database") were written out corrupted on Linux and Mac, resulting in crashes. Unfortunately, the only way to revert from this is to manually delete the affected database files (in userhome/database/).
The base shaders have been fixed to once again correctly render UI elements on older/obscure hardware.
Upcoming/Errata
There was no time yet to address the question of documentation and use-cases with HOWTOs. This has not been forgotten, and will be done before the next patch.
Specifically, things left to do before NWSync can hit the Stable branch include (Warning, technical stuff!): Handling out of space errors gracefully; Improvements to the serverside repository format and download process to use fewer, bigger files and range requests; Potentially automatically expiring and removing deprecated persistent world data when their data set updates; Providing tools for players to manually remove data inside NWSync they don't want anymore.
This is patch 8183 for the Development branch of NWN:EE!
This patch is a quick iteration on the previous Development patch. Build 8182 introduced NWSync, a feature allowing clients to download needed gameplay data from servers without having to manually hunt down HAK and TLK files.
Fixes
We fixed a performance issue indexing/reading NWSync manifests on Windows. Data should now load at least as fast as it would with HAKs.
Fix-tures
Gamma functionality is back: You can type `Gamma x` in the console, where x is a floating point value larger than 0. The default is 2.2. This value is persisted to nwn.ini (under a new configuration key). Gamma only affects the rendered ingame scenery, not menus or UI, as it is applied through a FBO shader, not GL/system-wide gamma correction. There is no slider yet, sorry.
NWSync memory management was refactored to be as zero-copy as possible, eventually paving the way for streaming compression (to deliver, e.g., full movies). Right now, the suggested optimal file size for individually synced files is under 20MB.
All shaders have seen revision to be more efficient and future-proofed for extensions like Roughness and Glow Maps.
The game now writes out all Aliases to nwn.ini even if the file already existed.
Features
NWSync now has an Alias entry in nwn.ini, which allows moving data to a more suitable place if so desired.
Downloading the initial manifest now shows progress, instead of letting the user guess. This is relevant for big manifests, which may grow to megabytes in size for servers using the CEP and more.
The multiplayer server browser now shows a progress window when downloading the server list. It also switched over to libcurl, which should prove to be much more reliable than the previous solution.
Coming Soon, and Known Issues
There will be a utility to introspect and prune the client-side storage.
There will be a utility to (pre-)download manifest/server data without the game running.
We're aware the NWSync documentation is very bare bones right now, and aimed at the technical user. It will see a lot of improvement before a stable release, including example use-cases and HOWTOs for each. Current work-in-progress documentation can be found here: https://docs.google.com/document/d/1eYRTd6vzk7OrLpr2zlwnUk7mgUsyiZzLoR6k54njBVI.
Server-side repository management utilities are exhibiting some issues on Windows. We're looking into the problem
We'd like to thank all the volunteers who have spent time testing the previous patch! We're not quite at the required stable feature list yet as far as NWSync goes, but we're getting there! Any testing, feedback, and criticism you provide is invaluable.
NWN:EE Development Update 8182 is being rolled out.
This patch contains a major new feature: NWSync, a mechanism to transparently and automatically download user content when joining a multiplayer server.
The first development release now exists to serve as an invitation to test for the community.
Features
The game now supports normal DDS textures without having to mangle the header.
The game now supports KTX texture containers.
We added a new console variable that enables a over-time FPS display. Type "fpstrace 1" into the game console to enable.
Optimisation
We added a (off-by-default) experimental toggle that prevents needless texture switching. It should yield about 10-20% more FPS depending on the scene rendered. You can toggle it, to see the difference, via the console variable glCacheTextureBinds 0/1.
Fixes
We fixed lighting uniforms not assigning properly on some GPU drivers. This should address some flickering and maybe improve stability.
Premium Modules don't show in "Other Modules" anymore.
Yet another crash for creature appearances with PERSPACE=0 has been addressed.
NWSync
We merged in experimental support for NWSync. The first implementation of this allows transparent multiplayer server content downloads, and requires special setup by the server admin to make functional.
Note that if you are from The Future, then this URL might have changed or disappeared. Please check the official documentation in lang/en/docs instead.
Again, this is experimental, and we are shipping this to the development branch early so persistent world admins and interested parties can test it. This is a very complex technical feature and we fully expect that we will have to ship more development patches to address found issues.
Ultimately, this feature can only grow into what it should be with feedback and guidance from those that will be heavily employing it.