Hello! Sorry for small build again, had a busy day, but I want to push out some more tweaks and bugfixes for the new screen mode before we push out the weekly update. The MTC strings were updated with new bindings and some other bugs fixed. You can now also toggle crouch with C!

Oh I also fixed the reporting of current mode, everyone showed as VR user in the last build! ^^; Now the reported mode will dynamically change as you toggle between VR and desktop.

I'll have more substantial stuff soon, but hopefully this helps alievate a few problems in the meanwhile!

- Added "On Saving" node (under Flow/Events) which fires before the saving process occurs (requested by @3x1t_5tyl3)
- You can now double-tap C/LeftBumper to toggle crouch (holding C without double tapping it quickly will still crouch only for the duration of holding it)

- Impulse Timeout node is now called "Local Impulse Timeout" to avoid some confusion over its function (based on report by @Toxic_Cookie)
- Small optimizations when searching in the Contacts list
- OrderOffset property is now public in the API (requested by @Toxic_Cookie)

- Merged new English locale strings for the MTC badges and updated desktop control strings by @Rue Shejn | Artist 3D

- Merged Spanish locale additions and tweaks (including MTC Avatar Lobby) by @Ruzert
- Merged Russian locale additions and tweaks by @Shadow Panther [RU/EN, UTC+3]

- Scaling grabbed objects in desktop mode (or more generally through direct scaling input) now respects the Scalable property (reported by @epicEaston197 and @Shifty | Quality Control Lead)
- Fixed Tween<T> not validating generic type arguments, allowing creation of invalid versions, resulting in a world crash (reported by @dfgHiatus)
- Fixed reporting of VR/Screen user online status reporting VR for users who are currently in the screen mode (based on report by @Aegis_Wolf | Art Director)
-- The reporting is now realtime, meaning even when running in VR and switching to Screen mode will report that user as using screen mode
- Fixed TextEditor inserting control characters into the target string in some cases, causing issues
-- This fixes ESC ASCII character being inserted when Esc key is pressed, which would cause contacts to disappear in the Contacts list among other things (reported by @Rue Shejn | Artist 3D)
- Fixed full body tracking being active when toggled to the Desktop mode and affecting desktop hand positioning (reported by @AshtonSparx)
-- This is done by extending the internal system to track whether given devices should be active. Tracker devices can now be mapped to track in VR, Screen or both (full body trackers are currently mapped only for VR at the moment)
- Switching to another tool in desktop mode with the alphanumeric keys will now preserve assets when destroying any previous equipped tooltip (reported by @ohzee)

Hello everyone! Sorry for the very small build, I was planning to do more today, but things unfortunatelly got in the way. I'm pushing at least a bunch of tweaks and fixes for the new desktop mode and few other things so they don't wait too long. I'll have more soon!

- Updated the new Desktop Controls graphic on the dash (it's now automatically activated when user toggles from VR to Desktop mode as well) (graphic by @Coffee)
- AvatarExpressionDriver now resets all the expressions to default state when the data source is removed
-- This fixes avatar's face becoming stuck in the last known state on dequip (reported by @3x1t_5tyl3, @Reactant, @Epicgames1, @Beaned)
- Camera Dialog is now accessible on the desktop mode
-- Note that you still can't use the camera mirroring right now, the UI needs to be reworked for this first

- Merged Korean locale tweaks by @MirPASEC

- Fixed touchable search not filtering raycasts properly when the interaction laser is maintaining distance (e.g. holding an object)
-- This fixes not being able to assign texture, material and other references from held items in the desktop mode (reported by @Turk)
-- This will also fix any odd interactions in VR - not being able to assign held references in some cases due to tooltip not being filtered properly, or assigning/clicking through raycast portals
- Fixed tooltip quickswitching working while the keyboard inputs are blocked (e.g. by active text field) (reported by @ChrisWarner103 (DelVR), @AnythingItDoesn'tReallyMatter and @Turk)
- Fixed Camera rendering to user's screen when the postprocessing is disabled (reported by @Sox, @Gourry, @Turk, @Epsilion, @Hayden)
- Fixed Draggables (Slider, Joint...) and other grabbables that don't parent the object not being able to be freeform rotated (reported by @Blaze and @H3BO3)

Hello everyone! Sorry for another smaller build, but this one has pretty significant change - the new desktop mode is now default and usable without VR!

I did another pass on the keybinds as well, they have been changed quite a bit and should be closer to what fits with Neos the most. I'm sorry that it's going to mess up with your muscle memory (it does with mine too), but I wanted to make sure we have a good set for what Neos is, rather than haphazardly chosen ones.

Notably the locomotion ones were changed, which are now similar to Minecraft - Shift for "sneak" (slow movement) and double tap directional keys (WASD) to run. Using Space (jump) and C (crouch) in Fly/Noclip now also moves you up and down.

There's still lots of work and tweaks to do on the desktop mode, this doesn't mean it's quite finished yet, but I felt it's enough for the swap. The old one is still available as "Legacy Screen" for the time being if you need/want to use it (but prepare for it going poof at some point in the future).

Some work went into cleanup of initialization and postprocessing issues too, I managed to find reason why the streamer camera and in-game ones didn't have a working motion blur effect, so it's finally back! Also some other bugfixes, notably for the audio stopping with Unity Native video playback when switching URL, sorry for the trouble!

- The new desktop mode is now the default mode!
-- Launching Neos with "Screen" or AutoDetect without VR device will use the new desktop mode
-- Old debugging mode is available as "Legacy Screen" -LegacyScreen for the time being, but remains officially unsupported
-- Please note that this doesn't mean that the new desktop mode is finished, there's still a lot that will come to it
-- Note that we're also still updating associated materials like the tutorials and descriptions

- Neos now uses VSync by default in the new desktop mode, preventing running at unconstrainted framerate and wasting GPU performance (prevously requested by @かず (kazu / GitHub: kazu0617), @Shadow Panther [RU/EN, UTC+3] and others)

- Added MinInputCloseness and MaxInputCloseness to EyeLinearDriver
-- These allow adjusting the range of input values that the eye closing responds to. This can be used to tweak the range so the eyes stay fully open even when partially closed (e.g. due to eye tracking issues, as reported by @Rukio)

- Shift key is now used to slow down movement
- You can run by double tapping any of the movement keys (WASD)
- Space & C on keyboard and A/LeftBumper on gamepad (jump & crouch) are now also used to move up and down in Fly/NoClip
- Up/Down movement is now relative to the user's space, rather than the head (e.g. looking straight down and holding Space will move you upwards, instead of forward)
- Desktop Interaction Relay now ignores interactions when in the new desktop mode
- The LOD bias is now automatically switched when toggling between VR and Desktop mode
- Save To Inventory and Save Screenshot items now have distinct icons and colors

- Merged rewording of ban buttons in the Contacts menu in the English locale by @Polaris (she/her)
- Merged Korean locale addition by @MirPASEC
- Merged Japanese locale addition by @Aesc
- Merged Czech locale addition and tweaks by @rampa_3 (UTC +1, DST UTC +2)

- Fixed Motion Blur not working on non-VR cameras or in Desktop mode when VR is running
- Fixed Ambient Occlusion trying to treat non-VR cameras as VR when VR is running, resulting in incorrect behavior
- Fixed VideoTextureProvider stopping audio playback after the URL has changed or when the video loops (reported by @Rukio, @Turk, @Alex from Alaska)
- VRIK now respects the Enabled property even when updated externally (e.g. from VRIKAvatar)
-- This fixes regression where some behaviors that depend on disabling the IK would no longer work (reported by @Dante and @Turk)

Hello, sorry for a smaller build this time, I just want to push out some fixes and tweaks rather than waiting on some bigger things to be finished first, particularly some regressions for the facial tracking blendshape auto-detection.

There's a few other smaller additions too, for example you can adjust the volume of people outside of your whisper bubble now.

Some smaller improvements and progress to the new desktop mode as well, with key bindings and legacy dialogs now working properly. I'll have more soon!

- Added "Whisper Voice Volume (outside bubble)" setting, which allows configuring the voice volume of users outside the whisper bubble (requested by @Robyn (QueenHidi))
- Added "StrengthMultiplier" to the AvatarRawMouthData, allowing scaling the strength of all the properties

- Added TongueRaise and TongueExtend expressions to the AvatarExpressionManager
-- Those are useful when there are two separate blendshapes for raising the tongue and then extending it out
-- The heuristics was updated to map the TongueLongStep1 and TongueLongStep2 to these two, resulting in better behavior for models following the HTC Facial Tracker naming convention

- AvatarExpressionManager heuristics now checks for the exact expression naming of the blendshapes first, before using heuristics (based on feedback by @GearBell)
-- Note that for this to work, you must name the blendshapes exactly the same as the expression names you see in the inspector, without any extra spaces, periods or underscores. It is however case insensitive

- Keyboard bindings that use Ctrl, Shift and Alt keys will now also accept the right Ctrl/Shift/Alt keys (based on requested by @Psychpsyo)
- Legacy dialogs (e.g. Hyperlink warning, Create New Directory, Credits Transfer and so on) will now open in the overlay in the new desktop mode so they can be easily interacted with
-- Note that you cannot grab and move them around, only click to interact. This might not be implemented before they're replaced with more modern variants
- Any focused elements (e.g. text fields) will now block keyboard interactions in userspace
-- This fixes the dash opening/closing when using keyboard to type (based on report by @Shadow Panther [RU/EN, UTC+3])
- Text fields will now unfocus when the Escape key is pressed

- Merged Russian locale additions (including MTC Avatar Lobby) by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese locale addition by @Aesc
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)
- Merged Korean locale addition by @MirPASEC

- Fixed regressions in AvatarExpressionDriver, BipedRig and other naming heuristics, causing some blendshapes/bones and/or their chirality to not be detected properly
-- This fixes the HTC blendshape naming convention not automatically setting up all the names anymore
-- If you setup your avatar recently, it's recommended to rerun the auto-assignment
- Fixed headless not measuring auto restart and auto save intervals properly, using a mix of UTC and local time, resulting in incorrect intervals (reported by @Shadow Panther [RU/EN, UTC+3])
- Fixed regression where it wasn't possible to interact with touchables with the same hand that's holding an object with the laser
-- This fixes not being able to invoke the Overwrite menu for world orbs
-- It also fixes not being able to assign references/values to the inspector fields by clicking (reported by @Turk)
- Fixed being able to use worlds that you don't have write permissions for to overwrite your own worlds with (reported by @sirkitree)
- Fixed Unity Native video playback returning audio data when paused
-- This should fix VolumeMeter reporting volume when the source video is paused (reported by @Toxic_Cookie)
-- It should also fix AudioSourceWaveformMesh showing undesirable visuals (reported by @AshtonSparx)

A whole bunch of new goodies for the new desktop mode! You can now rotate items (GMod style!), scale them, scale yourself, take in-game photos (even with timer), save items to inventory and quick switch common tooltips. I've also changed a bunch of the existing bindings too (e.g. the Dash is now opened/closed simply by pressing Esc) and there's a whole bunch of other tweaks, improvements and bugfixes to make the new desktop mode more usable.

Currently it is still usable only by pressing F8 after launching in VR, but I'll be swapping it out within next few days, after adding a few more crucial things. It's coming along!

Also added a bunch of bugfixes to this one (including broken Reflection Probes on Linux, which turned out to be just a stupid typo in a flag when uploading HDR textures to GPU, sorry!) and some regressions (sorry for the inconvenience!). More to come soon!

- Added "Save To Inventory" context menu option, which will save the held object to the current inventory folder
-- Note that you currently need to open inventory folder where you have write permissions first. This will be simplified with the upcoming inventory UI redesign
-- This also allows easily saving items in the new desktop mode

- Added an internal mechanism to load-equip and stash tooltips from an URL
-- Tooltips and other equippable items are now internally tagged as equippable when saved to inventory, which will allow future UI to allow one-click equip directly from inventory
-- When switching tooltips, if the current one is quick loaded, it will be "stashed" - disabled and hidden, preserving its state on the next equip in the same session
- You can now quick switch tooltips in the new desktop mode via tha alphanumeric keys. Bindings are following (the loadout will be customizable in the future):
-- 1 - Dequip
-- 2 - Developer Tip
-- 3 - LogiX Tip
-- 4 - Material Tip
-- 5 - Shape Tip
-- 6 - Light Tip
-- 7 - Grabbable Setter Tip
-- 8 - Character Collider Setter Tip
-- 9 - Microphone
-- 0 - Glue Tip

- You can now capture in-game photos in the new desktop mode
-- To capture instant photo, press Shift+P or Shift+PrintScreen
-- To start timer photo, press Ctrl+P or Ctrl+PrintScreen
-- The capture won't start until you release the combination, allowing you to hold it to see a preview

- You can now rotate held items in the new desktop mode
-- Hold the E key while holding an item and move the mouse around to rotate object around X and Y axes
-- Hold the Shift+E key to rotate only around the vertical axis
- You can now scale held items in the new desktop mode
-- Hold the Shift key while holding an item and use the mouse scroll wheel to scale it

- Added a mechanism to block inputs within InputGroup by specifying a priority level for each binding, simplifying setup of common control schemes (e.g. Ctrl+<Key>, Shift+<Key> and <Key> as modifiers)
- Reworked FingerPhotoGesture into a more general PhotoCaptureManager

- Dash is opened/closed by the Escape key now in the new desktop mode
- The locomotion switch and scaling options are now shown on the context menu even with equipped tooltip
- Keyboard & Mouse input is now fully supressed while the dash is open in the desktop mode
- Lowered the limit of how close you can bring grabbed items to your viewpoint from 10 cm to 5 cm in the new desktop mode
- Increased vertical mouse look angle limit to 89 degrees from 80 to allow looking near straight up and straight down
- You can now use PageUp and PageDown to quickly cycle between active locomotion modules
-- This will have a visual at some later point, not too usable at this point just yet, but it's been added as part of larger work
- You can now self-scale in new desktop mode by holding Ctrl and using the Mouse Scroll button
-- Scaling still needs to be enabled first in the context menu
- Tweaked finger photo gesture supression level, to be more tolerant of bent fingers
- Developer Tooltip laser now considers Gizmo controls as preferred targets and will pass through other objects and stick to them (based on report by @H3BO3)
- Undo & Redo context menu options are not generated when holding items with the active hand

- Merged Japanese locale additions for the new MTC Avatar Lobby by @Aesc
- Merged Czech locale additions for the new MTC Avatar Lobby by @rampa_3 (UTC +1, DST UTC +2)
- Merged Korean locale tweaks by @MirPASEC

- Context menu positioning in the new desktop mode now properly handles user's scale
- Interaction Laser now consider user's current scale when ignoring near raycast hits
-- This fixes the context menu not being usable when the user scale is too small in the new desktop mode
- Fixed SyncType inspector field not accepting any type strings after recent changes (reported by @3x1t_5tyl3)
- Fixed incorrect flags being set for BC6H textures when uploading them to the GPU through OpenGL, resulting in the texture data to be interpreted incorrectly on Linux and Android
-- This fixes ReflectionProbes being overly bright on Linux (@Shifty | Quality Control Lead, @happysmash27, @Polychrome, Silvea12, Enverex and @Develon)
- Fixed favorite avatar, keyboard, camera and home being unfavorited on manual log out (reported by @Flame Soulis and @H3BO3)
- Fixed gamepad mouse look being framerate dependent (reported by @Lewis Snow | Lead Audio Engineer)
- Fixed regression breaking the context menu flick functionality (reported by @Flame Soulis, @DeliriousJax, @ohzee, @H3BO3 and @Turk)
- Fixed regression when aligning laser-held objects to the vertical axis having a small offset from the aligned position, depending on the hand movement speed (reported by @orange)

An extra hotfix to fix some more issues that were found. Compatible with previous build, but update recommended for headless, since it mainly concerns bugs manifesting on those.

- Merged Korean locale translations by @MirPASEC (including new MTC Avatar Lobby)

- Fixed Neos trying to use VR bindings against VirtualController instead of VR one in some cases, causing the user's session to crash
-- This should fix some session crashes when joining worlds or switching locomotion in desktop, particularly with Oculus runtime (reported @epicEaston197, @Shifty | Quality Control Lead, @/home/JellyOsaurusPC)
- Fixed exceptions due to updating InputGroups for owners that have been removed in the input update cycle
- Fixed grabber transform computations using local user's space, instead of the user actually owning the grabber
-- This also fixes exceptions on the headless for the CommonTool, due to not having a user root
- Fixed VRIKAvatar trying to calculate distance to user's head when the head position is not available (e.g. during respawn or on headless)
-- This fixes IK avatars stopping to update when the user exits them on a headless session

Hello everyone, sorry it took a while to get this build ready, but it presents a major milestone in the develpment of the new desktop mode! You can now fully use Context Menus and Equippable items (Tools, Weapons, Gadgets...), making the mode fully usable for core interactions! With those in place, I'll start progress on making this the default desktop mode now. I'll keep the old one as legacy for a bit while it still gets more polish. Right now you still need to startup in VR and press F8 to activate it, but hopefully it'll prove very useful already!

There are some semi-major things that changed/expanded in this internally, the input handling has been restructures quite a bit, which reduced latency of some input handling by a frame (making things feel more responsive), but watch out for potential side effects. Thanks to the following danger testers for catching some game breaking bugs before this went live: @/home/JellyOsaurusPC, @Kulza, @Raith (CytraX), @Alex the pet peeve avali 🐦, @Epsilion, @Gawdl3y and @Shadow Panther [RU/EN, UTC+3]

This build has a whole bunch of tweaks, security improvements and fixes too, so hopefully that'll solve some of the pains and problems that you've been dealing with, sorry that those fixes got held up a while! I hope to have more ready soon, probably tomorrow. I'll probably have to release the weekly update tomorrow too instead of today, it took a bit longer than expected to get this ready and not sure how much longer I'll be able to stay up and focus.

But in the meanwhile, have fun! There's a lot more to come to desktop, but with this we should have all the necessary basics.

- Context Menus are now usable in the new desktop mode
-- Press the Middle mouse button or the "Y" button on gamepad to open/close them
-- They will also open when clicking on certain objects (e.g. Avatars or Tooltips/Equippable items)
-- When Context menu is open, the cursor will switch to a freeform one, allowing to easily select and click on the items without having to aim head at them

- Tools, Gadges and other equippable items are now usable in the new desktop mode
-- When equipped, the avatar's hand is positioned relative to the head so the item is visible in the first person mode
--- The positioning is based on the grip pose. If it's not setup properly, the item will be misaligned
-- The tool/item/gadget is automatically aimed at the point in world currently under the cursor reticle
--- For this to work correctly, the item needs to have a properly setup TipReference, that corresponds to the exact tip and direction that the tool should aim at (using Z axis as forward)
--- Existing RawDataTooltip items are auto-upgraded using heuristics, using sources of raycast or particles. You might need to adjust the TipReference on some manually if the detection doesn't work properly
-- Current bindings are (these are tentative and will likely change)
--- Primary - Left Mouse Button / Right Trigger (gamepad)
--- Secondary - R key / Mouse Button 4 (is present) / X button (gamepad)

- RawDataTooltip & StandardController now provide proper inputs in the desktop mode, making items that rely on those work
-- The values are fed through a VirtualController type from the current input bindings of corresponding actions

- Pointer interaction system now routes the first and second pointer (mouse cursor and/or touch) through the hand interaction system
-- This results in the intereractions posing the avatar's primary and secondary hand and being able to grab and scale with toushcreen
- Neos now renders its own cursor even in freeform cursor mode to provide visual consistency and contextual behavior and avoid the system cursor to be overlaid on top of the in-game reticle
- InputDeviceStreamDriver now disables streams for input devices that are currently inactive, saving network bandwidth
-- E.g. VR controllers when Screen is active and VirtualControllers when VR is active

- You can now self-assign the hearing impaired, color blind, mute and potato badges
-- To assign or remove, simply send following commands to the Neos bot account in your Contact list:
-- /addHearingImpaired, /removeHearingImpaired, /addColorBlind, /removeColorBlind, /addMute, /removeMute, /addPotato, /removePotato

- Restructured update order of the input handling components, avatar posers, VRIK and CommonTool
-- This reduces latency of input response by a frame (e.g. making the laser/cursor respond in the same frame that the input happened, rather than lagging a frame later)
-- It also allows for automatic interactions based on a laser hit (e.g. based on a screen touch that happens instantly)
- Desktop mode grabbing now uses user's head forward direction as rotation reference instead of the hand
-- This reduces slight rotation of the grabbed object upon grabbing due to the hand being procedurally moved
- Cleaned up and tweaked several user related components for driving the root objects based on inputs, tracking compensation and so on
-- Many of the components now implicitly work based on the user they're initialized on, rather than explicitly assigned
-- Where possible, the components only register events for the user actually using them, rather than for everyone
- Improved parsing of URL's when classifying asset types to search the URL query for a file extension in addition to the main data path
-- This makes certain types of URL to get detected properly and imported directly from the clipboard (e.g. IPFS file URL's, as reported by @Komdog)
-- E.g. https://example.com/somepath?file=MyFile.mp4&someparam=42 will now properly import as a video
- Near head hand gestures/shortcuts are no longer triggered in the desktop mode
- Saving world with Ctrl+S is now supressed when the new desktop mode is active (based on report by @Cyro)
- Crouch is now on the Left Bumper on gamepad

- Added new English MTC Avatar Lobby locale strings by @Ryuvi | Technical Artist and @Aegis_Wolf | Art Director

- Merged Estonian locale additions and fixes by @Meron
- Merged Russian locale addition & tweak by @Shadow Panther [RU/EN, UTC+3]
- Merged Korean locale addition by @MirPASEC
- Merged Japanese locale addition by @Aesc
- Merged Esperanto and Chinese locale additions and tweaks by @Melnus
- Merged Czech locale addition by @rampa_3 (UTC +1, DST UTC +2)
- Merged English store page fixes by @rampa_3 (UTC +1, DST UTC +2) and Enverex (will take a bit before it's pushed to store)
- Merged British English addition by Enverex

- Fixed SimpleAvatarProtection throwing an exception when checked against user who has been destroyed (based on report by @3x1t_5tyl3 and @marsmaantje)
- Fixed CommonAvatarBuilder not cleaning up user's default avatar if it fails to be equipped for any reason (based on report by @3x1t_5tyl3 and @marsmaantje)
- Improved security of touch sources (based on report & sample by @Psychpsyo and @Epsilion)
- Improved sanitization of held items being transferred through the userspace (based on report & sample by @Psychpsyo and @Epsilion)
- Reworked mechanism for cleaning up Userspace-only components, so placing them on protected Slots (e.g. Root slot) doesn't cause exceptions and failure to clean them up (based on report by @Psychpsyo)
- Fixed usernames not being sanitized in the World UI and on the world orbs, causing any RTF tags in the username to be parsed (based on report by @Kodey and @Psychpsyo)

- Fixed Neos locking the cursor when running in the VR mode (reported by @Hayden)
- Fixed headless running some actions that should only be ran by user under who is given behavior parented
- Fixed BeforeInputUpdate event being potentially executed twice on some components
- Fixed SmirkLeft and SmirkRight expressions breaking the AvatarExpressionDriver when activated (reported by @Alex the pet peeve avali 🐦)
- Fixed ContextMenu collider being enabled until it's first used
- Fixed some odd interactions when using the new desktop mode with the old one
- Fixed ScreenController update order being same as TrackedDevicePositioner, causing the simulated hand to freak out in some cases (where it ends up initialized after the TrackedDevicePositioner)
- Fixed ValueField<T> and other generic components that handle value types allowing certain non-value types (e.g. System.Type) and causing the session to crash (reported by @dfgHiatus, @Epsilion, @Psychpsyo and @Zyzyl)
- Fixed component exception handling misbehaving when placed on a destroy-protected slot (based on feedback by @Psychpsyo)
- Fixed Neos crashing when running on devices without a mouse present (e.g. Android) (based on report by @かず (kazu / GitHub: kazu0617))
- Fixed reliable user positioning breaking and throwing exception when the user is deleted in the middle of the process, causing other processes to break (based on report by @3x1t_5tyl3 and @marsmaantje)
- Added generic type validation to more components to prevent crashes when creating them with invalid argument (based on findings by @Zyzyl and @Banane9)
- Fixed some World event behaviors (e.g. focus change, on save, user join or leave) breaking due to not setting up proper update scope (e.g. when fields are driven)
- Added Type validation for setting SyncType and related operations, to avoid setting/processing of invalid types (e.g. type with invalid generic argument) and causing the session to crash (reported by @Psychpsyo)
- Fixed LipMoveLeftLower expression on AvatarExpressionDriver driving the target with movement to right, instead of left (reported by @umbran)
- Fixed nameplates using the IsPresentInHeadset state instead of IsPresent, causing them to become semi-transparent when the user switches to the desktop mode after existing their headset (reported by @Shifty | Quality Control Lead)
- Fixed Userspace having two instances of PointerInteractionController, resulting in cursor and touch interactions in userspace being processed twice
Known Issues:
- When interacting with context menu on gamepad, the camera view will move if you don't let go before the menu disappears
- Some tooltips are hard to use without additional behaviors (e.g. LogiX tooltip)
- Currently key bindings are tenative and will likely be tweaked as more behaviors are added

Hello everyone! I wanted to do a bunch more work on the new desktop before releasing this build yet, but there's an important bugfix that heavily improved the cloud API reliability (this should fix sync errors and other problems due to interminnent network problems) as well as some important improvements for the face tracking!

The heuristics got expanded and improved, adding support for more avatars. Autodesk Character Creator avatars should now get setup with lip tracking (not 100% support either, but a bunch of blendshapes will be auto-mapped). Also there's now a mechanism where the voice data supresses certain blendshapes to avoid overdriving the mouth when talking with the Vive Facial Tracker.

There's a bunch of underlying work on the interaction system for the new desktop mode as well. Not that much visible yet, but touchscreen now has official initial support! Still more work to do, but you can play with it a bit if you'd like! I've got more coming soon!

- Added Timestamp to AvatarRawEyeData, which is relative time in seconds of the current snapshot of the eye tracking data (requested by @SHFR_H)
-- This allows calculating things like the current angular velocity of the eye movement with much better accuracy
- Added missing "Squeeze" and "Frown" properties to AvatarRawEyeData (based on report by @Ryuvi | Technical Artist)
-- Note that it's strongly recommended to use EyeManager, rather than this component unless you're building specialized application, as EyeManager is significantly more efficient and modular

- Added "Tongue Up Left", "Tongue Up Right", "Tongue Down Left" and "Tongue Down Right" expressions to the AvatarExpressionDriver (based on request by @umbran)
-- These drive based on the combination of both directions. Typically used for corrective deformation blendshapes
-- The heuristics has been updated as well to auto-assign those blendshapes (you will need to re-run the heuristics to assign missing blendshapes on already setup models)
- Added SmileClosed(Left/Right) expression to AvatarExpression driver
-- This will drive a smile that's only active when the lips are covering the teeth and will go to 0 when the teeth are visible
-- This can be used when there are separate smiling blendshapes for when the teeth are visible and when not
- Added VolumeSource and SilenceSource to AvatarExpressionDriver, which allow supressing the face expressions when the user is talking
-- VolumeSource is checked first, then SilenceSource (inverse of the first) when calculating supression
-- Each Expression now has "VolumeSupressionStrength", indicating how much is given expression supressed by volume (1.0 full supression, 0.0 no supression)
-- Existing instances are auto-initialized using VisemeAnalyzer on the avatar default supression weights are setup
-- By default, only jaw open and lips raising and pouting expressions are supressed. This should prevent the mouth opening too much when talking

- Added initial multi-touch support - the new desktop mode now properly handles touch screen inputs and multiple pointers for interacting with the UI
-- While touch inputs are active, the mouse input is supressed (this is to avoid double interactions due to Windows simulating mouse inputs from the primary touch)
-- Note that it doesn't go through the laser interaction system yet, there's more work that needs to be done on this
- You can now adjust the FOV (field of view) of the new desktop mode in the settings

- The FOV increases slightly in new desktop mode when running with the physical locomotion
- Extended the heuristics of AvatarExpressionDriver to provide better support for automatically assigning Autodesk Character Creator models
-- Note that you might need to tweak strengh of some of the blendshapes, like smiling
- Improved blendshape/bone name splitting to support naming conventions that start with a chirality letter (e.g. "RlipDown", which would previously split to "Rlip" and "Down")
-- This heaivly improves auto-detection of blendshapes on the Autodesk Character Creator models
- AvatarExpressionDriver no longer needs "shape" name for the "Ape Shape" (Jaw Down) name heuristic (based on model by @GearBell)
-- E.g. "mouth_ape" is now sufficient for the blendshape to be mapped, instead of needing "mouth_ape_shape"
- Preset names when starting worlds on the headless are not case-insensitive (based on report by @Epsilion)

- Merged Korean locale additions by @MirPASEC
- Merged Russian locale additions and tweaks by @Shadow Panther [RU/EN, UTC+3]
- Merged Japanese, Esperanto and Chinese locale additions by @Melnus
- Merged Czech locale additions by @rampa_3 (UTC +1, DST UTC +2)

- Fixed Neos not retrying Cloud API requests when the server returns 429 or 500 codes
-- This should greatly increase the robustness of any cloud based functionality under load or interminnent network errors, fixing cases where different processes would get stuck
-- This should also significantly reduce number of sync errors due to network issues (e.g. connection briefly dropping) as reported by @PlasmaRaven, @Elektrospy, @Honeypotbutt and @Shifty | Quality Control Lead
- Added a mechanism to manually correct Patreon support for accounts where the amount of support stopped reporting correctly from Patreon API
-- If your Patreon account got messed up and you're no longer receiving rewards, even though the charges are going through, let us know and we'll update your account manually
- Fixed GrinLeft using the right side of the upper lip to modulate
- Fixed invite and restart headless commands throwing exceptions when no world is focused (reported by @Glitch)
- Fixed non-host users showing "Hidden Contacts Only World" as "Contacts Only World" (reported by @Shifty | Quality Control Lead and @Cyro)
- Fixed Neos adding invalid entry to the configuration file when a world is started with invalid template and then configuration is saved (reported by @Epsilion)

Hello everyone!

We have some exciting news this week! HTC has released the VIVE Facial Tracker to general public, giving everyone access to full realtime facial tracking inside of VR. Neos has had native support for this hardware for a while, but with the release we have added an extra level of polish and automated the avatar setup to make it easy to get started.

If you haven't seen it in action, check out the video below. Combined with eye tracking, this hardware offers an unprecedented level of natural expression on avatars (huge thanks to Rezillo Ryker and GearBell for providing us with the most expressive avatars for demonstration) and allows scripting custom interactions that are triggered purely with your facial movements, like flying by blowing up your cheeks or activating fire particles on your head when baring your teeth.

To learn how to setup an avatar, whether a brand new one or existing one and even do some basic scripting, you can watch our tutorial video as well. It is split into several segments, so you can only watch the parts you're interested in!

And if you're looking to just give the face tracker a try, you can try it out on the new Neos Facebot, a fully free avatar by GearBell with full support for the face tracking working out of the box. You can find it in the “Face Tracking FaceBot Avatar World”.

We can't wait to see what you'll do with this new tech and what kind of amazing avatars and interactions you'll build!

In others news, we have made some progress on the desktop mode as well, adding support for proper aiming (a necessary step towards tools), control over FOV and even multi-touch support! You can read more about what we're working on below.

And also last, but not least, we have just crossed 1000 supporters and 15K on Patreon! We're overwhelmed with this level of support for this project. Without you, we wouldn't be able to keep working on it every day and remain independent, keeping the vision of the metaverse ours and its goals on providing the ultimate creative freedom. Thank you again everyone!



Last week, HTC released the new VIVE Facial Tracker add-on for the Vive Pro headsets to the general public, providing a real time tracking of the lips, cheeks, jaw and the tongue. Combined with the Vive Pro Eye headset, this gives you a full real time facial tracking while using VR, giving your avatar an unprecedented level of expressiveness.

Neos has supported eye tracking for almost 2 years at this point and face tracking since we got access to the devkit last year. With the release of this hardware to the public, we have applied an extra level of polish and functionality to make the use much easier.

We’re happy that this technology is now available to you, our community, as it brings the level of avatar fidelity in Neos to a whole next level, increasing the immersion for both you and other users, as your natural expressions, both voluntary and involuntary, now transfer to your virtual representation.

To demonstrate some of the possibilities, we have showcased the tracker with several different avatars (huge thanks to {LINK REMOVED}RezilloRyker and GearBell for providing us with highly expressive avatars to showcase), as well as some examples of basic scripting - triggering particles and special effects on the avatar with only facial movements and even flying by puffing up your cheeks.



To make full use of the facial tracker, you need an avatar that has the necessary blendshapes (face deformations) to visualize your facial movements. At the moment, there aren’t many avatars with a full support, but GearBell, one of our prominent community avatar creators, has created the Neos Facebot, a completely free avatar with a full set of necessary blendshapes.



If you’d like to give it a try, you can find it in the Inventory by going to Neos Essentials -> Avatars -> Face Tracking Ready. Alternatively check out the “Face Tracking FaceBot Avatar World” in the Worlds tab, where you can equip the new face bot and take it for a spin.

The avatar has some cool features built-in as well, including customization UI as well as jets and grappling hooks to play with, to showcase some of the cool avatar interactions you can have in Neos.

A huge thanks to GearBell for building this avatar to showcase this new technology!

When setting up a new avatar, you can now find a new option called “Setup Face Tracking” in the avatar creator. By checking this option, Neos will use heuristics to automatically map any blendshapes on the avatar to the tracking data coming from the face tracker.

The success rate and tracking coverage will depend heavily on the avatar. For best results, we recommend adding all the blendshapes from the sample models by HTC and following the same naming convention, to ensure that you get full use of the face tracker and maximum fidelity.

However Neos will perform even partial mapping, but some of the face tracking features will be missing with the particular avatar. For example we have added the Ready Player Me avatars to the heuristics, making them work with the new face tracker, but some of the face shapes are missing, such as the tongue movement.

We have recorded the following tutorial which showcases how to setup a brand new avatar, how to add support to existing avatars, how to customize the weights (strength) and even how to script custom behaviors based on your facial movements.



As part of the polish, we have also added a few of the missing blendshapes to the face tracking and eye tracking, such as tongue roll, tongue movement (left, right, up and down), eye squeeze and eye frown (currently doesn’t seem to be tracked with Vive Pro Eye however) and corrective tongue shapes.

We have heavily expanded the list of supported expressions on the AvatarExpressionDriver component, which serves as the primary way to drive the blendshapes on the avatar from the face tracking and added automatic assignment using heuristics (the same used by the avatar creator talked about above) to ease the setup.

Currently the heuristics are aware of the HTC sample models, Ready Player Me avatars and some general common face shapes. We’re working on adding a few more conventions, like the Autodesk Character Generator to work out of the box. If you have an avatar source that doesn’t work well with the heuristics and needs manual setup, please let us know on our GitHub with a sample model/naming convention!

The component now also allows you to tune the strength of the blendshapes as well, in case they’re too strong or weak. By setting the value to a really large number you can also channel the Garry’s Mod spirit.

We have built Neos to offer a huge amount of freedom for expression and creativity and supporting cutting edge hardware like this is part of that. Our goal is to always design systems in a highly future proof way to make sure we can adopt new hardware as it comes and make it available to you with as little effort as possible.

The face tracking is part of this as well and we’re planning to support more solutions as they come, including webcam based face tracking for the desktop mode that’s currently in development, without requiring any changes (or at least any significant ones) on your end and instead exposing them through the same set of components, like the EyeManager and AvatarExpressionDriver.

This is just the beginning for face tracking in VR and we can’t wait to see where it leads and what amazing stuff you’ll build with it yourself!


On our last livestream, we have showcased the face tracking with the VIVE Facial Tracker and the Vive Pro Eye headset

While a lot of our effort this week has been focused around the release of the face tracking, we have made some important progress on the official desktop mode as well. Currently our focus is on building out interaction systems, allowing the use of context menus and tools from the desktop mode.



This has a few major prerequisites, most of them that have been implemented already. We have expanded on the hand posing system, which now uses the actual avatar’s laser offset and is able to aim the hand exactly at a particular spot in the world, making sure the laser goes in a straight line.

https://www.youtube.com/watch?v=t02AyebR_gc

This part is crucial for building the tool interactions, as it will ensure that the simulated hand aims the tool at the exact point that your screen cursor is pointing at, whether it’s in first person or third person mode. Other interactions will be built on top of this as well, like physical grabbing or placing the hand at a nearby target.

Another important piece is extending the pointer interaction system to allow for a free-form cursor, allowing you to interact with the world and items within while freely moving the cursor around. To enable this, we have extended multiple systems. One of the benefits is that the desktop mode now has explicit control over the field of view (this is necessary to be able to calculate where in the world the laser hits).

https://www.youtube.com/watch?v=FAxGIBrYnPM

The second one is that Neos now has support for multi touch interfaces! Any existing UI can now be interacted with using a touchscreen. This will be particularly useful on mobile phones and tablets, providing a natural way to interact on those devices, although it still needs more work in this area, such as on screen controls for movement.

https://www.youtube.com/watch?v=7CRCwSWCX60

One of our longer ongoing projects is transitioning the renderer from a gamma color space to linear and providing more explicit control over color spaces for textures and colors in Neos. Linear color space will provide more accurate and consistent lighting, while managing the color space for the assets and parameters will ensure that your content looks the same (or as close as possible) as in the software it was authored in.

There is much to do for this transition, from the asset variant system, data model and the renderer, as well as making sure that existing content doesn’t break or look significantly different. We showed some early screenshots of the lighting and we’ll likely have more in the upcoming weeks as we progress.




The Metaverse Training Center is continuing development as well, with the different rooms getting filled out with content and polish. Here are some latest screenshots of the Streamer room and the Creation Plaza:




Creator Jam is back again! This week we focus on the four elements earth, wind, fire, and water. There were many Fifth Elements references abound, but we had some nice creations in this jam, so stop by and check out what people have made. Thanks everyone!


In this zone created by the keywords above, you feel like someone’s always watching you. Inspired by the .hack franchise, we have a fan recreation of a notable map from the franchise. Come here and listen to a prayer, or check out the stained glass directly in VR. It’s quite a lovely map and I appreciate all the effort that’s gone into it! Thanks Storm!


This wonderful sculpture of a tooltip brought to life by Beaned, showcases the magnificent beauty that is the Developer Tooltip. A tool made to make you a literal god in the Metaverse comes to life as it animates itself. Be wary that the Developer Tooltip doesn’t look for you for disrespecting it. (Yes this is a JoJo Reference) Thanks Beaned!


While it’s been in development, the VR Workout Club has been developing exercise machines in VR! Currently they only have a “Squatatron” (name pending), but it really shows that VR + Fitness can really go hand in hand! I’ve participated myself and it’s definitely a good workout! You really can feel the burn! If you wanna join try contacting one of them, anyone can join! Squat Away!


--------------------------------------------

Anyway that's all for this week, we hope it was an exciting one and that you'll have a lot of fun experimenting with the new hardware in the upcoming weeks as well. We'll continue bringing you more improvements over the upcoming days and weeks, as well as focusing on getting the desktop mode to basic feature parity, so we can perform an official swap.

You can keep an eye on the development on our official Discord in the #devlog channel and on any releases and update notes in #neos-updates.

See you next week!

Hello everyone! With the release of the Vive Facial Tracker, here's a bunch more eye and lip/mouth tracking improvements, to make the setup of avatars easier and more automated.

Heuristics of the blendshape matching were improved significantly and there's now a checkbox in the Avatar Creator to automatically set it up for face tracking. It's off by default, as it can mess up avatars if they're not setup for it, but feel free to experiment and see how well it works. You'll get best results following HTC's naming scheme (tutorial & samples coming soon).

There are some important improvements for the new desktop mode as well, notably with hand posing, which is now exact and capable of aiming the laser perfectly straight at the target. This is an improtant pre-requisite for implementing tools, as it will allow the system to aim the tool exactly at the point where the screen cursor is pointing.

A bunch of other improvements, tweaks and additions too, the VIVE Hand Tracking SDK was updated too (they seem to have made quite good improvements on the tracking quality!) and a few new blendshapes for the eye tracking that Neos didn't register before.

More to come soon!

- Added "Setup Face Tracking" to the avatar creator, which will scan the blendshapes on the avatar and attempt to setup face tracking
-- This can be used to easily setup avatars for the Vive Facial Tracker
-- Note that success rate will depend on the available blendshapes, Neos will use name heuristics to try to find best matches. You can get best results by following the naming convention from the HTC sample model

- Added Squeeze and Frown eye tracking parameters (based on feedback from @Reactant)
-- Appropriate fields were added to EyeManager and EyeLinearDriver
-- Squeeze indicates how tightly is the eye closed
-- Frown indicates the eye frowning (note that from my testing this doesn't seem to be currently tracked with Vive Pro Eye)
- Added StrengthMultiplier to AvatarExpressionDriver, which allows applying a global scale to the driven targets/blendsdhapes
- Added "Show Laser in Desktop Mode" setting which will display the laser in the new desktop mode when pointing at interactable objects
-- I mostly use this for debugging, but you can use this if you prefer for immersion

- Heavily improved hand posing in desktop mode when grabbing items and interacting with them
-- The actual position/direction of the laser is now respected, posing the hand exactly so the laser is straight when at rest and hand is properly offset for different avatars
--- This is an important mechanism for tool support, as it allows to pose the hand to aim at exactly specific point in the world
-- When the target point is near face, the hand is pushed to the side and down to avoid it from intersecting/obscuring the face/viewpoint
-- The hand interaction transition now tracks velocity, preventing instant snapping of movement direction and has tweaked velocities/smoothing
- The cursor reticle now fades away after 2 seconds of inactivity in the desktop mode

- Sliding items all the way towards the face will no longer equip/physically grab in then desktop mode
- Improved internal moderation tools to allow for more finegrained control (based on feedback by the moderation team)
- Userspace laser in desktop mode now correctly overrides the world-space laser, rather than having both activate at the same time
- Holding items in desktop mode no longer uses the simulated hand's twist to rotate the item, to prevent unwanted rotations when moving the item around
-- Note that some partial rotation might still be transferred when initially grabbing
- Improved Avatar Expression driver blendshape assignment heuristics
-- The heuristics now detects more keywords, improving face tracking support for Ready Player Me models and others
-- The assignment system now also has a face target filtering, ignoring ambigous blendshapes if non-ambigious are present (e.g. "Mouth Smile" vs just "Smile") and preventing multiple similar blendshapes from being all assigned, causing over-driving (e.g. having both "Mouth Open" and "Jaw Open")
- EyeManager parameters now use sliders to represent values within the 0...1 range
- Increased Stiffness on DynamicBoneChain decimal points from 2 to 4 (based on feedback by @H3BO3)

- Updated VIVE Hand Tracking SDK to 0.10.0 (from 0.9.3)
- Updated VIVE SR Anipal SDK (for eye and face tracking) to 1.3.2.0 (from 1.3.1.1)

- Merged Czech locale additions and tweaks by @rampa_3 (UTC +1, DST UTC +2)
- Merged Korean locale tweaks by @MirPASEC

- Fixed Interaction Laser raycast portals not working properly when the laser goes through another object that's ignored, causing UI to be unusable
-- This fixes cases of UI not being interactable in desktop mode when using portal raycast and the initial interaction hit hitting the user's avatar head
-- This also fixes cases of UI not being interactable when laser passes through canvas that let's the hit pass through if it doesn't hit anything (e.g. dash not being usable in VR when laser passes through the notifications canvas)
- Fixed grabbing a VRIKAvatar causing the neck and hips position to be taken from the held avatar, even though it's not equipped
- Fixed Cameras rendering the overlay layer
-- This should fix the mirror facet from rendering the dash while the desktop mode is enabled (re-reported by @epicEaston197 and @Khosumi)
- Fixed DictionaryList getting into invalid state when an operation during enumeration results in a single remaining item
-- This fixes various random bugs and potential corruptions due to using a list that's been returned back to memory pool
- Fixed out of range inputs for the maxUsers command on headless throwing an exception (reported by @Glitch)
- Fixed RecordSyncStatus not reporting sync error when the error is anything other than Out Of Space (based on report by @PlasmaRaven, @Elektrospy, @Honeypotbutt and @Shifty | Quality Control Lead)