Attached is an example of a demo captured automatically on QoL which shows the "Demo playback failed for file" dialog while audio from the demo is playing in the background. Calling the function Canvas.setContent(PlayGui); shows the demo playing back (though none of the keybinds work).
It plays back normally on RC2a and is handled fine by TribesForever's very permissive demo parser. I'm able to reproduce this issue intermittently.
Attached is an example of a demo that shows the "Demo playback failed for file" dialog but audio plays in the background. Calling the function Canvas.setContent(PlayGui); shows the demo playing back normally. isPlayingDemo returns true.
Playback works fine on RC2a. TribesForever's very permissive demo parser has no issues either.
Krash, we're playing around with AI upscaling for textures and 2x-4x file sizes get kind of big. Is there a maximum size a vl2 can be? Seems after 2gb it stops loading textures.
Standard resource manager file sizes in bytes are limited to a signed 32 bit integer, so after about 2.14GB you'd be feeding the zip handler an incorrect stream length. If any of the textures are being CRC checked, you may also run into issues with the frame allocator being too small if the texture being checked exceeds 3MB; I don't remember offhand if that was increased already.
I can confirm this demo works fine on my end. Would have to look closer to spot any oddities, but generally it shouldn't show that abort message unless it fails to load entirely, I think the exception being a soft fail added to the patch for debugging when the terrain failed to load, but that shouldn't be the case here.
Only one or two new serverprefs will be added automatically after the first run/export (the boosted rate for sending datablocks faster being one), anything else will be optional additions. Would be curious to know what the dll warning was, but the installer is essentially just a compressed package extracting files and making sure that the license agreement is accepted; if the server runs it's probably fine, if required dlls are missing it won't start at all. AVX has been optional for a while, it should detect support and switch code paths on older hardware.
No avx probs here, server is replete with avx goodness, however I have tried pretty much everything to get t2csri.vl2 and attendant eula to overwrite by the patch and no go, patch error msg complains about not being able to overwrite. I gave the folder all permissions, I gave the files all permissions, I ran the patch as admin, I copied another t2csri.vl2 from another source and no go. I gave pretty much all entities listed in properties for the freaking thing full control and no go.
CRC checking isn't generally going to be the cause of a stutter, not because the computation is particularly fast, but because it's only done at load time, and isn't going to be performed on textures loaded during gameplay where the host isn't requiring it. High resolution textures do have quite a bit more work to do in runtime load processing though, because all those extra bytes need to be read from disk, decompressed (both from the vl2 if deflate compression is used and from the image format compression itself), converted from a raw bitmap to an optimal format and uploaded to the GPU by the driver, and have an entire mipmap chain generated — the last, while much faster than the original software implementation, can still be a fairly heavy process and the main thread will block until the driver finishes for each texture being loaded in a frame. A lot of this load time would be avoidable by supporting more modern texture containers like dds or kxt, which can both hold offline-generated mipmaps and store everything compressed in a format directly transferrable to and readable by the GPU... Technically it'd be possible to add something like ktx2 support, though it'd need to bypass most standard texture handling, and of course only really something that'd likely be noticed with very large textures.
Krash, I'm wondering if you can help me out with a very odd problem that I'm having. I often play on Blakhart's server, named Blair Lag Project. He has a lot of bots there. One of them is named Sami-FIN. That's also the name I use to play. Just some hours ago, it wasn't possible to join Blair Lag Project. The server was very broken. It wasn't even showing up here on the website's server list. I went to some other server then to play. It only had players and that demo bot that records gameplay. As I joined that server, I noticed an odd message in the chat: ''Sami-FIN has been muted.'' It was true: I couldn't use the Voice Game System. I checked out a few servers. I was probably muted on all the game's servers except for TRIUMPH BotAbuse 300Maps server. However, I managed to get myself unmuted by removing my pyrdb file in the base, prefs folder. I can play now, but I would like to know how to play using that pyrdb file again. As I understand, it stores muted players as numbers. I don't know if my name is in there.
What I'm guessing is going on is that often when I join Blakhart's server, and my bot version is already there, it gets named to Sami-FIN.1. I often mute that bot, because it uses the same voice as I and because it gets confusing listening to and reading its messages. Maybe the last time I played on Blair Lag (before this muting problem) I muted my bot version. I left, and later on Blakhart's server broke down, and somehow my muting of Sami-FIN.1 bot got saved somewhere permanently, and the game thinks Sami-FIN.1 and Sami-FIN are one and the same. I don't know. I do know that muting those bots isn't permanent: I can mute them, leave the game, come back to the server, and they are unmuted again.
Since bots don't have guids, you most likely accidentally muted your previous client after timing out and reconnecting.
The pyrdb format is pretty simple, you can open it up in any text editor, and each line is a guid and two 0/1 boolean values: mute text and mic voice, respectively. Remove the line starting with your guid (it'll match the name of the file after pyrdb), and you should be fine to keep using the rest.
You could in theory also host a local listen server and connect to it with a second instance of the game and mute/unmute yourself there, but note whichever client closes last will overwrite the prefs.
This won't have prevented other players from seeing your messages, of course; being able to mute yourself is just an unexpected quirk of that system.
Odd thing is about muting, when The Sami was in our server later on he indeed was muted in chat, no chat from him but did get the canned chat sounds. Dunno if a corrupted server had anything to do with it.
I checked to see if I somehow I muted him, he wasn't, dunno wtf was going on there. That said, I will do my best not to put up a server without immediate testing after serious changes have been made, it was well borked for a major portion of the day.
Can't reproduce that on my end anywhere. The scaling doesn't directly manipulate ui elements in a way that could cause this; the simplified basis of how it works is that it indicates to the existing gui canvas engine internals that you're running at a lower screen resolution. The only difference once you pass 150% scale would be that it will try to use textures/gui/[email protected] for that fill area if the file exists... but the stretched fill starts right at the bottom of that top gradient, and goes down to a flipped gradient of the same height on the bottom, as highlighted in green in this shot:
How these shll textures are combined here is hardcoded in the rendering of the ShellPaneCtrl gui element, using the raw bitmap dimensions of the gradients for the vertical offset. The whole area's drawn as two triangles, with no depth testing, so it's not like it can stop part-way vertically or clip through other elements, and if it were only drawing one triangle you'd see it cut off in a big diagonal accross line the screen.
To render as shown in your shot without altered assets, without anything else in that pane's rendering being affected... it'd have to somehow be sending very incorrect data or sampling incorrectly in your GL driver. Maybe try temporarily putting a bigger square texture at textures/gui/shll_fieldfill.png and see if it still does it.
Yeah, the alpha channel on that scaled texture is busted; only the bottom row has the original transparency, which explains what was in your shot. It's a solid color fill of course, so yeah it doesn't need a scaled version.
The decompression is going to be negligible in at that size, especially since zlib was replaced with a more modern implementation that's on average at minimum 1.5x to 2x faster than stock (although the original png decoder hasn't yet been replaced with accelerated modern versions to speed that part up), but at higher resolutions it couldn't hurt to either store them in the vl2 without compression or put them directly in your textures folder just to avoid going through the inflate process and allocations twice for each texture. .jpg textures could also decode a little faster, while .bmp textures would require no decompression, if disk space weren't a concern... the tradeoff being you couldn't use either format for anything you want the alpha channel on, and .bmp textures are going to take longer to read from disk if you're on an HDD..
The bigger delay comes in just how much data is actually being processed at those scales: at 1024²|2048²|4096²|8192² you'll be allocating (on the cpu), processing, then uploading roughly 3MB|12MB|48MB|192MB to the gpu, which then needs to allocate at least 4MB|16MB|64MB|256MB for the full mip chain (plus a bit more for alignment/padding), then do the mip generation for each level. The upload and mip generation doesn't generally block immediately on most drivers, but if it's still running when a draw call that binds that texture is run, it'll need to wait until it finishes and the gpu syncs... and because it's only being loaded because it's needed immediately for that frame, it shouldn't be too surprising if it blocks for very large textures.
At 1024² this whole ordeal should take well under 0.5ms with a modern system, but could be 1ms+ at 2048, maybe a few more ms at 4096², and maybe 10ms or more at 8192² – I don't have metrics for this on hand, though: this is just ballpark accounting for PCIe bandwidth and very rough alloc/draw times on the gpu. For things like skies, interiors, terrain, etc., huge textures are fine because they're not loaded mid-gameplay, but 4096 and above for custom player skins (which are loaded/freed on the fly) could easily take a big bite out of the frame budget for a visible delay on high refresh rate displays.
The other thing that could be done to avoid delays here, besides introducing a modern format, would be to add an asynchronous texture upload pipeline in a future version. This would mean it'd have to initially use a placeholder texture for at least one frame and pop in only when it's available, but that'd probably be fine for player/weapon/vehicle skins.
This shit is amazing, almost everyone is using the QOL patch now and loving it. You're literally saving T2. Awesome.
I have a question about the asset downloads, we were toying around with a silly mod and thinking it would be awesome if we could override some of the vehicle sounds (for example htransport_boost.wav). This can be done client side by loading the .wav in a .vl2 but with the new asset downloading we were wondering if you were to allow .wav files could we have the server send a client a htransport_boost.wav which would live in the cache folder and override their local setting while in that server, or until they restarted their client? or maybe we could have the server send them a differently named .wav file and the mod could play a different audio file when they're in the havock instead of htransport_boost.wav.
This shit is the lowest priority, don't waste your time on it.
Sorry, the spam filter for some reason thought you were a spammer and I just noticed it was sitting in the queue.
Unfortunately the reason the asset depot doesn't serve wav files is that the client expects these files to load from disk only as soon as they're needed, and audio file loads still run on the main thread (this will likely be replaced one day, but it's not currently a priority); if you've ever played with a very slow old/fragmented HDD, and the game freezes up for a second when you fire a weapon for the first time, this is why... and because the download system essentially acts as a remote hard drive when the file is not found locally, the game would lock up in much the same way while the download runs (only once, but still, you'd feel it). Technically it could be possible to pre-cache any audio files referenced by datablocks at mission load to avoid this happening during gameplay, but it'd need some consideration to avoid causing problems.
From a practical implementation standpoint of how you'd want to set up your mod were the wavs uploaded though, you couldn't override any existing files with content delivered from the depot: when building the resource table, .cache/ is checked first, then base/, then the active mod directories, and the last location a filename is seen is the one that the resource manager is going to use. This does mean the resource manager has capacity for overriding, but only in the opposite order: to use custom override wav files with the same filenames and switch between them, the game client needs to load it from a new mod path. To do it more or less as you described using what's there today, it could technically be pulled off with a client script that could download the overrides from a different source, but... putting it together that way would get a little overcomplicated just to serve a couple files when if clients would need to download the script anyways.
Using new filenames and just referencing those in the appropriate datablocks on the server would be the way it'd have to be done if they are eventually added to the depot – but, obviously, anyone who doesn't have the sound files and doesn't want to download assets wouldn't hear anything.
hey, we could really use clan tags, if anyway you can add them, or let us know if the endpoints are working and we can do now.. what does it take, as i thought i tried correctly before and the new endpoints were bogus
Comments
Hi Krash. Perhaps this is already fixed in the upcoming release. Some users have reported having issues playing back demos that were automatically recorded using this script: https://github.com/tribes2/scripts/blob/master/client/auto-record.cs
Attached is an example of a demo captured automatically on QoL which shows the "Demo playback failed for file" dialog while audio from the demo is playing in the background. Calling the function
Canvas.setContent(PlayGui);shows the demo playing back (though none of the keybinds work).It plays back normally on RC2a and is handled fine by TribesForever's very permissive demo parser. I'm able to reproduce this issue intermittently.
Hi Krash. This is likely fixed in the next patch. Here's a reproduction case anyway.
Demos captured with this script https://github.com/tribes2/scripts/blob/master/client/auto-record.cs don't always play back on QoL. I've had success with demos captured in a client/non-dedicated LAN server but issues with online client demos.
Attached is an example of a demo that shows the "Demo playback failed for file" dialog but audio plays in the background. Calling the function
Canvas.setContent(PlayGui);shows the demo playing back normally.isPlayingDemoreturns true.Playback works fine on RC2a. TribesForever's very permissive demo parser has no issues either.
Krash, we're playing around with AI upscaling for textures and 2x-4x file sizes get kind of big. Is there a maximum size a vl2 can be? Seems after 2gb it stops loading textures.
Standard resource manager file sizes in bytes are limited to a signed 32 bit integer, so after about 2.14GB you'd be feeding the zip handler an incorrect stream length. If any of the textures are being CRC checked, you may also run into issues with the frame allocator being too small if the texture being checked exceeds 3MB; I don't remember offhand if that was increased already.
I can confirm this demo works fine on my end. Would have to look closer to spot any oddities, but generally it shouldn't show that abort message unless it fails to load entirely, I think the exception being a soft fail added to the patch for debugging when the terrain failed to load, but that shouldn't be the case here.
Only one or two new serverprefs will be added automatically after the first run/export (the boosted rate for sending datablocks faster being one), anything else will be optional additions. Would be curious to know what the dll warning was, but the installer is essentially just a compressed package extracting files and making sure that the license agreement is accepted; if the server runs it's probably fine, if required dlls are missing it won't start at all. AVX has been optional for a while, it should detect support and switch code paths on older hardware.
No avx probs here, server is replete with avx goodness, however I have tried pretty much everything to get t2csri.vl2 and attendant eula to overwrite by the patch and no go, patch error msg complains about not being able to overwrite. I gave the folder all permissions, I gave the files all permissions, I ran the patch as admin, I copied another t2csri.vl2 from another source and no go. I gave pretty much all entities listed in properties for the freaking thing full control and no go.
Also any chance of getting a module built in to watch for and log crashes with as much detail as Krash desires?
Would the strategy to prevent CRC stuttering to keep textures under 3mb?
CRC checking isn't generally going to be the cause of a stutter, not because the computation is particularly fast, but because it's only done at load time, and isn't going to be performed on textures loaded during gameplay where the host isn't requiring it. High resolution textures do have quite a bit more work to do in runtime load processing though, because all those extra bytes need to be read from disk, decompressed (both from the vl2 if deflate compression is used and from the image format compression itself), converted from a raw bitmap to an optimal format and uploaded to the GPU by the driver, and have an entire mipmap chain generated — the last, while much faster than the original software implementation, can still be a fairly heavy process and the main thread will block until the driver finishes for each texture being loaded in a frame. A lot of this load time would be avoidable by supporting more modern texture containers like dds or kxt, which can both hold offline-generated mipmaps and store everything compressed in a format directly transferrable to and readable by the GPU... Technically it'd be possible to add something like ktx2 support, though it'd need to bypass most standard texture handling, and of course only really something that'd likely be noticed with very large textures.
Maybe the file was in use? Weird, but if you still couldn't get it in there, you could always just make another install folder and copy the mods over.
Krash, I'm wondering if you can help me out with a very odd problem that I'm having. I often play on Blakhart's server, named Blair Lag Project. He has a lot of bots there. One of them is named Sami-FIN. That's also the name I use to play. Just some hours ago, it wasn't possible to join Blair Lag Project. The server was very broken. It wasn't even showing up here on the website's server list. I went to some other server then to play. It only had players and that demo bot that records gameplay. As I joined that server, I noticed an odd message in the chat: ''Sami-FIN has been muted.'' It was true: I couldn't use the Voice Game System. I checked out a few servers. I was probably muted on all the game's servers except for TRIUMPH BotAbuse 300Maps server. However, I managed to get myself unmuted by removing my pyrdb file in the base, prefs folder. I can play now, but I would like to know how to play using that pyrdb file again. As I understand, it stores muted players as numbers. I don't know if my name is in there.
What I'm guessing is going on is that often when I join Blakhart's server, and my bot version is already there, it gets named to Sami-FIN.1. I often mute that bot, because it uses the same voice as I and because it gets confusing listening to and reading its messages. Maybe the last time I played on Blair Lag (before this muting problem) I muted my bot version. I left, and later on Blakhart's server broke down, and somehow my muting of Sami-FIN.1 bot got saved somewhere permanently, and the game thinks Sami-FIN.1 and Sami-FIN are one and the same. I don't know. I do know that muting those bots isn't permanent: I can mute them, leave the game, come back to the server, and they are unmuted again.
Since bots don't have guids, you most likely accidentally muted your previous client after timing out and reconnecting.
The pyrdb format is pretty simple, you can open it up in any text editor, and each line is a guid and two 0/1 boolean values: mute text and mic voice, respectively. Remove the line starting with your guid (it'll match the name of the file after pyrdb), and you should be fine to keep using the rest.
You could in theory also host a local listen server and connect to it with a second instance of the game and mute/unmute yourself there, but note whichever client closes last will overwrite the prefs.
This won't have prevented other players from seeing your messages, of course; being able to mute yourself is just an unexpected quirk of that system.
Odd thing is about muting, when The Sami was in our server later on he indeed was muted in chat, no chat from him but did get the canned chat sounds. Dunno if a corrupted server had anything to do with it.
I checked to see if I somehow I muted him, he wasn't, dunno wtf was going on there. That said, I will do my best not to put up a server without immediate testing after serious changes have been made, it was well borked for a major portion of the day.
Seems like this gui bg is getting messed up passed 150% UI scale or @2x
Can't reproduce that on my end anywhere. The scaling doesn't directly manipulate ui elements in a way that could cause this; the simplified basis of how it works is that it indicates to the existing gui canvas engine internals that you're running at a lower screen resolution. The only difference once you pass 150% scale would be that it will try to use
textures/gui/[email protected]for that fill area if the file exists... but the stretched fill starts right at the bottom of that top gradient, and goes down to a flipped gradient of the same height on the bottom, as highlighted in green in this shot:How these shll textures are combined here is hardcoded in the rendering of the ShellPaneCtrl gui element, using the raw bitmap dimensions of the gradients for the vertical offset. The whole area's drawn as two triangles, with no depth testing, so it's not like it can stop part-way vertically or clip through other elements, and if it were only drawing one triangle you'd see it cut off in a big diagonal accross line the screen.
To render as shown in your shot without altered assets, without anything else in that pane's rendering being affected... it'd have to somehow be sending very incorrect data or sampling incorrectly in your GL driver. Maybe try temporarily putting a bigger square texture at
textures/gui/shll_fieldfill.pngand see if it still does it.Ah, thanks Krash.
shll_fieldfill.pngbeing 2x2 pixel prob doesn't need a @2x of that. Not having one fixes it tho.I thought for sure it was still doing it without the upscaled version. I posted what I was using.
Do you think 1024x1024 low compression PNGs should be the target to prevent any hang ups in the thread?
Yeah, the alpha channel on that scaled texture is busted; only the bottom row has the original transparency, which explains what was in your shot. It's a solid color fill of course, so yeah it doesn't need a scaled version.
The decompression is going to be negligible in at that size, especially since zlib was replaced with a more modern implementation that's on average at minimum 1.5x to 2x faster than stock (although the original png decoder hasn't yet been replaced with accelerated modern versions to speed that part up), but at higher resolutions it couldn't hurt to either store them in the vl2 without compression or put them directly in your textures folder just to avoid going through the inflate process and allocations twice for each texture. .jpg textures could also decode a little faster, while .bmp textures would require no decompression, if disk space weren't a concern... the tradeoff being you couldn't use either format for anything you want the alpha channel on, and .bmp textures are going to take longer to read from disk if you're on an HDD..
The bigger delay comes in just how much data is actually being processed at those scales: at 1024²|2048²|4096²|8192² you'll be allocating (on the cpu), processing, then uploading roughly 3MB|12MB|48MB|192MB to the gpu, which then needs to allocate at least 4MB|16MB|64MB|256MB for the full mip chain (plus a bit more for alignment/padding), then do the mip generation for each level. The upload and mip generation doesn't generally block immediately on most drivers, but if it's still running when a draw call that binds that texture is run, it'll need to wait until it finishes and the gpu syncs... and because it's only being loaded because it's needed immediately for that frame, it shouldn't be too surprising if it blocks for very large textures.
At 1024² this whole ordeal should take well under 0.5ms with a modern system, but could be 1ms+ at 2048, maybe a few more ms at 4096², and maybe 10ms or more at 8192² – I don't have metrics for this on hand, though: this is just ballpark accounting for PCIe bandwidth and very rough alloc/draw times on the gpu. For things like skies, interiors, terrain, etc., huge textures are fine because they're not loaded mid-gameplay, but 4096 and above for custom player skins (which are loaded/freed on the fly) could easily take a big bite out of the frame budget for a visible delay on high refresh rate displays.
The other thing that could be done to avoid delays here, besides introducing a modern format, would be to add an asynchronous texture upload pipeline in a future version. This would mean it'd have to initially use a placeholder texture for at least one frame and pop in only when it's available, but that'd probably be fine for player/weapon/vehicle skins.
Krash,
This shit is amazing, almost everyone is using the QOL patch now and loving it. You're literally saving T2. Awesome.
I have a question about the asset downloads, we were toying around with a silly mod and thinking it would be awesome if we could override some of the vehicle sounds (for example htransport_boost.wav). This can be done client side by loading the .wav in a .vl2 but with the new asset downloading we were wondering if you were to allow .wav files could we have the server send a client a htransport_boost.wav which would live in the cache folder and override their local setting while in that server, or until they restarted their client? or maybe we could have the server send them a differently named .wav file and the mod could play a different audio file when they're in the havock instead of htransport_boost.wav.
This shit is the lowest priority, don't waste your time on it.
Thank you!
where'd my post go
Sorry, the spam filter for some reason thought you were a spammer and I just noticed it was sitting in the queue.
Unfortunately the reason the asset depot doesn't serve wav files is that the client expects these files to load from disk only as soon as they're needed, and audio file loads still run on the main thread (this will likely be replaced one day, but it's not currently a priority); if you've ever played with a very slow old/fragmented HDD, and the game freezes up for a second when you fire a weapon for the first time, this is why... and because the download system essentially acts as a remote hard drive when the file is not found locally, the game would lock up in much the same way while the download runs (only once, but still, you'd feel it). Technically it could be possible to pre-cache any audio files referenced by datablocks at mission load to avoid this happening during gameplay, but it'd need some consideration to avoid causing problems.
From a practical implementation standpoint of how you'd want to set up your mod were the wavs uploaded though, you couldn't override any existing files with content delivered from the depot: when building the resource table,
.cache/is checked first, thenbase/, then the active mod directories, and the last location a filename is seen is the one that the resource manager is going to use. This does mean the resource manager has capacity for overriding, but only in the opposite order: to use custom override wav files with the same filenames and switch between them, the game client needs to load it from a new mod path. To do it more or less as you described using what's there today, it could technically be pulled off with a client script that could download the overrides from a different source, but... putting it together that way would get a little overcomplicated just to serve a couple files when if clients would need to download the script anyways.Using new filenames and just referencing those in the appropriate datablocks on the server would be the way it'd have to be done if they are eventually added to the depot – but, obviously, anyone who doesn't have the sound files and doesn't want to download assets wouldn't hear anything.
hey, we could really use clan tags, if anyway you can add them, or let us know if the endpoints are working and we can do now.. what does it take, as i thought i tried correctly before and the new endpoints were bogus