The gumbo of stutters, pauses, freeze and other ingredients

[ACE]

Mine is an MSI GamingX card with a really good cooler on it, which is why I bought it. I'm at 2100 core, 4374 Memory, 108 power limit. I never break 65c in the Unigine stress tests.

If you're not experienced in overclocking read up on it before you just plug settings into afterburner. Baby steps and a lot of patience are key. My card and my rig and my cooling are gonna be different from yours. Your overclockability will likely vary from mine. Don't fry your card! Just a brief PSA about overclocking.

That and let's not hijack this thread. 

I got 2 fans on mine. Stock cooler I assume. I do wanna turn her up tho

[Pudgie]

Update:

I have ran AHIII Dx11 a couple of times w\ LatencyMon 6.5 running in the background to check if I'm getting hit w\ a lot of DPC's off ISR's being generated by any running drivers on my Team Red box, hoping that a screen pause\freeze would occur while LatencyMon was running to see if it would pick this up & identify the culprit..........

As you would have it, my box has ran flawless while this app was running so I haven't got anything so far except LatencyMon showed that there were a lot of ISR's w\ subsequent DPC's generated by this MS Win 10 OS file: wdf01000.sys (MS Kernel Mode Driver Framework) but the execution times were very short & LatencyMon reported that Team Red was running fine w\ no dropouts......but that's a lot of CPU context switching going on that is somewhat driver related...........it doesn't appear to be sound driver related but could be graphics driver related as this MS Win 10 OS file: dxgkrnl.sys (MS Directx Graphics Kernel) wasn't showing any ISR counts but was showing a few DPC's though.

Gonna keep testing to see if I can capture something.................... .

Snippets of LatencyMon results provided below.

FYI.......................... .

 

[Pudgie]

Update:

I finally caught a screen pause\freeze while flying AHIII Dx11 vers w\ LatencyMon running in the background.

Snippets of the results provided below.

It appears to be driver-related w\ LatencyMon reporting a potential buffer underrun w\ a potential CPU power management warning due to CPU throttling (I have AMD's Cool & Quiet CPU power\temp control enabled in UEFI & am using the Windows Balanced Power Plan but Powercfg.exe has been modified so that when CPU core usage % exceeds 40%--which is all the time once AHIII is running in full screen--Cool & Quiet will give this 1800X full power to all 8 CPU's so the CPU is running @ 3.7Ghz on all 8 CPU cores consistently and won't back off until CPU core usage % recedes below 30%--which won't happen until AHIII has exited back to desktop) w\ a suggest to update BIOS to latest vers (which I already have done so).

The only driver showing to be getting ISR's & DPC's is the MS wdf01000.sys (MS Kernel Mode Driver Framework) and MS dxgkrnl.sys (MS Directx Graphics Kernel) not showing any ISR's but quite a few DPC's. None of the Creative SBX7 drivers show any ISR's or DPC's to suggest that there is an issue w\ them but I'll check them for the allocated buffer size. Do the graphics drivers have a buffer allocated to them & is this buffer a dynamic one or a fixed size buffer? I'm sure there is 1 but I've never seen evidence of it OR where it can be set. I'll assume that this buffer is located within the graphics card's onboard mem & is accessible thru the vBIOS controlled thru the graphics card driver.

Maybe on to something here.

Got some homework to do.........................

 

[N95KF]

ever since VR my stutters have vanished

[Skuzzy]

Yes, the VR headsets have never suffered the issue.  That is what got us thinking about where the issue may be.

[Pudgie]

Update:

After doing some homework I think I may have at least come to some better understanding of what is occurring in the background.

From LatencyMon showing the 2 Windows 10 files wdf01000.sys (MS Kernel Mode Driver Framework) & dxgkrnl.sys (MS Directx Graphics Kernel) w\ wdf01000.sys getting a lot of ISR's (interrupt service routine) & DPC's (deferred procedure call) and dxgkrnl.sys getting no ISR's but several DPC's........

The wdf01000.sys is part of the Windows Driver Framework stack that the graphics card driver (AMD or Nvidia) is signalling Windows to signal the CPU to request data (interrupt) to be sent to the GPU (whether from the CPU's L3 cache or system mem cache) thus the ISR. The large amount of DPC's is due to something else blocking these requests from being executed in a timely manner at the CPU core level momentarily (higher rank thread priority due to thread scheduling by the Windows scheduler, 3rd party software execution, poorly written software...., damaged Windows files from bad\incomplete driver installations, thread held up waiting on another thread to finish to fetch the data from that thread to finish the other one, etc) which slows down the data transmit requests from the CPU to GPU. The dxgkrnl.sys is part of the Windows 10 Directx Graphics Kernel that processes the Directx 3D application of the graphics data requested earlier. The absence of ISR's is due to the kernel being embedded within the Driver Framework stack but needs CPU core processing time to do it's work & the number of DPC's indicate that there is something also blocking it from being executed in a timely manner at the CPU core level momentarily as well which also slows down the Directx application to the needed data which slows down this data getting to the GPU in a timely manner. This situation is referred to as "resource starvation". The amount of time duration for each individual DPC to complete is very small, but when you have a large amount of DPC's occurring the total duration time does add up to some significant slow downs of graphics data transmission when under a game load.

Now some of this does occur within the normal processes so the fact that there are DPC's at all isn't the issue per se....it's when there are an EXCESSIVE amount of these DPC's occurring within the graphics computation pipeline that can cause the graphics frame buffer to be underrun (frames are moved out of the buffer to be displayed faster than the finished frames are being moved in to be displayed) & when this reaches the 0 point the screen will stop momentarily as there are no finished graphics frames to be displayed to screen until there are enough finished frames refilled to the graphics frame buffer to continue displaying to the screen then the screen motion will restart.....as long as this happens within the 2 sec threshold then no TDR will occur (why we ain't getting .dmp files). This is why these events are so random as there is no way to reliably predict when the buffer underrun will actually occur & what exactly is causing it. The only thing I do know now is what part of the system is being affected. The number of causes are very large w\ most of these causes being created outside of the AHIII client's domain so Hitech & Co will be essentially limited in what they can do from their end. The fact that these screen freezes aren't occurring when the AHIII DX11 client software is running in offline mode is an indicator of the base level of AHIII DX11 client being intact & error free code-wise.

SO I've been out on the Internet looking for any & all info on using all the built-in Windows tools that can be accessed thru the CMD to start going thru the OS to see if I can find anything amiss on my end & if I find something I'll post.

At least now I have a little more to go on when I start looking........

FYI........................

Now it's time to get outside & pour some cement.................

 

 

[SirNuke]

very nice thanks for the writeup, I will go to sleep less ignorant tonight 

[Pudgie]

One other item I forgot to add.....................

When looking at LatencyMon results w\ the screen freeze\pause captured, the latency issue was occurring the vast majority of the time on CPU core 0 (or CPU core 1 if you prefer) w\ AHIII Dx11 version being run w\ CPU core priority\affinity applied which means that the game threads weren't being processed on CPU core 0 as I had instructed Windows to assign these threads to other CPU cores......which also moves the AHIII game client from being the cause in my mind........

CPU core 0 is handling the majority of it's processing time the OS 1st due to high priority being assigned to itself then any device drivers & related software, 3rd party software, etc.......everything except the AHIII game threads w\ CPU core 0 latency being 3 times worse than the rest of the CPU cores..........

Since Windows Scheduler schedules threads using a round robin method according to the binary CPU core numbering (which means that it will try to schedule as many threads, regardless of which process that generated them, as it can to CPU core 0 unless the CPU core is tied up) this can lead to CPU core contention on this core which can affect the Windows Driver Framework stack's operation & create these DPC's.

When you have a 8-core CPU available w\ Windows only trying to use less than 1\2 of them there is an issue that needs to be resolved there as well.

 

[Pudgie]

Ok, this is what I have went thru to date:

1. Ran the following thru the Windows CMD Editor to check Windows 10 image on my box:
sfc scannow----> Reported back clean w\ no errors found after scan was run
DISM.exe /Online /Cleanup-Image /ScanHealth----->Reported back image was repairable after scan was run (even though SFC reported all clean)
DISM.exe /Online /Cleanup-Image /RestoreHealth----->Ran it anyway......reported back all completed successfully so Windows 10 image checks out as clean.

Went in AHIII & flew around for awhile..........got a screen freeze\pause afterwards so not due to Windows 10 OS issues.

Ran LatencyMon.......still shows the 2 MS Windows files (wdf01000.sys\dxgkrnl.sys) getting hit w\ excessive ISR's\DPC's.

Bing'd these 2 files to see if any fixes\solutions available............

1. wdf01000.sys------->Got 1 suggestion to go into BCDedit /set IncreaseUserVA 3072 (this was setting to increase amount of addressable mem of 4Gb total system on x86 Windows OS to user mode from the std 2048 to the max of 3072--3Gb--to help alleviate mem resource starvation of the Windows Driver Framework Kernel....does me no good as I'm using Win 10 x64 OS which doesn't have this limitation), doesn't apply but I tried it anyway then retested...........and got a screen freeze\pause so validated this ain't it.

2. dxgkrnl.sys----------> Got 1 suggestion to go into BCDedit /set disabledynamictick yes (this setting would stop Windows from dynamically switching timers based on the application needs). Now since I have this AMD Ryzen 7 CPU that likes the HPET timer to be active & had already set this prior for Windows to use the HPET I set this to yes to keep all using the HPET just to see what would happen.........AHIII ran much snappier after this was done (LatencyMon did show that the ISR\DPC execution times did reduce across the board after making this setting change which verified the performance improvement) but I still got a screen freeze\pause to occur but only after a couple of hours of game play had elasped. Liked what I had but wasn't the culprit of causing the screen freeze\pause..........

So now this points to start shutting down stuff I have running in the background as follows:

1. Shut down HWINFO32------->Still got a screen pause\freeze to occur-----not the culprit (was glad to see this)
2. Shut down MSI AB\RTSS next----->Still got a screen pause\freeze to occur-----not the culprit (was glad to see this as well)
Note: These 2 programs make up the overlay that I'm currently using to do real time monitoring of my box while flying.
3. Shut down Gigabyte APP Center w\ all sub apps next-------->Still got a screen pause\freeze to occur------not the culprit
4. Shut down Logitech Keyboard & Mouse Control next------>Still got a screen freeze\pause to occur----------not the culprit
5. Shut down CorsairLink4 Monitoring\Control (H80i V2 AIO) next------->Still got a screen pause\freeze to occur------not the culprit
6. Shut down Samsung Magician SSD Monitoring\Maintenance Software next------->Still got a screen pause\freeze to occur-----not the culprit
7. Checked settings in Webroot SecureAnywhere AntiVirus for Gamers......found some settings that would allow AV to update itself automatically--turned this off, found AV set under Access Control to actively monitor HWINFO32 software & SB X7 DAC-AMP driver while they were running-?, set AV to allow both HWINFO32 & SB X7 DAC-AMP to run unhindered, found that AV System Optimizer has never ran due to not being enabled (this cleans up all the junk files left behind from system operations)...set this up to run upon system bootup as well as AV to scan all on system bootup & not on a schedule.

Am in the process of testing out all this at the time of typing this post.................

This is what I have covered on my box so far.

FYI..........................

 

[Pudgie]

Ok, this is what I have went thru to date:

1. Ran the following thru the Windows CMD Editor to check Windows 10 image on my box:
sfc scannow----> Reported back clean w\ no errors found after scan was run
DISM.exe /Online /Cleanup-Image /ScanHealth----->Reported back image was repairable after scan was run (even though SFC reported all clean)
DISM.exe /Online /Cleanup-Image /RestoreHealth----->Ran it anyway......reported back all completed successfully so Windows 10 image checks out as clean.

Went in AHIII & flew around for awhile..........got a screen freeze\pause afterwards so not due to Windows 10 OS issues.

Ran LatencyMon.......still shows the 2 MS Windows files (wdf01000.sys\dxgkrnl.sys) getting hit w\ excessive ISR's\DPC's.

Bing'd these 2 files to see if any fixes\solutions available............

1. wdf01000.sys------->Got 1 suggestion to go into BCDedit /set IncreaseUserVA 3072 (this was setting to increase amount of addressable mem of 4Gb total system on x86 Windows OS to user mode from the std 2048 to the max of 3072--3Gb--to help alleviate mem resource starvation of the Windows Driver Framework Kernel....does me no good as I'm using Win 10 x64 OS which doesn't have this limitation), doesn't apply but I tried it anyway then retested...........and got a screen freeze\pause so validated this ain't it.

2. dxgkrnl.sys----------> Got 1 suggestion to go into BCDedit /set disabledynamictick yes (this setting would stop Windows from dynamically switching timers based on the application needs). Now since I have this AMD Ryzen 7 CPU that likes the HPET timer to be active & had already set this prior for Windows to use the HPET I set this to yes to keep all using the HPET just to see what would happen.........AHIII ran much snappier after this was done (LatencyMon did show that the ISR\DPC execution times did reduce across the board after making this setting change which verified the performance improvement) but I still got a screen freeze\pause to occur but only after a couple of hours of game play had elasped. Liked what I had but wasn't the culprit of causing the screen freeze\pause..........

So now this points to start shutting down stuff I have running in the background as follows:

1. Shut down HWINFO32------->Still got a screen pause\freeze to occur-----not the culprit (was glad to see this)
2. Shut down MSI AB\RTSS next----->Still got a screen pause\freeze to occur-----not the culprit (was glad to see this as well)
Note: These 2 programs make up the overlay that I'm currently using to do real time monitoring of my box while flying.
3. Shut down Gigabyte APP Center w\ all sub apps next-------->Still got a screen pause\freeze to occur------not the culprit
4. Shut down Logitech Keyboard & Mouse Control next------>Still got a screen freeze\pause to occur----------not the culprit
5. Shut down CorsairLink4 Monitoring\Control (H80i V2 AIO) next------->Still got a screen pause\freeze to occur------not the culprit
6. Shut down Samsung Magician SSD Monitoring\Maintenance Software next------->Still got a screen pause\freeze to occur-----not the culprit
7. Checked settings in Webroot SecureAnywhere AntiVirus for Gamers......found some settings that would allow AV to update itself automatically--turned this off, found AV set under Access Control to actively monitor HWINFO32 software & SB X7 DAC-AMP driver while they were running-?, set AV to allow both HWINFO32 & SB X7 DAC-AMP to run unhindered, found that AV System Optimizer has never ran due to not being enabled (this cleans up all the junk files left behind from system operations)...set this up to run upon system bootup as well as AV to scan all on system bootup & not on a schedule.

Am in the process of testing out all this at the time of typing this post.................

This is what I have covered on my box so far.

FYI..........................

 

Update:

Have gone thru Webroot w\ a fine tooth comb working all settings that should have not allowed Webroot to interfere w\ the game but I still got a screen freeze\pause..............
8. Shut down Webroot SecureAnywhere Antivirus completely today next & ensured that it was completely shut down then ran AHIII Dx11 vers....still got a screen pause\freeze......AV is not the culprit (glad to see this). So I have restarted Webroot SecureAnywhere Antivirus software, reset all settings back to prior test setup but went ahead & set up AHIII in both Dx9 & Dx11 versions to be allowed to run unhindered (AV wasn't hindering the game anyway).

Put all my monitoring software back in pre-test setup as none of them were found to be the culprit.

So, after all this testing it appears that none of the software (OS, monitoring apps, AV, services, etc) that I have running on my box is responsible for these screen pauses\freezes occurring so unless I've missed something I'm gonna have to look into my hardware.....
9. SB X7 DAC-AMP------> Had this DAC's USB cable plugged into 1 of the onboard USB 3.0 rear ports that is designed w\ Gigabyte's USB DAC-UP2 power regulation control to provide clean power to assist external sound interfaces to operate smoothly. Had reset the line power calibration from normal to +.3v additional power then ran the game.........got screen pauses to occur at each power reg point.........not the culprit.
Installed a spare USB 2.0 rear adapter & plugged the X7 into this adapter to run the DAC off the internal USB 2.0 ports (this DAC was built to USB 2.0 specs--info on Creative's site of sound pops, static if DAC is plugged into USB 3.0 header....would get 1 or 2 pops every blue moon) then ran the game.....still got a screen pause\freeze.......not the culprit.

Reconnected the X7 DAC back to the USB 3.0 rear USB DAC-UP2 header.
10. Orico PUV3-7 USB-to-PCI-E add-in card-------->Have noted some spiking occurring w\ my CH USB HOTAS that I think is due to the VIA USB chipset that this add-in card uses. A quick in-game recalibration fixes it for a while but will start wandering & occasionally will spike my throttle w\ throttle handle pulled back (idle) so wondering if USB spiking is causing this..........
Since I got this Fractal Design Meshify C case the USB connectors from my HOTAS didn't want to fit thru the case expansion slots w\o some force applied to them (USB slots in this add-in card run perpendicular to the expansion slot) to plug into this card....this ain't good either so I have ordered a Sunix 4-port USB 3.0-to-PCI-E USB add-in card in that the USB connectors plug in parallel to the case expansion slots to eliminate the connector binding and this Sunix card uses a Renasaus USB chipset that from reading up these USB chipsets are reported to be good, clean, stable USB chipsets. I did have an issue getting the Orico PUV3-7's driver to install so was using a USB driver provided thru Win 10. Should have the Sunix USB add-in card by Wednesday.................we'll find out then. Was looking for a USB 2.0 version of this card but none to be found........at least on Newegg.

This is where I'm at to date........

 

[DaddyAce]

Wow Pudgie, you've done a lot of digging to try and ferret this out man!        

[Pudgie]

Update:

Worked w\ CPU as follows:

Enabled High Performance Power Plan so CPU will run at full clocks (even though AMD Cool & Quiet is still enabled in UEFI) then ran the game......played for approx 25 mins w\ steady CPu clocks @ 3.7 Ghz.........but got a screen freeze to still occur.

Got an idea............

Went & enabled the Balanced Power Plan again (this power plan's Powercfg.exe tables has been modified to allow the CPU to still be controlled by the OS thru AMD's Cool & Quiet) & reset the PERFINCTHRESHOLD & PERFDECTHRESHOLD parameters (CPU usage parameters that instruct the OS to instruct AMD's Cool & Quiet\TurboCore to ramp up\down CPU power\frequency) from 40\30 to 30\25 (lowered the adjustment threshold settings so that the OS will instruct Cool & Quiet to allow TurboCore to load up the CPU once the usage avg exceeds >30%...as long as the CPU temps stay below the TDP threshold (80*C) & will shut down TurboCore to allow Cool & Quiet to throttle CPU power\frequency if the CPU temps exceed the TDP limit OR if the CPU avg core usage recedes <25%) due to thought that the 40\30 setting range may be still causing CPU power\frequency to be oscillated due to CPU core usage dipping back below the 40% upper range in which Cool & Quiet might be trying to throttle CPU on a percentage basis when usage falls in between the 2 control points of 40\30 (which I have witnessed happening from time to time while playing). To test these settings properly I also went in the WBPP advanced settings & lowered the min processor power % setting from 75% to 50% (at this setting Cool & Quiet will throttle the CPU to the lowest power\frequency table setting @ 2.2 Ghz so any further lowering of the min processor % setting in the WBPP advanced settings is useless).

Ran the game & I immediately noticed that I had hit a setting curve that really made a difference on the Ryzen 7 1800X's performance. Evidently the CPU had been throttling but it isn't throttling now......CPU is running very stable @ 3.7 Ghz on all 8 CPU cores w\o fluctuation w\ TurboCore applying full power @ 1.45v-1.5v & CPU operating temps never dropping below 49*C at any time. Most of the time the CPU core temps held between 52*C-58*C (prior setting @ 40\30 CPU operating temps ranged between 44*C-52*C w\ CPU clocks bounce between 3.2Ghz-3.7Ghz) so I now know that the CPU is really running at full power now. The game ran much, much better than it has for some time so I've got some more performance to enjoy now! Played for approx 35-40 mins w\ all going good.......but then a screen freeze\pause occurred right as I had leveled out into level flight but the freeze duration was much shorter than the others so I think I'm starting to gain on it but time will tell.

Ongoing......................

FYI........................

 

[Pudgie]

Update:

Just got done playing AHIII Dx11 vers again & all was going very well w\ box performing subperbly w\o a screen freeze\pause then once I entered into landing glide path\angle w\ me ole Spitty (leveled out w\o a lot of movement) I had a screen freeze occur............

This is now 2 times in which all was looking good as long as there was a lot of movement, control input, etc that kept the CPU core usage above 75% & up but as soon as I went into level flight w\ CPU core usage dropping below 65% is when the screen freeze happened. GPU tach on the Fury X was pegged out thruout so the GPU was running flat out at full clocks @ 1050\500 mem (stock GPU boost clocks\mem clocks for Fury X\Fiji GPU). GPU usage% never got even close to 90% usage (mostly hovering in the 70's-80's) so what I'm seeing may be a function of something going on within this Fury X's Fiji GPU at the vUEFI\driver level.........maybe related to some type of GPU power fluctuation anomaly when changing power states when throttling back on power but not GPU clock speed. Read speculation here.................

Wished I had access to this Fury X's vUEFI to check into this...............

Check out this MSI AB graph's GPU frametime graph line.............that is the best frametime graph line that I have ever witnessed coming from this Fury X........the GPU wasn't missing any graphics frames rendering\displaying & was VERY steady doing it all.

FYI..........................

 

PS---It just dawned on me that this issue on my box just may be a symptom of this Fury X hitting its practical operational limit at which these kinds of anomalies may start occurring when running under Dx11 breaking the graphics rendering work into pieces to take advantage of this GPU's ACE's.....thinking parallel multiprocessing issue in which 1 ACE unit may be a little slow to get it's piece of the work done in time relative to the others. This is a complex GPU...............

Just a hunch..............

 

[Noir]

deleted

[SirNuke]

another approach would be to look how skuzzy configures his boxes, bios settings, drivers, services etc and work from there?