The gumbo of stutters, pauses, freeze and other ingredients

[Pudgie]

While I was dinking around online today I read up on the topic of CPU core affinity and came upon the process of going thru the Task Manager to assign CPU affinity to a process so I started up AHIII Dx11 then went into Task Manager, right clicked on the game process then set CPU affinity to use only the 1st 2 CPU cores instead of all 6 then went back into AHIII and carried on to play.

Upped at A1 and I immediately noticed that the petite freezes that Chalenge referred to seeing were now gone (I had always looked at these as GPU stuttering prior until recently). I saw none of this anymore even when making rapid movements in game. All screen motion was butter smooth. After flying around for a little longer I exited out (got shot down by OUTATIME on a 6 o'clock bounce), pulled up MSI AB graphs to see the CPU core usage....now I could see distinctly that 2 CPU cores were being used even though the CPU core usage % was still pretty low.
So I ran AHIII Dx11 again setting CPU core affinity to use 2 cores, upped from A1 again....I got none of the petite freezes this run as well and was getting pretty happy about all this. Tangled up w\ OUTATIME again and was working the fight when I got a 1.5 sec screen pause right when I was rolling over the top of an Immelman to come down on his 6.......went on and finished the fight and shot down OUTATIME then exited the game and pulled the graph up and moved the tracer line over the screen pause where GPU clocks dropped and frametime spiked then traced it down to the CPU usage % of Core 1 and Core 2 and it showed Core 1 at 34% and Core 2 at 21%.

Ran test 2 more times to verify the petite freeze pattern and verified that by setting the CPU core affinity to 2 cores instead of using all 6 cores, this stopped these petite freezes from occurring under AHIII Dx11 on my box but not the screen pause.

FYI.......................... .....

 

[MADe]

hmmmmm I remember skuzzy saying that AH3 could use 3 cores.

Now I gotta test this. I have played the affinity game before, never seemed to matter ultimately, but maybe..................

[Pudgie]

Just to clarify what I did to verify I ran AHIII Dx11 w\ CPU core affinity w\ all 6 cores active then ran AHIII Dx11 w\ CPU core affinity set to use 2 cores then made the in-game comparisons as 1 test cycle.

I did this a total of 4 complete cycles and witnessed the results that I posted on every cycle.

During this test cycle I also witnessed another screen pause occur beside the 1 that I posted on in my last post, this 1 came during test #4 while I was on the runway at A1 during the engine startup routine which lasted approx 2 secs. After exiting out I checked my MSI AB graph and noted that this screen pause occurred just after Windows made a core usage change to run the game (was using Core1 and Core4 prior then switched threads from Core4 to Core2 while still using Core1). This was odd to me as I had never witnessed this occurring since I started monitoring this using MSI AB until I remembered that I had just exited out of AHIII Dx11 when I had all 6 CPU cores active and had made the CPU core affinity setting change to 2 cores active, but I made the affinity change at the startup screen w\o fully exiting the game 1st (I had fully exited the game before I did this on the 3 prior sessions) due to me also noting that while AHIII Dx11 was at this screen the game wouldn't crash when opening Task Manager to set CPU core affinity to the game process (tested this using AHIII Dx9 as well but the game would crash when I tried to open Task Manager w\ the opening screen clipboard up.....solved this by minimizing the game before opening Task Manager. Doing this using AHIII Dx9 I saw no change at all...game ran just as flawless w\ 2 CPU cores as it has w\ all 6 CPU cores) so instead of exiting out fully then restarting AHIII Dx11 I exited to the opening screen then went in & changed the CPU core affinity w\o fully exiting out.

This last screen pause may have been my fault but it also does bring to light that some of these screen pauses just may be due to Windows switching threads across multiple lightly loaded CPU cores.....which I'll admit is really mostly a misnomer as if I've read this correctly Windows OS's are coded to do all this on a preemptive basis so this really happening should be very rare to none.....but I also saw evidence of this actually happening so.................

The whole reasoning for me doing this was to test the theory of consolidating the CPU core usage under AHIII Dx11 to less CPU cores that the actual CPU core usage per core would increase by stopping Windows from parsing game threads across all 6 cores hoping that this would also stop the on die Intel SpeedStep CPU control from potentially downclocking\underpowering the CPU cores due to very low core usage %. IOW's I hoped they would mimic the CPU core usage pattern I've consistently witnessed when running AHIII Dx9 (used 2 CPU cores exclusively w\ very high usage % across both cores), got the 2 core usage pattern but the core usage % was still pretty low relative to core use under AHIII Dx9........

Continuing on....................

 

[DaddyAce]

As I've been reading a couple of these posts I've been struck by a couple things.  My "mini-freezes" seem rare and quick to what at least some others are seeing.  Mine seem unpredictable, only show up occasionally, and resolve too quickly for me to react to do anything.  If they happen at an inopportune time, I suppose they could cause me to miss a shot, but because I get a bit better performance on DX11, I've been inclined to commonly run DX11 because my mini-freeze issue is pretty minimal.

I have an i3 6100 in my machine, a dual core processor, so if there is a core-swapping issue with cpus with 4 or more cores, my having minimal issue with my dual core processor seems consistent with that theory......

[Pudgie]

As I've been reading a couple of these posts I've been struck by a couple things.  My "mini-freezes" seem rare and quick to what at least some others are seeing.  Mine seem unpredictable, only show up occasionally, and resolve too quickly for me to react to do anything.  If they happen at an inopportune time, I suppose they could cause me to miss a shot, but because I get a bit better performance on DX11, I've been inclined to commonly run DX11 because my mini-freeze issue is pretty minimal.

I have an i3 6100 in my machine, a dual core processor, so if there is a core-swapping issue with cpus with 4 or more cores, my having minimal issue with my dual core processor seems consistent with that theory......

Hi DaddyAce,

Got some info here to interest you on this subject.

I just got done running AHIII Dx11 on my box and setting the CPU core affinity in Windows to run the game starting w\ a single CPU core, then upping the CPU core affinity to add a core each session then snipping the MSI AB graphs of each for comparison.

Please pay particular attention to the GPU frametime graph line on each 1......you'll note the change in the graph line smoothness across each successive CPU core added into the mix....you'll also note some of these graphs will show a single large spike in the graph lines...those represent a single screen pause where the screen motion froze for a sec then resumed motion. The myriad of mini spikes in the GPU frametime graph lines represent the petite screen freezes while the game is running.

What all this data is demonstrating is that a lot of issues we see and blame on the vid card is not the vid card at fault......it is actually the CPU operation that is causing the GPU to react in negative ways.

AHIII whether it is running under Dx9 or Dx11 runs the smoothest\best when it is running on 2-4 CPU cores TOTAL. Any CPU w\ less than 2 cores or more than 4 cores in use show on my box to exhibit the petite freezing issues due to the CPU operation being slowed down\made erratic too much causing the GPU to hesitate thus the petite freezing w\ the exception of AHIII running under Dx9. Hyperthreading will not help w\ this issue at all.....actually I believe it will make it worse (I have hyperthreading disabled on my I7 5820K Haswell-E 6-core CPU) unless you're running a 1-core or 2-core CPU.

From all this I am becoming even more certain that all these issues are arising from the CPU being underutilized due to Dx11 vs Dx9 causing Intel SpeedStep to interfere w\ CPU voltage\clocks settings causing CPU instability . If the CPU loading can be increased under DX11 to get the cores above and stay above the 50%-60% usage mark I believe this will be enough to stop Intel SpeedStep from interfering w\ the CPU voltage\clock speeds in which all this will go away.

Enjoy!

 

[Pudgie]

Here's the last graph snippet w\ all CPU cores active................

Enjoy.................

 

[Pudgie]

Skuzzy,

Here are 4 .dmp files that I discovered in the AH temp folder that were created yesterday.

Hope these help y'all out.

 

[Pudgie]

Update:

I got a hunch again on all this and went into my box's UEFI to look more closely into my CPU's advanced core settings and found the CPU EIST Function setting (this setting is what controls Intel Enhanced Speed Stepping at the CPU level from the UEFI) so I set the UEFI to disable this function in my I7 5820K CPU, saved it, rebooted then checked this thru Gigabyte's SIV......my CPU is showing now to be locked solely into Intel's TurboBoost function w\ CPU consistently clocking between 3900-4003 MHz (this is set in my UEFI's CPU Upgrade setting). So this will not be fully disabled thru the Windows Power Management scheme of High Performance....only at the UEFI\BIOS level.

Went up using AHIII Dx11 using all 6 CPU cores to test..............testing has now confirmed that it was Intel's EIST that was causing the petite freezes while running under Dx11 due to the low CPU core loading. Here is a snippet of MSI AB graph showing a pretty clean GPU frametime graph line now w\ all 6 CPU cores being used and Intel EIST disabled in the UEFI......

However, you will also note that a screen pause was also recorded and when I moved the white synch line over the large GPU frametime spike then traced down to the CPU core usage this showed once again to coincide w\ CPU core usage % to be less than 35% on any active CPU core which is following the same pattern of all other screen pauses that I've captured since I set up the individual CPU cores to be graphed.

I'm gonna run some more tests w\ CPU affinity cut to the optimum CPU core numbers that have shown the AHIII game client to run the best under w\ EIST disabled to see if the screen pausing ceases..........but I don't see this happening unless the CPU core loading goes up enough to keep up w\ a fully optimized GPU at full GPU clock speeds.....and this IMHO can only be achieved thru the game client software OR me making setting changes to try to slow down the GPU to better "synch" w\ the current CPU game loading w\ AHIII Dx11.

IOW's I believe these screen pauses are due to a timing issue created between the CPU\GPU being influenced by the AHIII Dx11 game client and\or other external factors that shows up once the CPU core usage % drops below 35% on all active CPU cores. Again I'm NOT making any claims that the AHIII game client is BROKEN under Dx11 so don't take this there.

What I will suggest is that maybe the AHIII game client could use a little more tweaking to make more use of the CPU cores relative to the current usage model to raise the CPU core usage or loading up some more OR raise the post-processing rendering levels\GPU work load levels as hard coded in the AHIII game client to work the GPU some more to effectively slow it down just enough to "synch" better w\ the current CPU loading when running under Dx11.....but this may not be feasible IF doing this breaks the game client from being able to qualify for VR certification.....which is what I'm kinda suspecting would be the quandry at hand at this time. As for running under Dx9 the game client needs no further enhancing work done to it that I can detect.

Just a hunch....................   

If I may ask, is there any way that I could talk the nice folks at HTC to consider to write into the game client coding the instructions to instruct the OS to assign CPU core affinity so when the game is started up the optimum number of CPU cores to be used will be automatically set up? I know I'm asking for a lot here but I REALLY like how the client performs when the CPU cores being used are limited to 2 total CPU cores and I KNOW that MS ain't gonna go there............

You do realize that Christmas IS just around the corner and Santa would love it as well while he's flying his sled in the game that he has put the insignia of Johnnie Johnson's Spitfire Mk IXe on the sides of it.......maybe w\ a case of a little something, something wet and over 81 proof thrown in to boot................. 

 

Yeah I know this was bad but I just HAD to try................

 

[Pudgie]

Here's a graph w\ AHIII Dx11 running on CPU core affinity of 2 CPU cores w\ EIST disabled along w\ Hyperthreading disabled in the UEFI:

As I suspected, all else was eliminated....except the screen pause. When I ran the timeline over the GPU frametime spike then traced it down to the 2 CPU cores the core usage on Core1 was down to 25%, Core2 was down to 14% when the pause occurred.

But other than this, the game performance was just spectacular to behold.............and the graph lines clearly give a picture of this..........

FYI.......................... .....

 

PS--Skuzzy, do y'all have all y'all's computers @ HTC set up in the BIOS\UEFI w\ Intel SpeedStep disabled? If y'all do then this would most likely be the reason why y'all haven't been able to duplicate all the petite freezes running the Dx11 version. As for the screen pauses I can't say but to ask: what is the CPU core usage load % on y'all's boxes when running AHIII Dx11 w\ a VR headset attached vs w\o a VR headset attached and is it consistently staying above the 35% threshold that I've identified on my box from my testing? If if is then this may also be the reason why y'all can't duplicate these screen pauses as well.....

Just laying out a thought of mine for consideration................ ........

 

[DaddyAce]

Hi Pudgie,

Reads like you're making progress tracking down clues......If I'm following you correctly, in summary, you were getting nearly constant mini-freezes, but got those ironed out by turning off cores, etc.

Now that looking at your graphs have me tuned in to the utility of "Frametime", I've set up my AB to include frame time in my next set.  I'm guessing my frametime graph, produced by running AH3 DX11 on my i3 dual core will look similar to the one you just put up....that pretty flat with just the occasional spike.  I'l try to run a graph or too later today when I fly....meanwhile should spend some time in RL activities.....

 

[Chalenge]

I have tried everything Pudgie is doing now. In shorts tests the "pause-and-play" petit freezes cleared up (in my case) but after extended play would always return.

My conclusion is that instead of trying to correct our systems to fix the problem, we need to record and film, and then post reports. Even if we find something like the speed-step as a possible cure it is not a fix, because there are many, many users that could never find the setting to even change it!

I tried the CPU affinity change and unless there is some mysterious combination of cores that will work I still get petit freeze. I applaud Pudgie for trying though.

[Pudgie]

Hi Pudgie,

Reads like you're making progress tracking down clues......If I'm following you correctly, in summary, you were getting nearly constant mini-freezes, but got those ironed out by turning off cores, etc.

Now that looking at your graphs have me tuned in to the utility of "Frametime", I've set up my AB to include frame time in my next set.  I'm guessing my frametime graph, produced by running AH3 DX11 on my i3 dual core will look similar to the one you just put up....that pretty flat with just the occasional spike.  I'l try to run a graph or too later today when I fly....meanwhile should spend some time in RL activities.....

 

Hi DaddyAce,

The main issue that I found to create the mini-freezes when running AHIII Dx11 is actually Intel's EIST CPU power\frequency control kicking in causing the CPU to cut power\core frequency and operate erratically which was affecting the GPU operation due to the light CPU core loading derived from running AHIII under Dx11 API. I also found out that this can't be reliably shut down thru the Windows Power Management scheme....only thru the UEFI\BIOS. Once I disabled this in the UEFI it doesn't matter about CPU core affinity being in the mix concerning the mini-freezes.....all was showing to create a clean GPU frametime graph line which indicates that the CPU and GPU are working together now in near perfect synch.

But the CPU core affinity really does matter to the AHIII game client as the AHII game client, according to HTC, was written to be run (or read as optimized) on 2 CPU cores and this was brought over to the new AHIII client initially. There was some discussion during the Beta that Hitech was thinking about upping this to 3 CPU cores for AHIII but I haven't heard anything since to solidify this. My initial testing by hard setting the CPU core affinity showed to improve on the GPU frametiming until I set the affinity on 1 CPU core or in excess of 4 CPU cores in which the mini-freezes reappeared. But an exchange of info from reading MADe's postings in another thread got me thinking to recheck the CPU core settings in my box's UEFI which lead to the discovery that I just typed about prior to this post.

My testing the AHIII game client running on the forced 2 CPU cores shows why this game client really needs to have CPU core affinity assignment written into it's coding IMHO nowadays to reflect the advent of it being run on platforms using multi-core CPU's in excess of the optimum CPU core count on which it was written to perform with. The less thread parsing across multiple CPU cores that Windows has to do the better Windows performs these functions thus the better the game shows to run.......but that is gonna be up to Hitech to decide if its worth the coding effort to incorporate it into the client software......thanks in no small part to Microsoft not providing provisions for this to be set thru the OS per app\game in a user friendly, permanent fashion so you don't have to redo this every time you start the game up. The perfect place to had provided CPU core affinity setting access thru the OS is thru the app\game shortcut icon Properties, Compatibilities tab..........

And your other point you've made is well taken, sir!

Signing off for a while........................ ............

 

 

[Pudgie]

I have tried everything Pudgie is doing now. In shorts tests the "pause-and-play" petit freezes cleared up (in my case) but after extended play would always return.

My conclusion is that instead of trying to correct our systems to fix the problem, we need to record and film, and then post reports. Even if we find something like the speed-step as a possible cure it is not a fix, because there are many, many users that could never find the setting to even change it!

I tried the CPU affinity change and unless there is some mysterious combination of cores that will work I still get petit freeze. I applaud Pudgie for trying though.

Thanks for the flowers Chalenge!

I get what you're posting here on all points and am certainly not advocating anyone else to follow suit just because I'm doing it.

What I'm hoping to accomplish is not to fix the issues w\ this game on my end per se, but to highlight these issues to hopefully assist HTC to identify operational realms from the good work that they've done so far w\ this new AH game client that can unintentionally cross over into areas of interoperability to ampify issues that sometimes may not be so obvious at the onset due to a single reported symptom that can point to multiple causes of said symptom....like remembering where the Intel CPU load threshold is that will invoke something like Intel's EIST to work against your client running smoothly then working your client to ensure that this line isn't crossed while your client is running on other's systems to remove it as a potential cause......because the results of my testing does expose it to be a potential cause of these symptoms.

So I will ask of you, have you tried all this on your box w\ SLI disabled using only 1 vid card running? I don't know as you've never posted to this effect (you have posted that you do run AH on SLI'd GTX 980's that I know even w\ Nvidia doing a LOT of work w\ their drivers to cure the GPU frametiming issues that occur from SLI'd GPU's which can create these same petite freezes due to interleaving) and I'm running a single vid card and from my testing since I've made these changes I've never witnessed the petite freezes showing up again...only the occasional screen pause. I also do realize that my vid card is an AMD R9 Fury X vid card and due to it's architectural design parameters\driver stack tuning may benefit more from this kind of work than your GTX 980's might............

Just asking...............

I'm not saying that you have to or even that you should just because I asked but I am curious of this for my own knowledge\understanding if nothing else as I do not have any intention of SLI\Crossfire usage on my boxes so I'll never really know this for myself unless someone else posts their results to this effect.

Anyway have a good holidays and I'm gonna give this a rest for a while.

 

[Chalenge]

Yes, I thought it could possibly be the SLI method, so I tried five different modes and still get the pause-and-play hesitations. You seem to be saying that petit-freeze and screen-pause are two different things, but they are the same thing to me.

What caused me to give up trying to configure the game to run without this problem was when I ran the game on a top end workstation system and still got the same events. The game runs great otherwise and looks amazing.

[Chalenge]

Windows 10 X64 Pro Edition.

Petit-Freeze / 1/2 sec. Screen Pause

Seems to be tied to Windows Audio, Windows events, AH3 audio. Not sure how to change the priority of that, or to allow for more/less buffer.