Author Topic: Aircraft performance and ENY  (Read 4067 times)

Offline Anaxogoras

  • Platinum Member
  • ******
  • Posts: 7072
Aircraft performance and ENY
« on: February 07, 2009, 12:17:46 PM »
So, after presenting the idea of popularity + k/d ENY and seeing it roundly rejected, I investigated ways to rank aircraft by performance.  The idea is to come up with some kind of objective way to assign each aircraft a number that reflects a wide cross section of various performance statistics.  442w30 encouraged my efforts and allowed me to borrow his scheme and tinker with it as I saw fit (thank you).

To assign a rank for a peformance stat, e.g. speed, you take the individual aircraft's performance stat, subtract that average for all aircraft, and then divide by the standard deviation.  This is called the Zscore:

Zscore = (x-avg)/stdv

The result is generally a number between -2 and +2, and anything higher or lower is extreme.  For example, for top speed at sea level the La-7's zscore is 1.58, while the A6M2's is -2.05.

These are the performance categories:

Top speed at sea level
Best speed at <10k ft (altitude varies by aircraft)
Best speed at >10k ft (altitude varies by aircraft)
Climbrate at sea level
Climbrate at 10k ft
Climbrate at 15k ft
Maximum sustained turn rate (flap use depends on aircraft)
Minimum turn radius without flaps
Minimum turn radius with full flaps
Primary weapon lethality
Secondary weapon lethality
Primary weapon firing time
Secondary weapon firing time
Primary weapon ballistics (meters/sec)
Secondary weapon ballistics
Combat flight time (no drop tanks)
Ordinance capacity
Roll rate at 200mph
Roll rate at 300mph
Roll rate at 400mph
Cockpit visibility (from http://www.jcsautomation.com/AH_View_Planes.asp)
Dive acceleration (from 150mph to 400mph using auto-speed)
Energy retention (cutting the engine at 400mph level flight, measuring time to 150mph)
Level acceleration 200-300mph

To emphasize or minimize the impact of certain categories, they are all assigned a modifier, like .25 or 2.5, depending on their degree of importance (these numbers are debatable).  With 24 categories, all modifiers added together equal 24, and then the average of the modified Zscores is that aircraft's rank.  After lots of tinkering and adjusting, these are the modifiers I have assigned and below you will see the results:

2.5 Top speed at sea level
1.5 Best speed at <10k ft (altitude varies by aircraft)
.25  Best speed at >10k ft (altitude varies by aircraft)
1.5 Climbrate at sea level
1.0 Climbrate at 10k ft
.25 Climbrate at 15k ft
1.5 Maximum sustained turn rate (flap use depends on aircraft)
.75 Minimum turn radius without flaps
.75 Minimum turn radius with full flaps
1.75 Primary weapon lethality
1.75 Secondary weapon lethality
.25 Primary weapon firing time
.25 Secondary weapon firing time
1.25 Primary weapon ballistics (meters/sec)
1.25 Secondary weapon ballistics
1.0 Combat flight time (no drop tanks)
.25 Ordinance capacity
.25 Roll rate at 200mph
.75 Roll rate at 300mph
1.25 Roll rate at 400mph
.5 Cockpit visibility (from http://www.jcsautomation.com/AH_View_Planes.asp)
1.0 Dive acceleration (from 150mph to 400mph using auto-speed)
1.0 Energy retention (cutting the engine at 400mph level flight, measuring time to 150mph)
1.5 Level acceleration 200-300mph



Some of the results are surprising, like the low score of the N1K.  If I were to emphasize its positive traits and minimize its negative traits with the modifiers, aircraft like the P-51D and 190D-9 would be very low on the list and the Spitfire XVI would be at the top.  This is to say that these numbers simply reflect performance on paper and do not reflect how aircraft are used in the arena.  The Hurricane IIC is more effective than its score reflects because people love to furball, and it scavenges on the aircraft the fast ones force to turn.  Likewise, the P-47N is very high on the list even though it is rarely used as a fighter in the arena: its roll rate at 400mph is tied for #1 with the P-38L, it has good firepower with high ballistics, it's fast and has a long loiter-time over a combat area; yet few people find the fighting style that makes use of its strong points worth pursuing.

The one statistic I wish I had, but which is difficult to measure, is durability.  If someone knows of a good way to test aircraft durability, please let me know.  Taking account of it would bump the Spitfires down a bit, but the XVI, XIV and VIII would still rank among the top aircraft.
gavagai
334th FS


RPS for Aces High!

Offline SEraider

  • Silver Member
  • ****
  • Posts: 1755
Re: Aircraft performance and ENY
« Reply #1 on: February 07, 2009, 12:24:39 PM »
Are the modifiers you assigned just for the LA7?  Or are they constants for all planes?  I'm a little lost on that, please explain.
* I am the embodiment of Rule #14
* History is only recent.
* Stick and Stones won't break my bones, but names could "hurt" me.

CO Screaming Eagles

Offline Anaxogoras

  • Platinum Member
  • ******
  • Posts: 7072
Re: Aircraft performance and ENY
« Reply #2 on: February 07, 2009, 12:29:31 PM »
Are the modifiers you assigned just for the LA7?  Or are they constants for all planes?  I'm a little lost on that, please explain.

The modifiers are for all aircraft.  I apologize if I did not explain them enough.

Let's use speed at sea level as an example, again.  With a modifier of 2.5, the La-7's Zscore is 1.58 x 2.5 = 3.95.  The A6M2's Zscore is -2.05 x 2.5 = -5.13.  All of the modified Zscores are averaged, and that becomes the aircraft's rank which you see in the list above.
gavagai
334th FS


RPS for Aces High!

Offline 442w30

  • Nickel Member
  • ***
  • Posts: 471
Re: Aircraft performance and ENY
« Reply #3 on: February 07, 2009, 12:49:01 PM »
I have to get over the shock of seeing my name in a thread...

My contribution was simply to introduce Z-Scores to Anax and give him a basic spread sheet framework to begin his project with.  The rest of you cannot imagine how much work he has done on this project and how much time he has put into it.   :salute 

Anax, I started a little project and need P39D and Q data.  Could you please send me that? Feel free to send me the spreadsheet too.  I'd enjoy looking it over.
Now go away or I shall taunt you a second time

"The plural of anecdote is no data."- statistician's axiom

Offline Lusche

  • Radioactive Member
  • *******
  • Posts: 23874
      • Last.FM Profile
Re: Aircraft performance and ENY
« Reply #4 on: February 07, 2009, 01:00:56 PM »
At first glance, I spot the major tool of the trade for town busting, the Me 110G somwhere in the middle.
The hordes would be ever thankful for that - like having ENY 20 and 110's still available to roll bases NOE with 30 110's ;)
Steam: DrKalv
E:D Snailman

Offline 442w30

  • Nickel Member
  • ***
  • Posts: 471
Re: Aircraft performance and ENY
« Reply #5 on: February 07, 2009, 01:27:09 PM »
The heavy gun package on the 110G is so powerful that it has a raw z-score of over 4.0  IIRC that puts it in the 99.9997% range.  In fact if this were a survey where the data was from a sample size rather than a whole population, that score would be labeled an "outlier" and throw out.  However this is a population driven work and thus all data must be included.  He has the lethality of the primary and secondary weapons at a 1.75 modifier which weights the gun package a lot. 
Now go away or I shall taunt you a second time

"The plural of anecdote is no data."- statistician's axiom

Offline E25280

  • Gold Member
  • *****
  • Posts: 3475
      • http://125thspartanforums.com
Re: Aircraft performance and ENY
« Reply #6 on: February 07, 2009, 04:11:11 PM »
Why no 262 or 163?  Do they skew the data that much?  Did you remove them from your averages?

It doesn't surprise me that the Nik is somewhere near the middle based on your dependence on averaging averages.  It does everything reasonably well and has no excessive flaws, but is also generally outclassed in something by most late war aircraft.

Nice attempt, but as you say, the weightings you give are easily debatable.  For example, someone looking for an aircraft in more of an attack role would certainly not assign only .25 to ord capability and firing time, and would not be quite as needful of a good top speed at sea level as the hefty 2.5 you give that category.
Brauno in a past life, followed by LTARget
SWtarget in current incarnation
Captain and Communications Officer~125th Spartans

"Proudly drawing fire so that my brothers may pass unharmed."

Offline Lusche

  • Radioactive Member
  • *******
  • Posts: 23874
      • Last.FM Profile
Re: Aircraft performance and ENY
« Reply #7 on: February 07, 2009, 04:28:02 PM »
The heavy gun package on the 110G is so powerful that it has a raw z-score of over 4.0  IIRC that puts it in the 99.9997% range.  In fact if this were a survey where the data was from a sample size rather than a whole population, that score would be labeled an "outlier" and throw out.  However this is a population driven work and thus all data must be included.  He has the lethality of the primary and secondary weapons at a 1.75 modifier which weights the gun package a lot. 


And that shows in my opinion how difficult it os to find a purely objective and quantifiable approach to ENY, because there are ever changing factorsthat you can't really put into numbers.
To stick to the 110G example, your arguments are not completely uncionvincing from a pure statistical approach. But it fails to take into consideration what the primary role of the 110 in MA gameplay really is, and that a much higher ENY for that plane would have a detrimental impact on gameplay.

Furthermore, like E25280 already said, while the comparison between the different planes within a specific category is quite objective, the weighting of all the categories to get any kind of ranking is not. It's largely subjective and can be debated a lot.
Steam: DrKalv
E:D Snailman

Offline Anaxogoras

  • Platinum Member
  • ******
  • Posts: 7072
Re: Aircraft performance and ENY
« Reply #8 on: February 07, 2009, 04:40:45 PM »
Furthermore, like E25280 already said, while the comparison between the different planes within a specific category is quite objective, the weighting of all the categories to get any kind of ranking is not. It's largely subjective and can be debated a lot.

Yes, but despite its imperfections, this kind of approach is superior to simply making an ad-hoc guess.  It's a fallacy to say that because it can't be done perfectly, it ought not to be done at all (which I know you didn't say, but it's nearly the conclusion I see you regularly make in these discussions).
« Last Edit: February 07, 2009, 04:44:57 PM by Anaxogoras »
gavagai
334th FS


RPS for Aces High!

Offline Bronk

  • Persona Non Grata
  • Plutonium Member
  • *******
  • Posts: 9044
Re: Aircraft performance and ENY
« Reply #9 on: February 07, 2009, 04:44:31 PM »
 :rofl Why ???? because you say so?  :rofl
See Rule #4

Offline Anaxogoras

  • Platinum Member
  • ******
  • Posts: 7072
Re: Aircraft performance and ENY
« Reply #10 on: February 07, 2009, 04:45:12 PM »
:rofl Why ???? because you say so?  :rofl

What?
gavagai
334th FS


RPS for Aces High!

Offline Bronk

  • Persona Non Grata
  • Plutonium Member
  • *******
  • Posts: 9044
Re: Aircraft performance and ENY
« Reply #11 on: February 07, 2009, 04:47:00 PM »
You claims of superiority. All but made me fall out my chair in laughter. :rofl
See Rule #4

Offline Anaxogoras

  • Platinum Member
  • ******
  • Posts: 7072
Re: Aircraft performance and ENY
« Reply #12 on: February 07, 2009, 04:48:18 PM »
You claims of superiority. All but made me fall out my chair in laughter. :rofl

And I stand by it. :aok  I'll stand by any decision based on quantitative data over a hunch, any day, any week, any year.
gavagai
334th FS


RPS for Aces High!

Offline Gabriel

  • Copper Member
  • **
  • Posts: 207
Re: Aircraft performance and ENY
« Reply #13 on: February 07, 2009, 04:52:43 PM »
Always easy to find exceptions or quirks ,

but on balance that is a very good list from Top to Bottom  :)

Offline Bronk

  • Persona Non Grata
  • Plutonium Member
  • *******
  • Posts: 9044
Re: Aircraft performance and ENY
« Reply #14 on: February 07, 2009, 04:53:22 PM »
The use of ac in rolls they were never really intended to do defies quantification.

But you just keep on a trying. :aok
See Rule #4