Aces High Bulletin Board

Special Events Forums => Friday Squad Operations => Topic started by: RedBeard on July 19, 2025, 05:20:36 PM

Title: A look back at the past 15+ years of FSO
Post by: RedBeard on July 19, 2025, 05:20:36 PM
I've been doing some data analysis on the past 15+ years of FSO and thought I'd post some of what I've found here.  All of this is based on FSO log data from the AHEvents site.  Here's a look at participation since Jun of 2009.

(https://i.imgur.com/7RiISls.png)

As you can see, there's some anomalies in the data.  I believe the large spikes are scenario events that were somehow logged as FSO, even though they weren't.  I dropped some logs from the collected data that appeared to be other events like King of the Hill as well.  Still, it's probably 95% good data.

The trend doesn't look good.  I would like to start or renew a topic elsewhere to discuss how to address this.

It shows a decline in population and total flight time.  Mostly, this is understandable as FSO used to be a single life event, but I didn't see a jump in total flight hours after multilife was enabled, which surprised me.  We do see an increase of sortie rates with multilife, but again that hasn't seemed to have much impact on total flight hours.  I may have to go back to check my data.

Anyway, if you are interested, I'll be posting more info here.

Title: Re: A look back at the past 15+ years of FSO
Post by: fudgums on July 19, 2025, 11:17:48 PM
Before the thread turns sour, we should be thankful for the CMs and the job they do/did. It’s tough.


And also 450 pilots were the days.

*insert The Wire Quote about the good ole days
Title: Re: A look back at the past 15+ years of FSO
Post by: hazmatt on July 20, 2025, 06:13:48 AM
Anybody know what caused that big drop at the beginning of the chart? The rest seems pretty gradual but that one was pretty drastic in relation to the rest of the chart.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 06:25:29 AM
Sorry, I didn't mean to turn things sour, nor did I mean to imply any fault with the CMs.  My original intent was to look at how to make missions more effective, but there are always other things you can learn along the way from the data you get.

In particular, I wanted to look at bomber effectiveness.  There's some data missing from the logs to get a real look at effectiveness, but the data does show the following:

PlaneTotal SortiesSorties With DmgTotal Landed SortiesSorties Landed With DmgChance to Make TargetChance to Destroy Target and Get Home
A-20G2499188453045075.39%18.01%
Ar_23450840439034379.53%67.52%
B-17G537347143431322287.73%59.97%
B-24J380330162616234779.31%61.71%
B-25C746560633485302681.22%40.54%
B-25H457310927267.83%15.75%
B-26B279622741902169681.33%60.66%
B-29293228131270125795.94%42.87%
B5N211533502177230.36%6.24%
Boston_III315724351353114577.13%36.27%
C-47A7422102.70%0.00%
D3A12760184145130066.70%10.87%
G4M1_Model_112240145483961364.91%27.37%
He_111H324324311956168674.96%51.99%
Ju_87D-33702262656440370.93%10.89%
Ju_88A-41111991815820522182.57%46.96%
Ki-672208166392474275.32%33.61%
Lancaster_III93787351950993.17%54.32%
Mosquito_Mk_XVI27620114411972.83%43.12%
SBD-53465245488362570.82%18.04%
TBM-3161975535317246.63%10.62%
Tu-2S2094172074064082.14%30.56%

There's a number of things we don't have in the published log data that would help with analyzing bomber  effectiveness.  These include formations used or not, load out carried, what ordnance was dropped and when, and who did damage, but didn't destroy a target.  I understand that some of this is actually available, but not published, and I may take a crack at exposing more of it at some point.
Title: Re: A look back at the past 15+ years of FSO
Post by: hazmatt on July 20, 2025, 06:58:06 AM
I think this number is way too high! lolz
                                                                               Chance to make target      Chance to destroy target and get home
G4M1_Model_11   2240   1454   839   613            64.91%                                27.37%

I remember one fso back in the day were I drew the short straw and had to fly a formation of these deathtraps and we all got slaughtered before ever getting sight of the target!
Title: Re: A look back at the past 15+ years of FSO
Post by: AKKuya on July 20, 2025, 07:24:56 AM
Quote from: fudgums on July 19, 2025, 11:17:48 PM
And also 450 pilots were the days.

I remember when we had 600 pilots and we were planning for 4 objectives to defend and attack.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 07:42:46 AM
Quote from: hazmatt on Yesterday at 06:13:48 AM
Anybody know what caused that big drop at the beginning of the chart? The rest seems pretty gradual but that one was pretty drastic in relation to the rest of the chart.

I suspect it was an anomalous set of data.  I have found numerous cases where the data just doesn't jive.  Pilots die without taking off.  Bomber formation pilots die after only two kill claims.  Pilots take off more than once.  Some of this may be due to when logging for the event is turned on or it might be data getting dropped as it is transferred from the servers to the AHEvent site or it might be logging is a low priority item on the server that was sometimes overloaded and couldn't keep up.  Real-time programming is an art and the data we get is mostly all we have and we just need to accept that. 

There might be cases where we lost a server or internet mid Frame and noone returned after it came back.  I do remember times when large chunks of pilots would disco.

There might be cases where the event design just didn't work out well and it was a slaughter that was over quickly.

There are some cases where multiple logs were produced for the same frame, one log for Axis, one for Allied, and I haven't checked yet to see if my data set collection has merged those two correctly or not.

I will say that there are possibly still some non-FSO event data mixed in and that may be the spikes up or down.  I have added some sanity checks along the way as I look at data to help filter out stuff that doesn't belong.  Things like, FSO is Friday Squad Ops, so if data is dated on a Thursday or Sunday, it's probably not FSO data.

The big V around late 2010 is really just a down spike with not a lot of event data on either side of it. 

What I have at the moment is data on 177 Events (months), 536 Frames, 244 Squads, 4273 Pilots, 152000+ sorties using 123 different planes, vehicles, PT Boats, and an RV-8.  Most of the data is going to be good, but some isn't.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 08:00:25 AM
My intentions in looking back at all this data is to understand what works and what doesn't.  To do that, I have been trying to recreate what actually happened based on log data.  In essence, I have been looking at automated after action report generation.  I have sort of been able to do that, but with mixed success.  There's a lot of info that can be picked up from the log data, but it's not reliable.  For instance, I can determine who are the defenders and who are the attackers with about 90% accuracy based on log data.  I can also determine active fields, field ownership, and most target fields, but I need better accuracy for targets to get the data I'm looking for.

I am starting to integrate some Objectives data into the mix.  I have about 2-3 years of objectives data and I have orders for a bit more than that.  The objectives data gives a little more info about active base and target proximity, required targets vs. what was actually destroyed and the orders give expected flight plans.  I was hoping to look at animated maps of flight paths, but that is a long way away at this point. 

If anyone has Objectives going back to 2009 and wouldn't mind sharing it, I'd like to talk to you about it.  Any more than the last 2 years or so, would help.

Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 08:13:30 AM
Here are some of the things I'm looking at producing and sharing from the data.

I will post separately on where I'm at with after action reports.

I have been looking at building up a "leaders board" of top squads and/or top pilots with some metrics just for fun, but haven't decided yet what meaningful metrics should be used.  This could potentially show a historical perspective of things the squad did to get where it is.  Some of what pushed me in this direction is seeing how many friendly fire incidents there were.  Departures and during engagements seem to be notorious for FF.  Another factor is I wanted to identify what squads were good at and use that in planning missions.

Another thing I'm looking at is writing up some software to produce pilot log books.  It seemed to be a natural side project that would be fairly easy to do from the after action report data.  It would be a mix of similarity to  WWII pilot log books and current pilot log books used today.  Part of this will be dependent on my either integrating Objectives data for everything or using software to intuit target information into the sortie description.  It would be nice to get the bombs dropped/target damaged info in this, but as it stands now, I can't tell the difference between didn't carry bombs, didn't drop, dropped and missed, jettisoned for escape, and dropped and damaged but didn't destroy.
Title: Re: A look back at the past 15+ years of FSO
Post by: Spikes on July 20, 2025, 08:55:20 AM
Quote from: AKKuya on Yesterday at 07:24:56 AM
I remember when we had 600 pilots and we were planning for 4 objectives to defend and attack.
I don't recall FSO ever having 600 pilots.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 09:01:24 AM
Here's a look at the kind of after action report I'm generating.  This one is hand generated based on the July Frame 1 log while trying to follow an algorithm in my head.  It doesn't include any flight path info from orders and does not include Objectives info such as coordinates, it is just what was shown in the logs.

22:05
    Allies
        1 defensive F4U-1C departed from Task Group #1 to cover Task Group #1
        14 defensive F4U-1C and 1 F6F-5 departed from #3 to cover Task Group #1
        8 offensive TBM-3, 8 offensive F6F-5, and 2 offensive F4U-1A departed from Task Group #2 for Airfield #16
        1 offensive F4U-1C departed from Task Group #1 for Airfield #16
    Axis
        6 offensive Ki-67, 7  Ki-61 departed from Airfield #16 for Task Group #1
        12 def N1K2 from Airfield #16 to cover Airfield #16
22:10
    Allies
        Pilot exchanges 1 F4U-1C for 1 F4U-1A out of Task Group #2
        1 F6F-5 is lost to defenders at Airfield #16
    Axis
        12 N1K2-J defenders at Airfield #16 engage 4 F6F-5s and shoot 1 down
        2 offensive Ki-61s depart from Airfield #16 for Task Group #1
22:20
    Allies
        4 F4U-1C + 1 F6F-5 defenders at Task Group #1 engage 2 Ki-61s and shoot 1 down
        4 F6F-5 jumped by 12 N1K2-J defenders at Airfield #16 and shoot 1 down, but lose 4
        10 F4U-1C defenders at Task Group #1 engage 3 Ki-67 formations and shoot 8 down and lose 1
        2 TBM-3 and 1 F4U-1A lost to defenders at Airfield #16
    Axis
        3 Ki-67 formations shoot down 1 F4U-1C, but lose 3 formations
        12 N1K2-J defenders at Airfield #16 engage 4 F6F-5s and 3 TBM-3s and shoot down 4 F6F-5s and 2 TBM-3s while losing 1 N1K2-J
        1 Ki-67 formation destroys a ship at Task Group #1 but the formation is lost
        3 Ki-61 damage Task Group #1 and sink 1 ship
22:30
    Allies
        3 F6F-5 departed Task Group #3 for Task Group #1
    Axis
        3 N1K2-J land at #13
        9 defensive N1K2-J, 5 Ki-61 depart Airfield #16 for Airfield #16
         10 offensive Ki-61 and 3 Ki-67 depart Airfield #16 for Task Group #1
...

I feel like this is better than the by squad,pilot time based facts, but is still a bit difficult to grasp all the moving pieces.  It is also the reason why I would eventually like to be able to replay with coordinates on a map.  Right now, we just know origination location, destroyed target location, and sometimes RTB location, and not a lot between unless an engagement occurs and then just who was involved in the engagement and not where.

I have some of this coded up, but I'm currently trying to deal with targets changing between hour 1 and hour 2.

If anyone has feedback on the style of the after action report or information presented, let me know.  I'd like to make it useful to someone other than just me. 

Once I get Objectives data integrated, I can add some additional distance and direction info, e.g. 6 offensive Ki-67, 7  Ki-61 departed from Airfield #16 for Task Group #1 100 miles to the NE
Title: Re: A look back at the past 15+ years of FSO
Post by: AKKuya on July 20, 2025, 12:19:44 PM
Quote from: Spikes on Yesterday at 08:55:20 AM
I don't recall FSO ever having 600 pilots.

Around 2009/2010, we hit 600 pilots for a few frames.  It was commented on.  i think I confirmed it looking at the logs.  15 years can make the memory a little fuzzy plus being on the other side of 50 now.  :old:
Title: Re: A look back at the past 15+ years of FSO
Post by: fudgums on July 20, 2025, 01:26:11 PM
I don’t anyone did yet, but it will inevitably turn on the CMs at some point lol
Title: Re: A look back at the past 15+ years of FSO
Post by: Devil 505 on July 20, 2025, 01:40:19 PM
Interesting data gathered.

There are a few elements that don't quite make sense though.

1. How are you calculating what a sortie is? I would expect to see the sortie number match the number of pilots except in FSO's where extra lives are given, but that seems to not be the case.

2. Your B-29 sortie count seems to be way too high. I can only think of 4-5 FSO's in total where the plane was used. No way their total use is 2900 sorties, even with drones and rearming counting as a new sortie.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 02:59:12 PM
Quote from: Devil 505 on Yesterday at 01:40:19 PM
1. How are you calculating what a sortie is? I would expect to see the sortie number match the number of pilots except in FSO's where extra lives are given, but that seems to not be the case.

2. Your B-29 sortie count seems to be way too high. I can only think of 4-5 FSO's in total where the plane was used. No way their total use is 2900 sorties, even with drones and rearming counting as a new sortie.

A sortie is defined as one pilot takeoff (launch) followed by one flight termination (shotdown, landed, ditched, bailed, disco).  You can get multiple sorties even in one life events when pilots are allowed to bail/replane at the start or disco/replane with CM permission.

There were 3 B-29 events: 2012-12, 2013-12, and 2014-10 with 3 frames each.  I just did a quick scan of the html files and you are correct.  There were only 318 sortie departures listed.  I just validated that the DB has 318 sorties, so now I need to go back and find what I did wrong with a SQL JOIN.
Title: Re: A look back at the past 15+ years of FSO
Post by: Devil 505 on July 20, 2025, 04:02:44 PM
Quote from: RedBeard on Yesterday at 02:59:12 PM
A sortie is defined as one pilot takeoff (launch) followed by one flight termination (shotdown, landed, ditched, bailed, disco).  You can get multiple sorties even in one life events when pilots are allowed to bail/replane at the start or disco/replane with CM permission.

Ok, so rearms don't factor in. Gotcha.

Quote
There were 3 B-29 events: 2012-12, 2013-12, and 2014-10 with 3 frames each.  I just did a quick scan of the html files and you are correct.  There were only 318 sortie departures listed.  I just validated that the DB has 318 sorties, so now I need to go back and find what I did wrong with a SQL JOIN.

Ketsu Go in April 2011 also had B-29's. It was the first one that used the Superfortress. Can't pull up the logs for that month for some reason.



Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 20, 2025, 04:42:23 PM
Ok, again, my apologies for the bad data.  I've fixed the SQL and rerun the queries.  This gets the count of "Departed" either matching or closely matching the sortie count.  There are still anomalies here and there where someone shoots down a plane that hasn't taken off (late start of logging?) or just weird data like a squad being listed twice for the same side with slight variations in the squad name and no merging of the two into one.

PlaneTotal SortiesSorties With DmgTotal Landed SortiesSorties Landed With DmgChance to Make TargetChance to Destroy Target and Get Home
A-20G10073921709038.93%8.94%
Ar_23416258924535.80%27.78%
B-17G129563663242349.11%32.66%
B-24J129751058431539.32%24.29%
B-25C25731171103357445.51%22.31%
B-25H21063341430.00%6.67%
B-26B100248051130547.90%30.44%
B-29318199948162.58%25.47%
B5N29121091742911.95%3.18%
Boston_III128656445825043.86%19.44%
C-47A7422102.70%0.00%
D3A112713522176627.69%5.19%
G4M1_Model_11111833235813229.70%11.81%
He_111H136555365138140.51%27.91%
Ju_87D-3182074432716640.88%9.12%
Ju_88A-4372117831607100847.92%27.09%
Ki-677732282819929.50%12.81%
Lancaster_III12864483850.00%29.69%
Mosquito_Mk_XVI12045532837.50%23.33%
SBD-5139738636911127.63%7.95%
TBM-310531892284717.95%4.46%
Tu-2S74236823213249.60%17.79%
Title: Re: A look back at the past 15+ years of FSO
Post by: Molsman on July 21, 2025, 12:45:38 AM
Red

How many FSO s had late war events with the Brits having lancs and the temps and typhoons against the late war German fighters. That was my first event I ran as a Cm and was just curious that was also when we had the numbers back in the day. not Now
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 21, 2025, 07:35:02 AM
Why do I feel like this is a test?

As far as I can tell, there's only been one FSO event with a Lancaster, Tempest, and Typhoon plane set.  It was Operation Clarion in April 2014, and yes, this did stretch my SQL skills.
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 21, 2025, 07:39:07 AM
By the way, I show plissken as the CM for all three frames for Operation Clarion, though perhaps it was multiple people and plissken just happened to be the one dealing with the logs?
Title: Re: A look back at the past 15+ years of FSO
Post by: icepac on July 21, 2025, 08:09:21 AM

The stats are skewed by the bomb and bailers who have no intention of getting back home.
Title: Re: A look back at the past 15+ years of FSO
Post by: hazmatt on July 21, 2025, 09:29:03 AM
Quote from: icepac on Today at 08:09:21 AM
The stats are skewed by the bomb and bailers who have no intention of getting back home.

Why would somebody bomb and bail in a 1 life event?  :headscratch:
Title: Re: A look back at the past 15+ years of FSO
Post by: RedBeard on July 21, 2025, 11:55:10 AM
Quote from: icepac on Today at 08:09:21 AM
The stats are skewed by the bomb and bailers who have no intention of getting back home.

Don't think so.  There appears to be a consistent fluctuation of bailing pilots throughout the history of FSO that stays consistent even after we started multi-life.

(https://i.imgur.com/r3q8gnR.png)
Title: Re: A look back at the past 15+ years of FSO
Post by: Spikes on July 21, 2025, 02:18:47 PM
Quote from: RedBeard on Today at 07:39:07 AM
By the way, I show plissken as the CM for all three frames for Operation Clarion, though perhaps it was multiple people and plissken just happened to be the one dealing with the logs?
Yeah the name on the logs is who ran them in the arena. It's a bit strange because Operation Clarion isn't in our site at all, but there seems to be a few monthly articles missing (I don't see FSO setups for Feb, March, or April of 2014). There can be a variety of reasons for this but usually it happens when someone updates/modifies an article rather than creates a copy or something, or it gets categorized wrong in Joomla.