Speed is speed, a fast drawer will be pretty quick out of any holster. You're right that when you're prepared to draw on a buzzer, it's nothing like a real life encounter with a threat, where much of the time they'll have surprise on their side, and you're reactions will play a huge part. Once you're recovered from your startle response though, your draw speed is what it is unless your draw is interfered with in some manner due to being in a cramped position, contact distance to the threat, whatever. I train mostly with a level 2 or 3 Safariland retention holster, that's what I was issued and what we used when I worked for a couple PMCs, I'd never want a retentionless kydex/whatever open top outside belt holster for anything other than a place to hold my handgun out plinking somewhere. I've trained many shooters to get under a 1 second draw with the Safariland holster in an open carry config. CCW with an IWB open top holster, sub second is something most shooters can get to if they want to put the time in.
I agree, draw speed is far from the most important skill, but I'd rather be faster than slower myself. The best school I've been to is hands down Rogers shooting school, I'd rate it probably #1 in the USA by a pretty fair margin in terms of straight up pistol speed skills, and right from the first minute, you're shooting against a timer vs reactive moving plate targets. In training the only way to simulate stress conditions is adding the pressure of time, outside of using sim/air munition and force on force (ie people shooting you with things that sting), but it's hard to teach and learn things in force on force. It's a good tool IMO for trying/testing skills out, but not for actually learning them.
Time-Distance-Cover, the equation/loop that rules all gunfights, all are important, but cover is the one which is most frequently...not ignored, but has the least amount of stress put on it in training. IMO it's a critically important part of the equation, as it more than the other two will be most likely to keep you alive, more so than draw speed (which is part of "time", but as you pointed out, distance as well, in terms of creating distance from a contact range threat), or anything else. I think spending time on identifying cover, how to get to it quickly, and how to effectively fight from it, is time better spent than learning how to "fast draw" ie the whole Tex video. You never see guys putting up videos of doing that though, moving at an angle on the way to cover.