Hard to say what went wrong. If you can't figure it out and correct it you can go to the HTC/AcesHigh/Settings folder and delete the Video8 file there. When you restart AH it'll put you back in the default video configuration and you can start over.
Mace