Check your manual (and if you can online) to find out where the power switch leads go. I'm nearly 100% certain this is the problem. (I don't know why, but I
always accidently put the switch leads on the wrong pins the first time I set up a new system.) It may also turn out that you have a bad switch in the case.
Those ATX switches are simply a momentary switch, so you can easily eliminate a bad switch from the equation by just getting a paperclip (or something else metal) and quickly contacting the two pins together that the switch leads go to (remember, you only need to touch them for a split second). This should power up the system. BE VERY CAREFUL TO ONLY SHORT THE SWITCH PINS - the system will turn on immediately when you touch them.
If that doesn't work, your powersupply could be bad. (Yes, defective new supplies do happen.) Testing an ATX supply is a little bit tricky, but it can be done. First of all unplug all the power connectors in the system and take the power supply loose from the case. Now locate the large ATX power plug that goes to the motherboard directly. On the side of the connector with the little clip you will see a small green (99% of the time) wire near the center of the plug. Make sure to have the power switch on the back of the power supply on if it is present. Now you can actually attempt to turn on the powersupply by shorting the green and black wire immediately next to it together. As long as these two wires are connected the power supply should be on (i.e. fan running). If you have a voltmeter it would be a good idea to measure the voltage on an IDE power connector (hard drive/cd drive power connector) to ensure the powersupply is working. The wiring on these plugs is: Center 2 black wires are ground, yellow wire is 12V+, red wire is 5V+.
_____________________________
________________
My guess is that it's as simple as the power switch leads being on the wrong pins. (USB header connectors often look identical to the jumper block for the reset switch, HD led, speaker, and power switch.)
Good luck!