First lets talk about our artificial refinery shortage.. back in the day there were plenty of refineries out there.. Mom and Pop operations were common and the majors were there in force also.. As a result of this free market supply would meet the demand rather well keeping fuel prices stable .(low) while that stability is great for the consumer it keeps the markets down for the producers due to the competition from the various other competitors..
The refinery reduction occurred after deregulation. Before that prices to the consumer were higher (in that period’s dollars) than they are today. Prices were stable (but high) because they were fixed by the government. Supply was immune to casual disruption and volitility (outside of OPEC issues) because there was serious overcapacity.
Also, most of those record profits that have sparked this discussion (and even the non record profits of years past) come from exploration and production (moving crude) and not refining:
Still, record crude oil prices -- which touched $70 a barrel in the quarter -- pushed earnings at its exploration and production unit to $5.73 billion, up $1.8 billion from a year earlier.
At its refining and marketing operations, profit rose to $2.13 billion, up $727 million from a year earlier. Stronger refining margins outweighed weak marketing margins and lower petroleum product sales. http://money.cnn.com/2005/10/27/news/fortune500/exxon.reut/Then the majors got a bright idea, "hey if we jump on the enviro bandwagon that was going on in the 90's we can lobby for tougher standards for the refiners to make gasoline heating oil etc..."
I would like to see the hard data that supports the industry pushing this as a master plan. It’s a nice theory, but IMO it gives the industry too much credit for evil brilliance. As a plan, it really failed to pay off for about 20 years. I can't imagine any industry having that long term an outlook on profits. And as noted, refning may not be the "tail of the dog" but it's certainly in the bellybutton end where profits and risk are concerned.
the plan is rather ingenious, by using lobby and campaign contributions they got their laws... and while the majors had the stock backing of their shareholders for the purchasing power to meet the new regulations the mom and pop operations were left in the wind.. unable to upgrade they were gobbled up by the majors but instead of keeping those operations running they then closed them limiting capacity to refine.. now this didn't happen over night so to the consumer little or no change was to be noticed.. that is until any disruptions to the new weakened refinery capability was to be experienced..
So thats why when you breath the wrong way at our refinery infustructre there is a major crisis in the laws of supply and demand doing exactly what charon has described (very good points on many of your posts.)
The industry found itself, as it left regulation, with significant over capacity in refining, and with a lot of mom and pops that couldn’t survive anymore (after 1973) in a non regulated market from both an efficiency and environmental upgrade standpoint. The same thing happened to many retail and marketer operations that had enjoyed artificially "set" high retail margins and suddenly found those artificial supports removed.
U.S. refining capacity, as measured by daily processing capacity of crude oil distillation units alone, has appeared relatively stable in recent years, at about 16 million barrels per day of operable capacity (graph). While the level is a reduction from the capacity of twenty years ago, the first refineries that were shut down as demand fell in the early 1980's were those that had little downstream processing capability. Limited to simple distillation, these small facilities were only economically viable while receiving subsidies under the Federal price control system that ended in 1981. Some additional refineries were shut down in the late 1980's and during the 1990's, always, of course, those at the least profitable end of a company's asset portfolio. At the same time, refiners improved the efficiency of the crude oil distillation units that remained in service by "debottlenecking" to improve the flow and to match capacity among different units and by turning more and more to computer control of the processing. http://www.eia.doe.gov/pub/oil_gas/petroleum/analysis_publications/oil_market_basics/Refining_text.htm The decision, for a long time, has been to have fewer refineries but more capacity at those refineries. We currently do not have under capacity, we just don’t have over capacity (a goal with any business or industry). It becomes an issue during peak periods when you have a refinery fire or a pipeline shut down (or in this case a natural disaster at the worst place and the worst time). It is not an issue otherwise. Refineries have made more money in the past few years, but there is no consensus that the “good times” are here to stay. Average returns have been in the 5 percent range, which is poor compared to industry in general. Low enough to discourage investment throughout the 1990s and at the same time encourage all of those integrated major oil companies to pass off many of those refineries they “gobbled up” to independent refiners and refiner marketers like Valero. Refining just didn’t (and doesn’t) bring in the money compared to exploration and production -- too much trouble for the returns because of all of those environmental regulations they “fought” for

Now im all for free market when everybody plays by the rules of supply and demand.. but when you decrease supply capacity to increase demand imo thats a trade violation in regards to a commodity that is a part of our lives in so many ways... in effect holding the consumer / nation hostage..
The FTC has consistently disagreed.
IMO with a commodity so precious to our western way of life, goverment should take over the responsibilty in the entire US energy industry.. Works for power companys.. (use FPL for a example) they are on the ball and held accountable by the florida govt..the state just approved a increase they requested and by peer review they needed it due to rising energy costs.. very few shenanagins from them..
Quite honestly I dont trust the energy security of the USA to a free market.. its just too valuable in our daily lives to leave open to the vulnrability of market manipulation, either domestic or foreign with the huge economic penalties that come with such tampering..
The difference between the oil industry and the electric industry is that one is regional in scope and self contained, while the other is global in nature (infrastructure, ownership, competition, demand for raw materials and finished products, import and export dynamics) and where the US is hard pressed to even be considered a marginal produce of the raw product that grossly influences the price of gasoline. It’s hard to imagine a functional solution within these realities that would work as well as the current one does, even with the problems inherent in the system.
Btw, even recently, with the exception of a few years (and this is one) oil industry profits tend to lag the average for industry. As long as crude stays above $40 bbl that will not be the case.
Charon