This abuse mess is way beyond just Rumsfeld resigning, but it's a good start. I'd tell you my thinking, but it's along the line of Max Boot's thinking, and he's got solid conservative credentials:

"Secretary of Defense Donald H. Rumsfeld's failure to offer his resignation over the Abu Ghraib scandal is sadly typical of the lack of accountability that permeates the U.S. government...What...is the case for Rumsfeld resigning? Simply that this scandal has caused devastating damage to America's moral standing in the world, and we need to recover fast....Having the Defense secretary resign might salvage some good out of this house of horrors by causing Arabs to ask why their governments tolerate torture and ours doesn't...." (Max Boot, LA Times, 5/13/04)
http://www.latimes.com/news/printedition/opinion/la-oe-boot13may13,1,4803979.column (I think you may have to subscribe to read. And you should read it because I don't want to infer by my selective quotes that Boot is completely down on Rummy.)

And does anyone really, really believe Victor Davis Hanson's schoolgirl-crush gushing about how splendidly, scrumptiously, wonderful Rummy Tum-Tum is as Secty of Defense? As Boot says, it "doesn't pass the laugh test."

My theory is some short-term thinking Republicans are putting partisan political worries ("Let's never ever do ANYTHING Teddy or 'Flapjack' recommends!") above the need to solve this mess.