But he's an ex-leader and therefore a man of some "standing" in the party despite his somewhat dubious past, plus he's a right winger and a possible focus for other, lesser known, right wing nut jobs, so, he was allowed to stay. In short because Cameron lacked the authority to sack him and get away with it.
But he has made an almighty mess of running his department, he has used doubtful statistics, and on several occasions he's just plain lied.
Liam Byrne (another incredibly unlikeable soul) has summarised IDS's performance in an article for the Telegraph.
And like or dislike Byrne, you have to admit that he is right on this. That said, Byrne is pretty much behind the policies. It's just the way that they are being implemented and the fact that it's IDS and not Byrne himself getting the ministerial salary and perks that bothers him most.
So what does he say?
He points out firstly that the Work Programme has missed every single target that it set itself for the last 3 years. That's a pretty damning indictment.
Then he moves to Universal Credit. The DWP promised only last year that a million people would be on this credit by 2014. Now they say that it is being rolled out to 10 jobcentres from October... and they are all in low unemployment areas. No one from the department is commenting on the number of people that this will cover, but it seems unlikely to be a twentieth of the number promised.
The Youth Contract has been a failure. It has missed 90% of its targets and the unemployment number for young people remains at around a million. A massive failure to the country and its young people.
Finally the Work Capability Programme has probably been the biggest ever fiasco.
We had people being taken off benefits only weeks before their death from cancer. There are stories of vulnerable people being told by someone they accept as a doctor (although they may be a nurse or paramedic or hospital cleaner from what I can tell) telling them that they are fit for work, them believing it (some people take what a "doctor" says as gospel), and dying of heart attacks because of over exertion brought on by the belief that they are now well!
Additionally, we have learned from whistleblowers, that Atos is working to targets and staff are under pressure, trained or not, to fail people to in order to keep to the company's profile. Both the BBC and Channel 4 have made documentaries (Panorama and Despatches) showing this. Of course these claims are denied, but I've never heard of a government department that doesn't work to targets and budgets, and I doubt very much that any outsourcing organisation doesn't have the same.
Atos has been audited independently and found to be failing to work to its contract. Hundreds of cases were found to have been determined without any logic or application of the rules (presumably in an attempt to meet the targets that the DWP and the Atos management swear do not exist). The company has done so badly that its monopoly is being taken away from it.
(It has, of course, only joined a long list of outsourcing companies which have been found to be dodgy, incompetent and stealing from the taxpayers.)
And of course the cost of tribunals and appeals has tripled since 2010.
So basically IDS 0/10.
Will the prime minister take his future in his hands and sack the monstrously incompetent cabinet secretary?
Probably not. After all, who else has he got to put there?
PS... BBC report that there is a 5% increase in people dying in England and Wales this last year. I wonder if IDS has anything to do with that.