Mr Letwin told people in Brixton of his dream

The shadow home secretary told BBC One's Breakfast with Frost that talk of plotting "makes a tremendous story... but it will drop away".

Instead the focus would turn to the new programme of policies which captured people's attention despite being unveiled at a party conference "which was marred by all this stuff about leadership".

Mr Letwin's comments came the day after Sir Patrick Cormack, an executive member of the backbench 1922 committee, called on Mr Duncan Smith to end the current plotting by calling a vote of confidence in his leadership.

The Staffordshire South MP said that if the party leader won the vote, his opponents should rally round him.

The Tory leader is currently being investigated by Parliamentary Standards Commissioner Sir Philip Mawer over the office role played by his wife Betsy.

These cowards have gone after my wife as a way of getting to me and I think it's intolerable, absolutely intolerable Iain Duncan Smith Analysis: IDS talks tough

Mr Letwin said he expected his party leader to be totally cleared by Sir Philip, and that would be the trigger for the party to rally round and concentrate on winning the next election.

He said that the party had made "massive improvement" in opinion polls over the past two years but still had to make "huge progress" in the 18 months or so before the next election.

"We have to do better than we have done in the past few weeks, I don't deny that," said Mr Letwin.

Labour had had a "six year honeymoon" from the British people but that was over and people were now thinking of "divorce" -"and we are putting ourselves forward as a new possibility".

Blunt message

Sir Patrick said on Saturday: "The fact of the matter is there are those from the shadow cabinet down who are not happy with what is going on at the moment."

The Tory leader should "bring the thing to a head" and "call the vote himself" he said.

"If he wins that, everybody should belt up and get behind him. If he doesn't we have to move on in a very seemly way to choosing a successor."

Sir Michael Spicer, the chairman of the 1922 Committee, is empowered to call a leadership contest if he receives 25 letters from MPs.

I don't think a combination of Disraeli and Churchill could run the Conservative Party effectively in the present circumstances Chris Patten Patten 'feels sorry for Tories'

He is not believed to have received that many letters, although some commentators believe Tory malcontents may be holding fire for the time being.

Edward Garnier, who also sits on the 1922 committee executive, said Sir Patrick was representing only his personal views and not those of the committee.

Shadow foreign office minister Alan Duncan said Tory divisions were "exaggerated and subsiding".

Mr Duncan told Sky News's Sunday with Adam Boulton: "Hand on heart I say stick with him and let's concentrate our efforts on advancing policies and trying to position the Conservative Party to being a credible opposition to Labour."

Parkinson

Crispin Blunt, who resigned from the shadow front bench in protest at Mr Duncan Smith's leadership in May, said a leadership vote was needed to end the speculation.

"The essential thing is we have either got to give our leader authority to lead us into the next election or we've got to initiate the process of choosing a new leader," he told BBC One's Politics Show.

A shadow cabinet team with Ken Clarke, Michael Portillo, Oliver Letwin, David Davis and Michael Ancram would be a "pretty formidable alternative government in waiting," he said.

Lord Strathclyde, Tory leader in the House of Lords, said once Mr Duncan Smith was cleared over allegations about his wife's employment "we can get on with what the Conservatives need to do which is to be a strong alternative government in waiting, a strong opposition".

On Saturday ex-Tory chairman Lord Cecil Parkinson said he did not believe there was a need for a vote of confidence, pointing out Ted Heath and Margaret Thatcher had both won elections after facing criticism as leaders in opposition.