By Martina Purdy BBC Northern Ireland political correspondent

Losing one party member is unfortunate, two appears careless and three is rather worrying if you are trying to rebuild popular support.

The recent defections of Peter Bowles, James Leslie and Bob Little from the Ulster Unionist Party to the Conservative Party has led to speculation about Tory intentions - and the prospect of other resignations.

Lord Trimble believes a deal may still be reached on devolution

One Ulster Unionist said he had recently been sounded out about his attitude to joining the Conservatives.

While he has rejected such a move, he believes others are being approached.

The names of two assembly members have been mentioned.

One name being bandied about in the tea rooms of Stormont and beyond is David Trimble.

The speculation is fuelled by his close relationship with the Conservatives, his confirmation this week that he had discussions with the Tories about restoring the link and the high esteem in which many Conservatives hold him.

Would the former Ulster Unionist leader defect?

Others in the party are not convinced that Lord Trimble - with the closure of the assembly - won't be wooed into the Northern Ireland Conservatives

When contacted, Lord Trimble declined to comment saying it would only lead to people taking such speculation seriously.

But others in the party are not convinced that Lord Trimble - with the closure of the assembly - won't be wooed into the Northern Ireland Conservatives.

This would improve his profile in the Lords, it is suggested, and would give the Conservatives a high profile, even if some unionists dismiss Lord Trimble as "electoral poison" among unionist voters.

'Not been interested'

Lord Trimble's fans say there is still much goodwill towards him for his efforts in trying to bring stability to Northern Ireland.

What is clearer perhaps is the changing attitude of the Conservative Party to Northern Ireland.

With a new leader in David Cameron, who has visited Northern Ireland once already and plans another visit later this month, there appears to be a new warmth from across the Irish sea.

At Bournemouth this week, the Conservative Party Chairman Francis Maude spoke about how issues such as education and the environment affect voters in Northern Ireland.

"Spin being put on Mr Maude's remarks was note-worthy"

In the past, local Conservative members have grumbled that Central Office has not been interested in them at all.

The spin being put on Mr Maude's remarks was note-worthy: "Party chairman... added his support for continued expansion of party in Northern Ireland."

Bob Little, the Ulster Unionist turned Tory, said no-one approached him join the Conservatives.

He said he made his decision because he wanted to build on David Trimble's work to secure the union by "adding value to the United Kingdom."

"We can't do that by remaining in our insular world."

He went on: "We have a sea change in attitude. The Conservative Party is now saying we want Northern Ireland to be full members of the party."

Like James Leslie, Little is supportive of Mr Trimble.

Some believe the Conservatives see opportunities to build a base here through the new super councils and in any devolved assembly should it return.

The opportunity, it is argued, is offered by the political and financial weakness of the Ulster Unionist Party - and an antipathy of some unionists towards both it and the DUP.

Mr Little does not subscribe to the view that the Conservatives will never recover from signing the Anglo-Irish Agreement.

David Burnside condemned the trickle to the Tories

He insists there is no party in Northern Ireland that doesn't have "baggage."

David Burnside, a UUP assembly member for North Antrim, condemned the trickle towards the Tories as counterproductive and unwise.

He said he had intended to kickstart a debate on rebuilding the formal link between the Ulster Unionist Party and the Conservative Party at the UUP's annual conference this autumn.

But the conference has been cancelled. Instead the party intends to merge its annual conference with its annual general meeting which is held in March.