RSA is an arm of the Royal Bank of Scotland

The RSA - which was founded in 1990 as a joint venture between the Royal Bank of Scotland and Scottish Equitable - has set aside �50m to cover claims for compensation.

Endowment mortgages - where the borrower takes out an insurance policy to pay off the loan - were at their peak in 1988, when they accounted for more than 80% of all new mortgages.

But the investment background is now very different, which means some policies look unlikely to produce a big enough sum to pay off the mortgage.

Mortgage debt

The FSA criticised the life insurance firm for "serious deficiencies" in the way it handled, marketed and calculated its flexible mortgage plan endowment policy between 1990 and 1994.

The authority said RSA "failed to act with due skill, care and diligence" in calculating the premiums.

These were set at such a level that policies would not repay the outstanding mortgage debt when it matured.

I am extremely disappointed that during the period in question the conduct of Royal Scottish Assurance's management was of a standard that was not remotely close to that demanded by the Royal Bank of Scotland group

Benny Higgins, RSA chairman

The FSA said: "RSA implied a level of expertise that was absent, as was evidenced by its incorrect pricing of its endowment policy premiums."

The authority also highlighted an incident in September 1996 when 300 plan holders were sent letters suggesting they increase their premiums to reach the sum needed at maturity.

"This letter misleadingly explained that the increase was necessary because of reductions in projected rates of return.

"In reality, the main reason was to address the shortfall that would arise as a result of the firm's undisclosed policy pricing errors," said the FSA.

Unreserved apology

The letter also failed to give customers the information necessary to make a balanced and informed decision.

The authority also criticised RSA's failure promptly to inform the Personal Investment Authority - now part of the FSA - about the problems.

RSA chairman Benny Higgins said he apologised "unreservedly" to policyholders, and regarded the failings as "wholly unacceptable".

"I am extremely disappointed that during the period in question the conduct of Royal Scottish Assurance's management was of a standard that was not remotely close to that demanded by the Royal Bank of Scotland group," he said.

Damages action

He said there was now a completely new management team in place at the business, and "far-reaching" changes had been made to its operations.

When RSA was created it was decided that policies - including the flexible mortgage policy - should be provided to RSA by Scottish Equitable.

"Whilst we fully accept that as the regulated entity RSA is accountable to both customers and the regulator, we have raised an action for damages against Scottish Equitable, who provided the product to RSA," added Mr Higgins.