Olympics £440m 'drain on culture'