Lord Livermore said it was his ‘personal view’ that the UK would rejoin the bloc