AdrianC is right, Dids, repeatedly restarting a router is interpreted as a 'drop' in connection, and the broadband circuit slows down in an attempt to compensate.
It's usual for broadband circuits to slow from their initial installation. They stabilise over the first week-to-ten-days and will then settle to the optimum speed for the line.
I suggest that you just leave it as is for a week or two and then report to BT if it slows any more.
