Have you waited the 24 hrs (or whatever time length it is) for the other manager to either accept or reject your offer?

You only get the tokens back after the offer is either rejected or the time limit expires.