This happened in an Association game to me as well. One of my team mates told me if your Association games start and you click away to watch a friend's game when you come back to your Association game you become just a spectator and can't make changes. Don't know if that's correct or not.