Yes yiou can do it but not when the tournament is ON, you must wait until the tournament is over and only then you will be able to do it. What can happen? Nothing, they will just be removed, as long you are the captain, you are free to do what you think is the best for your Association.