As long you will have him on facebook he will be in-game too I don't think is a way to remove only from game as long he is on your facebook too.