ATM if hes a friend on FB you can only remove him. you can also try talking/message him to stop. thats what i do with friends, both sides must agree via FB chat.

*edit*
if hes only a friend on...