if you cannot add someone with a button , it means he is in a different server.
Only with fb can add him.