I think that you are unable to add some of these people because they are on a different server and therefore the only way you could do it is to add them as a friend in Facebook.