Players without the invite tag will be on a different server to you. The only way to become friends with these players is if you become friends through Facebook.