You never switch servers that I know of. If you directly invite someone to the game they should be on your server, but perhaps not in your league.