I've had the same problem when playing from my desktop.

From mobile, I haven't had any problem being able to support non-same server friends.