It doesn't really matter.
I lost my keeper in a game a few seasons ago after about 7 minutes. I was already a goal down and decided not to put a sub on and leave no-one in goals, and I won 3-1.