I do that.
I've only one keeper in my squad and when he's injured or suspended then i put a defender in goal. It doesn't seem to make much of a difference.
Good example today of the worth of a keeper was the team just below me lost 5-3 to an opponent who had his keeper injured in the first minute, no sub came on and he proceeded to make 3 saves and get a rating of 7 even though he wasn't on the pitch.