If you agree that (both teams') mentality and pressing have some effect on GK performance then one possibility is:

Managers who anticipate not watching or "playing" matches could set their team to be more defensive than if they were actively managing.

If 3 is the most goals the unwatched team scored in your examples, not huge numbers, seems more like defensive than attacking. It seems possible these indicate tactical mismatches.

In 2 cases I lost or tied after having a small 1-0 lead before opponent arrived late 2nd half. I think they came, saw they had no time, needed a goal and changed to hard attacking, and applied a possession/ attack bonus; I didn't react the right way. In another 2 I arrived to see a lead, but then lost... these could be as easy as I changed the conditions to favor their counterattack, or they switched from attack to possession when I arrived, and that's all they needed.

/Rambling. I wish I knew the mystery! Just trying to figure it out