I agree with you, whenever I'm absent (usually I'm stronger side) , easily get a 3-0 4-0 win , but when I attend the match, my team struggle a lot and might end up with a draw or even a lost
I experiment this by not joining the game until the match almost over (second half) , I join in and see the score is 0-0 , not long after that they score 1 goal and win the match...