i feel you ahah, but its a rule: you lost two times with him so he passes despite goal difference
another scenario would be like this one
Click image for larger version.  Name: Screenshot_122.jpg  Views: 20  Size: 97.8 KB  ID: 91111
in this case we both won all matches and draw against each other, i have a better goal dif, dominance in goal scored, BUT, in the draws, the first one i was away and scored 0 goals, at home he scored one goal away, so he basically " beat " me and with the rules of goals away he goes after me.

Oh an you can't imagine how much i sweared XD