nope. all is ok.

A 2-3 B

B 0-1 A

A vs B = 3-3 the differance was +1 goal in both results but he scored 3 in your field and you 1, so he won.

the same if you win 1-2 and loose then 19-20, he scored 20 in your field, he won.