It is bad manners in my view especially when they are not your friends already. Of course you can cancel the fixtures if you catch them before they are played but remember that you do not lose any condition as the challenged team.
Nonetheless, You could message the guy or even add him as a friend?