This is one the things it is just impossible to control.
I can play a game with normal tackle and get yellow cards and even red, i play a game with hard tackle and have a clean sheet.