The 20 point difference is between HIGHEST and LOWEST of the three categories. That is why when you added to physical, it made no difference, because the gap between attack and defence hadn't chamged. It dosen't matter which two categories the difference is between.