Yes unfortunately you made a common mistake.. you should only upgrade on the day when you get new youth players which will technically only cut your wait time by 1 day because you are cutting the wait time by 4 days but waiting for the 'clock' to start after construction which takes 3 days. This all is, of course, assuming no token investment to speed up the process.