I made the mistake of building my stadium too fast, and in ways being penalized for having "relatively" small crowds. Its fixing itself as I advance, and I just upgrade other parts of my club.