Well... the fan base, the parking, even what Competition it is can affect how many fans show up.
The stadium needs to be big enough to hold the level's max attendance as well.