Stars are only relative for your level. it's the quality that matters..

As you get to higher levels, your players will each lose a star.