I read that the number of supporters doesnt really matter so dont worry too much.
Personally my support is always max so i cant tell the difference.
if it does affect you, just add a few active friends. 6 managers supporting him is a small number. You can quite easily get more.