You cant remove.

If you do not whant to sign more skill points, add them manually and not automatic.