Yes, the choice is yours: your player is going to progress in special ability OR in overall skills. But at any moment, you may pause the development of the SA for gaining skill points again.