I never go more than 4 days without assigning skill points. I see no advantage in keeping them.