It's the top 14 players that count for your 'average'.

I know that because last season I had 4 one * players to bring my average down, yet my cup draw was still with teams all 3 levels above me with roughly the same average as my top 14. The one * players were not counted sadly.