[NEFTA] Propose that the grading system’s rolling 20 should be lowered to a
i. Rationale: Currently grades hardly change.
c. [SWEFTA] Rolling 20 to become rolling 16.
i. Reason, from my experience over the previous years of doing and
monitoring Grading, the 2014 NEFTA proposal to reduce to rolling 10 is
far too drastic and will open up too many opportunities for "ringers" to
drop a grade far too easily. In truth, I Believe the rolling 20 is working
fine, but a reduction to 16 is a compromise that is worth trying for a
Let's talk about the above. Anyone who knows me will know I love a chart, graph and a load of maths, so I've been looking at the grades with some interest for a while now and the above really breaks down into 3 sections:
1. Number of scores used to derive at %
3. Maths used.
Looking at the above proposal we can see that the concern is that there's not much movement in the grading system.
This isn't caused by the length of the rolling number but by only updating the grades twice a year, once after the Winter League (11 shoots on average) and then after GP's (8 shoots on average). There's a couple of shoots in between that get graded but not too many.
So shorting the length of the rolling 20 to 10 or 16 won't have much if any effect as they are still only changed twice a year. You need to also increase the frequency of the updates as well if you want more movement on the grade tables.
The other problem with shortening the length of rolling 20 is you actually weaken it's accuracy because we use Mean average to calculate grades. The problem with using Mean is that it is very very susceptible to what's called Outliers
. These are pretty bad when using Mean to calculate anything, it only takes 3 outliers to keep a B Grade shooter in C and let's face it, it's not hard to have a bad day shooting FT.
Here's an example using Mean, this guy in the example gets off to a bad start to the season but one look at his recent scores tells you he's not a C Grade Shooter but the Maths we are using says he is.
I've also used the same maths for 16 rolling scores with one less Outlier and you can see that it actually makes the accuracy even less reliable and inaccurate because it takes less Outliers to break the score.
So as you can see shortening the rolling 20 actually doesn't help. So what will? There are two things we need to do to get the grades more accurate and moving. Change the frequency with which the grades are updated and change the maths used. I'd recommend every 5 shoots for grade changes and I have two ideas for using different maths that aren't really that complicated.
One is this: Time Fade gives more importance to recent performances from the competitor. It makes old data less important than new data. It does this while still rewarding consistency, but also doesn't punish competitors who show genuine improvement. The other effect is a single good, or bad, performance will skew the numbers less.
So using this method 80% of your grade is calculated for your last 12 shoots and 20% from the other 8, simple enough and more accurate.
The second method is to use P Factor maths to "tune out" the Outliers. Here you use a simple calculation to tune out the low scores so they have less impact.
I will say that there is no perfect way to calculate averages where you have Outliers but Mean is pretty poor in the first place.