Is it objective or subjective when judges decide how well a jump was executed?
Might one judge favor Sasha's air positions and another favor Miki's higher jumps? Do all judges agree on this and to what extent? Is a higher jump always better than a jump with terrific air position?
I think a first step is to get people to stop equating "objective" with God and "subjective" with the Devil. Yes, judging is subjective. Now that we've got that point settled, on we go with trying to come up with the best possible system. :yes:
The fall in skating's popularity has forced ISU to cut the judging panel down to five marks that will determine a skater's score. The previous system used nine sets of marks. Does this make any difference? Are mistakes likely to be amplified with such a small panel?
To get a feel for the effect of trimming the mean (using only the middle five numbers instead of all seven), here is a little bench experiment.
Suppose that the entire universe of Interpretation scores given to a particular performance is
{6.00, 6.25, 6.50, 6.75, 7.00}
That is, for the purpose of this experiment we are pretending that there are only five well-qualified, competent, honest, experienced, and well-trained figure skating judges in the entire potential judging pool, and these are the scores that those five judges would give to the particular performance under view if they were on the panel. The average taken over all judges in the universe is
6.50, and this is the best we can ever hope to do at defining the “correct” score for this performance.
Now let’s say that the actual judging panel comprises three judges. We will compare the full mean using all three numbers to the trimmed mean obtained by discarding highest and lowest.
Here are all the possible outcomes, for the ten different judging panels that might be chosen to officiate at the actual contest.
{6.00, 6.25, 6.50} Untrimmed mean = 6.25, trimmed mean = 6.25. Both are “off” by 0.25.
Tie.
{6.00, 6.25, 6.75} Untrimmed mean = 6.33, trimmed mean = 6.25. Untrimmed mean is off by 0.17, trimmed mean is off by 0.25.
Untrimmed mean is better by 0.08.
{6.00, 6.25, 7.00} Untrimmed mean = 6.42, trimmed mean = 6.25. Untrimmed mean is off by 0.08. Trimmed mean is off by 0.25.
Untrimmed mean wins by 0.17.
{6.00, 6.50, 6.75} Untrimmed mean = 6.42. Trimmed mean = 6.50. Untrimmed mean is off by 0.08. Trimmed mean is exactly right.
Trimmed mean wins by 0.08.
{6.00, 6.50, 7.00}. Both exactly right, 6.5.
Tie.
The other five are similar. In all, the untrimmed mean is better four times out of ten, the trimmed mean is better two times out of ten, and they are the same four times out of ten. The greatest difference between the trimmed mean and the untrimmed mean is 0.17 points, and the most that either is off from the “true and correct” mean is 0.25.
I have a dream to scrap the short program and have separate events for jumps, spins, skating skills/interpretation, and well-balanced freeskating, with separate medals for each. At the big championships, results/points from the individual skill events could be used as qualifiers for the combined-skills final.
But it would cost more to hold an event structured that way, so I'm afraid it will never happen.
Why would that cost more money?
I love the "skating skills/interpretation" combination.
