**0**

I agree. And I appreciate your post. But I think you said some interesting things about the CoP that I want to address, because I find them irreconcilable:I think these debates mostly serve to deflect attention from the real issues, namely, doing everything we can to catch cheaters and banning them for life when we do.

If I understand you correctly, we both agree that the CoP is inadequate. We both agree that a "true score" in skating cannot possibly exist. The only thing we can hope to achieve with some certainty is to rank the skaters. In effect, this is the first objective of both the ordinal and CoP systems. A secondary goal of the CoP is to give a meaningful score for each skater. Ironically, the secondary goal is the one being hyped by the ISU, but it is the first goal where the CoP is severely deficient. Deficient because results based on these scores -- of dubious value and meaning -- will themselves be dubious.But I have a little bit of a problem with the sampling theory model anyway. This depends on a tacit assumption that there is somehow a "true score" for every performance, that this "true score" is in principle quantifiable (as being the mean score of the hypothetical population of all possible qualified judges past, present or future, for instance), and that we can then treat the judges' panel as a sample of this population (like a political poll in trying to predict the outcome of national election).

...

But I am not convinced that this mythical "true score" actually exists. As we all admit, skating is subjective. In sober reality, the only thing that counts is, do these 9 judges like your performance of not. Thus the voting panel is the entire population, and everything that we learned in our statistics classes goes out the window.

I took your quote above to mean that it is impossible to sample a population for a true score that doesn't exist. I'm unclear however, if we both agree that we can indeed "treat the judges' panel as a sample of this population" under the ordinal system. In the ordinal system, if a different sample of 9 judges out of a population of 1000 was taken multiple times,the system could produce samples with consistent results, i.e. ranking of skaters (but NOT assigning individual scores as in CoP).

It is only true that a random draw is insignificantOne of the points that Dr. Rossano is particularly concerned with is the effect of the random draw. I agree. But this is a public relations issue, not a statistical one. The public says, hey, wait a minute, doesn't this introduce an unwanted element of random chance into the mix? Not really. If you have a sitting panel of, let us say, 9 judges, it does not matter statistically whether or not an additional 5 dummy judges, whose votes are predetermined not to count, are taking up space at the judges' table. It's rather silly, of course, and it does give the public something to howl about. But statistically speaking, choosing 9 judges out of a pool of 14 by computer 15 minutes before the competition starts, has the same effect as if the voting judges had been chosen months in advance by drawing names out of a hat, which is how they did it under the old system. No matter when the judges' draw takes place, or how, skaters face the same probability of obtaining a draw that is favorable to them, or not, for whatever reason.when we are talking about the ordinal system. And first, we have to bar the issue of bloc judging, which is a separate issue that confounds both systems, and in which case we do often say, "Oh but with a different set of judges, so-and-so skater would have won..." For a discussion of the ordinal system (as for the CoP), we have to assume that all judges in the panel are marking each skater the same, regardless of nationality and using the same standards and criteria. With these standards in place, we have to assume that the results of a judging panel with a 5-4 or 6-3 split are indicative of bloc judging, which is why the majority of judges' placements determines the final results under the ordinal system, and which is why the actual scores themselves don't matter. (Under the CoP, the scores matter, to the detriment of the final placements.) For the ordinal system only, the placements of the 9 judgesARErepresentative of the larger population and how the population would have judged the same event. Your argument is that a sample of a sample will be representative of the population, such that a sample of 5 marbles out of a sample of 9 marbles out of 100 would probably be equal to a sample of 5 marbles out of a population of 100.Only in the ordinal system would this be a correct assumption.Under the CoP system, however, every judge introduces error and variability through each little mark that they generate,therefore each judge can and will individually affect the final outcome regardless of the majority opinion of the panel. Under the ordinal system, 9 judges' placements are compared, but in the CoP system, each judge contributes MULTIPLE scores -- with error -- into a sum total.Because actual scores are being used to compute the results and not ranks, every time you take a random sample of the judges' marks in the CoP, the results of the competition will be different! As we have seen, the GP competitions have been coming down to within hundredths of a point margins. Therefore, random count of judges' marks has a significant impact in the CoP system.

The ordinal system uses a majority system of ranks to determines the results. The ordinals eliminate error that would be inherent in raw scores as used in the CoP. The unfortunate conclusion is thatit is not possible in any fair way to use an absolute point system in our subjective sport. It'd be nice if it were possible, but it's not, and the ISU is misleading people. The scoring can't be done systematically on such a micro level without enormous amounts of significant error introduced into the results. Secondly, the ISU claims that the point system gives a more meaningful measure of a skater's performance. Unfortunately, this cannot be true, because 20 out of 20 draws of a hat, the competition results will be different. This is not true with the ordinal system, except in the case of bloc judging, which must be dealt with irrespective of the judging system being used.

Under CoP, the random count of the scores by itself is a fault considering that the sample size is small to begin with. But in addition, even if we assume that the judges aren't cheating, there is the issue of human error in judgment. The ordinal system effectively eliminates this error by refining the judges' marks into relative placements. The CoP does not eliminate error, despite the trimmed mean, since the mean is not a robust measure of central tendency in small samples, and since EVERY judges' mark will have error and will introduce its own error into the sum. Therefore no amount of randomization or dropping scores will change the fact that the CoP's results are unreliable. In the ordinal system, the majority count should effectively eliminate human error and produce reliable results, consistent results time and again.

One of the reasons that the CoP is inherently faulty is that it is not complete in its objective. The first part, TES, consists of the technical components, which are judged element by element as the skater's program progresses. It is more objective than the 5 presentation component scores, which must be judged at the end of the skater's performance,just like the way it was in the ordinal system, except now there are 5 subjective marks instead of 1. These component scores are all as subjective and liable to human error as the single presentation mark in the 6.0 system. The difference is that these 5 subjective marks are added together to form part of a total that is touted to be an objective measure of performance. This is just not possible given the subjective nature of judging the presentation components. The total is not equal to the sum of its parts.

## Bookmarks