Oh I thought this thread was finished...
Despite all of the statistical objections to CoP, I fail to see why placements does anything but obscure the judges' reasoning for those placements that are so transparent to statisticians in CoP. CoP provides a lot of data, to determine whether judges are adhering to the code and to feed the objections of its critics.
I understand that many skating fans like having all these numbers to mull over. It gives an impression of a rigidly systematic approach to judging, so that you do see a sort of chain of events in the scoring process. But in fact, despite the apparent transparency of the system, each of the links in the scoring chain are very weak, such that the system has a good chance of breaking down at any point in the chain. In other words, you do get more information from the CoP. This makes the fans happy. On the other hand, the system itself is crude and error-prone, so that the skaters can be majorly screwed by the single hand of one judge or random chance. In effect, what most fans seem to be saying is, the system sucks for the skaters, but at least we get to see what the judges are doing to them. Yay for the fans!
My 2nd response is, if the CoP "provides data to determine whether judges are adhering to the code," then what disciplinary actions against cheating judges are available to us in a secretive, pooled scoring system? We have already seen numerous times in the GP series that judges have not been taking the proper deductions.
Under 6.0 all of the biases that are displayed in the statistics are encorporated into two scores.
Yes, and under CoP, bias will be a factor in ALL of the different element and component scores, and those biases are just as difficult to decipher or rationalize. Unfortunately, it turns out that each of the 5 program components are just as obscurely marked as the single presentation mark. That's why we have concluded that ranks are less error-prone than an absolute point scale, and a single ordinal introduces less error into the results than do 5 separate marks for program components.
What does it mean when one judge gives a skater 5.50 points for Skating Skills and another judge gives the exact same skater 7.25 points? Then, what are we supposed to think when the final results are decided by less than a tenth of a point?
Just because there is more information provided by the CoP, doesn't imply anything about the integrity of the enormous amounts of information being given. Judges in BOTH systems are going to be biased. The important thing is that in addition to the error from bias, the CoP also introduces random error into all the scores and calculations. The random error is far more serious, because bias cannot be completely gotten rid of in a judged sport, but random error should be minimized. A large amount of error means that the system is not producing reliable results.
All of the statistically insignificant differences between skaters are inherent in the placements -- Judge chooses Skater A over Skater B, regardless of the significance of the differences between the two skaters.
I'm not sure I understand. Do you mean that ordinals simply place the better skater ahead, regardless of the "quantifiable" difference between skater A and skater B? First of all, it's not always possible to quantify instead of qualify the differences among skaters. It would only be possible if skating was a jumping contest. But every skater has their own strengths and weaknesses that cannot be added and subtracted into totals that can be quantifiably compared meaningfully.
Really, skaters can only be compared relatively.
Ordinals give very little data with which to work...
Well I didn't know this was your job
j/k
...since nearly every decision can be explained as relative... "Oksana Bauil moved me, so I gave her great scores, which Nancy Kerrigan skated like an iceberg." "Nancy Kerrigan landed everything, so I gave her great scores, while Oksana Bauil had a technically flawed program under pressure." Etc.
First of all, I really hope you don't think that that's how judges judge. Obviously, bloc judging totally confounds the results process.
Like I said, the CoP is even more confounding than ordinals. Let's take a look at what might have happened with CoP:
Judge A gives Baiul:
8.00 SS 7.50 T 8.50 P/E 8.50 C 8.50 I
Same judge will give Kerrigan:
8.50 SS 7.00 T 8.25 P/E 8.25 C 8.25 I
Judge B gives Baiul:
7.50 SS 7.50 T 7.00 P/E 7.00 C 7.50 I
And same judge gives Kerrigan:
8.00 SS 7.50 T 8.00 P/E 7.50 C 7.50 I
I gave these numbers based on how I recalled their skates. And basically, that is what the judges do in the CoP: At the end of the skate, they assign 5 numbers to the program components.
The absolute scale is subjective and inaccurate enough that any WIDE range of scores by the judges will seem justifiable, legitimate, and uncontestable. I made the scores above so that one judge preferred Baiul, and one preferred Kerrigan. Basically, we see that one judge likes Baiul, because of her balletic style and choreography. The other judge preferred Kerrigan, because of her strong, clean lines and perfect execution. Now, how the competition turns out is up to anyone and no one under the CoP system, because of the variability in the judges' marks and the random count of the scores. Would you feel better now, knowing that one judge scored Kerrigan higher in Perf/Execution, even though another judge scored Baiul higher in the same mark? Or are we back where we started with the ordinals, except now we have 5 marks instead of one and a multitude of confounding variables, and no accountability.
Each year, the ISU calls up a series of judges to explain what they perceive as National bias. When was the last time that the judges were suspended as a result of the investigations? Or even given the less "plum" events? Back in the '70's when the Russian delegation was kicked out en masse? The ISU hasn't had much data that proves anything.
I don't know what your point is. The ISU or the judging system?
There are four things that CoP has over placements, in addition to transparency:
Let's be specific here. You mean a more transparent judging chain, a more transparent scoresheet for technical elements, but still equally untransparent marks for program components, and overall secrecy in the judges' marks.
a) Every defined technical element must be given credit when it is performed.
Yes, that is most definitely a positive. Now here is one for OBO:
Each judge marks the elements, including non-jump or spin elements, as THEY see them and not as some technical specialist calls them. In other words, in a 9-judge panel, there are nine judges making independent decisions, instead of a single person making calls for the entire group.
i. Double-footed and under-rotated jumps can't be "forgotten" in the heat of the judging moment
Do you mean Baiul at the Olympics? That was more likely bloc judging. If there's a group of collaborating judges working the CoP system, they can easily manipulate the system in more subversive ways.
Also, under CoP we have seen many times already in the GP that jumping mistakes are often "forgotten," by exaggerating the component marks.
ii. A missed element at the end -- like Pang/Tong's weak Pairs Spin at the end of their CC LP -- doesn't eliminate the excellence at the beginning of a program -- like their superior 3 Twist.
When did it ever do such a thing in the ordinal system? In both systems, the judges must take into account both the strengths and weaknesses of a skater's performance. The CoP seems to be more literal and systematic in its calculations, because of the TES mark, but people are forgetting that the TPC is highly subjective, and has been the deciding factor in numerous events in the GP. In addition, the problem of the systematic adding of the total elements and program components is that the marks are riddled with random and human error. Some consider the leeway with which judges have to award marks in the ordinal system to be a bad thing, but I consider it a good thing because it allows judges to check for error that would otherwise be present in the calculated results.
b) The relative difficulty of jumps and especially spins, footwork, and spirals is codified, not up to each judge at each competition for each skater.
Well some people consider this a bad thing. Some people think that the values that have been assigned to elements in the CoP don't make sense. I like the idea of a democratic judging panel (more reflective of the population) rather than a tyranny of what is considered valuable in skating.
Also, a technically and artistically rapidly evolving sport like skating NEEDS an adaptive system like ordinals, not a system rigidly coded with values for certain elements whose emphasis may or may not change with the times (even year to year or event to event!!)
c) The relative "goodness" of the skaters in one phase is not minimized by equal placements. If there is a statistically insignificant difference in one phase of the competition, it does not lock that skater into a place. If a skater is significantly better in that phase, the amount of the difference is not lessened.
We have also seen that skaters can build an insurmountable lead after the SP... But, that is not the main problem. As I discussed earlier in this thread, the ultimate goal of a judging system in skating
is to assign placements, not scores. And
there can never be such a thing as a "statistically insignificant difference" between any two skaters. This is a fallacy or myth generated by the CoP.
The strength of the ordinal system is that it is able to AMPLIFY the marginal differences between any two closely matched competitors into meaningful differences in placements. The CoP is unable to do this, because any skaters that finish within about 5 points of each others' totals will not have a clear and definite justification for their placements (due to error and variability), and statistically there is no reason why those placements wouldn't change with another random draw.
The only thing the scores of these closely ranked skaters tells you is that the system was unable to differentiate the skaters. The placements of these undifferentiated skaters are entirely due to chance. Not so with ordinals. The judges must make a conscious, deliberate, and thoughtful decision to put one skater ahead of another so that there is no misunderstanding that skater placed #4 was better than skater placed #5.
d) With discrete standards for each mark the ISU doesn't have to prove bias (overt or unconscious) or corruption. They just have to show that the judges aren't following the code.
And how do you prove that the judges "aren't following the code" when it is undetectable? I will quote myself here:
"The absolute scale is subjective and inaccurate enough
{due to norms in human acceptance of variability} that any WIDE range of scores by the judges will seem justifiable, legitimate, and uncontestable."
It's a little early to decide that what we have now is the end, and that's it's same old/same old. It's too early to see if the ISU will address the uselessness of the CoP standards for PE so far this year or what else they will do with the data. The IOC can make itself aware of the statistical objections, and decide to reject the ISU's solution and figure skating itself.
Yeah, who knows, maybe the ISU is just buying itself some time, and even if the CoP is passed at the ISU congress, maybe after years and years of CoP failures and continuous modifications, like maybe after a couple of years of Olympic scandals, the ISU will introduce a "new" system of ordinals to fix the problems of the "old" CoP system. So maybe the ISU is buying itself some time with this experiment... It's not so bad since they're only wasting tons of money and affecting the results of competitons in the meantime.
let me just add that:
ne ordinal.
