According to the ISU the CoP system was already being developed two years before SLC, but the implementation was expedited because of the SLC scandal. That's hardly the definition of "thrown together."As most people are aware, the CoP was thrown together rather hurriedly in a fairly blatant attempt by the ISU to salvage a reputation badly damaged by the Salt Lake City Olympic scandal.
If the data doesn't allow any judgment of the relative merit of the two systems, why does he come to this conclusion, especially when he doesn't compare the problems of CoP to the problems of 6.0/OBO? For example, he has already shown that the percentage of control that the skaters have over their score is relatively consistent within program type and discipline, while under 6.0/OBO, it was up to individual judges to decide, and, there was no way to tell whether the determination was consistent across all skaters. But he doesn't explain why, net/net, 6.0 is at least equal to this system.This seems particularly unfortunate because the main rationale for the introduction of the CoP has been that of providing a "better" and "more objective" scoring system than that obtaining for the older ordinal systems. While the present data do not allow any judgment of the relative merit of these two systems to be made, it is difficult to see how the present problems could represent an improvement over anything at all.
Under CoP the ISU can 1. Figure out which judges are off the mark, because there is an exact, not relative, score for the elements 2. Find out if the system itself is causing the issue -- i.e., the caller downgrades and the judges apply deductions to an underrotated jump, which the ISU has already said it's going to change -- which is a pretty instant feedback loop 3. Develop better training, including specialized training. For example, if a judge doesn't seem to have lifts or spins figured out, but the rest of his/her judging is in line, train the judge on that element and follow improvement. Under 6.0 there was no way to track this behavior through data. 4. Decide if there's a better way to score -- specialized judges, checking the criteria for a score and adjusting scores automatically based on missed errors, etc. 5. Change the relative weights to get different expected results; ex: ramp up both the upside and downside of landing/missing difficult jumps and combinations; increase the point value of footwork in the Men's LP. Note that changes of relative weight do not affect the judges' training, but happens automatically, so there is not a spectrum of rules interpretation and application.For example, in the Ladies competition at Skate America alone, one can find Shizuka Arakawa receiving scores ranging from -1 to +2 for her triple Lutz-double Toe combination, while Jenny Kirk gets three +1's, two -1's and one -2 for her triple Flip in the Short program, while the Free program shows four more instances of scores ranging from +1 to -2, or +2 to -1, in addition to nine instances of scores ranging from +1 to -1.
What Dr. Schaeffer is referring to is the tendency on the part of the judges to give blanket marks across the board both for GOE and (especially) for Program Components. That is, in practice they are mostly subverting the intent of the CoP and judging as they always have. "I liked this skater's performance the best, so I'll give her the highest marks throughout; I liked this skater second best," etc. Dr. Schaeffer thinks this is a good thing, and that if the judges didn't do this it would be a disaster.(The CoP) may possess some redeeming features, but at present, seems to avoid major embarrassment only by virtue of the fact that the judges appear to be ignoring it as best they can.
Why is it a major embarrassment for CoP for the scoring system to enforce the rules? It seems more of an embarrassment for the judges to "prefer" a skater who has broken them and give that skater extra credit for an unlevel playing field.Mathman said:This question came home to roost in the men's competition in the Grand Prix final. If the judges had been polled, most of them probably would have said that Plushenko gave the best skate overall and deserved to win the gold medal. But when the points were tallied, because Plushenko did not get any credit under the CoP for a third combination, Sandhu came out on top, frustrating the wishes and the intent of the judges. Presumably, this is the kind of thing that Dr. Schaeffer regards as a "major embarrassment" for the CoP.
Mathman said:As expounded by scholars such as Dr. George Rossano and Dr. Sandra Loosemore, the whole concept of adding up points is inherently less reliable than methods based on ordinals.
Mathman
hockeyfan228 said:ISU can 1. Figure out which judges are off the mark, because there is an exact, not relative, score for the elements 2. Find out if the system itself is causing the issue -- i.e., the caller downgrades and the judges apply deductions to an underrotated jump, which the ISU has already said it's going to change -- which is a pretty instant feedback loop
There are a couple of issues that could be in affecting this:Joesitz said:What you say, if it works will assist in preventing such inequalities in later competitions. But what about NOW? If the ISU can figure out which judtges are off the mark, will they take immediate action? This action taken immediatey will assist the skater who is not in first place if the review is going in that direction. If they wait until May, I doubt very much they would admit to an inequality and even if they did, they will not take back any medals already issued. Am I Correct?Joe
Hey, Gisele. Just a quicky for now. I guess I meant the first. The first is certainly true.giseledepkat said:I apologize for picking on you, Mathman, but the above quote is unclear to me. Do you mean to say that:
Rossano and Loosemore believe strongly that "the whole concept of adding up points is inherently less reliable than methods based on ordinals", and have therefore devoted their energies to advancing said beliefs
or...
The statement "the whole concept of adding up points is inherently less reliable than methods based on ordinals" is not up for debate in the mathmatical community at large.
I guess what I'm really asking for is your opinion, Mathman! 'Cuz I respect you so much! Oh, and while I'm at it -- [OT] Did you ever finish the statistical analysis of Michelle's jump percentages that you talked about doing a while ago? (Maybe I missed it!) [/OT]
Thanks, Pennie
OK, now I am a little confused. Say the judges is marking a spiral. Let's take a change-of-edge requirement for levels 2 and 3. For level 2 it is: Spirals curve mainly in one direction but at least one uses opposite curve, different edge combinations. For level 3, it is: Spirals curve equally in both directions with unassisted change of edges. Let's ignore all other spiral requirements for a second, and pretend like the whole GoE was only based on this. Now, if a judge is marking a level 2 spiral, he/she sees a change of edge and gives a positive GoE. However, if that same spiral is judged as level 3, they may say, well, sure, she did a change-of-edge, but it's not anywhere near equal in both directions; this may result in a negative GoE.c. They have been making changes to the system all along. When the judges said that seeing the levels affected them psychologically -- i.e., they gave better GOE's to higher-level elements -- they asked to have the levels suppressed and only the elements displayed. This was implemented right away. They may have enough data now to at least suggest whether the same phenomenon happens with jumps -- i.e., 3A's and 4's don't get the same deductions for the same problems in the jump
Good question. This is another potential overlap between caller and judge. When the GOE's were first written, the Levels were displayed. EitherPtichka said:OK, now I am a little confused. Say the judges is marking a spiral. Let's take a change-of-edge requirement for levels 2 and 3. For level 2 it is: Spirals curve mainly in one direction but at least one uses opposite curve, different edge combinations. For level 3, it is: Spirals curve equally in both directions with unassisted change of edges. Let's ignore all other spiral requirements for a second, and pretend like the whole GoE was only based on this. Now, if a judge is marking a level 2 spiral, he/she sees a change of edge and gives a positive GoE. However, if that same spiral is judged as level 3, they may say, well, sure, she did a change-of-edge, but it's not anywhere near equal in both directions; this may result in a negative GoE.
Ptichka said:Let's take a change-of-edge requirement for levels 2 and 3. For level 2 it is: Spirals curve mainly in one direction but at least one uses opposite curve, different edge combinations. For level 3, it is: Spirals curve equally in both directions with unassisted change of edges.