Are judges able to "rank" skaters while they are judging?

[Joesitz]

Judges are like Fans. The judges are human and apparently love the skating (it is a voluntary job? No?). Judges also know the very elite skaters they are about to judge, as do the Fans. They may even have a preference for some skaters for whatever reason and will up their individual scores for them. Fans would do that too, if they had the power of scoring.

The Elements have been given, arguably, base values, but even those are modified by Grades of Execution which are supposed to clarified by their descriptive notes. Are they? This portion of the competition is as close as we can get for quantifiable purposes.

By and large, the entire PC scores are a matter of opinion. The question should be: Are the judges qualified to give such 'truths' in that score? Do they know the difference between a Carravachio and a Botticelli for scoring purposes?

So my conclusion is the CoP continues the opinion method of the 6.0 albeit differently and in more detail. On the Plus side is that collusion is more difficult. Maybe they will find a way to collude again.

Buttle/Joubert and Plushenko/Lyacek remain questionable? or decisive? Personally, I'm mute.

[Mathman]

I don't know if I agree with that. It is possible to rank skaters according to e.g. performance...

That is the point that I am bumbling around trying to state.

It is possible to rank performances. First, second, third is a ranking. This is what ordinal judging called upon the judges to do.

I do not believe that it is possible to score a performance. In terms of choreography, say, this is a 6.25 performance. Period. It is not a 6.00 performance and it is not a 6.50 performance.

That is the what (nominally) the CoP asks judges to do.

[Robeye]

That is the point that I am bumbling around trying to state.

It is possible to rank performances. First, second, third is a ranking. This is what ordinal judging called upon the judges to do.

I do not believe that it is possible to score a performance. In terms of choreography, say, this is a 6.25 performance. Period. It is not a 6.00 performance and it is not a 6.50 performance.

That is the what (nominally) the CoP asks judges to do.

Completely agree with this.

-As far as the "artistic" components of PCS are concerned, which are by definition analogue in nature, they are potentially amenable to accuracy (eg comparative ranking), but are resistant to fine precision (that is to say, better by how much?). This will always be true.

-In practice, the "technical" components face similar issues (due to the real-time, high speed nature of many elements, and the very subtle and fine-grained distinctions that need to be made by human beings in the heat of the moment). In principle, however, and in contrast to the artistic aspect, the technical components should be susceptible to higher degrees of quantification and measurement than is currently the case.

Measurement: the bottleneck is the limitations of human perception. A strictly quantitative measure of rotation (3, or 2.75) is used for a triple-jump, for example, but the measuring device is the human eye and the human brain, which works qualitatively. Even with the aid of slow-motion, it's not always clear-cut. In the longer term, it may be feasible to alleviate this bottleneck through a form of machine scoring, using digital motion capture technology and software.

Quantification: As a consequence of such developments, it's possible to envision more strictly quantified definitions of many, if not all, technical elements, including jumps, spins, spirals, speed, deep edging, etc., as well as GOE bullets (speed, height, ice coverage, flexibility, et al). The current definitions, which are often neither completely fish nor fowl, are necessarily the way they are because they need to be applied by human beings using imperfect tools of measurement.

-The issues with going digital: Institutional conservatism, inertia and politics, for one. And the not inconsiderable cost and effort involved, for another. But I'm of the view that all sports will be going down this road eventually, although some of them may not yet know it. At some point, though (IMHO), such technologies will be sufficiently established in the wider sporting world, and the costs will have come down so significantly, that figure skating will find it difficult to hold out, given the benefits that it offers to such a technically opaque sport.

And, too, I wonder if committed skating fans shouldn't be careful what they wish for; sure, these developments might make skaters and general audiences happier, but I sometimes muse that we skating fans are a masochistic lot, and would feel positively deflated without controversial calls to parse endlessly.

[Poodlepal]

Thanks for answering my questions. I think that judges do play with the numbers in order to get "their" skater to win, just as much as they did under 6.0, but now it looks like the results are "scientific" and "statistically significant."

As a teacher, I don't (or at least try not to) grade the papers to make certain that a certain final average is reached, or so that student A gets a higher average than student B (to get the subject award, for example.) I grade each test or paper separately, and may not know whose average is higher until I put them in the spreadsheet. That's what the judges are supposed to be doing--just judging each skater's performance, checking those edges and transitions and so on, without regard to anybody else, and then being sort of surprised by the results.

In reality, I don't think they are doing that. I think they can kind of decide who they want to win and then prop scores up or keep them down, and I think with the anonymous judging, there is a lot of room for backroom deals and bloc judging. Maybe even more than before.

Interestingly, very few people outside of us hardcore fans even know about the code of points. If there's an offhand reference to figure skating in a commercial or during banter on a radio show, for example, they always mention getting a 6.0. In video games with figure skating, the little avatar gets a 6.0. The COP hasn't caught on at all in pop culture that I can see. I liked it a lot at first, when a lot of surprise winners came from behind, but now I'm not so sure.

[Joesitz]

^^^^^
That is the way it is in Modern Figure Skating, but there is really no fundamental difference between it and the 6.0 system.

[janetfan]

The COP hasn't caught on at all in pop culture that I can see. I liked it a lot at first, when a lot of surprise winners came from behind, but now I'm not so sure.

I just checked this out because I thought it was an example of 6.0 judging from a film.

http://www.youtube.com/watch?v=HdNO0...eature=related

Turns out it is about gymnastics, sort of, but the reference to the low score from a certain judge sure feels like figure skating.

[Mathman]

In the longer term, it may be feasible to alleviate this bottleneck through a form of machine scoring, using digital motion capture technology and software.

When that happens I think we will all be surprised and embarrassed at the number of actual degrees of air rotation in a "triple" jump. Also at how many edge calls are missed. In general, at how badly everyone does everything.

And, too, I wonder if committed skating fans shouldn't be careful what they wish for; sure, these developments might make skaters and general audiences happier, but I sometimes muse that we skating fans are a masochistic lot, and would feel positively deflated without controversial calls to parse endlessly.

That's one thing I don't think we need to worry about. When I watch a tennis match on TV I am constantly wuzrobbing the computerized line-calling technology. After a close call they show a purported picture of the ball landing clearly out.

Balony! That's a computer-generated virtual cartoon of a wholly imaginary flattened sphere crossing a line that exists only in the software's tangle of 0's and 1's.

[SkateFiguring]

Competitions are about ranking. In figure skating, it's the methodology employed that is debated. The 6.0 system, of course, was strictly about ranking. There were, however, several inherent problems:

1. Inaccuracy - It's impossible for judges to remember clearly the relative merits of each of the 25 programs performed over several hours.

2. Rigidity - The ordinals were marked with fixed numbers with little room for later insertions so that much prejudging happened necessarily to leave anticipated room for later performers.

3. Easy implementation of pre-decision - The simple ranking of skaters made it easy to place any skater exactly where a judge wished to regardless of performance, because of

4. Lack of accountability - No breakdown of scoring or explanation of how the ranking was arrived at by the judges was required or offered. It was all subjective. The judges ranked the skaters according to how much they liked each one.

CoP instead measures each element and each component of each performance with points and lets the ranking happen according to the total points of each performance. There are base values as well as guideline and bullets for the more subjective GOEs and, with averages of a whole panel, the system strives to achieve a relatively fair and accountable scoring. Manipulation for total points is too complicated and difficult. The major problem and cause of debates with the CoP system lie with the PCS. (There has not been any questioning of TES scores on the CoP Scoring On Olympic Jumps thread.) Ironically that is because of its similarity with the 6.0 system. A ceiling for each component's mark makes PCS somewhat of an ordinal system again. Unlike TES, PCS may be relatively scored, comparing each skater to the others. So we are back to the problems with 6.0, except for rigidity and needing to leave room for later insertion, since marks are not simple place holders and non-repeatable.

6.0 being strictly ordinal makes it impossible to compare programs from different competitions. CoP offers such comparisons even if imperfect. Though program requirements and BVs from different seasons may differ, adjustments may be made for comparison purposes, much like economic comparisons that are inflation adjusted.

[Krislite]

Completely agree with this.

-As far as the "artistic" components of PCS are concerned, which are by definition analogue in nature, they are potentially amenable to accuracy (eg comparative ranking), but are resistant to fine precision (that is to say, better by how much?). This will always be true.

-In practice, the "technical" components face similar issues (due to the real-time, high speed nature of many elements, and the very subtle and fine-grained distinctions that need to be made by human beings in the heat of the moment). In principle, however, and in contrast to the artistic aspect, the technical components should be susceptible to higher degrees of quantification and measurement than is currently the case.

Measurement: the bottleneck is the limitations of human perception. A strictly quantitative measure of rotation (3, or 2.75) is used for a triple-jump, for example, but the measuring device is the human eye and the human brain, which works qualitatively. Even with the aid of slow-motion, it's not always clear-cut. In the longer term, it may be feasible to alleviate this bottleneck through a form of machine scoring, using digital motion capture technology and software.

Quantification: As a consequence of such developments, it's possible to envision more strictly quantified definitions of many, if not all, technical elements, including jumps, spins, spirals, speed, deep edging, etc., as well as GOE bullets (speed, height, ice coverage, flexibility, et al). The current definitions, which are often neither completely fish nor fowl, are necessarily the way they are because they need to be applied by human beings using imperfect tools of measurement.

-The issues with going digital: Institutional conservatism, inertia and politics, for one. And the not inconsiderable cost and effort involved, for another. But I'm of the view that all sports will be going down this road eventually, although some of them may not yet know it. At some point, though (IMHO), such technologies will be sufficiently established in the wider sporting world, and the costs will have come down so significantly, that figure skating will find it difficult to hold out, given the benefits that it offers to such a technically opaque sport.

And, too, I wonder if committed skating fans shouldn't be careful what they wish for; sure, these developments might make skaters and general audiences happier, but I sometimes muse that we skating fans are a masochistic lot, and would feel positively deflated without controversial calls to parse endlessly.

It's entirely possible (though highly impractical at the moment) to have motion capture system in place with numerous high-speed cameras positioned around and above the rink, all controlled by a supercomputer measuring every possible aspect of a skater's technical skills. Such a system wouldn't even have to use an aggregate points system in place like CoP at all, since by its precise measurements it could rank jump for jump, spin for spin, which was higher, faster, more revolutions, etc. If we could somehow manage to agree on how to weigh each element in a program relative to others, and of course on the proper reward/penalty for the quality of execution, then such a system could be perfectly accurate in either an ordinal or cardinal scoring method. Either way, the technical marks could never be disputed.

But that would never eliminate contested results because figure skating by its nature requires judgment of a program's artistic merits. I am of the opinion that because such judgment relies on a person's own emotional reaction and aesthetic tastes, that such artistic marks will always have some element of subjectivity. As such, figure skating fans need not worry that they would no longer be able to bicker and fight over controversial calls.

[seniorita]

chances are in the 6.0 system or any system had Plushenko gotten anything but first place his nose would have been severely bent out of joint and he would have demanded his platinum medal just the same. :sheesh:

I really wont interfere in the conversation but many fans (and posters here) and some other skaters like Joubert and Lambiel -who have a small nose btw- have talked about holes in the new system and have expressed some judging about it. We are all used to judge anything in fs and CoP except for leaving the skaters do the same as we do all day.

[Olympia]

As long as human beings judge skating, there are going to be problems and disagreements. But having humans judge skating is the price we pay for having humans as skaters, and for having skating at all. It's not track and field, and it never will be. Besides, isn't it fun to fulminate about who wuzrobbed? I'm still doing that over Torvill and Dean in 1994. And Usova and Zhulin the same year, same medal, same event. (Figure that one out!)

[Mathman]

IMHO there is an unresolvable mismatch between the fundamental premise of the CoP and the heart and soul of the sport that it aspires to evaluate. Quality is not quantity.

To me, the ISU does not have a clear vision for where it wants the CoP to go. Instead, every year they look with chagrin over their shoulders to the previous season and say, "Well, we don't want that to happen again. We'd better raise (or lower) the values of quads, we'd better change they way we call and penalize under-rotations and wrong-edge take-offs, we'd better readjust the relative responsibilities of the judges and the technical panel.

Under 6.0, if you ask the Azerbaijan judge why he gave Tara a 5.9 and Michelle a 5.8, he will say, "They were both superb, but in my opinion Tara skated with greater zest and elan." OK, the guy's an idiot , but if you are not prepared to accept that explanation then figure skating is not for you.

Now ask that same judge why he gave a skater a CoP score of 6.25 in interpretation -- without referring to the performances of any other skaters, just going by the CoP rule book.Will he say that the skater hit the right musical notes 76% of the time (75% of the time gets only a 6.00)?

Figure skating is not a measured sport. We are trying to fit a round peg into a square hole.

[ImaginaryPogue]

For what it's worth, I tend to agree with Mathman. Of course, figure skating wasn't for me under 6.0. I'm glad it is now.

[Joesitz]

When I watch a tennis match on TV I am constantly wuzrobbing the computerized line-calling technology. After a close call they show a purported picture of the ball landing clearly out.

Balony! That's a computer-generated virtual cartoon of a wholly imaginary flattened sphere crossing a line that exists only in the software's tangle of 0's and 1's.

But you wouldn't dare question it if the ISU included the landing of jumps to be computerized for wrong edge takeoffs on the Jumbo Thon. I believe the Tech Panel has the sole privilege of watching this action for the scoring of competitions. That cartoon showing is how the Tech Panels sees the Wets, and the URs. Is that ok? Should the public see it too?

[seniorita]

what is zest and elan?
Michelle lost due to Azerbaijan judge then?