Analyzing the Code of Points
- Thread starter gsk8
- Start date
- Joined
- Jul 11, 2003
Quote:
a. An identified problem with assessing jumps is the overlap between the callers' and the judges' aegis. Until they remove that variable, it's hard to make an argument for any given judge being incompetent. The more the judges use the system, and the more they identify the crosswires, the more variables they can eliminate.
---------------------------------------------------------------------------
Understandably, but what if the Caller is incompetent? or just plain errs? Will his assistants come to the rescue? I believe this is a variable too.
Quote:
They admit to inequalities every time they publish the results of the official investigations. Even under secret judging last year, in an ISU communication, they published the names of the judges who had been brought up for bias.
---------------------------------------------------------------------------
What happens to those judges who are brought up for bias?
gkelly: Good post. The ongoing discussion of Spirals never ends. Many fans see the best spiral as the one executed with the highest free leg. That does show the best flexibility, but is figure skating a sport of the best flexibility? Imo, the edge on the skating foot is part of figure skating and how well that edge is maintained for a certain period of time. And there should be no bobbles in the free leg how ever high it is held. A smooth change of edge is obviously more difficult. A flat edge is inexcusable, imo.
Joe
a. An identified problem with assessing jumps is the overlap between the callers' and the judges' aegis. Until they remove that variable, it's hard to make an argument for any given judge being incompetent. The more the judges use the system, and the more they identify the crosswires, the more variables they can eliminate.
---------------------------------------------------------------------------
Understandably, but what if the Caller is incompetent? or just plain errs? Will his assistants come to the rescue? I believe this is a variable too.
Quote:
They admit to inequalities every time they publish the results of the official investigations. Even under secret judging last year, in an ISU communication, they published the names of the judges who had been brought up for bias.
---------------------------------------------------------------------------
What happens to those judges who are brought up for bias?
gkelly: Good post. The ongoing discussion of Spirals never ends. Many fans see the best spiral as the one executed with the highest free leg. That does show the best flexibility, but is figure skating a sport of the best flexibility? Imo, the edge on the skating foot is part of figure skating and how well that edge is maintained for a certain period of time. And there should be no bobbles in the free leg how ever high it is held. A smooth change of edge is obviously more difficult. A flat edge is inexcusable, imo.
Joe
- Joined
- Jul 26, 2003
The assistant caller's are supposed to come to the callers rescue, or at least call for the videotape. They also have transcripts of all of the communication that goes on over the headphones, with which to determine if a caller was ignoring the legitimate concerns of his/her assistants into account. I'm not sure if the judges have any recourse to call for the video. If the caller is incompetent, the ISU, which owns the positions, can decide not to appoint an incompetent caller to another competition.Joesitz said:
The ISU doesn't have the same control over the judges, because it's up to the Federations to supply judges, and the only way the ISU can get rid of them is to sanction them.
They are supposed to have hearings in which they're asked to justify their scores, but apart from the ISU announcing that they've been suspended, I have no idea.
Did you see the severe edge that Pavuk had on her hand to the ice spiral? Wow! She also had good positions, even though she held them only for seconds. On the tiny Windows Media file, it looked like she almost jumped from edge to edge, it was so fast and sudden.
- Joined
- Jul 11, 2003
hockeyfan228 said:
Unfortunately, I have not seen Pavuk' Euros routine and from what I understand she will not be going to Worlds this year. For next year, Sebastyen has to place top 10 for her to go. I hope she does. The Euro Ladies are on the move.
Joe
- Joined
- Jul 26, 2003
Bummer, it was available on SOMEBODYELSESLIFE.com the other day, but it's now offline.Joesitz said:
EDITED TO ADD: Her Exhibition, which I haven't downloaded yet, is available on http://www.somebodyelseslife.com/figureskating/
now (Wed, 1pm EST), along with Poykio's and Sokolova's FS. You need Windows Media Player to view and winzip to "unzip" the file.
There's a link to a photo that does the move only partial justice on Yahoo at:
http://story.news.yahoo.com/news?tmpl=story&u=/040208/ids_photos_sp/r1491791742.jpg
BTW: I hope you saw shine's link to a Real Player version of Lambiel's LP. If not you can view it by downloading (right click/Save As) shine's link towards the bottom of the page on the following thread:
http://www.goldenskate.com/forum/showthread.php?s=&threadid=4341
Last edited:
- Joined
- Jun 21, 2003
For Giseledepkat.
This is what I think (at least what I think I think) about the CoP and statistics.
First, in evaluating the claims of the CoP critics we need to distinguish between the CoP itself and Ottavio Cinquanta blowing smoke up our butts. Cinquanta is a politician and a public relations man. Blowing smoke is his job. He makes absurd claims about the CoP for the same reason that President Bush is still talking about weapons of mass destruction. That's what politicians do.
Two such claims are:
(1) The CoP will make it harder for judges to cheat. And
(2) The random draw and trimming process will protect the anonymity of the judges.
When people with mathematical savvy invest a lot of time countering these positions, they are tilting at windmills. Nobody believes (1) or (2) anyway, not even Cinquanta if he would tell the truth. So there is really no point in throwing a bunch of statistics at these straw men.
(About (2) for instance, after the Nebelhorn competition I sat down with my $3.99 calculator and after 20 minutes of trial and error arithmetic I was able to determine exactly which judges had been drawn for the panel and whose scores had been discarded.
(BTW, the random draw thing, however innocuous it might be from a statistical point of view, has been a public relations disaster for the ISU. I expect Cinquanta to toss it before the Olympics, grandstanding the decision as a big concession to the CoP critics in the hope that this will shut them up on more substantive issues.)
OK, so all that is smoke and mirrors. What are the statistical objections to the CoP?
(a) When you base the final placement on adding up scores for each element, there is necessarily a certain amount of statistical "noise," or "sampling error" in the result. In a close contest the noise can overwhelm the actual measurement. Like listening to the radio when the signal is overborne by static -- you can't hear the music for the static.
For example, if a triple Salchow is worth 4.2 points and the GoE scores range from -1.00 to +2.00 from one judge to another (which happened quite a lot in the Grand Prix series), it is hard to have confidence in the scoring of this element.
(b) When we look at the Program Components we see something even worse. Let's say Skater A has good transitions but bad choreography. Skater B is the opposite. We would expect that all of the judges would give skater A high marks for transitions and lower marks for choreography.
Instead, we find some of the judges giving uniformly high marks to skater A (i.e., for both components), while other judges give uniformly high marks to skater B.
This shows, say the statisticians, that the CoP is a sham because the judges are making no effort at a serious assessment of each component -- which, after all, is the whole idea of the CoP in the first place.
My opinion about all this is:
(i) The statistics are useful because they allow us to quantify assertions like (a) and (b). We can actually compute a number that tells us how much statistical noise is present in our total scores. And we can compute a number that measures the tendency of judges not to look closely at individual program components.
All this is useful. We can compare one contest to another to see, for instance, whether judging panels are getting better as they gain experience. We can look at individual judges and mark their performances against the norm. Lots of nice things. BUT...
(ii) At the end of the day, figure skating is still a judged sport.
I saw the Westminster Dog Show on television last week. Pretty cool. I thought the Pomeranian was robbed in the Toy Group, but never mind that. After all was said and done, after the judge had carefully compared each of these beautiful animals against the standards for the breed, she simply said,
"The Newfoundland is Best in Show. Thanks for tuning in, and see you next year."
To me, the CoP is figure skating's attempt at codifying the "standards for the breed." But at the end of the day it's still, "This dog is Best in Show (and this b. is Best of Opposite Sex)."
Mathman
This is what I think (at least what I think I think) about the CoP and statistics.
First, in evaluating the claims of the CoP critics we need to distinguish between the CoP itself and Ottavio Cinquanta blowing smoke up our butts. Cinquanta is a politician and a public relations man. Blowing smoke is his job. He makes absurd claims about the CoP for the same reason that President Bush is still talking about weapons of mass destruction. That's what politicians do.
Two such claims are:
(1) The CoP will make it harder for judges to cheat. And
(2) The random draw and trimming process will protect the anonymity of the judges.
When people with mathematical savvy invest a lot of time countering these positions, they are tilting at windmills. Nobody believes (1) or (2) anyway, not even Cinquanta if he would tell the truth. So there is really no point in throwing a bunch of statistics at these straw men.
(About (2) for instance, after the Nebelhorn competition I sat down with my $3.99 calculator and after 20 minutes of trial and error arithmetic I was able to determine exactly which judges had been drawn for the panel and whose scores had been discarded.
(BTW, the random draw thing, however innocuous it might be from a statistical point of view, has been a public relations disaster for the ISU. I expect Cinquanta to toss it before the Olympics, grandstanding the decision as a big concession to the CoP critics in the hope that this will shut them up on more substantive issues.)
OK, so all that is smoke and mirrors. What are the statistical objections to the CoP?
(a) When you base the final placement on adding up scores for each element, there is necessarily a certain amount of statistical "noise," or "sampling error" in the result. In a close contest the noise can overwhelm the actual measurement. Like listening to the radio when the signal is overborne by static -- you can't hear the music for the static.
For example, if a triple Salchow is worth 4.2 points and the GoE scores range from -1.00 to +2.00 from one judge to another (which happened quite a lot in the Grand Prix series), it is hard to have confidence in the scoring of this element.
(b) When we look at the Program Components we see something even worse. Let's say Skater A has good transitions but bad choreography. Skater B is the opposite. We would expect that all of the judges would give skater A high marks for transitions and lower marks for choreography.
Instead, we find some of the judges giving uniformly high marks to skater A (i.e., for both components), while other judges give uniformly high marks to skater B.
This shows, say the statisticians, that the CoP is a sham because the judges are making no effort at a serious assessment of each component -- which, after all, is the whole idea of the CoP in the first place.
My opinion about all this is:
(i) The statistics are useful because they allow us to quantify assertions like (a) and (b). We can actually compute a number that tells us how much statistical noise is present in our total scores. And we can compute a number that measures the tendency of judges not to look closely at individual program components.
All this is useful. We can compare one contest to another to see, for instance, whether judging panels are getting better as they gain experience. We can look at individual judges and mark their performances against the norm. Lots of nice things. BUT...
(ii) At the end of the day, figure skating is still a judged sport.
I saw the Westminster Dog Show on television last week. Pretty cool. I thought the Pomeranian was robbed in the Toy Group, but never mind that. After all was said and done, after the judge had carefully compared each of these beautiful animals against the standards for the breed, she simply said,
"The Newfoundland is Best in Show. Thanks for tuning in, and see you next year."
To me, the CoP is figure skating's attempt at codifying the "standards for the breed." But at the end of the day it's still, "This dog is Best in Show (and this b. is Best of Opposite Sex)."
Mathman
- Joined
- Jul 11, 2003
Bravo Mathman - Your post was quite understandable and I enjoyed that you had reservations about the CoP as everyone else has.
Totally unquantifiable from my point of view is the impossibility of human judges being consistent in awarding the +s and -s., and I am NOT speaking of hanky panky where there is plenty of room to manipulate the scores.
Your example of the triple salchow is proof.
As for being called up on the carpet which Hockeyfan will insist, well, let's see if that happens.
Joe
Totally unquantifiable from my point of view is the impossibility of human judges being consistent in awarding the +s and -s., and I am NOT speaking of hanky panky where there is plenty of room to manipulate the scores.
Your example of the triple salchow is proof.
As for being called up on the carpet which Hockeyfan will insist, well, let's see if that happens.
Joe
- Joined
- Jul 26, 2003
I disagree that human judges can't be consistent in the scoring. Because they have not been so far in a limited number of events in the first year of a system doesn't mean that they can't be. As I've mentioned before, one of the issues in the current implementation is that the judges' and callers' aegis overlaps, particularly in judging jumps. As I've shown earlier, for spins and spirals and footwork sequences, the judges were consistent -- i.e., all within one mark of each other after a single trim -- 99% of the time in the two competitions in which I checked and counted all marks in the Ladies' programs. While this might look "good" from the surface, the scores were mostly between -1 and +1, regardless of quality, with few elements given full credit for excellence or full deductions for bad quality.Joesitz said:
Whether judges will be called up on the carpet is not something I am sure of. What I'm sure of is that there is more data on which to make the call, and no lack of data to hide behind.
Again, I ask, if there was no CoP, how do you know what each judge is doing on his/her own to create an individual set of relative weights, values, and priorities that may not even be applied equally to each skater in the competition, let alone between competitions?
Last edited:
- Joined
- Oct 7, 2003
Thanks, Mathman, for both your posts! I guess the fact that I get lost so easily in the mathematical discussions, and only slightly less easily in the heavy technik dissections, makes me appreciate all the more your knack for [cliche warning]thinking outside the box, keeping things in perspective, and remembering the Big Picture.[/c.w.] For real. 
Speaking of cliches, though -- I always thought the expression was "blowing sunshine up your ass"...
I guess the fact that I get lost so easily in the mathematical discussions, and only slightly less easily in the heavy technik dissections, makes me appreciate all the more your knack for [cliche warning]thinking outside the box, keeping things in perspective, and remembering the Big Picture.[/c.w.] For real. Speaking of cliches, though -- I always thought the expression was "blowing sunshine up your ass"...
