Judging the judges

The ISU has promised that at the end of the season they will go over all the judges scores (not available to the public) and evaluate the performances of the judges. Those found lacking presumably will be required to undergo more training before they are assigned to any more major events.

One way to measure a judge's performance is to compare that judge's scores against the scores of the majority of the panel. For ordinal judging a standard way to do this is to compute a statistic called the Spearman rank correlation coefficient. Roughly speaking, this gives a percentage mark to each judges ordinal ranking which measures the degree to which that judge agreed with the rest of the panel.

I tried it out for the top six finishers in the ladies' long program in the 2002 Olympics. Here are the coefficients for each judge, identified by nationality, in descending order of apparent competency.

U.S.A. -- 1.00 (perfect match with the final placements -- take a bow, judge Joe!)).
Finland -- 1.00
Germany -- .89
Belarus -- .89
Italy --.83
Slovakia -- .80
Denmark -- .65
Russia -- .34 (the renowned Ms. Danilenko)

You can use the same method to compare the judges against each other, one by one, to see which judges tend consistently to vote together (possible collusion). Here are the highs and lows for each judge.

USA: Most compatible with Finland (1.00) and Canada (.94)
.........Least compatible with Russia (.54) and Denmark (.66)

Finland: Most compatible with USA (1.00)
..............Least compatible with Russia (.54) and Denmark (.66)

Germany: Most compatible with Canada (.94), USA and Finland (.89)
................Least compatible with Russia (.20) and Denmark (.49)

Belarus: Most compatible with Finland, USA and Slovakia (.89)
Least compatible with Italy (.60) and Germany (.66)

Italy: Most compatible with Canada (.94) and Germany (.86)
.........Least compatible with Russia (.37) and Denmark (.49)

Slovakia: Most compatible with Denmark (.94) and Belarus (.89)
...............Least compatible with Italy (.60) and Germany (.71)

Denmark: Most compatible with Slovakia (.94) and Russia (.89)
................Least compatible with Italy and Canada (.49)

Russia: Most compatible with Denmark (.89), Slovakia and Belarus (.77)
............Least compatible with Germany (.20) and Italy (.37)

********************************

The Miss Congeniality award: USA and Finland, with identical scores across the board of 1.00, .94, .89, .89, .83, .83, .66 and .54 (the last two reflect their low compatibility with Russia and Denmark). USA wins the tie-breaker, having a perfect record all the way down through the top ten (this study just includes the top six skaters).

Most cantankerous: Russia, with scores of .89 (v. Denmark), .77 (Slovakia), .77 (Belarus), .54 (USA), .54 (Finland), .42 (Canada), .37 (Italy), .20 (Germany).

*************************************

Conclusion: The Russian judge needs retraining before she is assigned to any more ISU events.

The Russian judge and the Danish judge appear to be in cahoots to skew the vote. Slovakia falls under suspicion, too.

************************************

Under secret judging we will not be able to do analysis like this any more. Thus we are counting on Mr. Cinquanta to do it for us, and to take the necessary and appropriate steps when evidence of incompetence and collusion are found.

MM

Under secret judging we will not be able to do analysis like this any more. Thus we are counting on Mr. Cinquanta to do it for us, and to take the necessary and appropriate steps when evidence of incompetence and collusion are found.

Click to expand...

Then there is no hope hwell: IMHO

Fuggeddabout math -- when judging the judges, I want to play Madame Defarge [insert flaming devil icon here].

Mathman said:

The ISU has promised that at the end of the season they will go over all the judges scores (not available to the public) and evaluate the performances of the judges. Those found lacking presumably will be required to undergo more training before they are assigned to any more major events.

MM

Click to expand...

In my opinion that is way too late for skaters in that specific competition! Of course rotten apples will gradually "disappear" along that system, perhaps.

I think the only hope for figure skating is to replace the speedskater as the head of the Union.

I have some concern as to whether, after these Olympics, if the sport will further decline in popularity. It will definitely remain a sport but so does that dance with the ribbons in the summer olys.

Joe

Joesitz

Jositz, Your question is a valid one. Will skating survive if the winner is not the one who skates the perfect program and its obvious the winner is overmarked. The general public wont go to colledge to get a math degree to figure out the new COP. I am being the broken record as I say Im going to enjoy the season as if its my last because I think the OLYS will determine the survival or not of the sport. I admit its a corrupt sport however I dont have to accept it. Perception is the name of the game like it or not and the skaters who are overmarked are obvious to the general public more so because they dont have favorites and can be objective. We the fans underestimate mr and mrs general fan they see what is going on and walk away so TV has little viewers. I sure hope that the winner deserves the win or we all lose.

ITA Cianni - and I will watch the Olys on TV and just declare the winner according to me. Why not? I don't have to suck up to the so called 'experts'. I've always said, give me another panel of 7 or 9 judges (experts) and the results will be different.

Joe

Judging

Jositz , I will do the same however according to me is usually according to most who dont take a stand to defend their favorite no matter how they skate. I think most fans know who should have won and who is overmarked and who the judges decided is going to win. It really doesnt take a rocket scientist to figure it out. The joke is changing the system to fool us and they make fools of themselves. This will be quite interesting the fun of watching the comp and how they skew the system to get their winner. Im showing my mistrust and wont I eat crow if Im wrong but I doubt I am.. Must add I do feel sorry for the skaters and esp. if the deserving winner doesnt get the Gold.

Maybe the USA and Finland are in cahoots. We should watch for foot tapping...

But if so, they are also in cahoots with the judgment of the panel taken as a whole.