Results 1 to 10 of 10

Thread: Judging the judges

  1. #1
    Custom Title Mathman's Avatar
    Join Date
    Jun 2003
    Location
    Detroit, Michigan
    Posts
    27,066

    Judging the judges

    The ISU has promised that at the end of the season they will go over all the judges scores (not available to the public) and evaluate the performances of the judges. Those found lacking presumably will be required to undergo more training before they are assigned to any more major events.

    One way to measure a judge's performance is to compare that judge's scores against the scores of the majority of the panel. For ordinal judging a standard way to do this is to compute a statistic called the Spearman rank correlation coefficient. Roughly speaking, this gives a percentage mark to each judges ordinal ranking which measures the degree to which that judge agreed with the rest of the panel.

    I tried it out for the top six finishers in the ladies' long program in the 2002 Olympics. Here are the coefficients for each judge, identified by nationality, in descending order of apparent competency.

    U.S.A. -- 1.00 (perfect match with the final placements -- take a bow, judge Joe!)).
    Finland -- 1.00
    Germany -- .89
    Belarus -- .89
    Italy --.83
    Slovakia -- .80
    Denmark -- .65
    Russia -- .34 (the renowned Ms. Danilenko)

    You can use the same method to compare the judges against each other, one by one, to see which judges tend consistently to vote together (possible collusion). Here are the highs and lows for each judge.

    USA: Most compatible with Finland (1.00) and Canada (.94)
    .........Least compatible with Russia (.54) and Denmark (.66)

    Finland: Most compatible with USA (1.00)
    ..............Least compatible with Russia (.54) and Denmark (.66)

    Germany: Most compatible with Canada (.94), USA and Finland (.89)
    ................Least compatible with Russia (.20) and Denmark (.49)

    Belarus: Most compatible with Finland, USA and Slovakia (.89)
    Least compatible with Italy (.60) and Germany (.66)

    Italy: Most compatible with Canada (.94) and Germany (.86)
    .........Least compatible with Russia (.37) and Denmark (.49)

    Slovakia: Most compatible with Denmark (.94) and Belarus (.89)
    ...............Least compatible with Italy (.60) and Germany (.71)

    Denmark: Most compatible with Slovakia (.94) and Russia (.89)
    ................Least compatible with Italy and Canada (.49)

    Russia: Most compatible with Denmark (.89), Slovakia and Belarus (.77)
    ............Least compatible with Germany (.20) and Italy (.37)

    ********************************

    The Miss Congeniality award: USA and Finland, with identical scores across the board of 1.00, .94, .89, .89, .83, .83, .66 and .54 (the last two reflect their low compatibility with Russia and Denmark). USA wins the tie-breaker, having a perfect record all the way down through the top ten (this study just includes the top six skaters).

    Most cantankerous: Russia, with scores of .89 (v. Denmark), .77 (Slovakia), .77 (Belarus), .54 (USA), .54 (Finland), .42 (Canada), .37 (Italy), .20 (Germany).

    *************************************

    Conclusion: The Russian judge needs retraining before she is assigned to any more ISU events.

    The Russian judge and the Danish judge appear to be in cahoots to skew the vote. Slovakia falls under suspicion, too.

    ************************************

    Under secret judging we will not be able to do analysis like this any more. Thus we are counting on Mr. Cinquanta to do it for us, and to take the necessary and appropriate steps when evidence of incompetence and collusion are found.

    MM
    Last edited by Mathman; 06-28-2005 at 12:17 AM.

  2. #2
    On the Ice
    Join Date
    Oct 2004
    Posts
    29
    Under secret judging we will not be able to do analysis like this any more. Thus we are counting on Mr. Cinquanta to do it for us, and to take the necessary and appropriate steps when evidence of incompetence and collusion are found.
    Then there is no hope IMHO

  3. #3
    Go NJ Devils
    Join Date
    Jul 2003
    Location
    Seattle
    Posts
    2,700
    Fuggeddabout math -- when judging the judges, I want to play Madame Defarge [insert flaming devil icon here].


  4. #4
    Custom Title
    Join Date
    Jul 2003
    Location
    Helsinki, Finland
    Posts
    3,546
    Quote Originally Posted by Mathman
    The ISU has promised that at the end of the season they will go over all the judges scores (not available to the public) and evaluate the performances of the judges. Those found lacking presumably will be required to undergo more training before they are assigned to any more major events.

    MM
    In my opinion that is way too late for skaters in that specific competition! Of course rotten apples will gradually "disappear" along that system, perhaps.

  5. #5
    Custom Title Joesitz's Avatar
    Join Date
    Jul 2003
    Location
    New York City
    Posts
    20,185
    I think the only hope for figure skating is to replace the speedskater as the head of the Union.

    I have some concern as to whether, after these Olympics, if the sport will further decline in popularity. It will definitely remain a sport but so does that dance with the ribbons in the summer olys.

    Joe

  6. #6
    Custom Title
    Join Date
    Jul 2003
    Posts
    281

    Joesitz

    Jositz, Your question is a valid one. Will skating survive if the winner is not the one who skates the perfect program and its obvious the winner is overmarked. The general public wont go to colledge to get a math degree to figure out the new COP. I am being the broken record as I say Im going to enjoy the season as if its my last because I think the OLYS will determine the survival or not of the sport. I admit its a corrupt sport however I dont have to accept it. Perception is the name of the game like it or not and the skaters who are overmarked are obvious to the general public more so because they dont have favorites and can be objective. We the fans underestimate mr and mrs general fan they see what is going on and walk away so TV has little viewers. I sure hope that the winner deserves the win or we all lose.

  7. #7
    Custom Title Joesitz's Avatar
    Join Date
    Jul 2003
    Location
    New York City
    Posts
    20,185
    ITA Cianni - and I will watch the Olys on TV and just declare the winner according to me. Why not? I don't have to suck up to the so called 'experts'. I've always said, give me another panel of 7 or 9 judges (experts) and the results will be different.

    Joe

  8. #8
    Custom Title
    Join Date
    Jul 2003
    Posts
    281

    Judging

    Jositz , I will do the same however according to me is usually according to most who dont take a stand to defend their favorite no matter how they skate. I think most fans know who should have won and who is overmarked and who the judges decided is going to win. It really doesnt take a rocket scientist to figure it out. The joke is changing the system to fool us and they make fools of themselves. This will be quite interesting the fun of watching the comp and how they skew the system to get their winner. Im showing my mistrust and wont I eat crow if Im wrong but I doubt I am.. Must add I do feel sorry for the skaters and esp. if the deserving winner doesnt get the Gold.

  9. #9
    Custom Title
    Join Date
    Jan 2005
    Posts
    675
    Maybe the USA and Finland are in cahoots. We should watch for foot tapping...

  10. #10
    Custom Title Mathman's Avatar
    Join Date
    Jun 2003
    Location
    Detroit, Michigan
    Posts
    27,066
    But if so, they are also in cahoots with the judgment of the panel taken as a whole.

Similar Threads

  1. Russian article on Euro judging
    By Ptichka in forum 2004-05 Figure Skating archives
    Replies: 16
    Last Post: 02-12-2005, 10:21 AM
  2. Ladies Short - Notes & Results
    By Rosaleen in forum 2003-04 Figure Skating archives
    Replies: 162
    Last Post: 03-29-2004, 09:36 PM
  3. New Judging System - End of Year Update
    By gsk8 in forum 2003-04 Figure Skating archives
    Replies: 6
    Last Post: 12-21-2003, 04:19 PM
  4. Statistical error in judging
    By mathman444 in forum 2002-03 Figure Skating archives
    Replies: 24
    Last Post: 04-10-2003, 05:46 AM
  5. Judging the Judges
    By Joesitz in forum 2002-03 Figure Skating archives
    Replies: 16
    Last Post: 12-28-2002, 07:37 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •