The Judging Controversy Thread

[Andre Smith]

YuNa vs. Sotnikova
http://imgur.com/XqXnps5

These are just the scores in ascending order. And it shows that if you removed Yuna's lowest 4 scores and Adelina's highest 4, Yuna would win.

Your observation is very correct!

 

[jenaj]

Sarah never won gold at such competitions, but she did win a bronze at worlds 2001 and the GPF the same year.

She also beat both Michelle and Irina at Skate Canada in the GP.

 

[Icey]

No, it is not.

http://static.isu.org/media/79156/2012_specialregulations_technicalrules_singleandpair_icedance.pdf

The scores in the columns belong to the same judge and there is no way of knowing exactly which judge that was (the judge number is randomly assigned); however, Each element line of the protocols is not randomized.

 

[Icey]

I thought only two scores (the highest and lowest) were dropped, not four.

Regardless, yuki90, you are incredibly rude. Accusing people of lying, blasting people for not knowing details, etc. -- Who do you think you are? I'm not even part of this conversation but I was shocked at the last couple of pages in this thread! :disapp:

You are correct that if there are 9 judges, only the highest and lowest are dropped. If there are fewer than 5 judges, no scores are dropped. Personally I think no scores should be dropped if the number of judges is 5 (or less)

 

[sk8in]

So if you were going to exploit the system you wouldn't underscore or overscore any skater drastically. You would just create a consistent few point gulf between the skaters in all elements. We at least saw that Sotnikova GOE's all tended to be like half a point higher than Kim's. That would be the smart way to impose a bias, or a straight up point conspiracy.

 

[OS]

What many failed to realize during the inception of COP is that to expect consistent high quality 100% human judging is ultimately futile - unless there are conscientious efforts to devise a strong ethic commission to managing the judges and its governing body. Something like Ombudsman for ISU and judges with their due diligence, reconciliation, audit deliberation and recalibration across different judging panel standards to promote impartiality, minimize subjectivity at major international events (WC, GP series, Olympics). It is particularly needed due to the crappy history this sport has in the public eye, and ISU's continuous failure to address these issues properly, maybe due to $$$$ This doesn't have to be a full time role, but a independent party consist of independent consultant professionals during key intervals of the season. Maybe funded by the IOC.

Judge judges events, Who are judging the judges?

The fact is as long as there are anonymity, ISU can easily manipulate any outcome without breaking any rules. They have all the information, they have all the power. Just place judges who has a history of awarding certain skaters high marks, certain style preferences, music tastes or national / political affiliation and biases, theoretically it can swing results to determine and improve the percentage of winning per particular skater per each competition. Even if it the judges are randomized, say draw 9 out of a list of 18, they can still ensure the 18 listed have a track history that befit their agenda. Is it any wonder 2 of the replacement judges at this Ladies FreeSkate also happened to be on the same panel as European Championship where the Russian skaters received wildly inflated PCS? The 3rd judge from Estonia judged Adelina at 2012 Jr. worlds, and the 4th judge the Ukraine judge has a shady past that should have seen him banned for life, instead he get to take part at another most controversial Olympics of all time. He was banned for a year at Nagano, he judged at Salt lake ice dance, and now Sochi ladies. Yet ISU don't seem to care.

Just like the skaters, each judges should have their own protocol scoring sheet that are available to public scrutiny. They should have a public profile fully disclose their judging history, scores they have given at which competition (whether were included or precluded being too high or too low), wrong calls made, corrective history (by ISU or Ombudsman), any penalties received etc... Only then there could be complete transparency and accountability to build public trust in this sport.

The skaters need to know they can trust the system and be judged fairly. Sochi Olympics has shattered that image of impartial judging since the Teams event, while the ladies was the pinnacle of those bias. If the worlds bests can't overcome a biased panel through sheer excellence, what hopes are there for the lower ranked skaters? Why bother training your heart out for an unfair sport?

Human beings are not robots. They can be inconsistent, fickle minded, emotional, sensitive, who all cope with pressures, environmental influences in their own way. Who all have built-in cultural, nationalistic/political biases that they fight to keep in check. Without supervision, without accountability, without transparency in the long run, no matter their age or nationality, they will always eventually succumb to these biases and poison the system. Without accountability and transparency, it always lead to corruption under any social organisation in any industry or business, it is no different than this sport of figure skating.

People like to pretend COP is better due to lower risk factors to protect 'too much' cheating through small incremental rewards that are marked by 'averaging' than 6.0 holistic way of judging, but as it has been proven, vulnerability has been identified and exploited. Somebody basically need to ensure judges need to be held accountable for all their marks given and be available to justify them.


---------------

By the way all these talk about missing a triple is the reason Kim has lost is truly laughable and convinient. Are jumps the only method to measure technicality? People forget Kim has always had 6 triples program since her back injuries, and with it she won multiple WC titles and Vancouver for several years through sheer quality and excellent delivery of her text book elements. That many Juniors have always had 7 triples but they were getting no where close to the seniors because they simply lack in maturity, artistry and seasoned skating skills. Things like Flutz, UR also hurt their scores, something Russian skaters are not called on at this competition at all, even if they had a history of flutzing and UR at previous competitions this season. People seem also forget it is not that Kim is deliberately trying less hard technical content, but there were rules changed after Vancouver which prevent her inclusion of 2A3T unless she risk injury to bring back the 3loop, ie/ ISU telling her to retire basically. Yet with it, she lead the field by 20 points last year with her Les Mis while being under marked in her SP. She has always maxed her best standard to have include 3lz3T in her Short and Long, and 2x 3luz in her long, never dumb down for a competition.

How convenient then all of a sudden to ignore all her other superior qualities. The fact there are hardly a gap between the 2 skaters in every thing else include GOEs and PCS indicate the marks are not done correctly. The system absolutely failed when we have Putin'mates running the asylum. Yes, COP died at this competition on the biggest night of 4 years in ladies free skate. It died even the reason it had been born was out of the necessity to be more objective and impartial to minimize subjectivity - to be less vulnerable to abuse. Yet at this competition, it seems deliberately maximized subjectivity by putting together a panel where subjectivity gains maximum reward beyond all reason, all objectivity according to sporting history particular the PCS corridor of judging.

To summarize. At this Olympics, a visibly flawed performance of an inferior Junior program managed to beat 3 world champions all put out practically flawless career best FS with superior programs, yet the 9th ranked WC skater from last worlds with no major international medals to speak of manage to beat the field by 5+ points, practically a land slide in this sport. And despite her visibly flawed performance of an inferior program, the panel decided to reward her with the greatest PCS in the history of ladies figure skating. Way to go COP, good luck in your future!

Thank you Patrick Ibens for speaking the truth, and hats up to Tony Wheeler for asking the tough questions.
http://figureskate.wordpress.com/2010/03/08/patrick-ibens-interview/

 

[Mrs. P]

The scores in the columns belong to the same judge and there is no way of knowing exactly which judge that was (the judge number is randomly assigned); however, Each element line of the protocols is not randomized.

Right. But the poster was saying Judge No. 1 for one skater was definitely Judge No. 1 for another skater.

 

[gkelly]

You are correct that if there are 9 judges, only the highest and lowest are dropped. If there are fewer than 5 judges, no scores are dropped. Personally I think no scores should be dropped if the number of judges is 5 (or less)

I know a local referee who prefers to use 4 judges instead of 5 at club-level events because high and low scores don't get dropped -- on the theory that 4 data points are better than 3. You just have to hope that none of the judges makes an egregious mistake, probably inadvertently, that overrides correct scoring by the other 3 on the panel.

But that would only be mistakes on one element at a time, or maybe the data input for one component.

 

[Mathman]

Thanks Yuki. So Sot received five scores over 150, which would be a world record score, despite a UR and an improper edge not called by the Russian controller. Yuna, despite a clean skate, gets only one score over 150, and gets a score of 134 from one judge. That is as convincing circumstantial evidence as you are going to get of a conspiracy.

Alas, that is not evidence of a conspiracy. That is evidence supporting the proposition that a lot of the judges liked Sotnikova's performance better than they liked Kim's.

 

[usethis2]

I really think we should not leave out tech panel when talking about the scores. They give out levels to skating elements. Even an honest point given by a judge can be skewed by the Techs. There are verrrrrrrrrrrry suspicious levels/calls/un-calls given to Yuna and Adelina. It boggles mind that Russian dude and "Olga" did tech calls for both the SP/FS.

 

[drivingmissdaisy]

At this Olympics, a visibly flawed performance of an inferior Junior program managed to beat 3 world champions all put out practically flawless career best FS with superior programs

Adelina had arguably the most intricate program in the entire field. Each of her jump entrances were more difficult, as in fewer crossovers and more turns in both directions, than either Caro or Yuna. As an example, Yuna did six crossovers prior to her lutz, whereas Adelina did only two but gained speed through other steps. A second would be their triple flips: look at how many steps Yuna and Adelina do before the jump, and how Adelina's more closely precede the jump. These things may not matter to you, and they might not look as clean and smooth, but they are more difficult and not at all indicative of "junior" skating. That Caroline Zhang does just crossovers and is much slower than Adelina shows how much work she has put into generate speed from moves other than basic stroking.

 

[usethis2]

Step sequence levels (lowest for Yuna's personal, Highest for Adelina)
Spin levels (lowest for Yuna's personal, Highest for Adelina)
Edge calls that Adelina did not receive
UR calls that Adelina did not receive (for very obvious URs)

 

[port79]

Five judges scored Adelina over 151, and all of those within a range of about 5 points. There were not two judges who scored her unusually high.

You friend Andy used an example: SS: 7,7,7,7,7,7,7,9,9

That is not what happened here. Adelina's lowest SS was 8.5 and she had 6 9.0+. She got all 9.0+ on PE, eight 9.0+ on CH and IN. The judges absolutely loved the program. To argue that two judges caused the result here is, frankly, stupid and ignores anything gleaned from a basic review of the protocols.

Please don't take my word for it: http://www.isuresults.com/results/owg2014/owg14_Ladies_FS_Scores.pdf

I don't understand how we are arguing whether two judges tilted this event when Adelina's third highest score was higher than all of Yuna's.

Let’s assume that out of four or five judges, who possibly did bloc-voting, two had the audacity to hold up Sotnikova and hold down Kim to the extreme scale. If we get rid of these two outliers, meaning factoring out the two highest scores from Sotnikova and the two lowest from Kim, then the FP scores of the two skaters would be virtually tied. That’s a huge swing from a 5 points gap.

Aside from technical analysis, an Economics professor at Dartmouth succinctly explained how a couple cheaters could influence an entire figure skating judge panel in this Washington Post article: (http://www.washingtonpost.com/blogs...dging-right-and-figure-skating-gets-it-wrong/).

The reverse happens in figure skating. Not only does a figure skater with a compatriot judge get a higher score from that judge, but they also get higher scores on average from the other judges, too (compared with events when they are not represented on the panel). This is evidence of vote trading, of the kind that occurred at the Olympics in 1998, 2002, and (allegedly) is occurring in 2014. Most of the benefit of having a compatriot judge actually comes through the vote trading. Skaters even benefit from having compatriot judges on the panels of other events, which is consistent with the fact that the vote trading we know about is often across events.

I'm not saying this whole conspiracy theory has been substantiated with much evidence, but I just don’t gather how you can act like your argument is indefensible to a point we can easily rule out the possibility.

 

[OS]

Adelina had arguably the most intricate program in the entire field. Each of her jump entrances were more difficult, as in fewer crossovers and more turns in both directions, than either Caro or Yuna. As an example, Yuna did six crossovers prior to her lutz, whereas Adelina did only two but gained speed through other steps. A second would be their triple flips: look at how many steps Yuna and Adelina do before the jump, and how Adelina's more closely precede the jump. These things may not matter to you, and they might not look as clean and smooth, but they are more difficult and not at all indicative of "junior" skating. That Caroline Zhang does just crossovers and is much slower than Adelina shows how much work she has put into generate speed from moves other than basic stroking.

So explain to me again how that justify 50 points improvement from Cup of China back in Nov 2013. Is that enough for to be awarded the greatest PCS ever in the history of this sport?

My opinion is movement should not be there unless the music calls for it. There are beauty in less is more, it is about the whole package and presentation. There's distinct lack of musicality in Adelina's performance, and I have no idea what she is trying to express and don't remember the program. Her lack in ice coverage also expose her great weakness in particularly the first 20 seconds she pretty much stuck in the same spot hardly skating, ie/ Time wasting for half way bonus.

 

[usethis2]

Ice coverage - Yuna
http://i.imgur.com/2I8PTPb.jpg

Ice coverage - Adelina
http://i.imgur.com/mEo6mFQ.jpg

What's more striking about these composite shots than the amount of area being covered, it's that Yuna's lines (paths) are clear and purposeful. Adelina's are almost random and hard to make out. There's no pattern to the choreography.

Don't forget that Adelina's shot is taken at an angle more favorable to her than Yuna's.

 

[Mathman]

I know a local referee who prefers to use 4 judges instead of 5 at club-level events because high and low scores don't get dropped -- on the theory that 4 data points are better than 3.

Strange as it seems, this referee's concerns are misplaced. The trimmed mean starting with 5, dropping highest and lowest, is statistically more reliable than four, counting all.

Distribution theory for trimmed means (the median is a "trimmed mean" with maximal trimming) is a topic of ongoing research in the branch of statistics called "non-parametric statistics." For such studies, all the results that we learn in Stat 101 that follow from the Central Limit Theorem (that is, all those formulas that have "Sigma/square root " in them cannot be guaranteed to work. Bottom line -- trimming from nine to seven has the same degree of statistical accuracy as using all nine (and has some other desirable features like eliminating the effect of obvious touch-screen errors.) In fact, there would be nothing wrong with using median marks throughout, eliminating all the adding and averaging altogether.

 

[drivingmissdaisy]

There are beauty in less is more, it is about the whole package and presentation.

I agree to an extent, but I acknowledge that the judging system is more about accruing points than beautiful skating. When a skater does something difficult it accrues points. Objectively, Adelina is doing more between, and into, the jumps and Yuna is doing less, but quality-wise what Yuna is doing is better. It's a legitimate dilemma for a judge how to fairly score this, and it isn't about past scores or trends, but rather what the athletes presented on that night.

 

[b-man]

Alas, that is not evidence of a conspiracy. That is evidence supporting the proposition that a lot of the judges liked Sotnikova's performance better than they liked Kim's.

So you are fine with five of nine judges giving world record, over 150 point scores, for a performance with an uncalled edge violation and an UR, with a russian controller? How many instances of questionable judging do you need to see before the results rise from the controversial to the level of a conspiracy? You are never going to get the 4 or 5 judges in the conspiracy to confess to their wrongdoing, all we have is the scores, which have risen to such levels that make any explanation other than a conspiracy an unlikely possibility.

 

[starrynight]

In my opinion, we do not even need to investigate how an individual judge scored the GOE and PCS for Adelina and Yuna. Please do any random combination matching judges' scores for Adelina and Yuna. In every combination, four judges stand out with blatant favoritism for Adelina. Sonia Bianchetti also pointed that out: "Can anyone explain to us why four judges placed Adelina ahead in every single component score, some of them with a difference of more than one point?" (See http://www.soniabianchetti.com/writings_sochi2014.html)



P.S. Yuna took the ice last and judges knew the scores of all others; one judge gave Yuna 134.09, which is less than Julia’s 135.34. Anyone agrees on this fabulous judge's insight - put your hands up and say big yay.

 

[drivingmissdaisy]

To clarify MM's point, it isn't necessarily that the judges liked Adelina's program better. They merely judged it to be better. That was the impression I got from Scott, was that he and Sandra very much preferred Yuna's overall performance, but it didn't accrue as many points as Adelina, who made more of an effort to tailor her program to the judging criteria, similar to how Irina would Bielmann everything to gain levels even though it made her program less pleasing.