Two Olympic Judges suspended by ISU

[Sam-Skwantch]

We've covered it millions of times and I am not usually provoked by yet another mentioning of Lakernik, Shekhovtsova, Sochi, etc. I do not share popular by some views of conspiracy and scandal - I think that they are delusional. But I do respect the people's right for their own point of view, including those on competency of Lakernik. I just cannot bear the labels like "fraudster".

I’ve always respected him. I think he has a true passion for the sport and believes in what he is doing. There was an article in like 2015 where he reasoned out and explained how and why he wanted to expand GOE and address BV. It was interesting and he certainly was well reasoned and competent.

I don’t like expanding GOE at all but I still took in his points and understand why he thinks it would help the scoring overall.

He probably wouldn’t be my first choice for a tech panel but his role at the ISU hasn’t really been a major concern of mine regardless of where I stand on the issues. He’s kind of in a no win situation given his job title. No one even ran against him for a very thankless job.

 

[Matt K]

rather than looking for corridors, ISU should simply add up the scores for each judge, and automatically investigate the folks who place teams too weirdly.
For example, while placing PC 1st and VM second at Olympics is fine, placing VM 4th is just not ok and requires some attention.
And so on.

Honestly, there is some utterly weird judge stuff, and it deserves more spotlight. We need to know our "heroes"

Very well said. Yes, and this is why an investigation should be opened up to include all judges and not just 2 handpicked Chinese judges.

It has been said numerous times that a US judge called Lorrie Parker must be investigated. I would like to include the addition of US Ice Dance judge Sharon Rogers. Who else?

 

[Mamamiia]

So the fact that USA judges give their own American skaters 2nd place is laughable and a very serious matter that requires much attention. Nathan Chen bombed at Pyeongchang and his marks from the USA judge should reflect that.

Nathan only bombed his short program and I didn't find any US judge in men's short so I don't know what you are talking about here. I might be looking at a wrong list...

 

[cohen-esque]

On a general note I don’t think that many people here, in their rush to gloom and judgement, have a) bothered to look at the OAC evaluation protocols, even though they’ve been posted; and b) realize just how wide the corridor is.

Anyway, if we use the OAC document as a guide, then the Chinese judge in the Men’s event was absurdly biased. Looking at the scores she gave to Boyang, her total deviation in GOE was high enough to trigger an evaluation for the SP marks. Her total deviation in GOE was high enough to trigger an evaluation again for the FS marks. Her total deviation was more than 18 marks! (The corridor allows 13.) SIX ELEMENTS in the Free Skate had high enough individual GOE deviations to trigger evaluations (and the individual GOE deviation bar is comparatively higher than the total deviation bar.) The cutoff for unjustifiable marks for the entire Free Skating segment of the competiton before triggering an evaluation was just four.

It’s hard to be that far outside the corridor. Like, extremely, ridiculously difficult. And yet she ended up about three halls over.

By contrast, *every single one* of Lorrie Parker’s marks for both Nathan and Vincent fell well within the corridor. Only one of her marks for Yuzuru was (barely) low enough to trigger an evaluation for that element (the 4S+3T). She also scored him more highly than the panel on two elements.

For all we know, she did receive a warning for that single GOE mark. But the threshold of unjustifiable marks for the Olympic Men's event (30 competitors) is four.

On a quick perusal, the Spanish judge in the Men's SP, who has been mentioned in this thread, also only seems to have given out one unjustifiable mark (0, for Kolyada's 3Lz). *cough* Perhaps he noticed the preceding steps. *cough* He gave generally low scores to most skaters.

No one’s PCS marks deviated enough to trigger an evaluation.

---

I have no idea how they can justify that Feng Huang received a warning at the GPF or the Olympics, much less a suspension. He did obviously lowball Savchenko/Massot at the GPF, but it doesn't look as if he did by enough to fall outside the corridor for any single element, for overall GOEs, for any single Component, or for the overall Components. (He also didn’t over-score Sui/Han by enough to fall outside the corridor, but the corridor is so wide that it’s just about worthless in establishing over-scoring evaluations of top skaters: there’s not enough room in the scores to deviate highly enough.)

None of his scores there should have triggered an OAC evaluation, despite showing bias. His Olympic scores for Sui/Han, S/M, and even Yu/Zhang are all within the corridor, especially in the FS. In fact, I'd say he took the warning seriously, because his Olympic scores... aren't really even that notable. He gave three teams comparatively higher scores than the Chinese pairs in the SP.

If his scores were enough to trigger disciplinary action, pretty much every ISU judge this season should have received at least a warning.

 

[Matt K]

As moriel and many others have stated here, corridor evaluation must go. Who cares that Lorrie Parker was "well within the corridor" for her marks to USA skaters? She still placed Vincent Zhou and others ahead of Hanyu etc. That is where the problem lies. Judges like US Ice Dance judge Sharon Rogers have been calculatingly placing skaters "within the corridor" despite favoring their own skaters (sometimes, massively) relative to other teams for quite some time now. The fact that Lorrie Parker and Sharon Rogers can breathe a sigh of relief from ISU knocking on their doors and get away with it simply because they stayed "within the corridor" shows that that is an easy way to cheat and continue cheating without ever getting caught. And that is the real problem here.

Another problem with corridor judging, is that marks especially for ice dancing among the top 10 or so, are so close that a PCS of 9.00 and 9.50 can be the difference between a podium position and 8th place. Corridor judging in that case clearly doesn't work.

Maybe a better argument you can offer is WHY corridor judging should stay.

 

[yume]

On a general note I don’t think that many people here, in their rush to gloom and judgement, have a) bothered to look at the OAC evaluation protocols, even though they’ve been posted; and b) realize just how wide the corridor is.

Anyway, if we use the OAC document as a guide, then the Chinese judge in the Men’s event was absurdly biased. Looking at the scores she gave to Boyang, her total deviation in GOE was high enough to trigger an evaluation for the SP marks. Her total deviation in GOE was high enough to trigger an evaluation again for the FS marks. Her total deviation was more than 18 marks! (The corridor allows 13.) SIX ELEMENTS in the Free Skate had high enough individual GOE deviations to trigger evaluations (and the individual GOE deviation bar is comparatively higher than the total deviation bar.) The cutoff for unjustifiable marks for the entire Free Skating segment of the competiton before triggering an evaluation was just four.

It’s hard to be that far outside the corridor. Like, extremely, ridiculously difficult. And yet she ended up about three halls over.

By contrast, *every single one* of Lorrie Parker’s marks for both Nathan and Vincent fell well within the corridor. Only one of her marks for Yuzuru was (barely) low enough to trigger an evaluation for that element (the 4S+3T). She also scored him more highly than the panel on two elements.

For all we know, she did receive a warning for that single GOE mark. But the threshold of unjustifiable marks for the Olympic Men's event (30 competitors) is four.

On a quick perusal, the Spanish judge in the Men's SP, who has been mentioned in this thread, also only seems to have given out one unjustifiable mark (0, for Kolyada's 3Lz). *cough* Perhaps he noticed the preceding steps. *cough* He gave generally low scores to most skaters.

No one’s PCS marks deviated enough to trigger an evaluation.

---

I have no idea how they can justify that Feng Huang received a warning at the GPF or the Olympics, much less a suspension. He did obviously lowball Savchenko/Massot at the GPF, but it doesn't look as if he did by enough to fall outside the corridor for any single element, for overall GOEs, for any single Component, or for the overall Components. (He also didn’t over-score Sui/Han by enough to fall outside the corridor, but the corridor is so wide that it’s just about worthless in establishing over-scoring evaluations of top skaters: there’s not enough room in the scores to deviate highly enough.)

None of his scores there should have triggered an OAC evaluation, despite showing bias. His Olympic scores for Sui/Han, S/M, and even Yu/Zhang are all within the corridor, especially in the FS. In fact, I'd say he took the warning seriously, because his Olympic scores... aren't really even that notable. He gave three teams comparatively higher scores than the Chinese pairs in the SP.

If his scores were enough to trigger disciplinary action, pretty much every ISU judge this season should have received at least a warning.

So ISU is saying "you can cheat, but cheat in the corridor". Absolutely absurd. US judge not only showed bias but also clear cheating by highly overscoring her skaters while
underscoring to death the most dangerous skater for them. But since she's in the corridor she shouldn't be punished.
But apparently even being in the corridor don't always save you when you are from some nationality. Laughable.

 

[alexaa]

Nathan only bombed his short program and I didn't find any US judge in men's short so I don't know what you are talking about here. I might be looking at a wrong list...

You are not reading wrong list. Some people just don’t bother to read before posting.

 

[cohen-esque]

As moriel and many others have stated here, corridor evaluation must go. Who cares that Lorrie Parker was "well within the corridor" for her marks to USA skaters? She still placed Vincent Zhou and others ahead of Hanyu etc. That is where the problem lies. Judges like US Ice Dance judge Sharon Rogers have been calculatingly placing skaters "within the corridor" despite favoring their own skaters (sometimes, massively) relative to other teams for quite some time now. The fact that Lorrie Parker and Sharon Rogers can breathe a sigh of relief from ISU knocking on their doors and get away with it simply because they stayed "within the corridor" shows that that is an easy way to cheat and continue cheating without ever getting caught. And that is the real problem here.

Maybe a better argument you can offer is WHY corridor judging should stay.

I haven’t offered any argument at all regarding keeping corridor judging one way or the other. Perhaps you should try to re-read my post without letting your sheer determination to be negative affect your comprehension. (Although in fact, this exercise has just further cemented my belief that corridor judging as it stands now is...useless, at best.)

This thread has 12 pages of moaning about hand-selected, hypocritical ISU disciplinary actions and explicitly calling out the other judges that the ISU supposedly showed favoritism to, by name, demanding to know why they weren’t also punished: and I am pointing out, that by the ISU’s own published standards that literally anyone in this thread could have spared 30 seconds of their life to glance over, there was not necessarily a reason to punish those judges—And also that, yes, there was also no apparent reason to punish Feng Huang by those same standards, either.

Which shows, I think, that the ISU have indeed not been consistent or even-handed in applying their own procedures. I’m baffled as to why you and others seem to want them to extend that disregard for their own rules to other judges, though.

And on that note, it’s The ISU who had better care that Lorrie Parker was within the corridor, because that’s the standard that they have decided upon to evaluate a judge’s performance. That Feng Huang was also within the corridor and yet he punished anyway should be concerning because of that. Of course, it also shows that being within the corridor is apparently *not,* quote, “an easy way to cheat and continue cheating with ever getting caught,” because he was.

---

Who cares that Lorrie Parker was "well within the corridor" for her marks to USA skaters? She still placed Vincent Zhou and others ahead of Hanyu etc.

Maybe a better argument you can offer is WHY corridor judging should stay.

If we’re going to determine the validity of the the results by scrutinizing the judges’ placements than we may as well go back to 6.0 with majority ordinals. In a numerical system supposedly based on mostly objective criteria, the amount deviation from the mean is *much* more important in evaluating the judging than the individual placements— especially in a sport where medalists have been separated by hundredths of a point.

Judges like US Ice Dance judge Sharon Rogers have been calculatingly placing skaters "within the corridor" despite favoring their own skaters (sometimes, massively) relative to other teams for quite some time now.

Another problem with corridor judging, is that marks especially for ice dancing among the top 10 or so, are so close that a PCS of 9.00 and 9.50 can be the difference between a podium position and 8th place. Corridor judging in that case clearly doesn't work.

Yes, as I alluded to in my post. The issue with the ISU corridor is not that they take a deviation-based approach, it’s that their cutoff for allowable deviation is so high as to be nearly useless in all but the most extreme cases of judging bias (such as that displayed by Chen Weiguang) and it has a very diminishing capability to determine bias as marks continue to get higher. If a skater earns average +1.5 GOE (under the +/-3.0 scale) and average 8.50s for PCS, it becomes mathematically impossible to establish any bias in their favor, only bias against them, using the ISU’s method.

The current calculation for GOE deviation would probably work a lot better under the +/-5 GOE system, and I haven’t seen anything to indicate the ISU is altering it so far. I believe, based mostly on having looked at countless protocols, the PCS corridor for each individual component should at at least be at halved to +/-0.75, though maybe others might think that’s a bit of a drastic cut.

A more straighforward approach might just be to compare judges against only:

1. The standard error of the panel
2. Their own average deviation in marks

In each individual competition, and consider those two pieces of information when evaluating judging bias. It’s essentially what the ISU has done with Feng Huang...in a very roundabout sort of way... by noting the discrepancies in his scores even if they don’t break the corridor. And it’s just a step above what Shanshani has already done with their spreadsheet, which posters in this thread have referred to to point out obvious instances of bias and the *amount* of bias.

 

[karne]

Sam..people aren't getting what you mean here. I think it's not as clearly stated as you think it is. Maybe you could clarify a little? Do you mean the whole Yuna-Adelina judging issue or something else.

Interested in your opinion but just want to get your true meaning.

Well, I'd like to add, I clearly asked why the poster I quoted felt Lakernik should be investigated. As this was not answered, I suspect that samkrut's response is accurate, that that person is still trying to proclaim issues with Sochi.

I'd be interested to know if Lorrie Parker would still be pursued with such vigour by some posters here if it had not been for the perception that she "marked down" Hanyu...

 

[Shanshani]

Huh, that corridor is absurdly wide (and Feng Huang falls well within it, as cohen-esque noted), as it doesn't catch some pretty blatant cases of nationalistic bias. Regarding my spreadsheet, I actually employed my own version of a corridor for flagging anomalous scores. Basically, I computed a measurement for how far a judge's score typically strays from the actual score in a given segment of the competition, which I then used to determine how likely it is for a judge's score to stray a certain degree from the actual score. I determined two thresholds for flagging score difference: First, a soft threshold, which caught any score outside of the range of score differences between which 75% of scores differences fell. Second, a hard threshold, which caught scores that fell out of the range of 95% of score differences. Essentially, the first threshold will catch the top ~25% of scores falling farthest away from the average, and the second threshold would catch the top ~5% of scores that are farthest from the average. Of course, there will always be top 25% and top 5% farthest-away-from-average score, but in a situation where judging is completely unbiased, we should expect those anomalous scores to occur randomly, rather than match up suspiciously with judges'/skaters' nationalities. So if we examine the pattern of judge's anomalous scores, we can have a good idea of whether or not there's bias. A one off 5% chance score deviation isn't much of a concern--it has a 5% chance of happening after all. But if those 5% frequency scores match up with certain skaters, especially more than once, then the chance of that being random goes down by a lot.

Taking the men's free skate for instance (since the men's free skate is coming up a lot), I found that 75% of the time, a judge's score will fall within 5.14 points of the skater's actual score, and 95% of the time, a judge's score will fall within 8.77 points of the actual score (keep in mind, this is valid only for the men's free skate--other competitions and other segments tended to have lower deviations). So a score more than 8.77 points away from the actual score has only a 5% chance of occurring. Note that Judge Parker had four and only four instances of scores that deviated more than 8.77 points from the actual score. They are:

Nathan Chen (over-scored by 10.22 points)
Yuzuru Hanyu (under-scored by 8.92 points)
Boyang Jin (under-scored by 10.02 points)
Adam Rippon (over-scored by 10.30 points)

(She also over-scored Vincent Zhou by 7.19 points.)

Even if you ignore her Vincent Zhou score, the chances of this occurring by accident are therefore roughly:
0.05*0.05*0.05*0.05=0.00000625=0.000625% or 1 in 160,000. (In actuality, the chances of this result occurring by accident are even lower, because the chances of a 10 point over or under-score are even lower than 5%). If you add in Vincent Zhou, it falls even farther. Any system that's unable to catch that is very broken.

 

[Shanshani]

Well, I'd like to add, I clearly asked why the poster I quoted felt Lakernik should be investigated. As this was not answered, I suspect that samkrut's response is accurate, that that person is still trying to proclaim issues with Sochi.

I'd be interested to know if Lorrie Parker would still be pursued with such vigour by some posters here if it had not been for the perception that she "marked down" Hanyu...

Lorrie Parker is an extremely obvious case of biased marking, and you don't need to be an uber of a particular skater to notice or complain about it. As per my post above, the odds that her particular pattern of scoring occurred by accident is something on the order of 1 out of 160,000, and that's making some generous computational simplifications in her favor. She is, without a doubt, the second most biased judge at the Olympics after Weiguang Chen and she was significantly more biased than Feng Huang was. It doesn't take more than some fairly basic statistical analysis to show this. Yuzuru Hanyu's fandom is irrelevant--what is relevant are the numbers.

 

[WineHerUp]

If your eyes are functional, then it should be clear as day as to why Vincent should not be scoring anywhere close to Yuzuru. You don't need to be a Hanyu fan to recognize quality .

 

[chopinskate]

On a general note I don’t think that many people here, in their rush to gloom and judgement, have a) bothered to look at the OAC evaluation protocols, even though they’ve been posted; and b) realize just how wide the corridor is.

Sure, but a protocol that "detects" the Chinese Pairs' judge (who was within the corridor as you and Shanshani have pointed out), but still somehow passes over the US Singles' judge (who was still the second most biased judge in the Singles' and Pairs' events, which even the fans were able to tell!)... Is there a real reason to study such a protocol? Not only is the "allowed" corridor ridiculous, it seems that it's not evenly applied to all judges. It makes you question the procedures, and absolutely it makes you question if there isn't some national bias.

Unfortunately, not as funny as the judges giving almost 130 to Zagitova with 3 falls, 2 URs and 1 downgrade.

Undebatable.

Fortunately, the judges got it unanimously correct with the World Champion ,

Very debatable.

I'd be interested to know if Lorrie Parker would still be pursued with such vigour by some posters here if it had not been for the perception that she "marked down" Hanyu...

Why? Because you were reminded of Sochi according to the rest of your post, and how people were vigorously complaining "just because" it was Yuna Kim? Can't it be that people care about fair judging? Or are we at the point that fairness falls simply within a post-modern realm of subjectivity, and we can all have our own interpretations of it?

It's great, though, that some seemingly can't even allow that people care about the state of judging more than a skater. Shows how doomed fairness really is, if people think fans cry about it only if it's affecting their favourites. But hey, at least anonymous judging is gone, and they can persuade others more easily, instead of just being accused of being "bots". That would be interesting to know, too, wouldn't it, how many more people would be called "bots" and "trolls" if anonymous judging were still around?

 

[xeyra]

So what the OAC guidelines tell us is that judges have a very large threshold for national/personal bias and only extreme cases are ever outside that wide corridor.

And the decisions taken by ISU also tells us that if you're Chinese, you're toast?

 

[russianfan]

The ladies discipline is hardly worth watching anymore

but you're still watching

 

[chopinskate]

but you're still watching

Barely. Doubtful with a non-olympic season coming up. Maybe the WCs.

 

[gkelly]

As moriel and many others have stated here, corridor evaluation must go. Who cares that Lorrie Parker was "well within the corridor" for her marks to USA skaters? She still placed Vincent Zhou and others ahead of Hanyu etc.

That's not how IJS scoring works. No judge "places" any skater anywhere.

Parker gave Zhou significantly lower GOEs and especially PCS than she gave Hanyu. However, Zhou's 18.03-point higher base value in the elements, which she had no control over, was enough to keep the total of her scores plus the tech panel/Scale of Value determinations for Zhou add up to more than for Hanyu.

Quite likely many judges would be surprised to learn how their scores plus the tech panel scores end up "placing" the skaters. They don't have time to do the math for each skater and keep track for the whole field. That's not their job.

 

[narcissa]

but you're still watching

I sold my tickets

 

[chopinskate]

I sold my tickets

 

[ancientpeas]

Maybe I'm an optimist but I'm looking forward to the new season and discovering new skaters and older skaters doing new things. I'm excited to see new routines and to enjoy skating.

I'm also willing to let the judging thing play out and hope that any fixes that are needed are made. I remain positive and hopeful.