South Korean federation's complaint to the ISU about judging

qwertyskates said:

However, you are *assuming* the most extreme scenario in your hypothesis, a difference of 12 points in (1), and then claim that the probability of that happening is *THE SAME* as a more plausible scenario,

Click to expand...

I made no such claim.

However, that would be an interesting study. Maybe we can cajole Vanshilar into undertaking it. Take all possible permutations of judges' scores (9 factorial divided by k factorial whenever there are k scores the same), and see what percentage of the time it comes up one way rather than another.

As for what is intuitively more likely, well … in this argument some people think it is more likely that some judges are stingier across the board, while others think it is more likely that some of the judges were biased. I do not have an opinion about which of these is more likely. We do know that in figure skating judging both happen. My only point is that we cannot decide by looking at the protocols, thanks to randomized anonymous judging.

qwertyskates said:

I can't help but laugh at your grandiosity in assuming that the whole world thinks like you. Sorry to say, your assertion is in the minority. It's only Koreans and a handful of loud, outspoken shills, the majority of the mainstream media supported Adelina's win, and were detailed their reporting. Whether you agreed with them is another matter. However the MAJORITY were fine with Adelina getting Gold, including me. Adelina skated better, period.

Click to expand...

Your statements are just as grandiose as Ven's. The majority of the mainstream media was fine with the results? Do you mean the Russian media? If not, then this can be easily disproven by statistics. Using exaggerations doesn't make your argument convincing.

qwertyskates said:

I can't help but laugh at your grandiosity in assuming that the whole world thinks like you. Sorry to say, your assertion is in the minority. It's only Koreans and a handful of loud, outspoken shills, the majority of the mainstream media supported Adelina's win, and were detailed their reporting. Whether you agreed with them is another matter. However the MAJORITY were fine with Adelina getting Gold, including me. Adelina skated better, period.

Click to expand...

I can't fathom the cognitive dissonance of someone who can put these two sentences side-by-side.

Mathman said:

However, that would be an interesting study. Maybe we can cajole Vanshilar into undertaking it. Take all possible permutations of judges' scores (9 factorial divided by k factorial whenever there are k scores the same), and see what percentage of the time it comes up one way rather than another.

Click to expand...

It's easy enough to write code to iterate through different possibilities, but I'm not sure what the study is exactly supposed to be. Is it trying to figure out that assuming each permutation were equally likely, what were the rankings for each skater from each judge? How many skaters would be compared (and which)?

Vanshilar said:

It's easy enough to write code to iterate through different possibilities, but I'm not sure what the study is exactly supposed to be. Is it trying to figure out that assuming each permutation were equally likely, what were the rankings for each skater from each judge? How many skaters would be compared (and which)?

Click to expand...

I don't know, either, what such a study would show. An earlier poster gave the opinion that matching high scores with high scores and low scores with low scores is inherently more likely than matching high scores with low scores and low scores with high scores. I just wondered how many of the latter kind of matchings would result in a situation where a minority of the panel dominated the ordinal majority. Maybe some sort of Bayesian analysis could then say something about the relative likelihood of (a) three judges conspiring to fix the results and (b) some judges were stingy and some generous across the board.

I don't think so. though. In this instance (as in almost all applications of statistics to figure skating judging) non-statistical considerations swamp anything we are trying to study. That is to say, judges' scores are not a sample drawn at random from a nicely distributed population -- certainly they are not if someone is playing sample demon by deliberately cheating or responding to national bias.

Mathman said:

I don't know, either, what such a study would show. An earlier poster gave the opinion that matching high scores with high scores and low scores with low scores is inherently more likely than matching high scores with low scores and low scores with high scores. I just wondered how many of the latter kind of matchings would result in a situation where a minority of the panel dominated the ordinal majority. Maybe some sort of Bayesian analysis could then say something about the relative likelihood of (a) three judges conspiring to fix the results and (b) some judges were stingy and some generous across the board.

I don't think so. though. In this instance (as in almost all applications of statistics to figure skating judging) non-statistical considerations swamp anything we are trying to study. That is to say, judges' scores are not a sample drawn at random from a nicely distributed population -- certainly they are not if someone is playing sample demon by deliberately cheating or responding to national bias.

Click to expand...

I meant I didn't know what I was supposed to do -- if it was just trying out each permutation of say skater X's judge 1 with any of skater Y's judge 1-9, then skater X's judge 2 with any of skater Y's remaining 8 judges, etc., and compare how often skater X was ranked higher than skater Y by the judges or something like that. In which case, at a first glance it seems like it's just a matter of iterating through the 9! = 362880 possible permutations, which seems easy enough. I guess adding a third skater would make it a lot more difficult since there'd be (9!)^2 ~ 1.3 e11 permutations which would take a lot longer, so it's probably better to just do pair-wise comparisons between different skaters depending on how efficiently I feel like coding it up.

But what exactly would I be comparing? Is it the sum of the PCS scores from each judge? Is it the sum of the GOEs? If I do something like the total score that each judge gave, then the technical calls and base values (which are based partially on the technical calls) could affect the rankings, but some of those are themselves under dispute. Yet if I only look at the sum of the PCS scores, for example, then it would only show how the judges would've ranked the skaters' overall impressions (artistry, etc.) and not consider the overall performance such as technical elements, which could change the judge's overall rankings. That would be true of any subset of the scores such as GOE's as well. So basically it could be done, but the implications are somewhat limited.

If we're looking to determine if there were any judging irregularities, whether high scores matched with high scores or high scores matched with low scores would be exactly the type of statistical analysis that would be performed, except that anonymous judging specifically removes this as an option. For example, "fair" scoring might look like (just making up numbers so they're not realistic):

Skater X: 10 9 8 7 6 5 4 3 2
Skater Y: 9 8 7 6 5 4 3 2 1

While judging with a few biased judges might look like:

Skater X: 8 7 6 5 4 3 2 10 9
Skater Y: 9 8 7 6 5 4 3 2 1

In the first case, of course, every judge felt skater X did better, while in the second case, every judge felt skater Y did better except the last two who threw the scores so that skater X would have a higher average. If the judges were known, it would be easy enough to do an X-Y plot (i.e. matching the scores from each judge) to determine this. Under anonymous judging, the scores shown in either case would be the same -- removing the ability to do this kind of analysis and to detect such biased judging. I'm not sure how you would construct a Bayesian analysis to determine how likely it is that you had some judges conspiring to fix the scoring, given that your beliefs on the number of conspiring judges would be the prior in the first place if I were to do the permutations -- so the results depend very much on your beliefs, which is, of course, different for different people and under disagreement in this thread.

of course qwertyskates is going to react from mathman's post..took it personally and now making an accusation w/ the poster.. how predictable.. and every poster who defended sot's ogm..

that 31 goes+ for sotnikova if it's indeed true is insane.. and it looks like someone who's in payroll..

ISU should reveal the scores!! there's nothing to hide.. besides russia and the US advocates to get rid of anonymous judging.. why not try to make an example of this?? after all it was a legit win.. right??

They should release the judge's marks but also the technical caller's marks. See if the Russians Lakernik and Baranova worked together against Gusmeroli.

Of course the ISU won't do this though because they have something to hide.

Vanshilar said:

I meant I didn't know what I was supposed to do -- if it was just trying out each permutation of say skater X's judge 1 with any of skater Y's judge 1-9, then skater X's judge 2 with any of skater Y's remaining 8 judges, etc., and compare how often skater X was ranked higher than skater Y by the judges or something like that…

Click to expand...

Just as a purely mathematical puzzle, I would be interested to know the following. I do not know what conclusion I could draw from the answer -- probably none.

Here are the total PCS for each of the nine judges for Sotnikova and Kim. (These are slightly different from what I posted earlier. I think these are right. I have a visual handicap that makes adding up lists of numbers difficult.)

SOT 48.25, 48.00, 48.00, 47.75, 46.50, 45.50 45.00 44.25, 44.25

KIM 48.75, 48.00, 47.75, 47.00, 46.75, 46.00, 46.00 45.25, 42.00

Fix Sotnikova's order and for each of the 300,000 possible ordering of Kim's, record the number x of judges that favored Sotnikova (half a point for a tie). What is the frequency distribution of x?

At the moment I cannot think of any reason why anyone would want to know this. But I think it might point my mind in the right direction to worry about it some more. :yes: I have a feeling that questions of this sort have already been extensively studied, and the result would be the same if we used the numbers 1, 2, 3, …, 9 instead. But the interesting thing would be, how does the distribution change as a function of the difference in the means? (In this example the means for the two skaters are about the same.)

Ven said:

They should release the judge's marks but also the technical caller's marks.

Click to expand...

It's this kind of ignorant comment that makes me angry and also proves that this is just Yuna fans being bitter that she didn't win, and not people who actually care enough about the sport to learn how the judging works.

The technical panel does not give marks.

Mathman said:

SOT 48.25, 48.00, 48.00, 47.75, 46.50, 45.50 45.00 44.25, 44.25

KIM 48.75, 48.00, 47.75, 47.00, 46.75, 46.00, 46.00 45.25, 42.00

Fix Sotnikova's order and for each of the 300,000 possible ordering of Kim's, record the number x of judges that favored Sotnikova (half a point for a tie). What is the frequency distribution of x?

Click to expand...

Eh okay but the numbers that I'll be using are:

SOT 48.25, 48.00, 47.75, 47.75, 46.50, 45.50 45.00 44.25, 44.25

KIM 48.75, 48.00, 47.75, 47.00, 46.75, 46.00, 45.75, 45.25, 42.00

I guess basically I'll tally up how often I had x = 0 to 9 in increments of 0.5.

While I'm working on it, does anyone want to speculate on what would it mean if I find that the distribution was generally above x = 4.5 (indicating more judges favored Sotnikova), or if I find that the distribution was generally below x = 4.5 (indicating more judges favored Kim)?

karne said:

It's this kind of ignorant comment that makes me angry and also proves that this is just Yuna fans being bitter that she didn't win, and not people who actually care enough about the sport to learn how the judging works.

The technical panel does not give marks.

Click to expand...

I think it's fairly obvious from the context that Ven is talking about the levels, URs, wrong edges, etc., regardless of if they're called marks, calls, indicators, or whatever.

Vanshilar said:

While I'm working on it, does anyone want to speculate on what would it mean if I find that the distribution was generally above x = 4.5 (indicating more judges favored Sotnikova), or if I find that the distribution was generally below x = 4.5 (indicating more judges favored Kim)?

Click to expand...

I am pretty sure it will be somewhat below 4.5, slightly favoring Kim. I do not think we can draw any conclusion from this unless it is much smaller, like below 4.

karne said:

It's this kind of ignorant comment that makes me angry and also proves that this is just Yuna fans being bitter that she didn't win, and not people who actually care enough about the sport to learn how the judging works.

The technical panel does not give marks.

Click to expand...

It's this kind of ignorant comment that makes me angry and also proves that this is just Yuna hater who just cannot see Yuna being in the right way, but in the end, it's pretty obvious there was manipulative and biased judging that caused this controversy. Flutz and UR not being called on Adelina was a dead giveaway. And if you cannot see( or purposely refuse to ) that, well sorry my friend, you need new set of eyes. Perhaps Yuna fans are just trying to make this sport more fair by setting her as an example and prove that she's a victim of this corrupted sport.


Plus you failed to understand what Ven was saying

Melon said:

Perhaps Yuna fans are just trying to make this sport more fair by setting her as an example and prove that she's a victim of this corrupted sport.

Click to expand...

Yes, by claiming the fact that the replay operator and data operator were Russian is proof of cheating. :disapp:

karne said:

Yes, by claiming the fact that the replay operator and data operator were Russian is proof of cheating. :disapp:

Click to expand...

My god you really need to go see an optometrist. Uncalled flutz and UR is waving at your face. Stop making a fool out of youself.

qwertyskates said:

I can't help but laugh at your grandiosity in assuming that the whole world thinks like you. Sorry to say, your assertion is in the minority. It's only Koreans and a handful of loud, outspoken shills, the majority of the mainstream media supported Adelina's win, and were detailed their reporting. Whether you agreed with them is another matter. However the MAJORITY were fine with Adelina getting Gold, including me. Adelina skated better, period.

Click to expand...

LOL, what a delusional, what is "majority" mean for you? You can only fool yourself period, stop bring NYT article or someone like Tara, Weir ... up here, we already know how nonsense about them

Mathman said:

I will start a new thread about this oddity in the ISU judging system, independent of nationalistic patriotism and unrepentant fandom. (Prediction: No one will want to read it or to contribute. )

Click to expand...

Well, at least you stated the fact that all this controversy was about nationalistic patriotism and unrepentant fandom. Can't agree more.

Mathman said:

Just as a purely mathematical puzzle, I would be interested to know the following. I do not know what conclusion I could draw from the answer -- probably none.

Here are the total PCS for each of the nine judges for Sotnikova and Kim. (These are slightly different from what I posted earlier. I think these are right. I have a visual handicap that makes adding up lists of numbers difficult.)

SOT 48.25, 48.00, 48.00, 47.75, 46.50, 45.50 45.00 44.25, 44.25

KIM 48.75, 48.00, 47.75, 47.00, 46.75, 46.00, 46.00 45.25, 42.00

Fix Sotnikova's order and for each of the 300,000 possible ordering of Kim's, record the number x of judges that favored Sotnikova (half a point for a tie). What is the frequency distribution of x?

At the moment I cannot think of any reason why anyone would want to know this. But I think it might point my mind in the right direction to worry about it some more. :yes: I have a feeling that questions of this sort have already been extensively studied, and the result would be the same if we used the numbers 1, 2, 3, …, 9 instead. But the interesting thing would be, how does the distribution change as a function of the difference in the means? (In this example the means for the two skaters are about the same.)

Click to expand...

The distribution of x:

x — Probability
2 — 0.5 %
2.5 — 1.3 %
3 — 1.0 %
3.5 — 9.7 %
4 — 32.4 %
4.5 — 12.9 %
5 — 24.4 %
5.5 — 3.7 %
6 — 4.4 %
6.5 — 0.2 %
7 — 0.1 %

Expectation: 4.28 (< 4.5)

Vanshilar said:

Eh okay but the numbers that I'll be using are:

SOT 48.25, 48.00, 47.75, 47.75, 46.50, 45.50 45.00 44.25, 44.25

KIM 48.75, 48.00, 47.75, 47.00, 46.75, 46.00, 45.75, 45.25, 42.00

Click to expand...

x — Probability
1.5 — 0.04 %
2 — 1.0 %
2.5 — 1.9 %
3 — 12.9 %
3.5 — 10.5 %
4 — 33.2 %
4.5 — 12.1 %
5 — 21.7 %
5.5 — 3.1 %
6 — 3.4 %
6.5 — 0.1 %
7 — 0.1 %

Expectation: 4.17.

Thank you!

So in 70% pf the trials there is no worse than a 5 to 4 split of the judges. That is no surprise, just from eyeballing the numbers -- pretty even, with a tiny edge to Kim in ordinals as in total. I think we are stymied now without further information from the ISU (which will not be forthcoming). If in fact it turned out that, say, there was a 6 to 3 split (probability only 2.8% in this calculation) then we could move forward.

cooper said:

of course qwertyskates is going to react from mathman's post..took it personally and now making an accusation w/ the poster.. how predictable.. and every poster who defended sot's ogm..

that 31 goes+ for sotnikova if it's indeed true is insane.. and it looks like someone who's in payroll..

ISU should reveal the scores!! there's nothing to hide.. besides russia and the US advocates to get rid of anonymous judging.. why not try to make an example of this?? after all it was a legit win.. right??

Click to expand...

Please take a look in the mirror. I didn't even bother to address any of your posts, because most of you are actually rather rabid and without content safe for a vindictive delight in attacking others. You, my dear, are as PERSONAL as you can be in your above post in your relentless attack on me.

Please you Yuna fans, have some class,

I point out that Mathman's scenario 2 of assuming cheating on the part of judges is based on a twisted pretzel logic that would be almost implausible. Pointing out the fallacy of this logic is NOT attacking him, ok?

Someone who ISN'T YUNA winning the OGM is not equal to cheating, ok?

As it turns out, statistically, based on PCS which are more evenly distributed, the probability of of 3 vs 6 judges acting to upset the entire contest is unsurprisingly low!

Another fact I want to remind is that the SP judging in fact favored Adelina MORE, as Yuna had a higher BV yet her margin of winning is very small. The panel of SP judges comprise mostly of non ex-Soviet Europeans. I have yet to see a blow by blow "proof" of cheating based on the SP judging and judges.