Irina Slutskaya and Olympics 2002

[Baron Vladimir]

How are you able to mind read Blades of Passion and know that he is debating for the sake of debate? Really, from my vantage point, it is you who is trying to prove your own point about him being biased - with absolutely no regard to the actual discussion of skating.

It is you who have a problem why I called BOP opinion biased. And because both of you were trying to incorporate this silly narrative how i'm the one who is biased (which is again, as always, based on my forum's name), i'm explaining (by using the actual facts) why i called his opinion as 'biased'. And i'm now finished with it Do I need to mention again that he said that at that Olympics 'Slutskaya looked like a bland slab of cow meat'?

 

[Skatesocs]

It is you who have a problem why I called BOP opinion biased. And because both of you were trying to incorporate this silly narrative how i'm the one who is biased (which is again, as always based on my forum's name), i'm explaining (by using the actual facts) why i called his opinion 'biased'. And i'm now finished with it Do I need to mention again that he said that at that Olympics 'Slutskaya looked like a bland slab of cow meat'?

 

[Blades of Passion]

Why are you talking about your forum name? That has nothing to do with anything and nobody brought it up. It's unacceptable how you make false accusations and take something out of context, and refuse to respond to the points being made, all while saying you're the one being misrepresented. Your opinion is detrimentally biased because:

1. You said American propaganda is the only reason someone could ever think Slutskaya should place lower at this competition
2. You keep making qualitative judgements about the skating that don't exist in these performances, even when it's been show your point was incorrect
3. You say we should judge the competition based on a non-6.0 system, and even when people do that and show how Slutskaya deserved to lose, you ignore it
4. You continually point at a single set of judges scores as if they are infallible, and you say all opinions that disagree with a majority are nothing but an error

These are some of the 'mistakes' you made in your judging:
1. Using certain criteria only (favorites one) and arriving at an average for the program
2. Forgetting to evaluate the direction of skating (and changes) as well as one foot skating
3. Forgetting variety of skating and variety in body movements
4. Forgetting that serious errors/falls impact the overall continuity and fluidity of the program and its delivery especially when there is a strong ending

This isn't true at all. You're just listing random things to make it seem like you have a point and ignoring all of the things that were written about these topics. And are you seriously trying to talk about people not looking at variety in body movements, as a defense of Slutskaya's performance? Really?

 

[Baron Vladimir]

1. You said American propaganda is the only reason someone could ever think Slutskaya should place lower at this competition
2. You keep making qualitative judgements about the skating that don't exist in these performances, even when it's been show your point was incorrect
3. You say we should judge the competition based on a non-6.0 system, and even when people do that and show how Slutskaya deserved to lose, you ignore it
4. You continually point at a single set of judges scores as if they are infallible, and you say all opinions that disagree with a majority are nothing but an error

1. I never said that is the only reason. I gave many possible reasons, and quoted that just as one of them.
2. It exists, both french and NBC commentators (in the short) noted Slutskaya was the fastest one.
3. I didn't say that, but your friend Skatesocs
4. Not all opinions, just one of them all (or two if you take both highest and lowest one!).

 

[Skatesocs]

3. I didn't say that, but your friend Skatesocs

Honey, YOU are the person who tried to say you don't know much about 6.0 but did know under CoP, Slutskaya would get more points for her one foot skating. Here:

It just looks like that. I don't have much knowledge how giving the marks worked in 6.0 era, but in many aspects of todays components scores, precisely 'more technical' ones Slutskaya was far ahead Sasha Kohen at that Olympics. Giving the fact Sasha fell in her program, her PErformance would be also higher. In some aspects of components, Slutskaya was ahead of Michele Kwan too, as speed, acceleration, one foot skating, usage of both feet etc. Her jumping elements were above Michelle and Sasha and she was on pair with Michelle in spins. So, i dont see how Michelle and Sasha with a fall could be ahead of her. Based on INterpretation of the music, which is like 1/10 (!?) of the total marks

Blades disproved that. I gave examples of who disagreed with you - and you never countered it and instead decided to switch to your rhetoric about bias.

I already did give my 6.0 analysis for the LP - and while not with the scores for the SP but definitely as I'd rank them. I was curious about how people would score it in CoP *for comparison* and because people were saying Hughes had under rotations without naming any.

People can read the thread. Please stop your revisionist propaganda and stop editing your posts after I quote them, you ain't fooling nobody.

 

[Mathman]

So, in that (maybe more statistical) sense, you can use word 'biased' to say why lowest and highest marks for a skater are not part of their final rankings anymore.

This is an interesting comment. Does the trimmed mean introduce bias or alleviate it? There is a strict statistical sence in which "bias" is introduced in a very technical fashion (in estimating parameters of skewed distributions, for instance).

On the other hand, here is what Wikipedia has to say about the truncated mean in the context of sports judging.

Discarding only the maximum and minimum is known as the modified mean, particularly in management statistics. This is also known as the Olympic average (for example in US agriculture, like the Average Crop Revenue Election), due to its use in Olympic events, such as the ISU Judging System in figure skating, to make the score robust to a single outlier judge.

In practice, in ISU judging, this procedure almost never makes any difference in the outcome. Its only practical effect is not to catch cheating judges, but to eliminate keypad errors, like when a judge intends to give a score of 9.5 but inadvertently touches 0.5 instead.

 

[Skatesocs]

This is an interesting comment. Does the trimmed mean introduce bias or alleviate it?

Yes. In the context of skating, I'd say it increases bias, in fact, since more judges might want to score a certain way so as not to have their scores thrown away (for instance) - tending them towards a popular opinion about someone's skating, say.

 

[Baron Vladimir]

This is an interesting comment. Does the trimmed mean introduce bias or alleviate it? There is a strict statistical sence in which "bias" is introduced in a very technical fashion (in estimating parameters of skewed distributions, for instance).


In practice, in ISU judging, this procedure almost never makes any difference in the outcome. Its only practical effect is not to catch cheating judges, but to eliminate keypad errors, like when a judge intends to give a score of 9.5 but inadvertently touches 0.5 instead.

One of the practical effect is to catch 'biased' judges. Thats how two Chinese judges were suspended from judging for two years. It is specifically mention in the last ISU seminar how one of the referee's job at the event is to catch those 'outliers' and to talk with a judges about it after the event, with mentioning it in their final report which should be sent to the ISU committee for further evaluation. Those 'informations' are certainly included in ISU process of chosing the judges for the biggest events. Some of those information could be positive, in terms of further developing of figure skating rules (because not every 'outliner' is a 'bad' one). The same procedure of scoring is used in many other sports even before it came in use in figure skating, as diving and ski jumping, in order to make scoring less subjective and more independible of individuals in the judging panel.

 

[Baron Vladimir]

Yes. In the context of skating, I'd say it increases bias, in fact, since more judges might want to score a certain way so as not to have their scores thrown away (for instance) - tending them towards a popular opinion about someone's skating, say.

The point of judging is not to judge how they want, but to try to judge most objectively they can (or according to some rules/recommendations). If that really towards them to popular opinion about someone's skating, than no lady but Yuna Kim wouldn't win Olympic Games (they won) from 1998 till today.

 

[Skatesocs]

The point of judging is not to judge how they want, but to try to judge most objectively they can (or according to some rules/recommendations). If that really towards them to popular opinion about someone's skating, than no lady but Yuna Kim wouldn't win Olympic Games from 1998 till today.

It is impossible to argue with that.

 

[Blades of Passion]

It exists, both french and NBC commentators (in the short) noted Slutskaya was the fastest one.

The SP has nothing to do with the LP's we were discussing. The commentators specifically said Slutskaya wasn't fast in the LP.

Assuming everyone thinks Cohen, Kwan, and Slutskaya deserved to be top 3 in the SP, then actually the SP with the 6.0 system is irrelevant to determining where they should place in relation to each other. Only their skating in the LP matters.

Not all opinions, just one of them all (or two if you take both highest and lowest one!)

There is far more than one opinion that has said Slutskaya deserved to place lower. Regardless, your mantra is problematic, because the validity of an opinion is not determined based upon it being a minority or not. 1 person can be correct about something and 8 other people wrong.

The "opinions" of sitting judges at competitions are difficult to take at face value no matter what, because there are constantly politics and other factors at play in how they score things. Who knows if they actually think something is better or worse, or if they are just marking it that way because it creates a competition result they have been told is "proper" or desirable, or if they weren't considering their own marks well enough and the numbers don't line up like they thought it should.

---

About the overall discussion of removing the highest and lowest scores, that's not something which needs to be in the CoP scoring system, and it doesn't do anything to "catch" biased judges. The scores can already be assessed just by looking at them; there is no extra information provided by tossing out highest/lowest scores in the competition score itself, as to whether a judge's opinion is negatively biased or not.

 

[flakyy]

Clearly this issue is still highly contentious after all those years. Probably because everyone on the podium had issues. Sarah: underwhelming SP, underrotations - Irina: tentatively skated LP, LP worked against her strenghts, the flip - Michelle: the fall on the flip, LP not skated with confidence. So I think it comes down to the SP.
Irina showed the most difficult content (combination lutz - loop, quite spectacularly done for that time) https://www.youtube.com/watch?v=ePYXpf4mKwE&t=1m23s
Michelle wasn't that stable on that flip https://www.youtube.com/watch?v=t-tfa1vO0jw&t=2m50s
So I would have changed: Irina 1st, Michelle 2nd.
I don't think the music choice for Sarah in the SP did her any favor, whereas the much more upbeat LP carried her through the 4 minutes and I would have placed her 1st in that portion. And at least she went for the 3-3s. But I'm convinced she should have placed lower in the SP, that would have closed the case under the 6.0 - system, no matter what happened between Irina and Michelle in the SP: Irina would have won the gold. The taste would have been somehow bitter because of the connection between LP and victory ceremony. That night it felt "right" because the best looking LP got decorated with the main price. Under the 6.0 - system Sarah was lukcy enough to end up in 4th after the SP - taking everything into account Irina should have won, the system was like that: one could bring down the house with the LP, but already be a loser overall.
Irina going into 2nd and Maria not into 4th in the SP perhaps had something to do with the competition held in the US :scratch3:

It would be really interesting to rescore this whole thing under a CoP - perspective, including both portions. Then it wouldn't matter where Sarah was after the SP, the question would be: could she have made up the missing points in the LP? Of course it's problematic to do that with programmes and elements not desingend for CoP

 

[Mathman]

One of the practical effect is to catch 'biased' judges. Thats how two Chinese judges were suspended from judging for two years. It is specifically mention in the last ISU seminar how one job of the referee of the event is to catch those 'outliners' and to talk with a judges about it after the event, with mentioning it in their final report which should be sent to the ISU committee for further evaluation. Those 'informations' are certainly included in ISU process of chosing the judges for the biggest events. Some of those information could be positive, in terms of further developing of figure skating rules (because not every 'outliner' is a 'bad' one).

All of that, however, is irrelevant to the question of using the trimmed mean or not. All the ISU machinery in place -- the referees' conferences, assigning "anomalies" to judges who give too many marks outside the "corridor, etc." -- would be just as effective whether the ISU used full sample means, trimmed means, medians, or any other measure of central tendency to compute the scores.

The same procedure of scoring is used in many other sports even before it came in use in figure skating, as diving and ski jumping, in order to make scoring less subjective and more independible of individuals in the judging panel.

I can't tell, from the words "the same procedure," if you are supporting the idea of trimming the mean or if you are opposed to it.

When you speak about "making scoring less subjective" do you mean that trimming the mean tends to accomplish this or do you mean that that not trimming the mean would be more likely to accomplish this goal?

 

[Mathman]

Yes. In the context of skating, I'd say it increases bias, in fact, since more judges might want to score a certain way so as not to have their scores thrown away (for instance) - tending them towards a popular opinion about someone's skating, say.

I do not think that this is much of a consideration in the minds of the judges. There is no shame in having your score disregarded in the final tally for being the highest or the lowest. (After all, somebody's score has to be highest and somebody's the lowest.)

Usually when this type of criticism is made of ISU judging, the focus is on "corridor judging" rather than on trimming the mean. The presence of the "corridor" might psychologically push a judge to score according to his guess about how the other judges will score the event. This bolsters "reputation judging," which is unfortunate but perhaps inevitable in the context of figure skating.

By the way, the theory of statistical inference does not give us very much help in discussing the trimmed mean. For instance, all of the formulas that have sigma/sqrt in them are inapplicable in the most general case. There is, in fact, no completely satisfactory technique for dealing with this problem, and in circumstances where it is really important to get it right, statisticians must rely on "boot-strap" procedures tailored to the particular data.

In the case of symmetrically distributed data it doesn't matter. Any method of trimming or not trimming gives the same result. (This is approximately the case with judges' scores in figure skating). For an unsymmetric distribution, one must know something about the amount of "skewness" in order to make suitable corrections (but if you know all anout that you basically already know the answer and you don't need statistics at all.)

In figure skating, all is well. The distributions of scores, even when judges cheat a little, is sufficiently "Gaussian" that is doesn't matter statistically. The only situation where it might make a difference is the one that I mentioned above, where one datum is a true humdinger of an outlier due to a typo, not just someone trying subtly to game the system.

 

[Skatesocs]

^I was wondering why I'd never dealt with an inference type analysis on trimmed mean. I had to look it up when Vladimir brought it up. His response didn't make sense, but it still made me wonder.

Yes, probably such an effect, if any, is negligible. When I saw your question, it did spring a scenario where some game, or consider it so trivial it doesn't matter (or human error; of course I went for the most grassy knollish one). Corridor judging is probably a lot more overwhelming psychologically.

I wouldn't say it alleviates anything either though. It just is there to remove those random errors (I'm happy with that).

 

[Baron Vladimir]

All of that, however, is irrelevant to the question of using the trimmed mean or not. All the ISU machinery in place -- the referees' conferences, assigning "anomalies" to judges who give too many marks outside the "corridor, etc." -- would be just as effective whether the ISU used full sample means, trimmed means, medians, or any other measure of central tendency to compute the scores.

The point of trimmed mean is to 'eliminate the influence of outliers that may unfairly affect the traditional mean'. So no, the other measures are not equally effective in the judging environment. Of course that trimming the mean tends to accomplish this goal by being made the way it is, in order to eliminate those influences (Edit: When you have an opinion which tends to unfairly affect the other opinions - One 5 can't do anything to three 4's, but one 6 will affect them. And it's not fair when one judges mark affects the marks of all of the other judges in that way. Because the point of the judging is not the opinion of one judge, but to equally involve the most possible number of judges opinions. And that's why that one judges opinion is being eliminated, to be able to include opinions of more of them). It is already being effective in other sports involving the larger number of judges decisions before figure skating started to use it. It's not something ISU invented for sure. Edit: In this situation, one judges score 'unfairly' affected the scores of the other judges. Thanks to the Chinese judge, difference between Sui/Han and Savchenko/Massot at the last Olympics was half a point and could affect the final rankings. Taking all her scores into the final scores Sui/Han would even be a winner of the event. Without her in the panel, Savchenko/Massot would be a clear winner. And that is the judge who is suspended by ISU after it.

 

[drivingmissdaisy]

I think any statistical analysis of 6.0 scores is meaningless, because the judge typically would decide the placement and manipulate the score to achieve the desired ranking. I don't think it was a situation where a technical score and artistic score were awarded in a vacuum and the judge might be mildly surprised at the resulting ranking they assigned the skater.

 

[Skatesocs]

The point of trimmed mean is to 'eliminate the influence of outliers that may unfairly affect the traditional mean'. So no, the other measures are not equally effective in the judging environment. Of course that trimming the mean tends to accomplish this goal by being made the way it is, in order to eliminate those influences. It is already being effective in other sports involving larger numbers of judges before figure skating started to use it. It's not something ISU invented for sure.

 

[Blades of Passion]

If there is an error of a judge keying in a 0.5 instead of the 9.5 they meant, then that should be corrected. Why just sit around and let these errors persist? Competitors and coaches should be able to immediately request a review of such things. Don't hand out the medals until everything has been decided fairly. We can afford to wait 5 minutes at the end of the competition to make sure everything has been done properly, and for the skaters who aren't in the final flight, the judges are just sitting around during the warmups and can deal with those possible errors during that free time.

 

[Skatesocs]

If there is an error of a judge keying in a 0.5 instead of the 9.5 they meant, then that should be corrected. Why just sit around and let these errors persist? Competitors and coaches should be able to immediately request a review of such things. Don't hand out the medals until everything has been decided fairly. We can afford to wait 5 minutes at the end of the competition to make sure everything has been done properly, and for the skaters who aren't in the final flight, the judges are just sitting around during the warmups and can deal with those possible errors during that free time.

To be fair, one COULD make the case that someone inputting 6.5 for transitions as a 100% non-erroneous opinion, if it strays from the rest of the readings, it'll still get treated as an error and be tossed out. So this is true: the best solution would be to just give some time to correct human errors, but keep all opinions. I wonder if
Mathman would agree this is an introduction of bias? If there's one brave judge who decides to not judge by reputation and scores significantly lower, we do want that opinion, but we won't have it.