Are judges able to "rank" skaters while they are judging?

[Poodlepal]

This is an offshoot to the conversation about Evan vs. Evgeni. I thought I'd start a new thread to ask my question, to separate it from the quad question.

Under 6.0, they did, correct? They would rank the skaters, saying "OK, I thought Tara was better than Michelle, so I'll give her a 5.9 because I gave Michelle a 5.8" for example. A judge would put the skaters in order as they liked them, and panel of crooked judges could manipulate the whole competition if they wanted to, by placing certain skaters above others en masse.

Under the COP, is this type of manipulation possible? Do they know that in order to make sure Skater A wins, they must not give Skater B more than 7.2 for transitions, for example? Or is it so confusing that they just add or take away decimal points as they see fit, with really no idea what it will all add up to in the end?

People complain about the judging all the time, but maybe there are so many numbers and so many categories, the judges themselves may have no idea what the final score will be for a skater (unless they can add in their heads), and may not know exactly who will be in first or second place. That doesn't let the COP off the hook, if the base values are off, however.

[Spun Silver]

I've been wondering the same thing for ages! Thanks for posting it.

[Mathman]

Great thread question!

My impression is that, no, the judges cannot compute fast enough in their heads to know that they need to give their favorite a 7.2 in transitions to make that skater come out ahead.

But a judge could certainly score the favored skater generously across the board, while low balling that skater's rival. Since all the points of all the judges are added up, this might be even more effective than just to give a high ordinal on the judge's own card. The rule about dropping the highest and lowest mark reduces the impact of this strategy, however.

There is also the potential for the technical panel to call errors more stringently for one skater than for another.

[janetfan]

Let's not forget that Cinquanta said anonymous judging is necessary to reduce cheating.

This means he believes federations and their judges have the same attitude about scoring under CoP as they had in the 6.0 era.

Personally I think the truest purpose of the CoP anonymous judging is to make it harder to get caught again, particularly at the Olympics which was such a major embarrassment for IOC in 2002.

As to competitions other than the Olympics ISU seemed comfortable enough with 6.0 or CoP judging.

"Judges are human" and changing a scoring system does not change human nature.

[Mathman]

On second reading of Poodlepal's post, I think I did not speak to the right question. It's not about cheating, but about whether judges are still using the scoring system to rank skaters as to who they thought was best, second best, etc. Right?

OK, in that case I think that at first the judges used the PCSs exactly like that. The skater that they thought was best, they gave that skater straight 8.5's. The skater they thought was second best got 8.25's, third best was 8.0's, and so on.

I think that over the years, as the judges got used to the new system, they began to make a more serious effort to score according to the guidelines. So that it would be possible (but still not common) for a skater to get high marks in interpretation but low marks in skating skills if the judge thought such marks were justified.

Still, I do not think that the ordinal approach can be completely eclipsed. At the end of the day, somebody wins, somebody is second and somebody third. The judges will give the higher marks to the skater that the judge thinks skated the best, as always.

[janetfan]

I think that over the years, as the judges got used to the new system, they began to make a more serious effort to score according to the guidelines. So that it would be possible (but still not common) for a skater to get high marks in interpretation but low marks in skating skills if the judge thought such marks were justified.

.

Do you think Joe Inman might have felt differently about this last season?

We know how Plushy felt about it when he remarked about TR and then again when said Evan's OGM was not about who skated best but which country/federation "needed it more."

When a well respected judge and skater of Plushy's stature make such remarks do you think they are satisfied with the scoring system?

Were they questioning the integrity or the basic nuts and bolts of the system

[Tonichelle]

When a well respected judge and skater of Plushy's stature make such remarks do you think they are satisfied with the scoring system?

chances are in the 6.0 system or any system had Plushenko gotten anything but first place his nose would have been severely bent out of joint and he would have demanded his platinum medal just the same. :sheesh:

[janetfan]

chances are in the 6.0 system or any system had Plushenko gotten anything but first place his nose would have been severely bent out of joint and he would have demanded his platinum medal just the same. :sheesh:

and what about Judge Joe?

Was he implying that too many judges were incapable of scoring the PCS properly?

If that is true then how are the same judges going to arrive at a good and fair point total?

[Mathman]

Do you think Joe Inman might have felt differently about this last season?

I think that Joe Inman (and almost everybody else who thinks about the CoP) believes that there is a gap between what the CoP says on paper and what the judges actually do.

Were they questioning the integrity or the basic nuts and bolts of the system

I do not think they were questioning the integrity of the individual judges. In Inman's case, I think it was more an accusation of laziness on the part of the judges, that they were not working hard enough to make distinctions between all the compartments and bullets that the program components are divided up into.

As for the basic nuts and bolts, I do think there is a big problem with the CoP. Underlying the CoP is the tacit assumption that it is possible to list enough criteria that a judge can discriminate, in a consistent and objective way, between a program that deserves 7.5 in Performance/Execution and one that deserves (objectively and consistently) 7.75.

Personally, I think this claim is absurd. But it is possible to say, of these two performances, Skater A's was superior to Skater B's in terms of performance values.

[janetfan]

I think that Joe Inman (and almost everybody else who thinks about the CoP) believes that there is a gap between what the CoP says on paper and what the judges actually do.



I do not think they were questioning the integrity of the individual judges. In Inman's case, I think it was more an accusation of laziness on the part of the judges, that they were not working hard enough to make distinctions between all the compartments and bullets that the program components are divided up into.

As for the basic nuts and bolts, I do think there is a big problem with the CoP. Underlying the CoP is the tacit assumption that it is possible to list enough criteria that a judge can discriminate, in a consistent and objective way, between a program that deserves 7.5 in Performance/Execution and one that deserves (objectively and consistently) 7.75.

Personally, I think this claim is absurd. But it is possible to say, of these two performances, Skater A's was superior to Skater B's in terms of performance values.

Let's say we are both judging and you think Michelle deserved a 5.9 for presentation.
I only gave her a 5.8 because I liked Sasha better and gave her a 5.9.

It's still subjective, just as GOE is subjective along with the PC points.
We even read complaints that the levels are called wrong on steps at times so I will also call that subjective.

Is 7.75 anymore subjective than 5.9?

I think it is because the human mind can only accurately process so much information.

I am sure I know who I liked better but to get it right (to the best of my ability) I would need to watch it 3-4 times and have a good 20-30 minutes to break it down the way the CoP does.

I see CoP as little more than a smokescreen at times and what certain fans like most about it is what I find most troubling.

Joe Inman was asking for what is extremely difficult to do. Watch a program once and then decide in a couple of minutes how to score it to the hundredth of a point in so many different categories.

Of course they get it wrong many times as they are human.

[Mathman]

Let's say we are both judging and you think Michelle deserved a 5.9 for presentation.

I only gave her a 5.8 because I liked Sasha better and gave her a 5.9.

Yes. That is the nature of judging. That is why judging is different from measuring. That is why quality is different from quantity. That is why figure skating is not downhill skiing.

[janetfan]

Yes. That is the nature of judging. That is why judging is different from measuring. That is why quality is different from quantity. That is why figure skating is not downhill skiing.

Not sure if I get your point here. Are you suggesting that

6.0 = judging

CoP = measuring

[Mathman]

Not sure if I get your point here. Are you suggesting that

6.0 = judging

CoP = measuring

Yes. In basic intent. (I do not think that the CoP succeeds in its intent, however.)

"Ordinals" means putting things in order: first, second, third. That's what 6.0 judging did, and was not ashamed to admit it.

The CoP says, you get so many points for this and so many points for that. The winner is determined by arithmetic. I think there is a fundamental mismatch between this scoring philosophy and nature of figure skating.

I am not really against the CoP, however. I just think it is a little bit naive to believe that it can do what it claims.

[prettykeys]

Yes. That is the nature of judging. That is why judging is different from measuring. That is why quality is different from quantity. That is why figure skating is not downhill skiing.

Not sure if I get your point here. Are you suggesting that

6.0 = judging

CoP = measuring

No, Mathman is stating that the nature of a judged sport such as figure skating can include some degree of differences in judgment, unlike measurement where people can agree on e.g. who was the fastest.

Edit: Oh, I see that that is indeed what Mathman meant.

As for [b]Poodlepal[b]'s question, I am sure that judges are able to rank skaters. The question is, should they? Some fans seem to say yes, because only by getting a general idea of rank could they make the outcome of the scoring/judging more or less "correct". Some fans say no, possibly because it brings too much bias and not enough analysis into the scoring/judging.

I really think the most damaging aspect of the IJS right now is the anonymity of the judges. I would like to see if I can reasonably see consistency and fairness in a judge's overall method of scoring. If the CoP's strengths include objective criteria to score with, then a judges' scoresheet should be justifiable.

Anonymous judging in either 6.0 or CoP, I would imagine, would make it easier to manipulate scores and get away with it.

[prettykeys]

Yes. In basic intent. (I do not think that the CoP succeeds in its intent, however.)

"Ordinals" means putting things in order: first, second, third. That's what 6.0 judging did, and was not ashamed to admit it.

The CoP says, you get so many points for this and so many points for that. The winner is determined by arithmetic. I think there is a fundamental mismatch between this scoring philosophy and nature of figure skating.

I am not really against the CoP, however. I just think it is a little bit naive to believe that it can do what it claims.

I don't know if I agree with that. It is possible to rank skaters according to e.g. performance or e.g. TES for a double axel or e.g. transitions. Although the differences in a PC score for one aspect may be just 0.25 of a point, making the overall impact of the difference on the scoresheet smaller than the impact on the overall performance (or vice versa.)