Judge the Judges and COP: What went wrong?

[gkelly]

Everyone starts off with a possible 10 in execution, and gets deducted from that. So, if one gymnast has a difficulty score of 5.9 and another has a difficulty score of 6.2. If they both, say have a 9 in the execution score, one gymnast is going to have a 14.9 as a final score and the other will receive a 15.2. The difficulty score has to do with elements done within a routine and how they're connected to one another which I need my sister around to explain why one thing is more difficult than another.

So if they both start with the same difficulty score and neither of them makes a mistake, do they get exactly the same score?

Or is there room for the gymnast who executes the same moves exceptionally well to score higher than the one who just executes them well?

Just as a basic comparison, you'd expect someone who does 3Z-3T, 3A, 3F to lose to someone who does 4T-3T, 3A, 3F, if they're both perfect even without factoring in any GOEs.

Yes. But if we're talking about a short program where two skaters both do the same jumps and the same levels of spins, with no mistakes, we'd expect the positive GOE to make the difference in TES.

That's not even taking into account program component scores. Especially Skating Skills, which is more fundamental to the sport of figure skating than the jumps are, and which doesn't have a direct correlative in gymnastics.

I think figure skating is more like golf sort of. I'm convinced the judges know what a skater's best scores are and then either adds or subtracts from it.

I'm not.

For one thing, an active judge is going to judge hundreds of different skaters each year. How do you expect them to keep track of everyone's best scores?

For another thing, skaters can gain points in one area and lose them in another area from one competition to the next. So just knowing what a skater's total was last month (assuming this judge spends more time memorizing scores from events he wasn't present for than paying attention to the rules and today's skating) doesn't tell the judge how many of those points came from base value, how many from GOE, and how many from PCS. Memorizing all that for hundreds of skaters per year, and multiple competitions per skater, would be next to impossible.

Third, even if judges estimate a skater's potential total score and aim to raise or lower GOEs and components to match or exceed or fall short, they could easily find themselves off by a point or more (and many competitions are won by smaller margins than that) just because the tech panel called a level 3 rather than level 4 step sequence -- which the judges have no way of knowing.

 

[OS]

I have been rewatching Men's FS event and rechecking the protocals for GOEs, and PCS.

The scoring for the non-Europeans: Hanyu, BoYang and Patrick vs Javier are surely interesting. If people are still in denial over how a European slant panel can affect scoring, they just have to see the details. Thank goodness Hanyu did not give anyone any excuse but to give it to him. How did BY's 4lz not get all +3GOEs, and then conservative GOEs on everything else. I still think Patrick Chan is undermarked (lol this sentence is unnatural i know), but how is his TES less than Javier's disaster? How did Javier (and Shoma) practically equalised Patrick's PCS at 94.52? It was one of Patrick's better FS, and a sublime original program. I would argue he should be in front of Nathen for the FS at least. His SS is really undervalued to the point of being meaningless.

If Shoma and Javier are at 94, I'd put Jason Brown up to 91 at least. The latter 2/3rd of his program was really spine-tingly good as he settles into the mood and delivered his best program ever. It is the reason why PCS should be half of the score. (Or supposed to be but currently has already reached its ceiling while TES have gone and broke the norm)

 

[GGFan]

I have been rewatching Men's FS event and rechecking the protocals for GOEs, and PCS.

The scoring for the non-Europeans: Hanyu, BoYang and Patrick vs Javier are surely interesting. If people are still in denial over how a European slant panel can affect scoring, they just have to see the details. Thank goodness Hanyu did not give anyone any excuse but to give it to him. How did BY's 4lz not get all +3GOEs, and then conservative GOEs on everything else. I still think Patrick Chan is undermarked (lol this sentence is unnatural i know), but how is his TES less than Javier's disaster? How did Javier (and Shoma) practically equalised Patrick's PCS at 94.52? It was one of Patrick's better FS. I would argue he should be in front of Nathen for the FS at least. His SS is really undervalued to the point of being meaningless.

If Shoma and Javier are at 94, I'd put Jason Brown up to 91 at least. The latter 2/3rd of his program was really spine-tingly good as he settles into the mood and delivered his best program yet. It was the reason why PCS should be behalf of the score. (Or suppose to be but currently has already reach its ceiling while TES have gone and broken the norm)

I think as fans we do a lot of reverse engineering of the scores: we determine where we think the scores should be and then determine how good the judging was depending on how close the judges' marks were to our standards.

I think gkelly does a great job of showing that's not how judging works. And our assumptions of what the judges know and have available to them are often incorrect.

My problems with the judging system are more fundamental: (1) lots of studies have shown that humans have limited multi-tasking abilities: this system requires too much of each judge. That wouldn't bother me as much if it weren't for, (2) there should be a built in margin of error. I'm not a math person but I'm pretty sure that those 1 or 2 point wins should be ties. But as humans we don't care for ties so we will continue to act as if there was a huge gap between placements and award medals based on distinctions that are hard to defend.

Generally I think the placements are defensible so I don't get up in arms too much.

 

[Ice Dance]

So if they both start with the same difficulty score and neither of them makes a mistake, do they get exactly the same score?

Or is there room for the gymnast who executes the same moves exceptionally well to score higher than the one who just executes them well?

In gymnastics, they only deduct. There are no pluses. A gymnast that executes less satisfactorily than a gymnast that executes exceptionally can be deducted more. (In other words, an athlete with exceptional toe point & height on his or her elements probably will probably receive fewer execution deductions than one who does not. As in figure skating, judges watch the practices & know what to expect heading into the performance).

Yes, gymnasts can get the same score. It used to happen all the time. Hence, the multitude of tie-breaking scenarios in gymnastics. Ties are rare now because there are so many more points than there used to be on the table. And because execution scores are carried out to a thousandth of a place, rather than just a hundredth of a place.

Nonetheless if you look at the execution score for the athletes on this link, you will see several ties.
https://www.usagym.org/PDFs/Results/w_16olympics_wqual.pdf