In Praise of the Judges

GGFan said:

Agreed. But then we also have to admit that even when individual judges distinguish averages tends to lessen the gap. Also, as experts like gkelly have pointed out the PCS has to cover all levels of skating. With the top skaters you're really just distinguishing between 7-9s. Once you throw out highs and lows, etc. things get close.

Click to expand...

This to me has always been one of the main facilities of the COP judging system. It makes no sense to me that you gain a PCS bump just for moving up a level of competition. Just because you are competing at the junior level doesn't mean that you have worse skating skills, presentation, etc. than senior level skaters. There are also plenty of Senior level skaters with weak skating skills and interpretation that aren't given the appropriate weight to balance out high tech scores, which places skaters who have spent the time to develop the skating side of the sport over the athletic side of the sport at an even bigger disadvantage because there is only a potential 3 point difference between PCS once you reach the senior level. It's not like jumps at the senior level are automatically worth more than the same jumps done at the junior level and the same should hold true for PCS.

Globetrotter said:

Personally, I don’t have any issues with the final placement. My bug here is that Tuks and Satoko’s extreme differences in strengths ought to have been much more obvious. Jumps = Tuks >> Satoko; spins and steps Satoko >> Tuks. With the new +/-5 GOE, this should be more obviously differentiated but alas, the jump GOEs were just not suitably different. PCS wise, Satoko >> Tuks for TR, CO; > for SS, IN; Tuks >> Satoko for PE. I just wish the differences were more suitably scored.

Click to expand...

Forget Liza and Rika for a moment. Let’s take a look at how Satoko’s TES compares to someone just off the podium.

PCS
Mariah 64.57
Satoko 71.50

TES
Mariah 71.42
Satoko 71.89

If we’re supposed to accept that the difference between these two skaters Components is being fairly represented here (not sure I agree with this gap) then are we also to believe the same can be said of jumps? Here is a more specific breakdown of TES points between the two skaters jumps.

Total GOE of all Jumps (factored)
Mariah 5.84
Satoko 4.45

Total Jump Elements (factored)
Mariah 49.72
Satoko 49.30

Keep in mind Mariah did not 3-3 in the FS but I think it’s still fair to ask if the scorecard reflects a fair comparison of skater’s efforts. Satoko is a masterful performer and deserves a PCS lead over the feild for her efforts. I think the question is if Satoko’s jump TES should also reflect the obvious deficiencies in her jumps in the same way it does in others PCS deficiencies? I’ll let you all decide for yourselves.

silver.blades said:

This to me has always been one of the main facilities of the COP judging system. It makes no sense to me that you gain a PCS bump just for moving up a level of competition. Just because you are competing at the junior level doesn't mean that you have worse skating skills, presentation, etc. than senior level skaters. There are also plenty of Senior level skaters with weak skating skills and interpretation that aren't given the appropriate weight to balance out high tech scores, which places skaters who have spent the time to develop the skating side of the sport over the athletic side of the sport at an even bigger disadvantage because there is only a potential 3 point difference between PCS once you reach the senior level. It's not like jumps at the senior level are automatically worth more than the same jumps done at the junior level and the same should hold true for PCS.

Click to expand...

If Alena Kostornaya was born two months earlier, she would beat many top skaters in PCS this season.
I think PCS are low in juniors to maintain the idea which is in all sports: Seniors are better than juniors. Which is not true.
If it was shown that many juniors are better than many seniors, why bither to watch seniors?
It is sometimes laughable to see a skater who competed both in seniors and juniors the same season, get good scores and results in seniors, but being nowhere near to that in juniors. I think about Konstantinova, Loena Hendrickx, or even Satoko.

silver.blades said:

This to me has always been one of the main facilities of the COP judging system. It makes no sense to me that you gain a PCS bump just for moving up a level of competition. Just because you are competing at the junior level doesn't mean that you have worse skating skills, presentation, etc. than senior level skaters. There are also plenty of Senior level skaters with weak skating skills and interpretation that aren't given the appropriate weight to balance out high tech scores, which places skaters who have spent the time to develop the skating side of the sport over the athletic side of the sport at an even bigger disadvantage because there is only a potential 3 point difference between PCS once you reach the senior level. It's not like jumps at the senior level are automatically worth more than the same jumps done at the junior level and the same should hold true for PCS.

Click to expand...

I used to have more of a problem with this until gkelly helped me to realize how focused my viewing is on elite skaters. Partly what is going in psychological--judges judging in juniors are going to think that they are working within a certain range because there's still another level above. But in general of course there will be some correlation between scores and levels. Seniors have been working on their skills for a longer period of time so it would make sense that the average junior would have lower PCS that they average senior. That's not to say that the scores of juniors should be capped.

Even if we acknowledge that the judges should distinguish more are there really any elite seniors that will get 3s or 4s in any categories, probably not. To get an idea of what 3s and 4s look like we would have to watch a lot of lower level skaters. We sometimes underestimate how good even the "bad" elite skaters are. Things that we deem horrible are really not that bad.

Once you consider that and all the averaging that is being done you end up with pretty compressed marks within a certain range.

Baron Vladimir said:

I think people should keep in mind how sports competitions work in general. By declaring the winner they are not saying litteraly who is better/or worse, just who scored more points in some exact competition. To make equation between more points and better player/team is not really what sports are trying to do in first place. But to declair the winner - the one who played the game better (according to the game rules) or who is better competitor (at that exact day) (not the one who is generally better).

Click to expand...

If that were true. But by not acknowledging an error in the judging, an UR or edge call that was called by mistake or not called, it sets an inaccurate presedence. I don't think an outcome to an event should be changed after the fact, but I do think that a review of elements should be acknowledged.

silver.blades said:

There are also plenty of Senior level skaters with weak skating skills and interpretation that aren't given the appropriate weight to balance out high tech scores,

Click to expand...

There are plenty of senior-level skaters who earn PCS in the 5s, and even 4s and sometimes 3s. You're just not usually seeing them in international competition, especially at the championship and Grand Prix level. You will see a few in the lower ranks of Four Continents, for example, and at Senior B events. Not to mention in national and lower domestic events.

SEA skaters (ladies) usually get 4s and 5s. A few season back Koreans like Na Hyun Kim and Dabin Choi were getting 5s. Sometimes Maé-Berénice Méité gets 5s and 4s (but she's mostly at 6s now).

gkelly said:

There are plenty of senior-level skaters who earn PCS in the 5s, and even 4s and sometimes 3s. You're just not usually seeing them in international competition, especially at the championship and Grand Prix level. You will see a few in the lower ranks of Four Continents, for example, and at Senior B events. Not to mention in national and lower domestic events.

Click to expand...

Also at Europeans there are skaters making the FS cut with some components below 6.
For example last year in the SP the following skaters qualified for FS with trimmed means below 6.0
Hayrapetian (Transitions: 5.89)
Reznichenko (5.68 for Transitions, 5.93 for Performance, 5.96 for Composition and Interpretation)
Boe (5.93 in Skating Skills, 5.68 in Transitions)
Ponsart (5.46 in Transitions, 5.82 in Performance, 5.93 in Composition) * Some individual judges' marks were in the 4s
Demirboga (5.64 in Skating Skills, 5.32 in Transitions, 5.68 in Performance, 5.61 in Composition, 5.64 in Interpretation) * Some marks were in the 4s
Virtanen (5.71 in Transitions, 5.93 in Performance, 5.89 in Composition)

In the Ladies SP, Ostlund finished 6th and she had 5.82 in Transitions and 5.93 in Interpretation. Paganini was 9th and she had all Components averaging in the 5s.
Brezinova, Khnychenkova, Dastisch, Gjersem, Cristini, Grm and Russo had some Components in the 5s. For some of them all of them were averaged below 6.0
Kropa, Hugec and Sorensen had some components in the 4s as trimmed mean and qualified for FS.
The bottom 3 girls went also below 4.0 for some judges.

Sam-Skwantch said:

Forget Liza and Rika for a moment. Let’s take a look at how Satoko’s TES compares to someone just off the podium.

PCS
Mariah 64.57
Satoko 71.50

TES
Mariah 71.42
Satoko 71.89

If we’re supposed to accept that the difference between these two skaters Components is being fairly represented here (not sure I agree with this gap) then are we also to believe the same can be said of jumps? Here is a more specific breakdown of TES points between the two skaters jumps.

Total GOE of all Jumps (factored)
Mariah 5.84
Satoko 4.45

Total Jump Elements (factored)
Mariah 49.72
Satoko 49.30

Keep in mind Mariah did not 3-3 in the FS but I think it’s still fair to ask if the scorecard reflects a fair comparison of skater’s efforts. Satoko is a masterful performer and deserves a PCS lead over the feild for her efforts. I think the question is if Satoko’s jump TES should also reflect the obvious deficiencies in her jumps in the same way it does in others PCS deficiencies? I’ll let you all decide for yourselves.

Click to expand...

Of course it should - ideally. If it’s up to me, most of Satoko’s GOEs at NHK will be 0 to 1 and negative for the ! And < calls. They were not quite as good as during Skate America, so let’s just call them out as they were done. The subtractions will be really to do with insufficient rotation and borderline techniques. But the same should apply to everyone else and also for PCS. If Satoko is a high 8s and low 9s for SS and IN, then I think no other lady skater really should score above that except for maybe CaroK or Marin Honda on a superb day.

the Korean judge at NHK was part of Elizaveta's fandom. She gave Liza 9's in PCS while the Russian judge went with low 8's to 7's.

yume said:

If Alena Kostornaya was born two months earlier, she would beat many top skaters in PCS this season.
I think PCS are low in juniors to maintain the idea which is in all sports: Seniors are better than juniors. Which is not true.
If it was shown that many juniors are better than many seniors, why bither to watch seniors?
It is sometimes laughable to see a skater who competed both in seniors and juniors the same season, get good scores and results in seniors, but being nowhere near to that in juniors. I think about Konstantinova, Loena Hendrickx, or even Satoko.

Click to expand...

That’s because Satoko and Loena GREATLY improved after being juniors.

As for Konstantinova...she got 4th at junior worlds and only reason she’s relevant is because RusFed overhype her and she went clean in a free skate for the first time in a long time , and Sakamoto had a horrendous short. Konstantinova’s always had talent but never delivered it ever but RusFed kept on supporting her even when they dropped other girls after one bad skate...that’s why she’s doing well now.

Is it not bizarre that the Australian judge gave the same overall PCS score to Mariah Bell and Satoko Miyahara at NHK? Whats even up with that? Did Satoko say she hates vegemite? Break it down for me scoring specialists. Break it down.

lanceupper1114 said:

Is it not bizarre that the Australian judge gave the same overall PCS score to Mariah Bell and Satoko Miyahara at NHK? Whats even up with that? Did Satoko say she hates vegemite? Break it down for me scoring specialists. Break it down.

Click to expand...

I think we all should agree that scoring is some dark magic, simply put.
That judges draw a pentagram on ice and a demon comes and delivers them the scores.
Imho that is the most logical explanation

GGFan said:

2. We assume that the judges have the amount of time to digest the performances that we have. The judges are making quick decisions. Yes, there is some prejudging but they also have to make decisions that I'm sure they would slightly revise with the benefit of time and hindsight.

3. We assume that the judges decide the placings following everyone's performance. They have to judge the event as it goes. That gives us a huge advantage over them.

Click to expand...

gkelly said:

If you want to get a sense of the experience of judging, try doing it yourself in a setup that closely replicates real judges' experience.

Click to expand...

I would agree that it would be very difficult for us to do judging to the standard the judges do, especially without the experience (and training) they have.

However one thing to remember is that many of them will have been judging for many a long year, and they literally put in thousands of GOEs and PCS scores in the course of a season.

For example, if a judge judges a singles discipline at a GP event they will put in 29 GOEs or PCS scores per skater over the 2 segments, that's 348 in just a single event with 12 skaters - at the World Champs if they happened to judge both segments that would be virtually 900 of them if there's 36 skaters in the SP and 24 in the LP.

Hence with that sort of volume I would expect them to be very accurate.

Indeed this may explain the uncanny pattern you get time and time again when it comes to over-marking own skaters and under-marking others. Indeed looking at the patterns within this you can also see further underlying patterns. For example I get the impression you get more over-marking than under-marking - it will be interesting to see what comes out of Shanshani's final analysis of this sort of thing, and my theory behind this is that you get a lot of judges just over-marking their skaters just a little bit more just to ensure a level playing field - if everybody's doing it then you're going to have to protect your own skater aren't you? (not excusing this sort of behaviour, just explaining it).

Also when it comes to inputting GOEs and PCSs I doubt very much whether judges can really keep track of what they've given other skaters, especially with the volume of data they input. Instead what I imagine what goes on a lot of time is that judges give their own skater the benefit of the doubt a lot more than they do opposition skaters, and by the time the scores come out it's all come out with the wash i.e. the judge has effectively boosted their own skater's GOEs and PCS's over a few elements, and that explains why time and time again, a judge's scores for their specific skater come out towards the top.

For example, I looked at 221 cases of own skater judging at the Olympics and following World Championships, but there were only about 10 to a dozen cases that really took me aback, including those of the 2 Chinese judges that got banned (though you do get a little bit numb to it after a while). However I also got 170 cases in total where a judge ended up giving their own skater a score in the top 3 scores for that specific skater/couple. Hence I’m sure this is where the above benefit of doubt/discreet boosting has come into play.

Finally, my final thought on it all is that some judges are absolute masters of knowing how far to push it without coming under suspicion, especially at a single event.

It's only when you look at their record over multiple competitions that you see their record is beyond anything that you could get by chance. However at a single event they've learnt how many GOEs/PCS's they can boost/mark down without coming under suspicion.

For example, at NHK the Russian judge scored Elizaveta 228.01 vs 219.02 actual, the Japanese judge scored Satoko 224.57 vs 219.47 actual, and when it came to marking down it was less, the Russian judge scored Satoko 215.59 vs 219.47 actual and the Japanese judge scored Elizaveta 214.80 vs 219.02, and this is exactly what you would expect from the above i.e. more marking up of own skaters and less marking down of others. The only good thing about it in this case was that both judges were on the panel for both segments and everything cancelled out.

moriel said:

I think we all should agree that scoring is some dark magic, simply put.
That judges draw a pentagram on ice and a demon comes and delivers them the scores.
Imho that is the most logical explanation

Click to expand...

I'm sure some kind of ritual sacrifice must be involved.