Evaluating judging trends: A study on GPF

[bara1968]

To CanadianSkaterGuy^^ Yeah, I used Li's example to show how judges were used to be super conservative to the skater who was very new to the field and also to show (at least partially), that how inflated the PCS are, nowadays. (it's been like that since the beginning of the time, even for the relatively well known skaters i.e. Flatt, Nagasu etc, who were very good during their Junior years - but they barely scored 55-60 PCS even if they were relatively clean during their first year as seniors...) I would not score their PCS there anywhere near 70 anyways. But with today's standard, I wouldn't give them less than 60 either. (--> which reflects these days' PCS inflation.)

 

[mrrice]

gkelly, this comment is correct

"International judges are nominated by their national federations but must receive training by the ISU to get international appointments."

I'm not a skating judge but, I have been in performing arts my entire life and have coached and judged dance for a VERY long time. Every judge on any panel knows who Frank and Mishin are. They also know that both coaches know more than they do about skating and its rules. By the time I stopped teaching and became a full time judge, I had as much, if not more experience than 90% percent of the coaches who would come to me to complain about a number I had given. Judges are trained to Justify the numbers they give by quoting the "Manuel" that is given to every coach and choreographer before the season begins. I hope that made sense.

 

[gkelly]

gkelly, this comment is correct

"International judges are nominated by their national federations but must receive training by the ISU to get international appointments."

I'm not a skating judge but, I have been in performing arts my entire life and have coached and judged dance for a VERY long time. Every judge on any panel knows who Frank and Mishin are. They also know that both coaches know more than they do about skating and its rules. By the time I stopped teaching and became a full time judge, I had as much, if not more experience than 90% percent of the coaches who would come to me to complain about a number I had given. Judges are trained to Justify the numbers they give by quoting the "Manuel" that is given to every coach and choreographer before the season begins. I hope that made sense.

Your experience in a different kind of competition may not always be directly relevant to the details of what happens in figure skating competition. It's interesting when you tell us what your experience is, but I don't think you can state as a fact that the way things worked in your dance competitions is the same way things work in figure skating.

Coaches don't always know more than judges about the rules. The good ones know just as much.

Coaches do know more about the nitty-gritty technique.

The PCS guidelines/rules don't change from year to year, although there may be further judge training, different reminders about what to emphasize.

There are usually some changes to the GOE guidelines, especially the reductions, every year or two. Both judges and coaches have access to the ISU communications those are published in.

Similarly, there are usually some rule changes each year that affect the way that the technical panel calls certain elements. These are also published in the ISU communications and tech panel handbooks. Coaches and technical specialists/controllers can have discussions about why a particular element was called according to a particular rule -- judges are not involved in the calling at all.

Coaches often ask technical specialists and controllers about why an element was called a certain way, or during preseason monitoring may ask how it is likely to be called. They will sometimes ask judges about GOEs on elements.

As far as I know, most communication between judges and coaches/skaters about program component scores comes during the monitoring process and focuses on areas where the skater is already doing a good job and areas where there is room for improvement. It doesn't focus on specific numerical scores because there are no written rules that if you do X you will earn exactly Y score in this program component. Nor do I know how there could be -- If you can give examples of how that was done in your dance competitions, I'd be interested to know.

This thread has been mostly about PCS. And it's just not common, from anything I've ever heard in a skating context, for coaches to challenge judges about specific program component scores.

And I don't know what the connection is between coaches and the people who assign (not hire/fire) judges to officiate at competitions.

Maybe your dance association was structured very differently from the ISU and from national federations, especially USFS which is the one I'm familiar with.

So I'm not sure what point you're trying to make.

 

[mrrice]

"G.Kelly
Coaches do know more about the nitty-gritty technique."

This statement is not accurate. Especially when it comes to the tech caller. Now, by the time I was judging, I was well my past my performing days. However, I still know how a body works and I can appreciate the fact that some skaters can make difficult moves look easy and IMO, they should be rewarded for doing so.

Also, it's been my experience that most judges are recruited from Coaches who have retired and are eager to stay involved in the activity. That's exactly how I started judging. I was called by the association and asked if I'd like to attend a training session. Following our written training we were asked to do what's called "Trialing" which basically consists of sitting next to real judges during an event and giving a recorded analysis of what you see.

 

[Mathman]

I think there is an ISU rule that tech specialists must be former skaters with a certain level of accomplishment. (?)

 

[gkelly]

Also, it's been my experience that most judges are recruited from Coaches who have retired and are eager to stay involved in the activity. That's exactly how I started judging.

That is true of some judges in figure skating but far from "most." Another way that skating is different from your dance experience.

In the US, which I'm most familiar with, the main difference is that coaching is generally a full-time job by which former skaters can earn a living, although some do it on the side in addition to a full-time job (or spending most of their time raising children with a few hours of skating coaching).

Judging is a volunteer activity that former skaters of all levels -- or parents or adult skaters -- can undertake as a way to stay connected to the sport they love, with as much or as little commitment as they are able to give. But in general it costs people money to become judges and the reimbursements for expenses/per diem stipends might yield a tiny profit to active judges but is nowhere near what anyone can live on. Therefore judges must have other sources of income -- if those sources are jobs, they probably spend significantly more time working at their jobs than volunteering as judges.

And they are not allowed to coach, with a few minor exceptions.

Elite skaters can fast track into judging while they are still competing, or as soon as they retire, or several years later after coaching or going to college/starting a career or raising children.

Same for mid-level skaters.

Lower-level skaters and parents don't have the skating skills themselves to become coaches to begin with but they can learn to be judges, with more years of training than it takes for experienced skaters.

In the US it is now possible to become a test judge by age 17 and a competition judge just a couple years later, especially for those on the elite track.

Those who used to be elite skaters are most likely to become national and international judges and have a quicker path to reach those levels. But it's not required.

And it is absolutely not required or especially common to have professional coaching experience.

Now, technical specialists are required to have been high-level skaters, and they are allowed to be professional coaches at the same time that they serve as tech specialists -- although there are rules regarding who they can or cannot serve on panels for in terms of conflicts of interest. It's not required to have any coaching experience, though. Some former skaters become tech specialists for the same reasons people become judges, especially if they have other significant time commitments like jobs and families.

But this thread is about judging, not technical calling.

I was called by the association and asked if I'd like to attend a training session. Following our written training we were asked to do what's called "Trialing" which basically consists of sitting next to real judges during an event and giving a recorded analysis of what you see.

Yes, skating judges also need to go through a trial judging process.

 

[mrrice]

^ Typo. You mean "Judging" in the first word of the third paragraph, right?

I'm sure Gkelly meant Judges. This is another huge difference from mine. I love dancing and judging but, there is no way I would have done either one for free. I was never into being a starving artist and if I hadn't been able to support myself, I would have taken a real job and finished college.

 

[AprilS]

It seems if the skater is popular anything goes. High TES wasn't ok if Jin scored them honestly, without e or ur, with gorgeous difficult jumps and inching close to 100, but not a beep when Hanyu broke all TES records doing the same. It boggles the mind...

This contradicts the spirit of sports, where fairness and a level playing field is essential.

I honestly can't remember anyone making a big deal about Jin's TES?? Other than to say he's amazing at it but that some people don't want him winning on TES alone. While I disagree, that's really not a TES complaint. I mean, where are you even getting this from?

 

[chuckm]

For those who say Medvedeva is overscored (and yes, she slightly was) on PCS... I think people are holding a skater like her to a higher standard simply because she's just out of the junior ranks. Yes, she is out of the junior ranks, but she has good speed, quality elements, huge difficulty (two triple-triples), many transitions in her program, and she performed it well.

Compared to Uno, who got 89.58% PCS for his clean skate, Medvedeva scored 90.46% PCS for her clean FS -- and nobody is really complaining that Uno's PCS score was so high after going clean.

It's honestly a lame argument at this point to say "They're just out of juniors... how can they be scoring higher than [insert senior skater]?" To me, if they have all the requirements and are executing things well, it's unfair to slap them with a PCS threshold, just because they're not established yet. That being said, the judging still needs to be fair and reflect the level and quality of performance.

BTW, I would say Medvedeva's performance was definitely on the calibre of Mao's Sochi FS (which was quite underscored on PCS), and Kim's OWG2010 FS from a technical standpoint -- 7 triples, two 3-3 combos (Kim did one triple-triple, a 2A+3T and only 6 triples; Mao's 3F+3L and 2A+3T were UR but she did have a 3A)... and 5 jumping passes in the 2nd half, compared to Kim (3) and Asada (4).

No matter how you cut it, Medvedeva delivered one of the best senior SP+FS of all time -- oh but, she just came out of juniors so let's trivialize her performances for it. If you had swapped her face with Lipnitskaia's (who's established), people would have been saying her PCS was underscored.

Technically, Medvedeva delivered an amazing performance---but not a particularly exciting one.

OTOH, Shoma Uno gave a performance that was not only technically excellent, but electrifying---he had the crowd on its feet long before the performance was over. IMO, his PCS scores should have been a lot higher.

 

[moriel]

Technically, Medvedeva delivered an amazing performance---but not a particularly exciting one.

OTOH, Shoma Uno gave a performance that was not only technically excellent, but electrifying---he had the crowd on its feet long before the performance was over. IMO, his PCS scores should have been a lot higher.

I disagree, Medvedeva's performance was pretty exciting. For a measure, it was superior enough to Gracie's performance to get 5 points extra PCs.

 

[qwertyskates]

I honestly can't remember anyone making a big deal about Jin's TES?? Other than to say he's amazing at it but that some people don't want him winning on TES alone. While I disagree, that's really not a TES complaint. I mean, where are you even getting this from?

Can you see the problem with such a statement? If he cheated, under/pre-rotated his jumps, etc., I can understand their sentiments. If he won fairly, if he is honestly scored, based on strong TES, so what? He won fair and square. Either the rules of the sport apply equally to everyone, or it's dishonest and biased. This happened when Jin first appeared on the scene, before Hanyu upped his jump layout to include more quads.

If Patrick's PCS is = or more than Hanyu, and Hanyu beat Patrick through TES, is that not the *same* thing? Imagine the uproar if people start saying "Hanyu won by TES alone...! PCS is maxed out at 100 while TES went over 120+! Hanyu won by TES, that's unfair to Patrick and the rest of the field. Let's change the system to cap TES so that the competition is more "balanced"."

After Hanyu pushed TES to over 120+, there has been complete silence about "balancing" TESCS to keep it 100:100. :sarcasm:

 

[andromache]

Can you see the problem with such a statement? If he cheated, under/pre-rotated his jumps, etc., I can understand their sentiments. If he won fairly, if he is honestly scored, based on strong TES, so what? He won fair and square. Either the rules of the sport apply equally to everyone, or it's dishonest and biased. This happened when Jin first appeared on the scene, before Hanyu upped his jump layout to include more quads.

If Patrick's PCS is = or more than Hanyu, and Hanyu beat Patrick through TES, is that not the *same* thing? Imagine the uproar if people start saying "Hanyu won by TES alone...! PCS is maxed out at 100 while TES went over 120+! Hanyu won by TES, that's unfair to Patrick and the rest of the field. Let's change the system to cap TES so that the competition is more "balanced"."

After Hanyu pushed TES to over 120+, there has been complete silence about "balancing" TESCS to keep it 100:100. :sarcasm:

Regardless of who is pushing the TES limit, I think once it happens more frequently--say, when 5-6 men are doing it fairly regularly--TES and PCS will need to be balanced. Figure skating has always been 50% technical and 50% artistic and I would prefer it to stay that way. Give skaters with excellent presentation and weaker jumping ability--like Jason Brown--a shot against skaters with great jumps but weak presentation. Though of course skaters with great EVERYTHING will still have the advantage, as they ought to.

 

[avantgardener]

It's definitely an upward trend since IJS was first rolled out... And it made sense at first, since—at least in theory—the skaters who were skating mid-2000s would, a few years later, have improved skating skills, more transitions, and perhaps better choreography and interpretation ability. I suspect, though, that these higher PCS points bleeds over into a higher PCS baseline at which younger/new-to-the-scene skaters, by default, will start at—even if their 'components' qualities are mostly on par with first-year senior skaters from a decade ago.

Ha, it's kind of like economic inflation, no? Perhaps we'll one day say things like: Ten years ago, Yuna Kim won the Olympics with a record-smashing 228 points—which, adjusted for inflation, is a whopping 240 in today's points.

 

[Ice Dance]

It's a serious problem because PCS has a ceiling and TES does not. If this is not changed, the only gains to be had will be technical. Period. This is a major issue in gymnastics right now, and the sport has lost many fans. I would not be surprised if it has also lost many coaches, choreographers, and athletes who valued the more artistic side of the sport. I hope figure skating will take the need for balance very seriously. To me, it is already bizarre to see an athlete like Denis put out a program like his last season's SP and score ten points higher on the technical side of the sport. This would NOT have been the case under 6.0. Denis, Hanyu, Patrick--they would all have been scoring higher on the second mark. The reverse was actually quite rare, and clearly sent a message to athletes like Timothy that this was the area they needed to work on in order to improve. Not today. Boyang can certainly afford to work on PCS. It is his weakness. But he could add another quad instead and likely earn more points for it. That was never an option for Timothy. He had to raise that second mark or his score was not going to go up. I believe the best athletes should be balanced, and I think particularly within the men's discipline, we have really seen these athletes rise to the top under IJS. We all have our own taste, but Ten, Hanyu, Chan, and Fernandez are all performers in their own right. I do not think we should take this for granted with PCS reaching it's limit and the baseline appearing to climb.

 

[Mathman]

After Hanyu pushed TES to over 120+, there has been complete silence about "balancing" TESCS to keep it 100:100. :sarcasm:

Actually, there was quite a bit of discussion on this topic when Hanyu broke 110. Now that he has broken 120, it won't be long before that thread is resurrected.

 

[Interspectator]

Actually, there was quite a bit of discussion on this topic when Hanyu broke 110. Now that he has broken 120, it won't be long before that thread is resurrected.

I think it certainly will. But you know, the difference between Yuzu and Boyang is that if the cap is lifted for PCS it will only raise Yuzu's scoring potential through the roof or into the stratosphere...more than any other skater. Whereas for Boyang, it will put him further behind the current leaders.

 

[merleice]

Can you see the problem with such a statement? If he cheated, under/pre-rotated his jumps, etc., I can understand their sentiments. If he won fairly, if he is honestly scored, based on strong TES, so what? He won fair and square. Either the rules of the sport apply equally to everyone, or it's dishonest and biased. This happened when Jin first appeared on the scene, before Hanyu upped his jump layout to include more quads.

If Patrick's PCS is = or more than Hanyu, and Hanyu beat Patrick through TES, is that not the *same* thing? Imagine the uproar if people start saying "Hanyu won by TES alone...! PCS is maxed out at 100 while TES went over 120+! Hanyu won by TES, that's unfair to Patrick and the rest of the field. Let's change the system to cap TES so that the competition is more "balanced"."

After Hanyu pushed TES to over 120+, there has been complete silence about "balancing" TESCS to keep it 100:100. :sarcasm:

I think the concern about Jin's TES arose because of the significant disparity between his TES and PCS that made some people concerned that he would win everything based on TES alone and push the sport in an "unbalanced direction", thus the suggestion made by some people to raise the maximum possible PCS to match the maximum possible TES to ensure "balance". Also he did push some skaters who were more complete skaters and had higher PCS but could not compete with him technically off the podium at both his GP events. Personally I have no problems if Jin won on the strength of his TES and did not see it as reason to change the scoring system.

Hanyu's situation is completely different, he is a balanced skater who is capable of topping the field in both TES and PCS, as demonstrated in NHK Trophy and the GPF. He did not win based on TES alone and it would be ridiculous for anyone to claim he did when he clearly won in both TES and PCS. Even if his skating is not someone's cup of tea, the superb quality of his skating is undeniable. If the maximum PCS and TES were balanced at the GPF, Hanyu would still win and with an even higher score than 330. In addition, Jin was beaten by Uno and Chan in the FS and overall due to Jin's weaker PCS, an outcome which highlighted the importance of being a balanced skater. Also, in a scenario in which two skaters have similar PCS but one wins because of higher TES, it is not the same thing as"winning by TES alone", it is a close victory won by having an edge in TES, which is quite common.

There has been some discussion on the possibilty of raising the PCS ceiling after Hanyu broke 120 TES, but the discussion is in the vein of how much higher can he score as the general view is he can get more than 100 PCS to match his TES. I would be curious to see what happens if changes are indeed made to make TES and PCS balanced, maybe Hanyu or another skater could score over 350, or 370 and very soon? It will also have the effect of further emphasising the gap between the top tier of skaters and the rest of the field, which can be a good or bad thing depending on one's point of view.

 

[zebobes]

If PCS were raised to 120 for the men, it might cause judges to lower PCS for men, too. Basically, what used to be a 10 performance could now be a 9 performance, so that the PCS is relatively balanced for great skaters. I don't think judges would give the same kind of numbers that they do now, they would adjust it downward slightly, more like how ladies PCS are scored (or used to be scored).

Since Hanyu can score 120 in TES, they could give him 110 in PCS, and it would be more balanced then it used to be, and they could save 120 for Olympic performances.

 

[blackey]

anyone thinking about maybe they could change the way of giving goe?
I've been thinking this lately, a fairly good 3a/quad could be give around +2.5 goe without any hard transition, and hanyu's spread angle 4s in sp got +3, it's only 0.5 point of different. IMO, that 4s should got at least +4 compare to other +2.5 jumps (not to mention hanyu's ridiculous 3a). I saw some people say they should lower the bv for quad, and I think that's not the situation we want to see. So how about lower the bv for quad, but increase the possible goe? And put more weight on the transition, then the skater must be very good on the jumps with transitions in order to get high tes.

 

[HanDomi]

anyone thinking about maybe they could change the way of giving goe?
I've been thinking this lately, a fairly good 3a/quad could be give around +2.5 goe without any hard transition, and hanyu's spread angle 4s in sp got +3, it's only 0.5 point of different. IMO, that 4s should got at least +4 compare to other +2.5 jumps (not to mention hanyu's ridiculous 3a). I saw some people say they should lower the bv for quad, and I think that's not the situation we want to see. So how about lower the bv for quad, but increase the possible goe? And put more weight on the transition, then the skater must be very good on the jumps with transitions in order to get high tes.

well it's hard to judge, because there are many skaters, someone lands earlier very good quad and gets +2.5, and then at the end of order Hanyu lands something ridiculous like that 4S from spread to spread, and max they can give him is +3. But if GOE would be raised it still propably would end like that