The abhorrent state of PCS judging

[gkelly]

It is all very well that the commentators say, but he did a quad flip out of three turns and held his hand over his head! That's really hard and earns him a lot of points. But the bottom line is,"They gave the gold medal to that guy? He fell down twice! What kind of a sport is this?"

And yet, many of the same casual fans do understand that quads are much more difficult than triples and would not be happy to see a guy with no quads come out ahead of a guy with several quads and two falls.

Of course the casual fans may not understand the difference between quad toe and quad flip or lutz -- it all looks the same to them, but bad landings are more obvious.

And that's a problem regardless of falls, if say one skater did two 4T and two 4S and one 3A, while another skater did one each of 4Lz, 4F, 4T, and 4S and also two 3A. Just listening to commentary and learning to distinguish 4 from 3 revolutions in the air might make someone think that all four-quad programs are pretty much the same (assuming clean programs and comparable performance quality from a lay perspective), but this example shows that the base values can be very different.

And, back on topic for this thread, the Skating Skills might be very different, such that a Patrick Chan really does deserve a full point higher on that component than guys who land more quads or include more highlight moves between elements or express the music better, so maybe he doesn't deserve the highest scores in other components. Judges can always do a better job of separating the various components, and the scoring system a better job of balancing the value of the various components and technical elements against each other.

But there's no way that any scoring system can guarantee that skaters who are best at whatever is most obvious and important to Fan A will always win, especially if Fan B thinks different skills should be more important. Not to mention Judge C and Judge D disagreeing with each other as well as Fans A and B as to what's most important. There are just too many different variables in a sport this complex.

 

[andromache]

That's a good point. Although, I don't know. I do want figure skating to be a well-funded sport, but honestly I personally couldn't care less how many people get "converted" to non-casual fans. Like, I feel like beauty of skating, including the difficulty of the quads, speaks for itself? But then again, a lot of my personal favorite performances had major mistakes/falls, but it doesn't detract from the performance THAT much for me (if the skater doesn't let it, of course). Maybe that's just me.

For me, there's something magical about watching a magnificent, clean performance, that is in part due to the hum of the audience. You feel their excitement (even through video) as the skater builds more and more momentum through skating well and landing jump after jump (see: Jason's Riverdance at Nationals, Shen/Zhao 2004 Worlds, Rudy Galindo 1996 Nationals - like, these legendary performances that I'll rewatch over and over again). A fall, or even a pop, totally disrupts that momentum that the skater builds in getting the audience into it.

Then again, I'm totally opposite from you. I HATE rewatching programs with mistakes. The "best" version of a program is, for me, the clean one.

 

[Hevari]

It seems that the discussion transformed into the "cleanness vs. difficulty" and how the PCS have to correlate with TES... And the other hand - correlation between PCS and non-sportive factors (such as tier, federation, consistency and so on)...

For me there is no problem when a skater with less difficult jump passes is higher, that the one with more difficult jump passes, because the first one's strenghts were enough to beat the harder jumps of the second.. It's normal because figure skating is not only about jumps.. So the figure skating competition is about comparison of skaters' strenghts and to figure out whos strenghts are strongest and enough to beat others...

 

[ramurphy2005]

For me, there's something magical about watching a magnificent, clean performance, that is in part due to the hum of the audience. You feel their excitement (even through video) as the skater builds more and more momentum through skating well and landing jump after jump (see: Jason's Riverdance at Nationals, Shen/Zhao 2004 Worlds, Rudy Galindo 1996 Nationals - like, these legendary performances that I'll rewatch over and over again). A fall, or even a pop, totally disrupts that momentum that the skater builds in getting the audience into it.

Then again, I'm totally opposite from you. I HATE rewatching programs with mistakes. The "best" version of a program is, for me, the clean one.

I agree with you, but like everything in life, there is one exception to the rule. I can't count the number of times that I've watched Matt Savoie's FS from Nationals in 2006. There's just something magical about that performance, aside from that one fall on the triple lutz.

 

[drivingmissdaisy]

And yet, many of the same casual fans do understand that quads are much more difficult than triples and would not be happy to see a guy with no quads come out ahead of a guy with several quads and two falls.

Of course the casual fans may not understand the difference between quad toe and quad flip or lutz -- it all looks the same to them, but bad landings are more obvious.

It's unfortunate to see events like the Sochi Men's competition, but the judges have to reward what they are supposed to. We see a lot of subtly watered-down choreography at the Olympics, so the skaters can focus on performing clean. That no longer works; holding out moves for musical effect (and rest) is losing an opportunity to accrue points.

 

[MaiKatze]

Ashley got a huge boost in PCS over Anna, surpassing Anna's 4.5 point TES advantage, so I agree that it could have gone either way. Anna is not as engaging a performer as Ashley, Evgenia, or Kaetlyn, so I think she will always have a bit of trouble creating great programs to showcase her artistry.

Ashley I could agree on, on some days, but not as engaging as Evgenia and Kaetlyn, I don't agree with. Her Bolero SP at last years Worlds was extremely engaging and her programs this year really highlight her style. In the last few competitions she had leg problems, but when she is on, she sells her skates like business. Look at her eye contact with the judges. She's grown up heaps since 2014.

 

[drivingmissdaisy]

Ashley I could agree on, on some days, but not as engaging as Evgenia and Kaetlyn, I don't agree with. Her Bolero SP at last years Worlds was extremely engaging and her programs this year really highlight her style. In the last few competitions she had leg problems, but when she is on, she sells her skates like business. Look at her eye contact with the judges. She's grown up heaps since 2014.

Anna is actually pretty good. I guess it's more of me, personally, not connecting to her performances. Part of it may be that I'm never comfortable watching her jump after seeing some of her very hard falls when she was struggling before.

 

[MaiKatze]

Anna is actually pretty good. I guess it's more of me, personally, not connecting to her performances. Part of it may be that I'm never comfortable watching her jump after seeing some of her very hard falls when she was struggling before.

Haha, yes understandable. We all have out favourites and that's okay. She's been cleanly consistent for over a year now, but I admit I sometimes still worry, too.

 

[LRK]

I don't. Anna gives me a good sense of... solidity & reliability these days.

 

[MaiKatze]

I don't. Anna gives me a good sense of... solidity & reliability these days.

She isn't as squeaky clean as I would like her to be, at the moment. See Europeans long, or at Russian Nationals she had a ugly fall during the warm-up. I just hope that when she falls in competition it won't automatically kill all her momentum, PCS and most importantly her confidence in her program.

 

[LRK]

She isn't as squeaky clean as I would like her to be, at the moment. See Europeans long, or at Russian Nationals she had a ugly fall during the warm-up. I just hope that when she falls in competition it won't automatically kill all her momentum, PCS and most importantly her confidence in her program.

She held it together, though - and I expect her to do that going forward as well. I think the days of epic meltdowns are behind her.

 

[andromache]

Ashley I could agree on, on some days, but not as engaging as Evgenia and Kaetlyn, I don't agree with. Her Bolero SP at last years Worlds was extremely engaging and her programs this year really highlight her style. In the last few competitions she had leg problems, but when she is on, she sells her skates like business. Look at her eye contact with the judges. She's grown up heaps since 2014.

I think Anna's presentation is more sophisticated than Evgenia's - she isn't acting out a specific role or part, but she's connecting with her music and being a kickbutt lady - it's not as "obvious" as Evgenia's presentation, which is very well done, but relies more on acting than on music or presence.

Kaetlyn's SP is, presentation-wise, top-notch. There's musicality and sophistication and you get the sense she is acting out a part. She does it all (not unlike Ashley). On the other hand, Kaetlyn's generic LP does her no favors - just going out there and smiling through the skating and the music doesn't do it for me. She's definitely engaging when she does it, but there's no depth to it the way there is to her SP.

But anyway, I think Anna's style is less accessible than the styles of the other ladies, because she doesn't really play a character or act a role.

 

[gsyzf]

Denis Ten was considered a 2nd-tier skater at 2013 Worlds (where he skated light years better than anything he did prior to it) and that was one factor that prevented him from winning. While that is controversial then, Top tier skaters "only having significant GOE and PCS advantage over 2nd tier skaters" should also be deemed controversial now, as well.

There have certainly still been shady judging/controversial results, albeit not many to the extent of 2013 Worlds men....

It is unfair that 1st tier skaters are automatically favored over 2nd and 3rd tier skater regardless of how they perform. But 1st tier skaters don’t just become 1st tier skaters for no reasons. They usually have some qualities better than the rest of the field. Controversies usually happen when a 1st tier skater give very flawed performances but still outscore a 2nd tier skater who gives much cleaner performances. If 1st tier skaters can consistently deliver, controversies are less likely to happen. So a 1st tier skater who is favored due to his/her consistency is less likely to get controversial placements than a 1st tier skater who is favored due to his/her potential with no consistency/ delivery.

Right now, competition result is more likely to be controversial at smaller competitions like GPs because there are usually only 1-2 first tier skaters in these competitions and the rest of the competitors are all 2nd or 3rd tier skaters. When 1st tier skaters don’t deliver while 2nd tier skaters do, it often results in controversial podium, e.g. COR, COC. (At COC2014, Hanyu was the only 1st tier skater. Dornbush was in the 2nd tier. If judges gave both equally stingy or equally harsh goe & pcs, the bronze & silver could’ve switched places. But Hanyu would still be on the podium. I don’t think he would be off podium because COP gives many points for rotating jumps.)

The results at major championships like WC, GPF, Olympics and to a less extent Euro & 4CC, are less likely to be controversial these days because there are usually 4-5 first tier skaters competing in the same competition. Unless all 1st tier skaters bomb, 2nd tier skaters usually don’t have much any chance to get on the podium. The only post-2013 major championship podium that is really controversial to me is the ladies’ competition at WC2014. I believe judges already decided on the podium before the competition, i.e. Asada, Lipnitskya and Kostner in some order, and simply ranked these 3 skaters based on their performances. Judges didn’t intend to give medals to anyone else. Anna Pogorilaya’s PCS was 10 points behind Kostner’s PCS. 10 points is usually the gap between 1st tier skaters and 2nd tier skaters regardless of their performances. The PCS of skaters in the same tier are usually just 1-3 points apart, no more than 4-5 points apart. IMO, the other major championship results you mentioned are in fact close wins. The results could’ve gone either way. Whoever skates later will place higher than the person skates earlier. Skating later always gets more generous scores, especially skating last. Skate order depends on luck. Nobody will always be lucky to skate later than his/her rivals. But for most people, it is harder to skate well when you know your rivals already skate well before you. So there is some justification for giving a bit more generous scores to later skaters.

Here is a list of close wins that could’ve gone either way, IMO.

1. WC 2014 men’s competition is a close call between Machida and Hanyu. Machida was judged as a 1st tier skater in this competition because the competition was held in Japan. If he was judged as 2nd tier skater, his PCS would be 10 points lower than Hanyu’s PCS, not just a fraction lower than Hanyu’s PCS. Machida’s PCS was almost 10 points lower than Hanyu’s in Sochi Olympics. That’s an indication that Machida was judged as a 2nd tier skater. Hanyu won WC2014 by a small fraction because he skated later than Machida. Machida would’ve won WC 2014 if he skates his FS well after Hanyu. But there is no guarantee that Machida could actually skate as well as he did had he skated after Hanyu knowing Hanyu gave a clean FS before him and he was in the winning position after the SP. Fernendez didn’t skate very well after Hanyu in the FS and dropped to 3rd overall.

2. 4CC 2017 men’s competition is also a close call between Chen and Hanyu. Chen won because he skated last, after Hanyu.

3. WC2016: Wagner outscored Pogorilaya because she skated last.

4. GPF2016: Miyahara outscored Pogorilaya because she skated after Pogorilaya (I’m only talking about the GOE and PCS, not the tech calls).

5. WC2012: Chan outscored Takahashi because he skated last. The margin should’ve been much closer though. WC2013 men’s competition was much more controversial, similar to WC2014 ladies’ competition.

 

[fallingsk8er]

How did Chen skating last make Hanyu pop a jump in the short program?

 

[Ares]

I think Anna's presentation is more sophisticated than Evgenia's - she isn't acting out a specific role or part, but she's connecting with her music and being a kickbutt lady - it's not as "obvious" as Evgenia's presentation, which is very well done, but relies more on acting than on music or presence.

Kaetlyn's SP is, presentation-wise, top-notch. There's musicality and sophistication and you get the sense she is acting out a part. She does it all (not unlike Ashley). On the other hand, Kaetlyn's generic LP does her no favors - just going out there and smiling through the skating and the music doesn't do it for me. She's definitely engaging when she does it, but there's no depth to it the way there is to her SP.

But anyway, I think Anna's style is less accessible than the styles of the other ladies, because she doesn't really play a character or act a role.

Great analysis andromache! Moreover I concur with points you made.

 

[MaiKatze]

I think Wagner vs. Pogorilaya was basically only because of the home advantage. Not saying that Ashley didn't perform the hell out of that program, but I think after Gracie off-podium and Evgenia far away, they had to make sure that Ashley got at least on the podium. And they did. I'm not happy about the result, but I'm actually more okay with it than with Carolina winning bronze over Anna in 2014. But here the second tier skater rule comes into frutition as Anna was a new Russian teen girl and Carolina was coming from the Olympics as bronze Medalist. They also desperately wanted Lipnitskaia on that podium, so in the end it was only 4th for Anna, although she skated 2 clean programs, with very high technical content. And..well I loved Mermaid Anna, although a lot of people did dislike that program back then.

 

[gsyzf]

How did Chen skating last make Hanyu pop a jump in the short program?

Hanyu popping a jump in the SP has nothing to do with Chen skating last. But both Hanyu and Chen made mistakes in the LP. So the question is whether Hanyu's LP lead can make up for gap in the SP. Chen had a few more visible mistakes while Hanyu had a big pop and tried to make up points by changing his layout. So the overall result was quite close between Chen and Hanyu. Had Hanyu skated last, he may get a bigger lead in the LP score and close the SP gap. Usually, skating later, especially skating last gets more generous scores than skating early. Usually when the result is this close, whoever skates later will win.

 

[gkelly]

It is unfair that 1st tier skaters are automatically favored over 2nd and 3rd tier skater regardless of how they perform. But 1st tier skaters don’t just become 1st tier skaters for no reasons. They usually have some qualities better than the rest of the field. Controversies usually happen when a 1st tier skater give very flawed performances but still outscore a 2nd tier skater who gives much cleaner performances. If 1st tier skaters can consistently deliver, controversies are less likely to happen. So a 1st tier skater who is favored due to his/her consistency is less likely to get controversial placements than a 1st tier skater who is favored due to his/her potential with no consistency/ delivery.

To some degree, it often happens that good skaters who make mistakes end up performing less well throughout the program than they had during their best performances. In those cases, their PCS should be lower during the flawed performances than during their best performances.

But element GOEs are given separately for each element. The fact that a skater fell on element 4 should have no bearing on whether s/he deserves +3 for element 10, or vice versa.

And each program component is supposed to be evaluated independently, as a measure of how well the skater met the criteria for that component over the course of the whole program.

Many technical errors on elements that lead to significant loss of TES points on those elements have little or no effect on how well the skater meets the PCS criteria throughout the program as a whole.

So a skater who, on average, is a "1st tier skater" on the various Skating Skills criteria, or Composition criteria, or any other program component, is likely still to be a 1st tier skater on those criteria even with a couple of failed jumps. They might deserve program component scores somewhat lower than their best performances, but the overall quality doesn't suddenly become 2nd or 3rd tier just because of a mistake or two. Occasionally they might even relax after an early fall and skate the rest of the program with more speed and freedom and precision than they usually do and deserve even higher PCS than usual.

Or a 1st tier skater might deliver a 2nd-tier performance in terms of one or more components (especially Performance and Interpretation -- but sometimes even slowing down and executing skating skills cautiously) during a clean performance because they're so busy focusing on executing the technical content successfully. This is probably most common early in the season while skaters are still getting used to new programs. In that case, the TES will probably be high but some of the PCS may suffer.

It shouldn't be about "potential" but rather about what the skater actually executes on the ice that day.

On the one hand, both judges and fans do have expectations about what known skaters are capable of, so that probably does unconsciously color perceptions of what the skater does on the ice.

On the other hand, often judges are reacting aspects of the actual performance that meet the component criteria especially strongly (or especially weakly) that fans overlook because they don't have the training to look (and listen) for those details, or because they're watching on video and the recording doesn't convey what was obvious live.

So when more established skaters earn better scores even with errors in their programs, is hard to know how much of that was due to unconscious expectations/"potential" effects, and how much to the fact that they just skated really well aside from the errors.

 

[drivingmissdaisy]

It is unfair that 1st tier skaters are automatically favored over 2nd and 3rd tier skater regardless of how they perform.

I don't think it is always intentional. To some degree, you have to be on the judges' radar to be a contender. When Denis won the WSM, he came in with 6th and 9th place GP results and 12th at 4CC, so he was only a 2nd tier skater because his previous competition performances were relatively weak. When someone who normally doesn't skate brilliantly all of a sudden does, it can be difficult for the judge to mark the skater because he probably hasn't given a lot of thought to what PCS scores he might assign to the skater if clean.

 

[LRK]

I think Wagner vs. Pogorilaya was basically only because of the home advantage. Not saying that Ashley didn't perform the hell out of that program, but I think after Gracie off-podium and Evgenia far away, they had to make sure that Ashley got at least on the podium. And they did. I'm not happy about the result, but I'm actually more okay with it than with Carolina winning bronze over Anna in 2014. But here the second tier skater rule comes into frutition as Anna was a new Russian teen girl and Carolina was coming from the Olympics as bronze Medalist. They also desperately wanted Lipnitskaia on that podium, so in the end it was only 4th for Anna, although she skated 2 clean programs, with very high technical content. And..well I loved Mermaid Anna, although a lot of people did dislike that program back then.

I liked it too. Or rather, I didn't "get" it during the GPs, but when I saw Anna at Worlds', suddenly her programs "clicked" for me - I don't know how or why that happened (I'd not seen her in between), but I really ended up enjoying Mermaid Anna in particular. It was a strong, powerful performance, I thought.