ISU releases official agenda with proposals for 2024: age limits, jump limits and more

[Mathman]

I think the focus really actually needs to be on improving the judging, not the scoring system, if we are talking about skaters getting or not getting the result we feel they deserve.

The trouble with saying we lwant better judging is -- well of course we want better judging! Let's take a vote: how many here want better judging? How many want worse judging? OK, that's unanimous. Now... how do we go about achieving better judging?

Plus, is the definition of "better judging" really "giving skaters the scores WE think they deserve?"

 

[Mathman]

As for Brown's mannerisms, I just don't like it.

Oh dear. And Jason says such nice things about you.

 

[gkelly]

I think it's quite impressive (in a negative way) that the change from 3 to 5 components has not really made any change at all.
You might just as well reduce it to one because that's more or less what the judges are still doing.

In that case you would really need to use smaller increments between scores, or else all the best skaters (or any group of skaters in the same skill range) would all be getting practically the exact same score.

I could see an argument for having only two scores, one for Skating Skills and one for Performance/Presentation, with everything currently under Composition absorbed into either of those two. Though personally I still prefer a finer breakdown.

Not what they should be doing, though.
I think the focus really actually needs to be on improving the judging, not the scoring system, if we are talking about skaters getting or not getting the result we feel they deserve.

Of course the judge training should always keep improving.

But if the value of giving 0.5 or even 1.0 higher on all components is less than the value of one quad, that makes it very hard for judges to commensurately award stronger skaters and performers enough to make a difference if they they're using the scale appropriately.

I am somewhat hestitant about that solution because the reasoning applies only to the very top of the top skaters.

The top is where the outliers will be. Under the current system, there is more room for jump rotation outliers to overwhelm the medal stand without opportunities for everything-else outliers.

But the same principles would apply at lower skill levels.

E.g., for elite but not medal contending women with no falls, which should prevail -- 7 triples and "good" components (low 7s) vs. 6 triples with several underrotation calls and "very good" components (low 8s)? Should they be close enough that the spin and step sequence levels and GOEs be the deciding factor? Or should jump rotation always be the deciding factor because it's more objective?

For a more representative sample, we could look at look at the bottom ten at Worlds instead of the top ten -- elite skaters all -- and ask whether the TES and PCSs are too high or two low for "perfect balance."

Men: TES > PCS: Lee, Daniliants, Kim, Chiu. Egaadze
PCS>TES: Nordeback, Sadovsky, Pulkinen. Economides, Fangripani

Women: TES>PCS: ,Joos, Schild
PCS>TES: Pelkonen, Gomez, Ting, Landerbaur., Taljegard, Petrokina,Schizas, Pinzcaone.

This gives a different picture tfrom considering Malinin, Fa and Kagiyama. Granted, I have never heard of most of the skaters on these lists (my bad), but that's the point. If anything the ladies' PCS factoring should be reduced.

You're just naming names here, so I don't know how that argues in favor of increasing or reducing PCS, one way or another.

There will be some skaters who are better at skating and performing than at rotating in the air. There will be some whose best skill is rotating in the air. For those who rely most on the jumps, success or failure of those jumps on any given day will have the biggest impact on their results. Some kinds of failures are very costly in TES, others are more costly in PCS, others both.

 

[Mathman]

Or should jump rotation always be the deciding factor because it's more objective?

My preference would be to reduce the value of jumps across the board. It might seem like this is equivalent to just raising PCS across the board, but I don't thnk so. For one thing, it would relatively elevate non-jump tech as well as program components, which I think would be a good thing.

You're just naming names here, so I don't know how that argues in favor of increasing or reducing PCS, one way or another.

I would think that the test of whether we have the balance right would be this. If it's right, then the distribution or TES-PCS over all skaters should be symmetric and centered around 0. In any reasonably-sized random sample of skaters, whatever their skill level, we would expect to find 50 percent with TES more than PCS, and 50 percent the other way. In this sample the men were 50-50, but the ladies were 20-80.

 

[icewhite]

In that case you would really need to use smaller increments between scores, or else all the best skaters (or any group of skaters in the same skill range) would all be getting practically the exact same score.

I could see an argument for having only two scores, one for Skating Skills and one for Performance/Presentation, with everything currently under Composition absorbed into either of those two. Though personally I still prefer a finer breakdown.


Of course the judge training should always keep improving.

But if the value of giving 0.5 or even 1.0 higher on all components is less than the value of one quad, that makes it very hard for judges to commensurately award stronger skaters and performers enough to make a difference if they they're using the scale appropriately.

I definitely don't want to decrease the number of components further. I think it's bad enough as it is. Just wanted to say with the way it's done in praxis it would probably not change anything if we only had one score.

I don't think all the top skaters need to get the same scores, even if I am clueless about non elite skaters and how they are judged. But I can see that even at the big events 5s are given, so it's not like they are not given to elite skaters at all.
So if there is a general range from 5-10 we can use for this group of skaters, why should it not be possible to hand them out according to what we see.

I don't know if better training will do because I don't know if the judges are not cabable or not willing. I find it hard to believe the judges should be so clueless, but okay, maybe they are.

I've talked about this in other threads, so it's probably just redundant, but my main starting point would be to get rid of the "it's subjective" premise. Everything that is subjective (and of course there are things that are questions of taste and cultural priming) should simply be left out of judging. For everything else purely objective criteria should exist. And for judges to follow those criteria I think we need to start erasing any talk of "it's subjective anyway, one person prefers this, the other that". Everything that's different, but of equal difficulty should be scored equally.

But spending more time on this forum I realize this will not happen. There are too many defenders of "it's subjective and I wouldn't want it to be any other way because that would kill the artistry" even among people who are really knowledgable about the sport.

But I struggle to see how things are going to be better when this is the starting point: "in the end everything but levels and jump rotations is subjective and as long as you just score like the others everything is fine." It's a premise that in my eyes kills every possibility to seriously improve judging from the start.

 

[gkelly]

Depends on the sample that you choose, though.

We'd have to study many full competitions to see what the patterns are across the field.

Looking at the women's free skate from 2024 Junior Worlds, I see that 24 out of 24 skaters had higher TES than PCS.

If not for the current change in the age rules, that tells us something about how younger jumpers with less refined PCS might score against more "mature" female skaters if competing against each other in senior events. Of course, for all skaters, if PCS factors were higher, the skaters might put more energy into improving those skills and less into trying to add more jump content. But it's often easier for younger skaters to add the difficulty before maximizing the quality.

The rankings by TES are pretty similar to the rankings by PCS, so changing the factors wouldn't have made much difference in the results in this particular event.

At 2024 Four Continents Women's FS, 11 had higher TES, 13 had higher PCS. Which is about what you predicted.
The skaters with the higher PCS tended to be the ones with more mistakes, who would have placed higher if they were having a better day. Fair enough. Unless the fans and TPTB would rather see skaters with higher on-ice skill levels aiming to skate easier clean programs and be rewarded for doing so.

Looking at a wider range of competitions would give more insight.

 

[gkelly]

I definitely don't want to decrease the number of components further. I think it's bad enough as it is. Just wanted to say with the way it's done in praxis it would probably not change anything if we only had one score.

I don't think all the top skaters need to get the same scores, even if I am clueless about non elite skaters and how they are judged. But I can see that even at the big events 5s are given, so it's not like they are not given to elite skaters at all.

I didn't mean all skaters at Worlds would get the exact same scores. I mean that all skaters in the same skill range subset within the field would get very similar scores.

E.g., if there are five or more skaters in the field that judges agree deserve scores "in the mid-7s" right now different judges might give different mixes of 7.25, 7.5, and 7.75 for the three components -- if they feel strongly that one of those skaters is stronger than the others in that group, in general or on one specific component, they can tilt toward the 7.75 scores. But it's more likely that more than one of them will end up with exactly 7.5 as their single component, as opposed to seeing ranges from 7.3 to 7.8 across those skaters.

In which case, the PCS will not make any difference in close calls among those skaters.

I've talked about this in other threads, so it's probably just redundant, but my main starting point would be to get rid of the "it's subjective" premise. Everything that is subjective (and of course there are things that are questions of taste and cultural priming) should simply be left out of judging. For everything else purely objective criteria should exist. And for judges to follow those criteria I think we need to start erasing any talk of "it's subjective anyway, one person prefers this, the other that". Everything that's different, but of equal difficulty should be scored equally.

Any suggestions on how to rewrite PCS criteria to make them more objective?

 

[TontoK]

On the other hand, it would provide more options for skaters eager to improve their results, Suppose a triple Lutz got 4 points and a quad toe 5,5. So one possibility would be for a skater to work hard on his quad and beat his rivals by 1.5 points. Or he could work on presentation or skating skills and try to get his 8.5s up to 8.75. and win by 2.5 points. Or he could concentrate on spins and footwork to make up the gap.

My beef with this is the discounting of difficulty and risk. How many men at the recent WC successfully completed a 4Z, 4F, or 4Lp? Only a handful, which is evidence that these are difficult and risky elements. Of those that attempted them, not everyone succeeded. For those who did take the risk and succeeded, the payoff was in a higher score.

In contrast, how many skaters attempted combination spins? All of them. Every last one. And they were all successful to varying degrees. This suggests that in relation to higher BV elements, they're just not that difficult. Now, that is not to say they're not important, but rewarded them to the same degree as a quad jump does not make sense.

 

[Jumping_Bean]

I am somewhat hestitant about that solution because the reasoning applies only to the very top of the top skaters. For a more representative sample, we could look at look at the bottom ten at Worlds instead of the top ten -- elite skaters all -- and ask whether the TES and PCS are too high or two low for "perfect balance."

Men: TES > PCS: Lee, Daniliants, Kim, Chiu. Egaadze
PCS>TES: Nordeback, Sadovsky, Pulkinen. Economides, Fangripani

Women: TES>PCS: ,Joos, Schild
PCS>TES: Pelkonen, Gomez, Ting, Landerbaur., Taljegard, Petrokina,Schizas, Pinzcaone.

This gives a different picture tfrom considering Malinin, Fa and Kagiyama. Granted, I have never heard of most of the skaters on these lists (my bad), but that's the point. If anything the ladies' PCS factoring should be reduced.

Nina Pinzarrone had a higher PCS score than TES score exactly once this season - In the FS at Worlds. She had an usually difficult time with the technical elements due to health issues.

Nella Pelkonen had 4 pops in her FS (and higher TES than PCS in the Short), which affect TES much more than PCS. The other time this season she scored higher in TES than PCS was at Euros in the Free, where she had the same amount of pops.

Basically, what I'm trying to say is that the sample is not necessarily representative - Mistakes will cause TES and PCS to drop by different amounts, and different panels will lead to more or less generous scores in either of the categories.

 

[Jeanie19]

I didn't mean all skaters at Worlds would get the exact same scores. I mean that all skaters in the same skill range subset within the field would get very similar scores.

E.g., if there are five or more skaters in the field that judges agree deserve scores "in the mid-7s" right now different judges might give different mixes of 7.25, 7.5, and 7.75 for the three components -- if they feel strongly that one of those skaters is stronger than the others in that group, in general or on one specific component, they can tilt toward the 7.75 scores. But it's more likely that more than one of them will end up with exactly 7.5 as their single component, as opposed to seeing ranges from 7.3 to 7.8 across those skaters.

In which case, the PCS will not make any difference in close calls among those skaters.


Any suggestions on how to rewrite PCS criteria to make them more objective?

For me I can overlook some of the pc bias. What I cannot overlook one is GOE on jumps. I watch a ton of skating, and just because you are in the first 2 groups does not mean a GOE gap of plus 1, when that 3A is as good as a plus 2 or more. There should not be bias on jump technique. An up and coming skater can have beautiful jumps and a favorite skater can have a bad day and deserves negative GOE.

 

[Jeanie19]

My beef with this is the discounting of difficulty and risk. How many men at the recent WC successfully completed a 4Z, 4F, or 4Lp? Only a handful, which is evidence that these are difficult and risky elements. Of those that attempted them, not everyone succeeded. For those who did take the risk and succeeded, the payoff was in a higher score.

In contrast, how many skaters attempted combination spins? All of them. Every last one. And they were all successful to varying degrees. This suggests that in relation to higher BV elements, they're just not that difficult. Now, that is not to say they're not important, but rewarded them to the same degree as a quad jump does not make sense.

I agree to a point. But just attempting a 4lutz, 4flip, and 4loop should not matter unless it is a successful jump with good GOE.
I don't like the well this skaters BV is more than that skaters. It doesn't matter until the skate is competed and successful.

 

[TontoK]

I agree to a point. But just attempting a 4lutz, 4flip, and 4loop should not matter unless it is a successful jump with good GOE.
I don't like the well this skaters BV is more than that skaters. It doesn't matter until the skate is competed and successful.

We can agree that the penalty for a failed element, any failed element, should be greater.

To borrow from another sport - in diving, there is such a thing as a failed dive. If a diver fails in an attempt at a dive, that diver receives a score of 0 of a possible 10. He doesn't get a 5 of 10 because he made a valiant attempt.

 

[Mathman]

Basically, what I'm trying to say is that the sample is not necessarily representative - Mistakes will cause TES and PCS to drop by different amounts, and different panels will lead to more or less generous scores in either of the categories.

Solution: take a larger sample. Yes, any individual sample will contain a few misleading data points. In a sufficiently large and random sample these anomalies, plus and minus, will wash out -- all but guaranteed. )

 

[4everchan]

I am somewhat hestitant about that solution because the reasoning applies only to the very top of the top skaters. For a more representative sample, we could look at look at the bottom ten at Worlds instead of the top ten -- elite skaters all -- and ask whether the TES and PCS are too high or two low for "perfect balance."

Men: TES > PCS: Lee, Daniliants, Kim, Chiu. Egaadze
PCS>TES: Nordeback, Sadovsky, Pulkinen. Economides, Fangripani

Women: TES>PCS: ,Joos, Schild
PCS>TES: Pelkonen, Gomez, Ting, Landerbaur., Taljegard, Petrokina,Schizas, Pinzcaone.

This gives a different picture tfrom considering Malinin, Fa and Kagiyama. Granted, I have never heard of most of the skaters on these lists (my bad), but that's the point. If anything the ladies' PCS factoring should be reduced.

this means very little in my opinion.
Why ? Because many of the skaters with higher PCS than TES just had bad skates. So in that sense, it is normal you would see their TES lower than PCS.

Statistics can be useful but they need to evaluate somewhat measurable consistent data.

Skater A has good skating skills with lower base value. Skater A cleans a good program, not perfect but decent. Skater A gets 78 in TES and 81 in PCS.

Skater B is a bit rough but has good quads... but then, pops and falls a few times. Skater B gets 70 in TES because of the many mistakes but manages 72 in PCS.

Are Skater A and B's results equivalent ? Not at all, yet you bunched them in the same group in your statistical analysis.


ETA I see that @Jumping_Bean had already given you a similar answer... so just see mine as reinforcement but also, for me it's not just about a small sample but the quality of evaluation.

 

[Mathman]

To borrow from another sport - in diving, there is such a thing as a failed dive. If a diver fails in an attempt at a dive, that diver receives a score of 0 of a possible 10. He doesn't get a 5 of 10 because he made a valiant attempt.

On the other hand, just because divers do it that way, that in iteself doen't make it right.

In horseshoes you get credit for a "leaner." You didn't ring the post, but you get a point for snuggling up to it. (It is not clear that lessons we can draw for figure skating, though.)

 

[4everchan]

The trouble with saying we lwant better judging is -- well of course we want better judging! Let's take a vote: how many here want better judging? How many want worse judging? OK, that's unanimous. Now... how do we go about achieving better judging?

Plus, is the definition of "better judging" really "giving skaters the scores WE think they deserve?"

Everyone wants better judging, except those who may be favoured by deficient judging... So it's not unanimous

But the point made, not only by @icewhite but many others in countless threads is that many fans wished that judging was more consistent and more objective when possible. There will always be a subjective part of figure skating judging, but even that part can have more educated judging.

I could also ask a question that may get close to 100% YESses.

Do you wish for judges to have even better training ?

 

[gkelly]

Yes-or-no questions are usually easy to answer.

The real question is, HOW can judges be better trained to evaluate the qualitative aspects of figure skating?

Are the current PCS guidelines fine as is, it's just a matter of getting judges to do a better job of using the existing rules and guidelines?
Or should there be different ways to measure some of these qualities and different questions to ask judges to answer?

(And also, how can fans be better trained to understand those aspects?)


In terms of looking at the balance of TES vs. PCS, it's probably best to start by looking at "clean" programs to see how the balance comes out.
Easier to find in short programs, but perhaps more relevant to long programs where there are more opportunities to build up big leads with harder jumps.


Also, can PCS be scored accurately across eras so that strong skaters with doubles could meaningfully be compared to strong skaters with triples and with quads? If so, is there a way to standardize the criteria so that judges would be less swayed by the presence of quads when evaluating the PCS?

 

[icewhite]

I didn't mean all skaters at Worlds would get the exact same scores. I mean that all skaters in the same skill range subset within the field would get very similar scores.

E.g., if there are five or more skaters in the field that judges agree deserve scores "in the mid-7s" right now different judges might give different mixes of 7.25, 7.5, and 7.75 for the three components -- if they feel strongly that one of those skaters is stronger than the others in that group, in general or on one specific component, they can tilt toward the 7.75 scores. But it's more likely that more than one of them will end up with exactly 7.5 as their single component, as opposed to seeing ranges from 7.3 to 7.8 across those skaters.

In which case, the PCS will not make any difference in close calls among those skaters.


Any suggestions on how to rewrite PCS criteria to make them more objective?

I mean everything remotely close to the truth (and personally I believe there is a truth in figure skating ) is okay for me. I won't start to argue about 7.5 or 7.75 even if it can be decisive. I am talking about the performances where I would go at least 1 point lower - or higher- in one of the components.

I'm not really having much issues with the PCS criteria. They should really say how much each bullet point is worth, though, in comparison. Are they equally important? Or is there a leading bullet point which overshadows the others? I don't know if this is done in judges training. It's not clear from the chart and I feel there are often discussions stemming from this.

Apart from that, the criteria seem pretty fine (although maybe they could be more elaborated on publically so the audience has better insight into the judging). It's mostly that what is written there and what is given out as scores simply does not fit. So for me it's the evaluation process that needs targeting more than the PCS criteria.

 

[Mathman]

this means very little in my opinion.
Why ? Because many of the skaters with higher PCS than TES just had bad skates.

Agin, take a bigger sample. If your sample includes 100 performances instead of ten, some skaters will have bad skates that day, some will have good.

Now repeat the experiment with a different humdred performances taken at random. Some skaters will have nad skaes, some will have good.

That is what gives statistics its predictive power.

The other thing is that in statistics the only thing that counts is the actual numbers, not the setting or context. The same numbers might be the heights of 100 Martians selected at random. Yes, some Martians are taller than average and some are shorter. Maybe one had his head cut off and hasn't had time to grow a new one. Doesn't matter -- we can still compute the average

In figure skating the goal is that, on the average, over good performances and bad, over results both expected and bizarre, TES and PCS should work out to be more or less the same.

 

[4everchan]

Agin, take a bigger sample. If your sample includes a00 performances instead of ten, some skaters will have bad skates that day, some will have good.

I don't think so. If you try to compare apples and oranges, it won't matter if you have one million more apples and one million more oranges. Still apples and oranges..

Now repeat the experiment with a different humdred performances taken at random. Some skaters will have nad skaes, some will have good.

That is what gives statistics its predictive power.

The other thing is that in statistics the only thing that counts is the actual numbers, not the setting or context. The same numbers might be the heights of 100 Martians selected at random. Yes, some Martians are taller than average and some are shorter. Maybe one had his head cut off and hasn't had time to grow a new one. Doesn't matter -- we can still compute the average

In figure skating the goal is that, on the average, over good performances and bad, over results both expected and bizarre, TES and PCS should work out to be more or less the same.

If your analysis aimed at evening PCS and TES, then you need to look at the maximums skaters can get. Looking at lower ranked skaters like you did in your analysis will not achieve that.