Multiplicative PCS scoring?

[Blades of Passion]

This system is better, but how much better I don't have the answer.

If we take a score of 160 on the original system.

For (TES-PCS) adding to 160: Old system --> New system

100-60 --> 75
80-80 --> 80

For any same score given on the old system, the new system values parity more. And there are diminished returns on increasing TES while PCS is lower, whereas on the current system there is no such thing.

I don't think that's a desirable thing to have, it just pushes people to homogenize their skating. Instead of trying to be brilliantly artistic or brilliant at jumps, it encourages people to just be "okay" at everything. 80-80 doesn't deserve to beat 60-100 or 100-60.

BTW, how are you arriving at those numbers of '75' and '80'? It should be 60 and 64 (which is the same on a percentage basis, so I'm curious where you got 75 and 80 from).

 

[Mathman]

...it encourages people to just be "okay" at everything.

This is true. The name of this procedure (up to an irrelevant constant multiplier) is the "harmonic mean" To maximize the harmonic mean of two numbers, be OK at both. If you are a genius at one and a doofus at the other, you are out of luck.

The same is true in any sport that has two parts. In biathlon it is better to be a reasonably good shot and a reasonably fast skier. In triathlon the person who is OK at swimming, OK at bicycling and OK at running will win over the specialist.

BTW, how are you arriving at those numbers of '75' and '80'?

100+60 --> 100x60/80 = 75
80+80 --> 80x80/80 = 80

The 80 in the denominator reflects the current 1.6 factor for the ladies LP. The results would turn out the same whatever factor is used.

 

[Mathman]

Here you're addressing a judging problem not a scoring system problem.
It makes no sense that PCS goes up when TES goes UP but we see it everyday.

To me, that question is not so cut and dried.

Everything else being equal, a skater who puts on a brilliant display of technical fireworks, woven together in an esthetically pleasing and emotionally satisfying program and with appropriate attention to the music -- well, maybe that is more impressive that the skater whose program is equally well choreographed and musical, but the overall impression is underwhelming.

 

[McBibus]

To me, that question is not so cut and dried.

Everything else being equal, a skater who puts on a brilliant display of technical fireworks, woven together in an esthetically pleasing and emotionally satisfying program and with appropriate attention to the music -- well, maybe that is more impressive that the skater whose program is equally well choreographed and musical, but the overall impression is underwhelming.

It is, and it's ok in performance and presentation.
But what has it to do with Skating Skills and transitions?
That leads me to the last question: Why the vote on the 5 pcs categories are always so close?
It's like thei're just braking down a single PCS vote they have in mind more than judging every single PCS.
Maybe the system is too complex.
They have to assign GOE to any single element (using "bullets"), and judge 5 different PCS.
Maybe like we have a tech panel we should have different judges for TES and PCS.

 

[Skatesocs]

The same is true in any sport that has two parts. In biathlon it is better to be a reasonably good shot and a reasonably fast skier. In triathlon the person who is OK at swimming, OK at bicycling and OK at running will win over the specialist.

I wouldn't say that skating is like a triathlon.

ETA: I love this discussion!

 

[Skatesocs]

I don't think that's a desirable thing to have, it just pushes people to homogenize their skating. Instead of trying to be brilliantly artistic or brilliant at jumps, it encourages people to just be "okay" at everything. 80-80 doesn't deserve to beat 60-100 or 100-60.

Curiously, what we have in the men's field is a 160 for all of those - the "tiny LP gold medal" goes to 60-100 on PCS, but they are just numbers. We'll have to look at the SP.

In the ladies field we have in comparison 100-48, 80-64, 60-80. That puts someone who was "perfect" on PCS last in the ladies field, even though I agree she should be ahead of 80-64 - AND there's a differential so it could affect the overall ranking.

Even in 6.0 (now for both men and women, if we assume 100=6.0, 80=5.8, 60=5.6), we have in the LP 5.6/6.0 > 5.8/5.8 > 6.0/5.6 and now the tech loses, even though I agree it should be ahead of 5.8/5.8 - AND this changes the LP ranking, so it could affect the overall ranking.

I never thought about these flaws :think:

 

[gkelly]

That leads me to the last question: Why the vote on the 5 pcs categories are always so close?
It's like thei're just braking down a single PCS vote they have in mind more than judging every single PCS....

I think there are basically three general ways that judges could approach assigning the 5 component marks:

*Get a general sense during the program of what range the components should be in (e.g., "8s"), and then when it's time to assign specific marks decide which component should be highest and lowest, how far apart, and fill in the others in between

*Start with assigning a score for whichever component makes the strongest impression first, most likely the strongest or weakest component for that performance, and then decide how far up or down to go for the other components. (This is likely to be the Skating Skills component for a couple of different reasons, but some judges may habitually start with a different component for every skater, and others will start with whatever is most salient for that particular performance.)

*Have clear mental benchmarks for each numerical value for each component, and assign scores for each component purely with reference to those benchmarks and with no mental comparison between one component and another.

Judges who use the last method would be most likely to have larger gaps between components.

Wide gaps would be appropriate for the skaters whose skills are more unbalanced.

On the other hand, even when judges make a conscious effort to separate their scores, many skaters are pretty well balanced in their performance skills, so it's not appropriate to separate the scores too far just for the sake of separating them.

For example, a skater with strong skating skills will tend to be more secure on their blades throughout the program (occasional errors aside), which would make everything look stronger than a skater with shakier skills. Even if the weaker skater makes a concerted effort to be artistic and the stronger skater does not.

Also, it depends what your point of reference is. Your mental benchmark for "average" or "poor" would be different if the worst skater you've ever seen is an average international-level competitor having a bad day, or if your mental image for "poor" is a below-average novice, or a beginner with limited skills doing their best.

In a field of world-class skaters, the scores are likely to range from 6s to 9s, and within that range the stronger skaters in the group will tend to have scores clustering toward the upper end of that range, the weaker ones (compared to the world-class elite; above average compared to all skaters) would have scores clustering toward the lower end of that range.


If we believe larger gaps should be very common rather than rare, then it would be helpful to give judges better training or better tools for thinking about the separate components separately. Maybe that would include judging (some components only).

Maybe like we have a tech panel we should have different judges for TES and PCS.

When people suggest this, I always say that the TES panel should also be tasked with scoring the Skating Skills and possibly the Transitions as well. Make these the technical judges, and let a separate panel concentrate on the performance aspects.

In addition to giving the technical judges something to think about between elements to keep them from getting bored out of their skulls, it would also make sure that no judges is anchoring their Performance or Interpretation scores directly their own Skating Skills scores, since they wouldn't be assigning SS scores at all.

How a further division of labor within the judging panel might work best -- assuming that practical aspects could be resolved -- would be an interesting discussion for another thread.

 

[Blades of Passion]

Curiously, what we have in the men's field is a 160 for all of those - the "tiny LP gold medal" goes to 60-100 on PCS, but they are just numbers. We'll have to look at the SP.

In the ladies field we have in comparison 100-48, 80-64, 60-80. That puts someone who was "perfect" on PCS last in the ladies field, even though I agree she should be ahead of 80-64 - AND there's a differential so it could affect the overall ranking.

I'm confused about what you're saying with both of those things? In the second part you're using different numbers that don't add up to the same, I would say the 80-64 deserves to beat the 60-80, and the 100-48 should beat both.

 

[Skatesocs]

I'm confused about what you're saying with both of those things? In the second part you're using different numbers that don't add up to the same, I would say the 80-64 deserves to beat the 60-80, and the 100-48 should beat both.

Maybe I misunderstood you or explained badly. I'll try again.
When I read this

80-80 doesn't deserve to beat 60-100 or 100-60.

You were talking about the men's field? Because the ladies field doesn't have PCS out of 100. And then you say that 80-80 doesn't deserve to beat 60-100 or 100-60, so I assumed you meant even though they add up the same, there's a hierarchy there for you.

So I tested what would happen in the ladies field - by scaling the PCS out of 80. So that changes "80/100 PCS" to "64", and "100/100 PCS" to "80" and "60/100 PCS" to "48" - percentage values.

I kept tech the same, because ladies can achieve those exact numbers. So the grouping now becomes "100/48", "80/64", "60, 80". Unlike with the men, these don't add up the same. But we have a hierarchy where "100/48">"80/64">"60/80", even though this is simply a translation from the men's field to the ladies field, because they add up differently. Further, this definitely places "60/80" below the other two, even though what I got from your post is that "someone who does 60 on tech, but gets 100% PCS deserves to beat someone who gets 80 on tech and only achieves 80% PCS".

Please correct me if I'm wrong.

 

[jersey1302]

Im not strong in math lol so TBH this all makes absolutely zero sense to me haha. I will go back to the original post about the idea of having a GOE essentially on PCS.. problem with that is with a GOE you need concrete ways to do things etc. Artistry and presentation is completely subjective to whos watching it. The judges cant even agree more than half the time of a technical element which has very specific scoring criteria. They wouldn't be able to fairly or accurately score PCS with GOE.

What I think they could do however is possibly make the factor a bit different. PCS could be worth 1.5 x the score more than now so that it can have a greater impact on the scoring. I also think you need to have a panel of referees that have the ability to scrutinise scoring before its submitted to be announced, if scores seem completely out of wack and its clearly evident the rules of the scoring are not adhered by, they would be able to step in and essentially say hey.. whats with your score, can you justify it. This goes for PCS and TES. There have been far too many times where a single element has been scored by most judges 0-1 and then 1 or 2 judges have it listed at a +4. Same goes for PCS. An average across the board for an area of PCS would let say be 8.50- 8.75 and then one judge has them 9.50.

 

[gkelly]

What I think they could do however is possibly make the factor a bit different. PCS could be worth 1.5 x the score more than now so that it can have a greater impact on the scoring.

Yes, if indeed we want PCS to be worth more.

Or at least, it's time to raise the PCS factors to 1.0 SP and 2.0 FS for ladies and maybe 1.2 and 2.4 for men, in recognition of the great increases in jump content since 2003.

I don't know that 1.5 and 3.0 are necessary yet.

There have been far too many times where a single element has been scored by most judges 0-1 and then 1 or 2 judges have it listed at a +4. Same goes for PCS. An average across the board for an area of PCS would let say be 8.50- 8.75 and then one judge has them 9.50.

That's why the high and low scores are dropped for every element and component.

 

[Blades of Passion]

You were talking about the men's field? Because the ladies field doesn't have PCS out of 100. And then you say that 80-80 doesn't deserve to beat 60-100 or 100-60, so I assumed you meant even though they add up the same, there's a hierarchy there for you.

Oh yeah I was just talking about the numbers in general, I don't think there should be a concrete advantage one way or the other. Except in rare tiebreaker scenarios, where either the technical score or PCS score needs to be chosen to break the tie, and I really don't care either way with CoP which it is, because it's not like the 6.0 system where the numbers are much more whole. But, having technical mark as tiebreaker in the SP and PCS as tiebreaker in the LP is still pretty fitting too.

 

[Mathman]

Im not strong in math lol so TBH this all makes absolutely zero sense to me haha.

May I try?

You have two numbers, the TES and the PCS. Somehow you want to come up with an idea of a total or "average" of the two. Let's take the hypothical case where both of the two numbers are 100. There are many kinds of averages. For example.

(1) The artithmetic mean = (X+Y)/2. The average of 100 and 100 is (100+100)/2 = 200/2 = 100.

(2) The geometric mean: squareroot(XxY). The "average" of 100 and 100 is squareroot(100x100) = squareroot(10,000) = 100.

(3) The harmonic mean. 2*(1/x + 1/y)[SUP]-1[/SUP] = 2*(1/100+1/100)[SUP]-1[/SUP]= 2x(1/100 + 1/100)[SUP]-1[/SUP] = 2x(2/100)[SUP]-1[/SUP] = 2X(100/2) = 2X50 =100. (This is the method under discussion on this thread.)

Any way you look at it, if your two scores are 100 and 100, then your average score is 100.

If the two numbers are different, then you get slightly different answers, depending on the method.

 

[auser]

So true. But on the other hand, the ISU could remove the PCS multiplier anyway, whether they made any other changes to the IJS or not.

To me, the most important question that the proposal raises is this: what do we mean by "program components" anyway, and how should they be related to technical elements scores, if at all. In (current) theory, element scores are element scores, and program scores have to do, not with the individual elements performed, but with the overall excellence of the program.

If a skater does a quad, should that skater automatically get a higher score for musical intepretation than a sktetr who is equally musical but not as strong technically? Could an artistically exquisite and emotionally moving pperformance get a 9.5 even thoughi t contains only double jumps (or no jumps at all)? If a skter falls on a jump, to what extent dshoulod that reduce the choreography and presentation score? Are we double-dipping if we penalize a technical mistake by lowering the base value, AND giving negative GOE, AND lowering the compenentsd across the board?

(IIRC there was one ice dance couple who had to withdraw from the free dance due to injury. Out of respect for the audience they skated out to center ice and bowed, then withdrew. They got 0 tech and 0.25 (from one judge) for Choreography. )

Great post, Mathman. This is what bothers me most about scoring. And how the scoring of elements effects program content. So I tend to focus on the beauty of the skating and pay less attention to scores. But as this is competition, I realize my approach would not satisfy many others .

 

[Mathman]

*Start with assigning a score for whichever component makes the strongest impression first, most likely the strongest or weakest component for that performance, and then decide how far up or down to go for the other components. (This is likely to be the Skating Skills component for a couple of different reasons...

I believe that one of the reasons is simply that SS is listed first, so judges key in the SS score first, and this then becomes the benchmark for the others.

When people suggest this, I always say that the TES panel should also be tasked with scoring the Skating Skills and possibly the Transitions as well. Make these the technical judges, and let a separate panel concentrate on the performance aspects.

Are you proposing that we keep the 3-person technical panel and the 9-person judging panel as is, but let the technical panel take over the job of evaluating SS and TR in addition to their duties of calling individual elements?

To me, this goes somewhat against the idea that the great division in figure skating scoring is that between "Individual elements, like jumps and spins:" versus "Program taken as a whole." Instead, this proposal would revert to the old "Tech" versus "Artistry" dichotomy that was abandoned with the 6.0 system.

 

[Mathman]

I wouldn't say that skating is like a triathlon.

True. It's more like biathlon. TES = hitting the terget. PCS = skiing on to the next target. :yes:

 

[gkelly]

I believe that one of the reasons is simply that SS is listed first, so judges key in the SS score first, and this then becomes the benchmark for the others.

I don't know if that was the intention. But by default it has become the practice for many judges, even if some of the training recommends other approaches.

Are you proposing that we keep the 3-person technical panel and the 9-person judging panel as is, but let the technical panel take over the job of evaluating SS and TR in addition to their duties of calling individual elements?

No.

Some fans have suggested having three panels: the tech panel to call the elements (lets call it the Calling Panel), an elements judging panel that only awards GOEs, and a separate components judging panel to award the PCS.

There are other issues* with that proposal that would need to be addressed if we're going to take that proposal seriously.

But the first one I'm bringing up is that scoring GOEs only, with no input about the base values/difficulty of the elements and no input into evaluation of the program as a whole including everything that happens between the elements, is not enough for experienced judges to do.

So my proposal would be to have what we might call a Technique Judging Panel responsible for element GOEs and also Skating Skills and Transitions components, and a Performance (or Artistic, if you prefer) Judging Panel responsible for Performance, Composition, and Interpretation components. In addition to the Calling Panel, which is not changed from current practice under either version of the split judging panel proposal.

*The main other issue is that an additional panel might require additional officials, which would add expenses to running competitions.

Since elite championships tend to use 9 judges, it would be easy enough to use 5 for the tech judging panel and 5 for the performance judging panel. With 5 on each panel, you could still drop the high and low.

But less important events often use 5-7 judges. It would be harder to just split those panels in half and still have statistically robust results. And too expensive to just add more judges to all those events.

So then the decision might be to split the panels for elite events and make the judges at smaller events continue to judge GOEs and PCS at the same time.

If the argument is that judging both elements and components is too much of a burden for the judges, then why lighten that burden only for the most experienced ISU judges while requiring double duty of the less experienced international and domestic judges?

 

[Skatesocs]

True. It's more like biathlon. TES = hitting the terget. PCS = skiing on to the next target. :yes:

Haha. No, I wouldn't say it's like a biathlon either. You're supposed to hit targets while skiing. It would be a cool sport, or a video game at least, if people were supposed to hit targets while continuing to ski (you stop to shoot in a biathlon IIRC).

 

[Skatesocs]

Imagine someone trying to shoot a target while doing a level 4 step sequence that's definitely something I'd buy as a video game.

 

[Skatesocs]

If the argument is that judging both elements and components is too much of a burden for the judges, then why lighten that burden only for the most experienced ISU judges while requiring double duty of the less experienced international and domestic judges?

I must say I didn't think of it this way. Still, I think time is what's most important. And so I think it's reasonable to argue that the most experienced ISU judges are assigned to the most important competitions of the season, with much more at stake, with the task of scrutiny likely being much harder - splitting panels (and hence apportioning more time) becomes more important at those levels.