Multiplicative PCS scoring?

[lzxnl]

In the midst of the drama around the hypothetical age increases, people have generally agreed that figure skating scoring needs to be tweaked so that artistry and presentation are valued more equally and so that the ladies field isn't dominated by pre-pubescent girls whose skating's sole selling point is the number of quads. A potentially controversial idea came to mind:

What if PCS was, like GOE, one big final multiplicative factor? Convert the PCS percentage to a multiplier. For starters, it provides an alternative scoring mechanism for skaters to whom quad jumps don't come naturally. Here's some numbers.

Then, consider a fairly strong 7-triple skater with a TES of 76 and strong components of 70/80, or a multiplier of 0.875 to give a total score 67.5. Compare now with Trusova scoring a TES of 100. Suppose her components are judged a bit more harshly; 60/80. This would give her a total score of 75. Now, she still beats the 7-triple skater, which is reasonable, but the margin of victory is much smaller than under the current system, and suggests that if you want to go for just quads at the expense of presentation/artistry, you better hit them all to win. I.e. if Trusova falls on one quad, her TES drops by around 6, fall deduction of 1, presentation score will drop a bit, and so she likely isn't going to win. Thus, it would then be better for her to work on raising that component score first.

Pros:
1. More incentive to balance out work on TES and components
2. Skaters that are prodigiously talented in either aspect can still compete and score well

Cons:
1. PCS scoring can be opaque, to say the least, and easily manipulated. We would need separate PCS judges
2. Scoring at lower levels might be disproportionately affected by PCS, when tiny changes in PCS become much larger multipliers

Thoughts? Would love to hear people's comments!

 

[gkelly]

What if PCS was, like GOE, one big final multiplicative factor? Convert the PCS percentage to a multiplier.

I don't understanding how what you're proposing would work, mathematically. What is being multiplied by what?

Then use examples from a variety of skill levels, not just comparing two potential medal contenders.

How does it work for skaters who are by far strongest on jump content, on TES generally including GOEs, on PCS, or well balanced across all skills.

How does it work among skaters in each of the above categories who can do multiple quads (and triple axels) (mostly men so far) vs. those who can do all triples and 3-3 combinations up to 3Lz, vs. those who can only do a few triples, or no triples at all.

Personally, I wish there were a system that would just take into account the best part of 6.0 (ranking; fewer things to score; inherent understanding of difficulty of certain elements)

Remember if you go back to using rankings, you also go back to standings flipflopping within a competition phase (and probably even moreso during the final phase, depending how results from the two or more phases are combined).
Which means reeducating audiences about unintuitive math that was rarely well-explained in 6.0 days to begin with.

"Inherent understanding of difficulty" either requires all judges, and everyone else, to share that understanding on a very detailed level, or else to accept that there will be large differences of opinions among judges as to what was more difficult than what.

The problem with IJS in this regard is that some skills are built into the scale of values and guarantee points, whereas other skills are not explicitly listed at all and are entirely up to judges to find ways to reward, if anyone bothers to demonstrate them. Which there is therefore less incentive to do.

The problem with 6.0 in that regard was that everything including difficulty was subjective.

So create a system that ranks skaters but makes sure the difference in scores matters somehow.

Any suggestions how that might work?

 

[Skatesocs]

Hi gkelly, I deleted my response precisely because I'm trying to think about how it would work. I was planning on making a more detailed post later.

Lzxnl is proposing that TES (after it is calculated via average) is multiplied with PCS (after it is calculated via average) to give us the TSS, but that the PCS is scaled against the maximum PCS (so in his example, he does 70/80 = 0.875 - for a woman scoring 70 PCS, it is scaled against the maximum of 80 PCS she can score).

 

[Mathman]

Thoughts? Would love to hear people's comments!

Would the proposal be to eliminate GOEs and fold quality of elements into the PCS multiplier? Or would the multiplier be applied to the total TES, including GOE?

 

[Mathman]

Remember if you go back to using rankings, you also go back to standings flipflopping within a competition phase...

For some reason this has never bothered me. Flip-flopping simply reflects the fact that projected or temporary placements change when the later skaters take their turn.

For example, SP = Michelle 1st, Irina 2nd, Sarah 4th. In the free program, Sarah and Michelle go first, with Sarah beating Michelle.

If we stop right now, the factored placements are Michelle 1st, Sarah 2nd. Michelle is in first place overall, ahead of Sarah.

Now Irina skates, beating Michelle but not Sarah. Recalculating the factored placements to include this new information, Sarah flipflops over Michelle.

To me, there is nothing to see here. The only "mistake" was in announcing prematurely that Michelle is winning overall in the middle of the LP. All the announcer has to say is, we don't know who is winning until all the skaters have skated. (Oh, the excitement, oh the building suspense!)

(The other kind of flipflop, where it is just barely possible (in a slightly different scenario) that Irina could flipflop Michelle and Sarah while losing to both (but beating Sasha) is more troublesome, but quite rare.)

Anyway, if the problem is "explaining the math" the solution is don't make premature announcements. Then there is nothing to explain or to backtrack from. (Politicians, take note.) It is, however, an advantage of the CoP that running totals are meaningful, even sacrosanct. Real-time sports audiences enjoy this aspect of competition. (Unless we consider it a "flipflop" when one skater is ahead for a hot minute in the little score box, but then drops when the elements are reviewed.)

 

[Mathman]

Hi gkelly, I deleted my response precisely because I'm trying to think about how it would work. I was planning on making a more detailed post later.

Lzxnl is proposing that TES (after it is calculated via average) is multiplied with PCS (after it is calculated via average) to give us the TSS, but that the PCS is scaled against the maximum PCS (so in his example, he does 70/80 = 0.875 - for a woman scoring 70 PCS, it is scaled against the maximum of 80 PCS she can score).

I believe that diving uses this type of scoring. It might be useful to think about how figure skating is similar to or different from this sport.

 

[Skatesocs]

I believe that diving uses this type of scoring. It might be useful to think about how figure skating is similar to or different from this sport.

I'll probably look into why that works. I am also discovering that an analysis of ranking and voting is a big hole in my education :hslap::dumb:

 

[Mathman]

(C)onsider a fairly strong 7-triple skater with a TES of 76 and strong components of 70/80, or a multiplier of 0.875 to give a total score 66.5. Compare now with Trusova scoring a TES of 100. Suppose her components are judged a bit more harshly; 60/80. This would give her a total score of 75. Now, she still beats the 7-triple skater, which is reasonable, but the margin of victory is much smaller than under the current system...

The margin is smaller in absolute numbers. But that's mostly because all total scores are reduced. Since we are proposing to use multiplicative language instead of additive, I think the analysis should go like this.

Current (additive) method)

Trusova gets 100 TES and 7.5s for PCSs. Her total segment score is

100 + 60 (with the current 1.6 LP factoring) = 160.00

Other skater: 76 + 70 = 146.00

Trusova's score is 10% higher than that of the second skater.

Proposed multiplication method

Trusova: 100x.750 = 75

Other skater: 76x.875 = 66.5

Trusova's "margin of victory = 13%

The margin of victory is about the same -- if anything, it is greater than before.

It would be interesting to do a lot of examples from actual competitions and see how it works out.

 

[Arbitrary]

It's quite a challenge to create a pseudo-objective scoring system which hits only a specific skater or team without affecting others in a negative way...


Sent from my ONEPLUS A6003 using Tapatalk

 

[Harriet]

I think I'm missing the point of this suggestion, though in my defence it is quite late here. What information would this method of scoring give to the skaters about what they did well, what they did poorly and where they need to improve better than the current system? How would that information be communicated to them?

 

[Skatesocs]

I think I'm missing the point of this suggestion, though in my defence it is quite late here. What information would this method of scoring give to the skaters about what they did well, what they did poorly and where they need to improve better than the current system? How would that information be communicated to them?

Both TES and PCS are intact - so exactly the same as before.

It's quite a challenge to create a pseudo-objective scoring system which hits only a specific skater or team without affecting others in a negative way...

Why would you conclude this?

 

[McBibus]

Hi gkelly, I deleted my response precisely because I'm trying to think about how it would work. I was planning on making a more detailed post later.

Lzxnl is proposing that TES (after it is calculated via average) is multiplied with PCS (after it is calculated via average) to give us the TSS, but that the PCS is scaled against the maximum PCS (so in his example, he does 70/80 = 0.875 - for a woman scoring 70 PCS, it is scaled against the maximum of 80 PCS she can score).

If you do moltiplication of scores you can remove the actual PCS multiplier.
Actually part of the umbalancing is that BV of women is now close to men's BV, while the women PCS multiplier is stuck at when there where no quads and trixel where a rarity, so it's very far from an ideal 50/50 where the difference (in percentage) between TES and PCS are weighted the same.

I look at percentage because it reflects the real difference and not a scoring system difference.
For example if we run a 100 m and a 400 m races it will be crazy to sum the time difference because the gap over 400 meters will always be bigger that the gap over 100 meters and the winner of the 400m event will always be the final winner.

This can also apply to SP vs FS, but here I like how it is with the FS being the "big" event. Sum is ok because the scoring system is the same.
The problem with TES and PCS is that one is a cumulative (virtually limitless) score while the other is a fixed maximum score (that will never be reached)

 

[1111bm]

It's quite a challenge to create a pseudo-objective scoring system which hits only a specific skater or team without affecting others in a negative way...

I dont't think the goal of the OP was to have an objective scoring system?

If the objective of this system is to increase the value of PCS (as the OP stated), than by its very nature it can't be objective, or at least not more objective than what we currently have, since PCS is the most subjective part of scoring (which is why I don't get why people want its value even increased, but oh well...).

 

[Mathman]

I'll probably look into why that works.

I am not very knowledgable about the sport of diving and how it is scored. But my understanding is that it goes something like this. Each dive gets a difficulty score (presumably there is a standard base value listing for different type of dives, how many twists and somersaults it has, type of take-off, etc.).

Then a panel of 7 judges give the dive a "GOE" from 1 to 10. The trimmed mean is taken by eliminating the bottonm 2 and the top 2 scores. This produces a "multiplier," say 8.5. Now multiply 8.5 times the difficulty score, and that is the total score for the dive. This is similar to the proposal on this thread.

The way to think about it is (ignoring the decimal point ), 8.5 means the dive is 85% as good overall as a perfect dive would be. So the diver gets 85% of the amount he would get for a perfect dive of that difficulty.

I am also discovering that an analysis of ranking and voting is a big hole in my education.

Strangely, this topic is usually discussed not in mathematics courses but in courses on political science and economics. (Mathematically, the topic is too intractable, but it might be touched upon under the heading "non-parametric statistics.") Serious study of the topic began with political theorists in France in the aftermath of the French Revolution, when the problem arose of how to organize election systems that were as fair as possible, that provided for rule of the majority without suppression of the minority, that prevented a dictator from seizing control (as in fact Napoleon did within a few years), etc.

Along about the 1950s economists started applying the same ideas to populations of buyers and sellers instead of voters.

 

[cohen-esque]

In diving it’s:

1. Start with seven judges.
2. Trim the lowest 2 and highest 2 scores.
3. Add the remaining three scores.
4. Multiply by the “base value.”

So for instance a 4 point dive with straight 8.5s gets 25.5*4 = 102 points.

Functionally the same basic idea as MM is saying but with a little less rounding error added by the middle step thanks to adding, instead of averaging.

 

[Mathman]

If you do moltiplication of scores you can remove the actual PCS multiplier.
Actually part of the umbalancing is that BV of women is now close to men's BV, while the women PCS multiplier is stuck at when there where no quads and trixel where a rarity, so it's very far from an ideal 50/50 where the difference (in percentage) between TES and PCS are weighted the same.

So true. But on the other hand, the ISU could remove the PCS multiplier anyway, whether they made any other changes to the IJS or not.

To me, the most important question that the proposal raises is this: what do we mean by "program components" anyway, and how should they be related to technical elements scores, if at all. In (current) theory, element scores are element scores, and program scores have to do, not with the individual elements performed, but with the overall excellence of the program.

If a skater does a quad, should that skater automatically get a higher score for musical intepretation than a sktetr who is equally musical but not as strong technically? Could an artistically exquisite and emotionally moving pperformance get a 9.5 even thoughi t contains only double jumps (or no jumps at all)? If a skter falls on a jump, to what extent shoulod that reduce the choreography and presentation score? Are we double-dipping if we penalize a technical mistake by lowering the base value, AND giving negative GOE, AND lowering the compenentsd across the board?

(IIRC there was one ice dance couple who had to withdraw from the free dance due to injury. Out of respect for the audience they skated out to center ice and bowed, then withdrew. They got 0 tech and 0.25 (from one judge) for Choreography. )

 

[Blades of Passion]

As was pointed out by Mathman, a multiplicative system doesn't necessarily do anything.

The PCS just need to be worth more. Increase the current factoring from .8 ---> .9 in the SP and 1.6 ---> 1.8 in the LP.

Jumps also need to be given less points, they get too much via GOE right now.

 

[Skatesocs]

I don't think the multiplicative system works, BUT I also don't believe the additive system would work if you just increased the PCS factoring, especially if the judges aren't good. I'm curious why people think so actually. You can make a philosophical argument for how PCS and TES should be "balanced" I guess... But Trusova was already getting 68 PCS, IIRC? She'll just get 68*1.125 = 76.5.

TES kept same, all the margins due to PCS will just become 1.125 times greater. That's... not much? But then I have proved in multiple threads in two days that my brain is dead for now :hslap:

One thing that *has* passed my mind before is what would happen if we made the judges give scores out of 20 for each component - and allowed gradation of 1 point. I think it'll change how they score.

 

[Mathman]

TES kept same, all the margins due to PCS will just become 1.125 times greater. That's... not much?

Every little bit helps, I guess.

At the very top level a really fine performance might get 9.5, while a blah one (by a top-ranked skater) will get 8.5. This gives the first skater an 8 point advantage, which would increase to 10 with the higher factoring. So about an extra 2 points gain for the artist trying to catch the technician.

One thing that *has* passed my mind before is what would happen if we made the judges give scores out of 20 for each component - and allowed gradation of 1 point. I think it'll change how they score.

I doubt that much would change. This proposal would give the judges 20 gradations to work with instead of the current 40. The argument has been made, how can a judge possibly decide whether a performance is worth 4.75 points or only a piddling 4.5, and the same argument would question whether a judge can consistently discriminate between a performance worth 9 points out of 20 compared to one that deserves only 8 points out of 20, in musical interpretation, say. (Psychologists and learning specialists assert that 7 gradations is the most that humans are capable of handling -- that's why GOEs from -3 to +3 were so cool. A typical human mind can tell the difference between a +1 performance and a +2, on a scale from -3 t0 +3; but that same person cannot tell the difference between +2 and +3 on a scale from -5 to +5.

By the way, the last time this question came up on the board, GKelly gave an awesomely convincing argument why this is all wrong and, when applied to figure skating, the 10 point scale with gradations of a quarter of a point in each f five components is not only OK but in fact is just what the doctor ordered.

 

[gkelly]

If you multiply an open-ended score times a capped score, at the top of the field won't the skater who excels in the areas covered by the open-ended score always have an advantage over one who excels in the areas covered by the capped score? The same is true for adding, but multiplying might just compound that advantage. It certainly wouldn't eliminate it, at least not at the upper end of the scores.

I don't understand why one would want to multiply scores for one kind of skills times scores for a different kind of skills. It seems to make as much (little) sense as multiplying the spin scores times the jump scores, or the SP scores times the FS scores. They're measuring separate things. So what would it mean to multiply them?

Scoring systems for sports like diving, or gymnastics vaulting or any other sports scored in terms of discrete elements, could be useful models for scoring individual skating elements. It doesn't make much sense for scoring skating programs, because those sports don't have anything comparable to what is being measured in the PCS (or most of what was measured in the 6.0 Presentation score).

I don't think the multiplicative system works, BUT I also don't believe the additive system would work if you just increased the PCS factoring, especially if the judges aren't good.

Yes, the validity of these qualitative scores certainly depends on the ability of the evaluators to do a good job of evaluating.

And how do we evaluate how good a job they're doing?

We'd need not just a workable rubric for evaluating the quality of the skating performances, but another rubric for evaluating the quality of the judging.

And the best people to evaluate how "good" the judging is would be people who have done that job and done it very well, and who are as unbiased as humanly possible.

But how do we identify who are such unbiased expert evaluators if we are not equally or more expert ourselves, if we are subject to our own biases, and if as yet the tools that we want such experts to develop have not yet been developed for us to use to identify them?

One thing that *has* passed my mind before is what would happen if we made the judges give scores out of 20 for each component - and allowed gradation of 1 point. I think it'll change how they score.

That could be interesting.

With more numbers available, it would be easier to reserve scores above 19 for truly exceptional examples even among the best of the best and plenty of scores for judges to play with in the 16s, 17s, and 18s for medal contenders, while remaining in double digits even for the lower ranked seniors.

The factors and/or the number of different components would also need to change to remain balanced with TES similarly to how they are now, or to improve the balancing.

Because we ten-fingered beings tend to think in decimal terms, we could achieve the same effect by leaving 10 as the upper limit per component (assuming an upper limit is desired), allowing judges to use increments of 0.05 (finer distinctions than the current 0.25), and doubling the PCS factors (so that 9.65 x 2.0 would work out to the same as 19.30 x 1.0).

ETA:
Oh, or did I miss understand, and you don't want to allow individual judges to use decimal places at all in the 20-point scale? In that case, ignore everything I just said and refer to Mathman's post above.