Judging math details | Page 6 | Golden Skate

Judging math details

GGFan

Record Breaker
Joined
Nov 9, 2013
Statistically, the correlation between SS and the other 4 is the highest among the 5 components. If you are given the SS score you can predict the other 4 with great confidence. I think that thiis is because the judges enter the SS score first. In any kind of exercise of this type, the judges will enter one mark and then will key on that mark for the rest. It is like a political poll. You ask a leading question first, then the answers to the rest of the questions will all tend to turn out the way you want them to. IIRC the ISU did some experiments in which the judges entered some other component first. Whichever was entered first had the greatest predictive power for all the rest.

It anchors the rest. The order could play a part and/or the importance of skating skills in the minds of the judges. Both can work together to increase the anchoring, which depending on what the system is trying to achieve may be a good or bad thing. If skating skills are the base for everything then it makes sense. My only quibble is that this has never been articulated as such.
 
Joined
Jun 21, 2003
Which mathematical point of view is important to you regarding numbers in PC?

Speaking for myself, the "mathematic point of view that is important to me" is this: I hate it when poor little mathematics is abused. :) I do not like it when numbers are coerced against their will in an attempt to prove something when in fact the numbers do not support the hypothesis. I do not like it when people use numbers to claim a faux accuracy and then use these numbers to pull the wool over peoples' eyes.
 
Joined
Jun 21, 2003
My only quibble is that this has never been articulated as such.

Actually, I think that this point has been articulated when this subject comes up (as it does every month or so.:) ) If you have good skating skills then you can use those skills to do difficult and varied transitions. You can use the same skills to translate the music into movement, to perform with confidence, etc. It is no surprise that someone with good SS is equally good at everything else.
 

GGFan

Record Breaker
Joined
Nov 9, 2013
Actually, I think that this point has been articulated when this subject comes up (as it does every month or so.:) ) If you have good skating skills then you can use those skills to do difficult and varied transitions. You can use the same skills to translate the music into movement, to perform with confidence, etc. It is no surprise that someone with good SS is equally good at everything else.

No, I mean it has not been articulated that the SS component is the most important one and explicitly influences the others. I totally understand how it can be the base for everything else.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Country
Saint-Barthelemy
Speaking for myself, the "mathematic point of view that is important to me" is this: I hate it when poor little mathematics is abused. :) I do not like it when numbers are coerced against their will in an attempt to prove something when in fact the numbers do not support the hypothesis. I do not like it when people use numbers to claim a faux accuracy and then use these numbers to pull the wool over peoples' eyes.

I agree with that. But also, you must keep in mind that rules defined their way of using numbers and math there, and in many ways they never claimed some of the things people think of them. Example: If Zhenia got 9 for SS and Caro 8.75 in one competition, that doesnt mean Zhenia is better in SS than Caro, and better for 0.25, which will be the most common conclusion in peoples minds. In the Rules' reality numbers mean that Zhenya that day fullfill more judges criterias defined in SS- lets say 9 of 10 and presented them more seconds of her programme, while Caro fulfill only 6 of them, not with that amount of duration, but all of them with exceptional quality. Rules are not saying that with that marks Zhenya generally has better SS than Caro. Numbers here doesnt have pretension to decide who is 'best skater', just who is the winner of some competition. So sometimes the one who is reading the numbers is one who is abusing them, not the one whos using them :biggrin:
 
Joined
Jun 21, 2003
I agree with that. But also, you must keep in mind that rules defined their way of using numbers and math there, and in many ways they never claimed some of the things people think of them. Example: If Zhenia got 9 for SS and Caro 8.75 in one competition, that doesnt mean Zhenia is better in SS than Caro, and better for 0.25, which will be the most common conclusion in peoples minds. In the Rules' reality numbers mean that Zhenya that day fullfill more judges criterias defined in SS- lets say 9 of 10 and presented them more seconds of her programme, while Caro fulfill only 6 of them, not with that amount of duration, but all of them with exceptional quality. Rules are not saying that with that marks Zhenya generally has better SS than Caro. Numbers here doesnt have pretension to decide who is 'best skater', just who is the winner of some competition. So sometimes the one who is reading the numbers is one who is abusing them, not the one whos using them :biggrin:

That's a good point. However, I still think that it is tricking people to say that Zhenya got 9 and Caro got 8.75, but this does not mean that Zhenya exhibited better skating skills. The average person would think that is does mean that Zhenya skated better: 9.00 beats 8.75.

On a separate topic, if we look at the criteria for the five components

http://www.usfigureskating.org/content/ISU program-component-chart_sandp-and-id_08-16.pdf

it makes me wonder why Composition and Interpretation are two separate categories.

Composition: Phrase and form (movements and parts of the program to match the musical phrasing)

Interpretation: Expression of the music's character, feeling and rhythm, when clearly identifiable.

And so on. Actually this criterion under Interpretation is kind of funny. The ISU seems to be saying, most skating music does not have any identifiable character, feeling or rhythm -- but if it does, the skater should respect it.

There is another problem with the Composition (Choreography) component. The actual choreography is done by someone else. A criterion like "multidimensional use of space and design of movements" is the choreographer's work, not the skater's. What the skater should get credit for is being able (thanks to her Skating Skills and talent for Interpreting Music) to perform complex choreography and fully to commit to the music while doing so. I do not really see the value in having a separate Composition component.
 
Last edited:

moriel

Record Breaker
Joined
Mar 18, 2015
For some of you those are just numbers with no significant difference between them. But in this case your view is not really important. For skaters who of course knows what their range of skating is, those could be significant informations. In example of Wakaba who knows that her range of skating is lets say 7.25 to 9, difference between 8.2 and 8.65 (as scores she got) may be big, as also the difference between 8.2 and 9 (her own maximum).

There is literally no significant difference between those numbers. By definition of significant difference.
This is not a point of view, this is a fact.
 

moriel

Record Breaker
Joined
Mar 18, 2015
Speaking for myself, the "mathematic point of view that is important to me" is this: I hate it when poor little mathematics is abused. :) I do not like it when numbers are coerced against their will in an attempt to prove something when in fact the numbers do not support the hypothesis. I do not like it when people use numbers to claim a faux accuracy and then use these numbers to pull the wool over peoples' eyes.

And this is why i hate 2 digit precision in FS. It is entirely misleading and tends to create a sense of "exactness" - people really think that there is a difference between 8.1 and 8.2, while its the same difference as between 8 and 8.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Country
Saint-Barthelemy
On a separate topic, if we look at the criteria for the five components

http://www.usfigureskating.org/content/ISU program-component-chart_sandp-and-id_08-16.pdf

it makes me wonder why Composition and Interpretation are two separate categories.

Composition: Phrase and form (movements and parts of the program to match the musical phrasing)

Interpretation: Expression of the music's character, feeling and rhythm, when clearly identifiable.

And so on. Actually this criterion under Interpretation is kind of funny. The ISU seems to be saying, most skating music does not have any identifiable character, feeling or rhythm -- but if it does, the skater should respect it.

There is another problem with the Composition (Choreography) component. The actual choreography is done by someone else. A criterion like "multidimensional use of space and design of movements" is the choreographer's work, not the skater's. What the skater should get credit for is being able (thanks to her Skating Skills and talent for Interpreting Music) to perform complex choreography and fully to commit to the music while doing so. I do not really see the value in having a separate Composition component.

Skater interpret the music in terms that skates faster with faster bit, stop when its pause in the music, use whole body to express rythm changes (not just parts of it), basically interprets tacts and tempo of the music. For composition is more important to express type of music - dark/light music through your character, using some nuncies, facial expression and gestures and covering the whole ice rink with that. Thus, the impression that skater can fill the rink is important for CO, and skater can achieve that also through more other things. You must keep in mind that ice rink is the place where actual performance is happening (not the TV :biggrin:). Even programmes are choreographed in a way that when skaters perform them should look the same(as best possible), they dont always look the same in ice rink from competition to competition - cause all ice rinks are not the same, skaters dont skating always with same speed, flow and acceleration, mistakes can change the places of elements in the ice and timing of them, some skaters some of their harder elements are placing only in one corner of the rink and when they feel unsecure tends to skate more to the 'safe part of the rink'. That will make choreographed pattern to look less good/more simple/empty/with no covering of full ice rink. Those kind of perceptions of skaters motions through the ice surface and ice rink, basically the usage of place where skater performs is judged in Composition mark and what makes it distinctive from IN.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
That's a good point. However, I still think that it is tricking people to say that Zhenya got 9 and Caro got 8.75, but this does not mean that Zhenya exhibited better skating skills. The average person would think that is does mean that Zhenya skated better: 9.00 beats 8.75.

Well, there used to be one mark that covered
*Harmonious composition of the program as a whole and its conformity with the music chosen;
*Variation of speed;
*Utilization of the ice surface and space;
*Easy movement and sureness in time to the music;
*Carriage and style;
*Originality;
*Expression of the character of the music

Did a higher score for Presentation under 6.0 necessarily mean that the skater was more artistic or better at all those criteria? Of course not. Just that in that performance, they met more of those criteria better.

It was very possible for a skater to be significantly stronger at some of those criteria than others, which is why it made sense when designing the new system to break them apart into separate scores and to expand the definitions of some of those scores. Along with making scores for technical skills demonstrated throughout the program including outside of scored elements.

But any time there's more than one criterion for a single score, there will be times when the skater who is weaker at the criterion that's most important to you is enough stronger in the other areas to earn a higher score. E.g., sometimes difficulty will trump quality, and sometimes vice versa, in SS or TR . . . or CO or IN, for that matter.

On a separate topic, if we look at the criteria for the five components

http://www.usfigureskating.org/content/ISU program-component-chart_sandp-and-id_08-16.pdf

it makes me wonder why Composition and Interpretation are two separate categories.

Composition: Phrase and form (movements and parts of the program to match the musical phrasing)

There's a lot more to Composition than that, though.

Interpretation: Expression of the music's character, feeling and rhythm, when clearly identifiable.

And so on. Actually this criterion under Interpretation is kind of funny. The ISU seems to be saying, most skating music does not have any identifiable character, feeling or rhythm -- but if it does, the skater should respect it.

Yes, that is funny.
Which would be worse, from Interpretation point of view? Choosing music that doesn't give much to interpret, or choosing music that does have a clear character or rhythm and then ignoring it?

Either way, though, don't expect very high scores in this component.

There is another problem with the Composition (Choreography) component. The actual choreography is done by someone else. A criterion like "multidimensional use of space and design of movements" is the choreographer's work, not the skater's.

To some degree, yes.

If the choreographer asks for a big reach up here and a tightly curled-in movement there, but the skater can only achieve a slight upward gesture and a slightly inward gesture, that's on the skater. If they're really bad at executing, it might not even be possible to understand what the choreographer intended.

If the point of this component is to demonstrate ability to control the body and the skates in a variety of purposeful ways in support of a concept or theme, and the skater does not demonstrate much of that ability (whether because they are unable or because the program doesn't ask that of them), they shouldn't score very high.

But yes, if they execute well enough for the judges to tell what was intended, they can reward that intention even it wasn't the skater's own work.

Also, different skaters are involved in their own choreography to different degrees. Sometimes a lot of the ideas do come directly from the skaters themselves. Or they show the choreographer what they can do and the choreographer builds the program around the skater's strengths.

The biggest place where someone else's work would be rewarded would be if the choreographer is especially skilled at camouflaging weaknesses. Or has such a clever concept that even a half-hearted execution would be enough for judges to be impressed and increase that score.

What the skater should get credit for is being able (thanks to her Skating Skills and talent for Interpreting Music) to perform complex choreography and fully to commit to the music while doing so. I do not really see the value in having a separate Composition component.

So under which component would you reward the "Harmonious composition of the program as a whole" and "Utilization of the ice surface and space" criteria that had been part of the second mark considerations long before IJS?
 
Joined
Jun 21, 2003
So under which component would you reward the "Harmonious composition of the program as a whole" and "Utilization of the ice surface and space" criteria that had been part of the second mark considerations long before IJS?

We could ould put it in the combined "Interpretation and Choreography" component.

But I have an even better suggestion. If we look at the bottom of the official chart

http://www.usfigureskating.org/content/ISU program-component-chart_sandp-and-id_08-16.pdf

we see

7 Good
8 Very good
9 Outstanding
10 Outstanding

There seems to be some ambivalence about what score "outstanding" deserves, a 9 or a 10.

I propose to clarify this as follows:

7 Good
8 Very good
9 Outstanding
10 Over-scored
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Country
Saint-Barthelemy
There is literally no significant difference between those numbers. By definition of significant difference.
This is not a point of view, this is a fact.

How on earth when differential (max possible difference) is 2,5 points or 2 points, difference of half point is not significant. Its 20% or 25% of max possible difference.
 
Joined
Jun 21, 2003
How on earth when differential (max possible difference) is 2,5 points or 2 points, difference of half point is not significant. Its 20% or 25% of max possible difference.

You and Moriel are using the word "significant" in different senses. If you have any collection of numbers, from whatever source, there are standard statistical tests for determining whether the difference between two statistics derived from these numbers are "(statistically) significantly different" or whether the difference fails the appropriate test of significance. Typically, for instance one might use the "95% level of significance": if the test fails, then "we cannot be 95% confident that the numbers represented by the two estimates represent true differences in the underlying parameters," as opposed to us being fooled by the random noise (sampling error) that always exists in any statistical setting.

The advantage of using these statistical tests is that they apply universally no matter the setting.

Putting statistics aside, we have the informal definition of "significant" that is found in the dictionary and used in ordinary discourse: important, conveying valuable information,"it matters" (to someone) etc.When you argue that the scores, even when they are close, still convey useful information to the skaters, you are using this second definition of the word "significant." It is like if you say to your friend:

You matter.

But if you accelerate yourself up to the speed of light, then you energy (physics nerd joke -- sorry abut that ;) )
 
Last edited:
Joined
Jun 21, 2003
The second reason why this discussion is going nowhere is that we do not agree on the question of what the purpose is of giving scores in the first place. One view (the one that I hold) is that the purpose of scores is to determine who wins. Whoever has the highest score at the end of the day is the winner, just like whichever soccer team scores the most goals is the winner.

If you hold this view then you are attracted to the idea that we might as well combine all the five program components into one, since this produces the same total score and the same winner.

The alternative to this view is that we want the scoring system to do more that just decide who won, who got second, etc. Maybe we think that the purpose of the IJS is to give feedback to skaters about how to improve their performances. Maybe we think that the purpose of the IJS is to educate audiences as to how figure skating is judged, or to provide transparency to fans about what the judges are thinking about when they render their scores.

I can appreciate all this. But I do not believe that any such consideration can possibly rival in importance the basic requirement that the scoring system determine the true winner of the contest.
 
Last edited:

moriel

Record Breaker
Joined
Mar 18, 2015
How on earth when differential (max possible difference) is 2,5 points or 2 points, difference of half point is not significant. Its 20% or 25% of max possible difference.

Where do you see a difference of 2,5 points or 2 points?
Please link protocols, I never seen anything like that.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Country
Saint-Barthelemy
The second reason why this discussion is going nowhere is that we do not agree on the question of what the purpose is of giving scores in the first place. One view (the one that I hold) is that the purpose of scores is to determine who wins. Whoever has the highest score at the end of the day is the winner, just like whichever soccer team scores the most goals is the winner.

If you hold this view then you are attracted to the idea that we might as well combine all the five program components into one, since this produces the same total score and the same winner.

The alternative to this view is that we want the scoring system to do more that just decide who won, who got second, etc. Maybe we think that the purpose of the IJS is to give feedback to skaters about how to improve their performances. Maybe we think that the purpose of the IJS is to educate audiences as to how figure skating is judged, or to provide transparency to fans about what the judges are thinking about when they render their scores.

I can appreciate all this. But I do not believe that any such consideration can possibly rival in importance the basic requirement that the scoring system determine the true winner of the contest.

Of course its determine the winner. Winner is the one who satisfies rules and game the most in one exact competition. But its not determine 'true winner' of figure skating. And it cant... True winner can be someone totally else who for different individuals satisfied different categories better, everybody can have their own 'true winners'. Winner of the competition is the one who is awarded with gold (because he is best competitor of the day or he better play by the rules or). Im just saying that we (in any sport) cant generalize the winner of the competition out of context of exact competition and competitions rules. EDIT: And i dont think the point of the competition is just to determine the winner, but also for the audience to 'enjoy the game'. Athletes are not there only for themselves (and their own goals), but also for the public (and public goals). So, does public watch Cristiano Ronaldo just because they want to see how his team won, or because they enjoy his way of playing the game?
 

moriel

Record Breaker
Joined
Mar 18, 2015
Now, i cannot see any true winners except the audience maybe.
Else, the definition of winner is kinda simple to me: its the person who won the competition according to a set of rules.
I see how it can be debatable if the rules are vague, or are just guidelines like the Brethren Code, or are not entirely followed.
 

bobbob

Medalist
Joined
Feb 7, 2014
I suggest we go back to the 6.0 scoring system, with a few modifications put in place. It would put more of the focus on the skating, instead of nitpicking small things and scores, and judges have the ability to judge things in the complete context of the skate. Sometimes, we can intuitively tell which of two performances we prefer...judging each individual element may not come to the same result...the base values and GOE weightings and PCS weightings are arbitrary numbers to begin with...who is to say a 3F is 5.3 while a 3Lz is 6.0...to some a flip might be harder than a lutz...who said this jump GOE is factored by 0.7 and that one by 1 and that spin by 0.5....why isn't a fall -2 instead of -1???/

In the long run, (and we are getting close to it) the PCS scores will become so close to perfect that there will be no choice but to go back to a "ranked" scoring system. The TES too, with its constant tweak-ages, might as well, too. The question is how be can best implement it to avoid the pre-2002 issues. To begin with, judges that are not sent by countries with a vested interest that could politicize and advocated for skaters.

In the end, there is no perfect judging system. Each judge, each person, each viewer can only say which skate they preferred in several different categories, and I think a bigger picture scoring system may solve such problems the OP mentioned.
 

Miller

Final Flight
Joined
Dec 29, 2016
Thank you all for replying for my posts and pointing me in the direction of what the components really mean, very interesting (and thought provoking).

I’ve had a look at the Ladies LP at CoC as promised and come up with revised PCS scores based on a 3 component SS, TR and combined PE/CO/IN one, with ratios 25%, 25%, 50%. Results were surprising, but then not surprising once I’d thought about them. BTW if you wanted to try it for yourself the new PCS would (SS * 2) + (TR * 2) + (Avg of PE, CO and IN * 4).

Results were Alina Zagitova 68.14 vs 68.35 actual, Wakaba Higuchi 67.74 vs 67.89, Mai Mihara 64.58 vs 64.74, Marin Honda 64.34 vs 64.58, Elizaveta Tuktamysheva 63.38 vs 63.54, Elena Radionova 66.76 vs 66.91, Gabrielle Daleman 65.08 vs 65.13, Xiangning Li 52.05 vs 52.19, Dabin Choi 52.83 vs 53.01, Amber Glenn 53.91 vs 53.87, Ziquan Zhao 46.43 vs 46.57.

As you can see all values have dropped except for that of Amber Glenn, and the range in differences is -0.24 for Marin Honda to +0.04 for Amber Glenn, a new spread of 0.28 marks, something which I would have thought was acceptable.

The reason for the fall for most skaters BTW is the % for PE/CO/IN has dropped from 60% to 50% in the new system, and the values of these tend to be towards the top of the range, hence their less weighting results in a generally lower PCS. Amber Glenn’s has gone up because her transitions were relatively high, and the % of these has gone up from 20% to 25%, plus she had a low PE meaning her average of PE/CO and IN was relatively low.

In summary I think the above proves you could go to a reduced 3 component system. The spread of PCS hasn’t changed by much, SS and TR could be input as now, and all you would need is an accurate PCS figure for the PE/CO/IN part that took account of the reduced PE that can sometimes occur - it would appear as in earlier posts that CO and IN are always very, very similar to each other, max spread 0.25 across all 40 skaters/couples at CoC, a difference of 0.3 marks in the LP for Dance, 0.4 for ladies/pairs, and 0.5 for men – has there ever been a case where the difference has been any size at all (as above 0.25 was the biggest difference)? Also it would allow for a system that allowed more separation to occur – the thought that you get an ‘anchor’ SS that in turn determines the other values seems very likely IMO, but with 3 values it might be less so, plus it might be a bit easier for the fans to follow.

One final thought. It would very be interesting to see how you guys would rate Wakaba’s and Alina’s programs at CoC. There was quite a bit of controversy over whether Wakaba or Alina should have won with the various arguments being basically based on PCS. Personally I think both programs are brilliant and would have them scored much higher in PE/CO/IN now that I’ve got a clearer picture, something like 9.50’s for Alina, and 9.75 or even 10’s for Wakaba, she really is a Bond Girl, femme fatale variety. Only reason I would have Alina lower is that her skating skills aren’t strong enough to get the absolute most out of the program e.g. after her final 3F she should really be accelerating at full speed in time with the music for her final 2A which she then takes just a touch of care with. I’d really be interested to see what you guys think because I think both are brilliant programs.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Country
Saint-Barthelemy
Where do you see a difference of 2,5 points or 2 points?
Please link protocols, I never seen anything like that.

You werent listening quite well. I said that in skaters own range of possible PCS he/she may get, and which is around 2 points, 0.5 difference in scores is big enough. That point range can change throu the years and with changing of programmes/their own skating, but at one point of the time when skaters need to compete is more or less defined. Every skater knows (by getting feedbacks) which marks can expect in PCS, he cant get 5 in one category and 9 in another, or 5 in one competition and 9 in another, because knows skills he/she has in that programme he/she skates again and again are for example from 7,5 to lets say 9 at his/her best, but also lets say may drop to min of 6.5 is he/she skates really bad. So, in that point range where every individual skater could possible get scores, 0.5 difference between categories and between one score in category and his/her own potential max is big. That 0.4 difference is significant for that individual skater in any possible universe, including statistical one, because sample of scores for individual skater is not 10 point range, but 2 point range.
 
Top