"Your" scoring of the 2016 Worlds' Ladies

[gkelly]

So, anyone who wants to discuss this, please don't point at the ISU guidelines as if that is some kind of iron-clad argument. Those guidelines aren't meant to be exactly followed like a lemming and they are incorrect in the first place. We are trying to talk about how things should be objectively scored here.

Well, since your definition of "objectively" is "what Blades of Passion believes," anyone who disagrees with you is wrong, and any opinion you hold is a fact, it's impossible to have a conversation with you.

A sport that values and rewards so many different criteria and so many inherently qualitative criteria as figure skating does will always involve some subjectivity -- in visual and aural perception of the skating, in opinions about how to weight different criteria, in defining the borderline between "good" vs. "adequate." That is precisely why it is important for quality to be scored by multiple judges, and why it's interesting to me to share and respect evaluations between different posters.

If recognition that evaluations will differ is not welcome in this thread, we can start another thread to discuss the scoring of the ladies event or other events without pretensions of objectivity.

 

[Blades of Passion]

Well, since your definition of "objectively" is "what Blades of Passion believes," anyone who disagrees with you is wrong, and any opinion you hold is a fact, it's impossible to have a conversation with you.

That is not what was stated at all. You've conveniently ignored all of the many points I just made and I am quite happy you have shared your viewpoint. If you still want to disagree then you may do so. However, it is a "fact" that bigger jumps are more difficult. It takes more strength/energy to launch yourself into the air for them, it makes them harder to control, and it makes them harder to land. Similarly, it is factually more difficult to have less pre-rotation on a jump and to complete the rotation entirely in the air. These things are indisputable and the scoring system does not reflect it, thus the scoring system is currently wrong objectively.

 

[GGFan]

Given the range of skills and qualities that figure skating has valued, what kind of system do you think cognitive scientists would recommend to account for as many of those qualities as possible?

I am certainly not a cognitive scientist and have not devised a new system, but I think you could probably follow certain principles. And remember that this is not about cheating, which I think is a separate problem. For example:

1. Having judges focus on one or two aspects of skating. I do not think just splitting the panel is enough, since no one judge could handle all of the PCS categories. Rather they should further break down those PCS categories amongst the judges. I think a judge could focus on footwork and transitions, for example (I am not an expert on all the details).

2. Admit there's a margin of error. While you have to make these judgment calls and give out medals, I would assume that differences under 3 points or so are meaningless. Embrace the subjectivity.

3. Correct me if I'm wrong but given the complexity of the system, I would guess that what happens is that the judges rely on prejudging (practices, prior competitions, reputation). If that's the case they could bring it into the open. Assessments could be made to let us know ahead of time to let us know what the maxima are in several categories (like in the old gymnastics system). For example, Hanyu might have a skating skills maximum of 9.5 and Max 7.5. Judges would then be limited to that maximum and deduct from there. I actually do not mind prejudging because there is less time pressure. I just want it to be transparent.

4. There should be efforts to standardize scoring across the system. There has to be statistical analysis, with judges retrained, suspended, and fired if they're not in line. After each competition some judging committee should have to score random samples of elements. This should be compared to the judges' actual scored and judges who are not within range should face consequences.

These ideas are still half baked. My bigger point is that there is a lot of science and expertise out there that is completely ignored.

 

[Li'Kitsu]

I have to admit that when I play armchair judge and try to score different skates for myself, I don't necessarily stick to the GOE guidelines either. They're a good "starting point" of what to look for, but I don't agree with some of them completely, just as I don't always agree with the GOE handed out by the judges (which is basically why I try to check what my own scores would be anyway).
Number 4 is one of those for me: "4) good height and distance". I'd rather judge those apart and not squeezed together into one bullet point. There are some skaters who have good height, but not the best ice coverage (I think Maxim Kovtun is a good example), and there are others who cover a lot of ice but don't get good height (Julia Lipnitskaya's 2A for example was always a lot 'longer' than it was 'high'). And then there are some skaters who have both. So how do those that get both actually get rewarded for that additional quality compared to those who have 'only' height or ice coverage? Do you give out 'half' points, or is it bad luck for those who put in the extra effort?
If you already judge a skate for yourself, I think it's fine to not take the guidelines at face value and re-evaluate at least up to a certain level what you think should be rewarded and what shouldn't. Yes, it adds more subjectivity, but as long as you can still reason why you did what you did, I think it is no less valid than completely going by the GOE guidelines.
(And with that I'm not saying people going completely by the GOE guidelines are wrong. I'm just trying to explain what I do and why I do it ).

I'm also in the same group as BoP I guess when it comes to how strict I am with handing out +3's. There is certainly a level were you can be too strict, but I really want +3's to be reserved for stand-out elements and highlights. There is a long range of possibilities with this though before it becomes scoring inflation (I didn't think gkelly was giving out +3's that easily either, personally). The main problem here is rather to stay consistent with these criteria between all the different skaters (and I don't find that an easy task, even without any biases involved).

 

[drivingmissdaisy]

This is where we are having a fundamental difference and it's part of what causes so much scoring inflation. When you have a maximum score possible for an element, handing out that maximum score NEEDS to specifically be a statement of "this was one of the best executions ever for this element." If you aren't judging like that, then there is no real differential in place. If skater A is receiving a +3 for an element that is significantly inferior to the way skater B executes that element, then where is the reward for that obviously superior element from skater B? What incentive do they have to keep executing it that well (and thus create a special WOW moment for us all to appreciate)?

I understand your viewpoint but I am inclined to agree with gkelly here. For me it comes down to the limited number of marks that a judge can give a skater. The lower marks (0 and below) tend to be reserved for a flawed or mediocre jump. That leaves +1 or +2 to differentiate a wide array of well-executed elements if the expectation is that +3 means one of the best elements executed ever. Two marks, or even three, aren't enough IMO but I think if three are available the judges are better able to differentiate between good and great elements.

 

[Blades of Passion]

For me it comes down to the limited number of marks that a judge can give a skater. The lower marks (0 and below) tend to be reserved for a flawed or mediocre jump. The leaves +1 or +2 to differentiate a wide array of well-executed elements if the expectation is that +3 means one of the best elements executed ever.

Two points here:

1.) A GOE grade of 0 is supposed to mean an acceptable element with a baseline of decent quality. Anything less should be -GOE. The problem with the way the judges have been handing out GOE is not just the overabundance of +3's, but that all of the GOE's are generally too high. It's like a +1 GOE is considered the "base mark" much of the time. And then we have situations where perfectly fine/good jump combinations get slammed with -GOE just because one of the jumps was underrotated. Very disconnected from reality.

2.) This is why the ISU will hopefully be implementing the more specific GOE range.

 

[gkelly]

The planned footwork sequence is very good on its own technically, but too much of the timing is missing.

Too much for you. For me, she was with the music notably more of the time than she was not, more than enough to qualify for this bullet point by my standards. I see so many bad and mediocre elements that I choose to reward good ones. If you rarely watch non-elite skating, you may have a different perspective.

So you can not honestly give this sequence the bullet point of "enhances the musical structure".

Are you calling me dishonest?

However, it is a "fact" that bigger jumps are more difficult. It takes more strength/energy to launch yourself into the air for them, it makes them harder to control, and it makes them harder to land. Similarly, it is factually more difficult to have less pre-rotation on a jump and to complete the rotation entirely in the air.

It is also a fact that it is more difficult to do a jump with a difficult entry or air position, and it is factually more difficult to reverse direction and execute one or more turns on the same foot you've just landed a jump on than it is to hold the back outside edge for the same amount of time (or, of course, less time).

Of course there will be some disagreement as to what variations qualify as sufficiently difficult to deserve a reward, or whether ugliness (which is often in the eye of the beholder) should partially or completely negate any rewards for added difficulty.

You have certain qualities that are most important to you, that you believe should be rewarded most highly or that must be present before you would even consider awarding specific high scores. Other qualities that have historically been valued in the skating community, some of which have rewards built into the IJS and others that are not officially specified, that are not important to you, so you never reward them.

That's your prerogative.

But those are your preferences, your personal weightings as to what should carry more or less weight. They are not any more or less "objective" than any other experts'.

The sport is complex. There will be honest disagreements.

 

[anyanka]

Give Mao the 10 points that were docked from her technical score (dropping from 75 to 65??) and push her to about 210 overall, which should put her in 4th or 5th. The end.

 

[boskil]

06.) ASHLEY WAGNER - 131.16 points [...]
Skating Skills - 8.0 / Well, she tries.
Skating Skills - 8.25 / Not the best edges, but solid ability.

Yet judges rated her skating skills highest overall with a unmatched 9.07(!) in FS.
They sent the message to all young aspiring skaters out there: watch, learn and move your blades like Ashley Wagner, her skating skills are today's benchmark. :hopelessness:

I'm not naive, I know this is figure skating, you mess with marks all the time, you mess with placements all the time, it's part of the sport and you need to accept it as a fan, but of all the skaters and all the components in the world, you pick Ashley Wagner's skating skills to boost the score and set the standards by the way?
Good lord have mercy on us.

 

[Mathman]

I understand your viewpoint but I am inclined to agree with gkelly here. For me it comes down to the limited number of marks that a judge can give a skater. The lower marks (0 and below) tend to be reserved for a flawed or mediocre jump. That leaves +1 or +2 to differentiate a wide array of well-executed elements if the expectation is that +3 means one of the best elements executed ever. Two marks, or even three, aren't enough IMO but I think if three are available the judges are better able to differentiate between good and great elements.

I agree with this. Think about it. There are only seven possible GOEs. If there are 8 skaters or more, then at least two skaters will get the same GOE regardless of whether one was better than the other on this element.

This effect is mitigated by having 9 judges and by giving GOE on many elements in a program.

 

[Blades of Passion]

I see so many bad and mediocre elements that I choose to reward good ones. If you rarely watch non-elite skating, you may have a different perspective.

This doesn't make sense. If an element is bad, then give it -GOE. Don't inflate a "good" element to +3 GOE just because you're happy to see a good element. Like, what.

For me, she was with the music notably more of the time than she was not, more than enough to qualify for this bullet point by my standards.

I'm not sure what your standard is there. The ISU does not have any specific definition of their standard in this regard either. So, that's part of the problem. To me, a "musical element" means it is either entirely or almost entirely constructed and executed exactly as the music dictates. This is also the long-standing understanding and operation of how figure skating worked for decades, until the new scoring system threw everything in the air and people started to forget the qualities that had been held as important. And this definition I just provided is how it works in most any other discipline of dance and performing arts as well. I'm not sure why figure skating is supposed to immune from this universally held standard of excellence.

It is also a fact that it is more difficult to do a jump with a difficult entry or air position, and it is factually more difficult to reverse direction and execute one or more turns on the same foot you've just landed a jump on than it is to hold the back outside edge for the same amount of time.

Ask figure skaters across the world to try and replicate Yu-Na Kim's 3Flip+3Toe from 2007 Worlds. And then ask them to replicate Medvedeva's. More people will be able to achieve the latter case than the former. The same goes for the Midori Ito / Ashley Wagner 2Axel comparison, with an even larger degree of disparity. This is how universal standards of "difficulty" are established, it's why the different types of jumps are weighted as they are.

 

[Skye]

Bullet points for jump elements:
1) unexpected / creative / difficult entry
2) clear recognizable steps/free skating movements immediately preceding element
3) varied position in the air / delay in rotaiton
4) good height and distance
5) good extension on landing / creative exit
6) good flow from entry to exit including jump combinations / sequences
7) effortless throughout
8) element matched to the musical structure

With the bullet points listed like that, I guess it explains why some ladies are getting +1~2 GOEs for their "not-so-much-jumping-there" jumps. I'm not saying your (or the majority of the judges who reward the points) logic is crooked, but IMO this system is an epic failure when Tsurskaya's 3Lz+3T gets a measly 0.7 points higher than Miyahara's.

 

[drivingmissdaisy]

Ask figure skaters across the world to try and replicate Yu-Na Kim's 3Flip+3Toe from 2007 Worlds. And then ask them to replicate Medvedeva's. More people will be able to achieve the latter case than the former.

I'm not sure about this. I think the air position variation has been awarded since the inception of COP, and AFAIK still she is the only woman ever to have landed a tano 3F in competition. I'm not saying what Yuna did wasn't great, but both are difficult in different ways. However, Evgenia didn't even get that many +3 on the jump elements in the LP, maybe a total of 7.

 

[Dan]

I so appreciate the original poster's devotion to this sport. The time, effort and thought put into this deserves very high PCS marks between 9.5 and 10. Thank you for your performance in this forum. I disagree here and there with some marks for some skaters (for example, I find the hunching of Hongo way more atrocious than the original poster did) but overall I really feel more comfortable about their resulting order of skaters than in real life.

However, I give the poster some minor TES deductions. Here is why. The GOE is factored. That means that a GOE level of +3 doesn't necessarily mean that three points are added to the base value. Most spins are factored by 0.5 for positive GOE levels and 0.3 for negative GOE levels and most jumps, except for 3A and Quads are factored at 0.7 in both positive and negative GOE levels. That means the most additional GOE points that you can receive on a spin are 0.5 X 3 = 1.5 points. An exceptional 3Lz with a plus three level GOE at most can receive 0.7 X 3 = 2.1 points. For the most part, the poster keeps all GOE on jumps less than or equal to 2 which is in line with reality. Occasionally, the poster gives 2 points of GOE on spins where that is not possible in real life. Also, choreographed sequences and step sequences are also factored at 0.3, 0.5 and 0.7 depending on base value level of 1, 2, 3 or 4 and GOE level. Unless the step sequence is base level 4, the highest GOE points possible also 1.5 points. This slight divergence from reality is understandable as the poster seems to award GOE in 0.5 point increments, a reasonable assumption to simplify the ridiculously complex exercise they have gone through in this forum.

The main question I have for the original poster is in awarding the GOE levels on the jumps. Did the poster consider the eight bullet points that judges must for awarding GOE level on a jump? That is for GOE level of +1 = 2 of 8 bullets, level +2 = 4 of 8bullets and level +3 = 6 of 8 bullets. For ladies where the max GOE points awarded for the jumps is 2.1 (minus Mao's 3A) that means every time we see the poster giving +2 points for GOE then the poster saw at least six of the eight following bullets. I think both the poster and the judges are overly generous with these bullets. Was that the case poster or did you like me just give a level based on the overall look and feel of the jump without actually identifying the bullets?

1) unexpected / creative / difficult entry
2) clear recognizable steps/free skating movements immediately preceding element
3) varied position in the air / delay in rotation
4) good height and distance
5) good extension on landing / creative exit
6) good flow from entry to exit including jump combinations / sequences
7) effortless throughout
8) element matched to the musical structure


Can't figure out how to get Blades of Passion's name in here so I used the title original poster instead. Thank you again for this great discussion.

 

[Blades of Passion]

^ The values listed in the original post are the GOE grades themselves, not the amount of extra points being awarded from the GOE.

I think the air position variation has been awarded since the inception of COP, and AFAIK still she is the only woman ever to have landed a tano 3F in competition.

That's because people, especially women, didn't really train tano Triples at all. It just wasn't something people cared as much about. Only in recent years, when the judging started getting extremely literal with regards to the CoP rules and very inflated in general, have people been focusing hard on how to squeeze out every little bullet point for GOE. For longer than not, big jumps with speed and perfect rotation and flowing landings were the standard. And in all the years while that was the standard, nobody did a 3Flip+3Toe as well as Yu-Na. Carolina Kostner on a good day could do a great one, but still not quite that same amount of amplitude (and straight body line) across both jumps.

Medvedeva's ability to tano so well is a worthwhile talent, but it's no replacement for the fundamentals of jumping. Especially since the smaller a jump is, the easier it is to tano it.

 

[AsadaFanBoy]

Scoring the 2016 Ladies Event Accurately

Medvedeva doesn't jump small. She might skate a little small, but the flip toe combo she does isn't small.

The sal toe is small though, but that might be because of fatigue.

 

[Sam-Skwantch]

The main question I have for the original poster is in awarding the GOE levels on the jumps. Did the poster consider the eight bullet points that judges must for awarding GOE level on a jump? That is for GOE level of +1 = 2 of 8 bullets, level +2 = 4 of 8bullets and level +3 = 6 of 8 bullets. For ladies where the max GOE points awarded for the jumps is 2.1 (minus Mao's 3A) that means every time we see the poster giving +2 points for GOE then the poster saw at least six of the eight following bullets. I think both the poster and the judges are overly generous with these bullets. Was that the case poster or did you like me just give a level based on the overall look and feel of the jump without actually identifying the bullets?

1) unexpected / creative / difficult entry
2) clear recognizable steps/free skating movements immediately preceding element
3) varied position in the air / delay in rotation
4) good height and distance
5) good extension on landing / creative exit
6) good flow from entry to exit including jump combinations / sequences
7) effortless throughout
8) element matched to the musical structure


Can't figure out how to get Blades of Passion's name in here so I used the title original poster instead. Thank you again for this great discussion.

If you go to the communications and read the guidelines on awarding jump GOE it clearly states that the amount of bullets used to achieve positive GOE is merely a suggesting and the judges are allowed to use their description. In other words a judge can weight the bullets differently and it's perfectly within the rules.

Top of page 12
http://static.isu.org/media/207718/1944-sptc-sov-communication-2015-2016.pdf

To establish the starting GOE Judges must take into consideration the bullets for each element. It is at the discretion of each Judge to decide on the number of bullets for any upgrade, but general recommendations are as follows

 

[russianbratz]

I understand your viewpoint but I am inclined to agree with gkelly here. For me it comes down to the limited number of marks that a judge can give a skater. The lower marks (0 and below) tend to be reserved for a flawed or mediocre jump. That leaves +1 or +2 to differentiate a wide array of well-executed elements if the expectation is that +3 means one of the best elements executed ever. Two marks, or even three, aren't enough IMO but I think if three are available the judges are better able to differentiate between good and great elements.

Really though? Do the judges need that much fine control over how each element gets scored? I'm personally not really comfortable with giving judges more direct control over placements. They have 7 points of variation between -3 and +3. It seems to me that if 75% of clean elements get a 0 or a +1, that's fine. I'm more concerned about accurate technical calls.

 

[Sam-Skwantch]

Really though? Do the judges need that much fine control over how each element gets scored? I'm personally not really comfortable with giving judges more direct control over placements. They have 7 points of variation between -3 and +3. It seems to me that if 75% of clean elements get a 0 or a +1, that's fine. I'm more concerned about accurate technical calls.

I'm 100% opposite. I like the idea of multiple judges grading the elements and far more in favor of removing power from the tech panel. I'd even go as far as to let judges award positive GOE for falls and let the TP take away 2-4pts at their discretion for falls as more and more or just lazy falls occur.

 

[Dan]

If you go to the communications and read the guidelines on awarding jump GOE it clearly states that the amount of bullets used to achieve positive GOE is merely a suggesting and the judges are allowed to use their description. In other words a judge can weight the bullets differently and it's perfectly within the rules.

Top of page 12
http://static.isu.org/media/207718/1944-sptc-sov-communication-2015-2016.pdf

Thank you for pointing that out Sam. I am familiar with this paragraph. I call these worlds "weasel" words. I think with any judging system, the judges should be free to award whatever values they want based on whatever criteria they want. Judging is after all subjective and while you want it to be fair, you don't want to remove subjectivity completely. The way I look at the bullets, it is the ISU trying to help judges who either might be struggling with consistency or judges who would like direction from the organization. I don't think all judges feel they need such detailed guidelines to do a quality job. My question for Blades of Passion was if they used this suggested criteria for GOE levels or applied their own (perfectly acceptable) or didn't use specific guidelines but went with a feeling (also perfectly acceptable).