New Season, New Rules, & Judging: Ice Dance | Page 3 | Golden Skate

New Season, New Rules, & Judging: Ice Dance

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
The technical panel determines the levels for all elements -- the judges don't have input into the levels nor even know what levels are being called by the tech panel.

The calling guidelines that the tech panel applies aren't deciding "who has better technical ability" or "who performed the element better" but merely whether the element met the requirements for the attempted level as spelled out in the rules.

They need to make yes/no decisions about each of the requirements attempted. Most of the time the answer is obvious. In "gray areas" -- where the skaters' attempt sits on the borderline between almost but not quite or just barely meets the requirements -- then different tech panel members who might be more or less strict in their understanding and application of the rules might disagree which side of the line the attempt ended up on.

The quality decisions are left to the judges and are more subjective by design.

Yes. And my question was why skaters who only meet reqirements (levels) better but dont meet GOE bullets should be pronounced as winners and the not opposite ones? I was arguing what is percieved as a better skater (by more broader audience) and that is not necessarely the one who hit the highest levels, but those with better GOEs.... Thats why i find this reworked system better where GOEs in ice dance weight more...
 

BillNeal

You Know I'm a FS Fan...
Record Breaker
Joined
Jan 10, 2014
Pardon me for quoting myself but I think this post from another thread is useful quantitatively and allows us to see the proportion of scores that are influenced by the technical panel (BV) and the proportion of scores influenced by judging panel (GOE+PCS):

So I looked into what percentages of the overall score are made up of BV, GOE, and PCS. The GOE proportion in the overall score has greatly increased due to the decrease of the two others. This is consistent with them emphasizing more on the quality of the elements, similar to singles and pairs. A good thing about this change is that the tech panel, which has only 3 officials versus 9 judges on the judging panel, has a lower influence on the score.

SD (2017-2018)
Max. BV = 33.30 (39.0%)
Max. GOE = 12.00 (14.1%)
Max. PCS = 40.00 (46.9%)

RD (2018-2019)
Max. BV = 33.29 (36.4%)
Max. GOE = 18.23 (20.0%)
Max. PCS = 40.00 (43.7%)

FD (2017-2018)
Max. BV = 44.90 (36.0%)
Max. GOE = 19.80 (15.9%)
Max. PCS = 60.00 (48.1%)

FD (2018-2019)
Max. BV = 45.83 (33.2%)
Max. GOE = 32.01 (23.2%)
Max. PCS = 60.00 (43.5%)

Overall (2017-2018)
Max. BV = 78.20 (37.2%)
Max. GOE = 31.80 (15.1%)
Max. PCS = 100.00 (47.6%)

Overall (2018-2019)
Max. BV = 79.12 (34.5%)
Max. GOE = 50.24 (21.9%)
Max. PCS = 100.00 (43.6%)

BV, determined by the tech panel, still comprises a sizable proportion of the score though, despite the decreased proportion from last season. I think what would also be interesting to look at are the increment changes in BV between levels. For example, the difference between level 2 and level 3 in a step sequence used to be 1.5 but now is 0.5.
 

markovai

Rinkside
Joined
Jan 16, 2015
Pardon me for quoting myself but I think this post from another thread is useful quantitatively and allows us to see the proportion of scores that are influenced by the technical panel (BV) and the proportion of scores influenced by judging panel (GOE+PCS):

I am sorry, but the way you present the numbers is not quite helpful. Comparing the max values doesn't help illustrating the problem. The interval the GOEs (for the RD for example this season) vary is from -18.23 to +18.23, i.e. the difference between the worst and the best that can be done by the judges as GOEs is of 36.46 points. Nobody can get a 0 for BV, because practically all competitors do all (or almost all) required elements at Basic level in the worst case scenario. So the BV of the RD given by the technical panel varies from 14.60 (I added the values of the 5 required elements if executed at basic level) to 33.29, i.e. the difference is of: 33.29 -14.60 = 18.69.
In summary, the technical panel could potentially influence the score in the RD and make a difference of 18.69 points, while the judges can make a difference in that same dance by 36.46 point by the GOEs only and by potentially 40 more points for the PCS. I personally think that 18.69 versus 36.46 +40 = 76.46 (19.6% versus 80.4%) is a way too big of a difference between the roles of the technical panel (difficulty of the elements) and the judges (quality).
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
I am sorry, but the way you present the numbers is not quite helpful. Comparing the max values doesn't help illustrating the problem. The interval the GOEs (for the RD for example this season) vary is from -18.23 to +18.23, i.e. the difference between the worst and the best that can be done by the judges as GOEs is of 36.46 points. Nobody can get a 0 for BV, because practically all competitors do all (or almost all) required elements at Basic level in the worst case scenario. So the BV of the RD given by the technical panel varies from 14.60 (I added the values of the 5 required elements if executed at basic level) to 33.29, i.e. the difference is of: 33.29 -14.60 = 18.69.
In summary, the technical panel could potentially influence the score in the RD and make a difference of 18.69 points, while the judges can make a difference in that same dance by 36.46 point by the GOEs only and by potentially 40 more points for the PCS. I personally think that 18.69 versus 36.46 +40 = 76.46 (19.6% versus 80.4%) is a way too big of a difference between the roles of the technical panel (difficulty of the elements) and the judges (quality).

Yes. But isnt the better solution that average opinion of 9 judges matters more (with some of those marks which system find as an error dont count at all!) than yes/no decision of 3 judges in the tech panel whose every decision count, no matter of possibility of mistakes? Now, the only argument i see for opposite stand of point is that GOEs decisons are more subjective. But the thing is - what system does to them after that to come with a final score is more mathematically correct (objective) than how system of tech panel works...
 

gkelly

Record Breaker
Joined
Jul 26, 2003
I don't think the question should be "Which do we want to count more: the decisions by 3 people or the decisions by 9 people?"

But rather "Which do we want to count more: what the skaters did or how well they did it?"

The reason there are fewer people making the "what" decision is because it's a question of fact, which is only hard to determine in rare borderline cases. For "what" calls, the answer wouldn't be any more accurate if you asked more people to make it.

With the exception that determinations that rely on viewing angle might benefit from having people viewing from multiple angles. Or having multiple camera angles for the few people consult. But just putting more people on the same tech panel stand isn't going to give better answers.

The "how well" decision is a judgment call, and that's why it's preferable to take a consensus of many people's judgment rather than relying on just a few.
 

markovai

Rinkside
Joined
Jan 16, 2015
I don't think the question should be "Which do we want to count more: the decisions by 3 people or the decisions by 9 people?"

But rather "Which do we want to count more: what the skaters did or how well they did it?"

The reason there are fewer people making the "what" decision is because it's a question of fact, which is only hard to determine in rare borderline cases. For "what" calls, the answer wouldn't be any more accurate if you asked more people to make it.

With the exception that determinations that rely on viewing angle might benefit from having people viewing from multiple angles. Or having multiple camera angles for the few people consult. But just putting more people on the same tech panel stand isn't going to give better answers.

The "how well" decision is a judgment call, and that's why it's preferable to take a consensus of many people's judgment rather than relying on just a few.

I do not discuss the number of people in the technical panel vs. the number of judges. Obviously if you have more judges it will be a more accurate judging of how well the dancers did. I do not defend nor reject the IJS system. I simply want to tell you, looking at numbers, looking at the currently used (this season) base values and GOE values, that the whole idea of the IJS becomes more and more useless, as the decision actually depends almost entirely of the judges, not of the technical panel. I am not giving my opinion about is this good or bad, I am simply saying that as it was before the IJS, we have again the judges who decide the final results. And my question is:if the technical panel can change (in theory) the final score for a rhythm dance by 18.69 points only, while the judges - by 76.46 (just an example) why bother to still have this whole IJS system ???
I think that the numbers they use for base values of the elements , and the increases of points they give for each level up versus the points they add/subtract for GOEs should be more balanced, in order for the whole system to make sense. That's all.
 

BillNeal

You Know I'm a FS Fan...
Record Breaker
Joined
Jan 10, 2014
I am sorry, but the way you present the numbers is not quite helpful. Comparing the max values doesn't help illustrating the problem. The interval the GOEs (for the RD for example this season) vary is from -18.23 to +18.23, i.e. the difference between the worst and the best that can be done by the judges as GOEs is of 36.46 points. Nobody can get a 0 for BV, because practically all competitors do all (or almost all) required elements at Basic level in the worst case scenario. So the BV of the RD given by the technical panel varies from 14.60 (I added the values of the 5 required elements if executed at basic level) to 33.29, i.e. the difference is of: 33.29 -14.60 = 18.69.
In summary, the technical panel could potentially influence the score in the RD and make a difference of 18.69 points, while the judges can make a difference in that same dance by 36.46 point by the GOEs only and by potentially 40 more points for the PCS. I personally think that 18.69 versus 36.46 +40 = 76.46 (19.6% versus 80.4%) is a way too big of a difference between the roles of the technical panel (difficulty of the elements) and the judges (quality).

Well, if we are going to assume that almost all competitors can achieve a minimum base value of Basic level elements, we can also assume that almost all competitors are good enough to be above a certain level in program components. Looking at this season's challenger series, the lowest RD PCS score came from the Estonian team at Lombardia of 17.76. Looking at last season's ISU championships and Olympics, the lowest PCS score recorded of 15.78 came at Euros from the Swedish team. I think we can assume that almost all teams competing in seniors at an ISU event would obtain at least that score. So a more realistic range of PCS would be 24.22 not 40.

As for GOE, it gets complicated because level B elements have their own GOE range. The amount of GOE ranges from -10.95 to 10.95 for all B elements in the RD to -18.23 to +18.23, if all elements are called level 1 or higher. Actually, the only times that I have commonly seen level B elements called are for the two pattern sections this season, when no key points are satisfied. Also, unless there are more than 6 negative features for the elements or falls, -4 and -5 aren't given. Therefore, the lower end of the negative range wouldn't apply unless there are serious mistakes.

I did note that the increment in points between levels have been reduced in my post and I think that may be more useful to look at in terms the power the technical panel has. Before, teams that got the level 4 step sequences had a 1.5 point advantage per StSq vs. those that only got level 3. For teams that contend for the podium but are bunched up together in PCS, the StSq levels were a deciding factor for the colour of the medal. For comparison, the difference in GOE (eg. +2 and +3) on the step sequence was 1.1 points. This season, the difference between a level 3 and level 4 step sequence is only 0.5 points vs. the difference of 0.97 points per GOE increment. In that sense, the technical panel's power has been greatly reduced. For the FD, also note that the choreo elements has very little BV and the amount of points is determined entirely by quality. Therefore, yes they are moving towards putting a greater emphasis on how well an element is performed, which is consistent with their approach in the other disciplines.
 

draqq

FigureSkatingPhenom
Record Breaker
Joined
May 10, 2010
One thing that is a bit blurry for me in the system is how steps are judged for level and for GOE in a similar fashion.

For instance, for pattern dances if the technical panel determines that a key point was not achieved, is that not also a "quality" determination to some degree? From what I gleaned from the ISU Development channel on ice dance, the standard judging panel is also supposed to consider whether a step was done correctly or not and reflect that in the GOE. That's why I sometimes get confused when I see pattern dances with two to three Ns for key points that still get +3 GOEs. Or in the case of step sequences (specifically H/D's diagonal step sequence in their free dance at the US Classic), how a Level 2 step sequence still gets +4 across the board.

Yes, the judges don't know what the technical panel calls the difficulty of a pattern and there are technically enough bullet points for GOE that the judges can still give out a +3 even with step errors. It's still a bit weird.

----

On a separate note, it would really help viewers (me included) if the ISU were to provide some sort of example explanation of how a particular pattern dance is supposed to look like. I can look at at the diagrams of steps all day but it just doesn't really gel until I actually see what they mean for a particular hold or step.

Or just on a basic level, what it means for a step "to count", like what it looks for a step NOT to count for some technical reason like a hopped step or a step that was too flat.

Perhaps videos like this exist somewhere already and aren't public or not easily accessible?
 

gkelly

Record Breaker
Joined
Jul 26, 2003
One thing that is a bit blurry for me in the system is how steps are judged for level and for GOE in a similar fashion.

For instance, for pattern dances if the technical panel determines that a key point was not achieved, is that not also a "quality" determination to some degree?

Sometimes -- e.g., if the skater is attempting to get onto the required edge but it ends up flat (or worse on the opposite edge), the "what" they did was wrong because of poor execution.

Still, the tech panel's job is to determine what the skaters actually did and the judges' job to determine how well they did it.

From what I gleaned from the ISU Development channel on ice dance, the standard judging panel is also supposed to consider whether a step was done correctly or not and reflect that in the GOE. That's why I sometimes get confused when I see pattern dances with two to three Ns for key points that still get +3 GOEs. Or in the case of step sequences (specifically H/D's diagonal step sequence in their free dance at the US Classic), how a Level 2 step sequence still gets +4 across the board.

With the elite teams, they're probably aiming for Level 4.

But even one tiny mistake that a casual viewer won't notice might result in the tech panel calling a lower level. That doesn't necessarily affect the overall quality of the step sequence.

If there are no negative features and more than enough positive features, the team can earn higher GOE. If they have enough positives for +5 and one small negative, they could still end up with +4.

And lower level teams or teams that were having trouble with specific moves they needed to earn Level 4 might deliberately plan to omit that move and do an easier step sequence. In that case the lower level call would not reflect any errors at all. If they execute it close to perfectly, why shouldn't they get high GOEs?

On a separate note, it would really help viewers (me included) if the ISU were to provide some sort of example explanation of how a particular pattern dance is supposed to look like. I can look at at the diagrams of steps all day but it just doesn't really gel until I actually see what they mean for a particular hold or step.

Or just on a basic level, what it means for a step "to count", like what it looks for a step NOT to count for some technical reason like a hopped step or a step that was too flat.

Perhaps videos like this exist somewhere already and aren't public or not easily accessible?

This old Golden Skate thread might help:
https://www.goldenskate.com/forum/showthread.php?36598-Ice-Dance-Educational-Videos-Pattern-Dances

I think some of the links are no longer active.
 

silver.blades

Medalist
Joined
Jan 4, 2007
Country
Canada
One thing that is a bit blurry for me in the system is how steps are judged for level and for GOE in a similar fashion.

For instance, for pattern dances if the technical panel determines that a key point was not achieved, is that not also a "quality" determination to some degree? From what I gleaned from the ISU Development channel on ice dance, the standard judging panel is also supposed to consider whether a step was done correctly or not and reflect that in the GOE. That's why I sometimes get confused when I see pattern dances with two to three Ns for key points that still get +3 GOEs. Or in the case of step sequences (specifically H/D's diagonal step sequence in their free dance at the US Classic), how a Level 2 step sequence still gets +4 across the board.

Yes, the judges don't know what the technical panel calls the difficulty of a pattern and there are technically enough bullet points for GOE that the judges can still give out a +3 even with step errors. It's still a bit weird.

The answer is because many judges are looking at the big picture, ie. how does the skater present the element as a whole, not if the element was correct or not. When I was working on my coaching certification, one of the tasks I had to complete was to observe a test day and shadow a judge. I sat through a dozen Rocker Foxtrots and only one girl actually skated a clean rocker. Every other skater did a 3-turn, most of them weren't even close to getting the turn clean. The girl with the best technique failed because she didn't have the correct "foxtrot" presentation (perfectly reasonable given the level), however, most of the girls who couldn't even skate a reasonable attempt at a rocker turn did pass. I asked the judge afterwards how they had passed if they can't do the main section of the dance. The judge told me that it didn't matter that the turn wasn't clean. They had done a beautiful job of presentation and skated the dance "well". The judge actually told me that presentation was more important than technique at the higher levels.

This was a club level test day, but I assume that the same mentality is present among higher level judges as well.
 

WeakAnkles

Record Breaker
Joined
Aug 1, 2011
The answer is because many judges are looking at the big picture, ie. how does the skater present the element as a whole, not if the element was correct or not. When I was working on my coaching certification, one of the tasks I had to complete was to observe a test day and shadow a judge. I sat through a dozen Rocker Foxtrots and only one girl actually skated a clean rocker. Every other skater did a 3-turn, most of them weren't even close to getting the turn clean. The girl with the best technique failed because she didn't have the correct "foxtrot" presentation (perfectly reasonable given the level), however, most of the girls who couldn't even skate a reasonable attempt at a rocker turn did pass. I asked the judge afterwards how they had passed if they can't do the main section of the dance. The judge told me that it didn't matter that the turn wasn't clean. They had done a beautiful job of presentation and skated the dance "well". The judge actually told me that presentation was more important than technique at the higher levels.

This was a club level test day, but I assume that the same mentality is present among higher level judges as well.

Why does this not surprise me?

:think:
 

draqq

FigureSkatingPhenom
Record Breaker
Joined
May 10, 2010
The answer is because many judges are looking at the big picture, ie. how does the skater present the element as a whole, not if the element was correct or not. When I was working on my coaching certification, one of the tasks I had to complete was to observe a test day and shadow a judge. I sat through a dozen Rocker Foxtrots and only one girl actually skated a clean rocker. Every other skater did a 3-turn, most of them weren't even close to getting the turn clean. The girl with the best technique failed because she didn't have the correct "foxtrot" presentation (perfectly reasonable given the level), however, most of the girls who couldn't even skate a reasonable attempt at a rocker turn did pass. I asked the judge afterwards how they had passed if they can't do the main section of the dance. The judge told me that it didn't matter that the turn wasn't clean. They had done a beautiful job of presentation and skated the dance "well". The judge actually told me that presentation was more important than technique at the higher levels.

This was a club level test day, but I assume that the same mentality is present among higher level judges as well.

Well, that's quite telling.

I suppose then the ISU video on ice dance explaining the new +5/-5 GOE system, which had two women from the ISU explaining that incorrect steps should be reflected in the GOE for pattern steps, is a sort of ideal case. But I guess in practice the presentation of an element is more important than the technique for many judges, as opposed to the more stringent technical panel.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
Well, that's quite telling.

I suppose then the ISU video on ice dance explaining the new +5/-5 GOE system, which had two women from the ISU explaining that incorrect steps should be reflected in the GOE for pattern steps, is a sort of ideal case. But I guess in practice the presentation of an element is more important than the technique for many judges, as opposed to the more stringent technical panel.

If steps are visibly incorrect then im sure it would be reflected in GOE. But if judges can hardly see if that one edge on the turn/steps in one key point of the whole pattern was deep enough to satisfy levels, why that should be reflected in their GOE? Does that one detail should weight that much comparing to all the other things skaters do? I think the poster was trying to say (and what was the judges feedback) that how the whole pattern is skated is more important than one key point which is just a part of it...
 

draqq

FigureSkatingPhenom
Record Breaker
Joined
May 10, 2010
If steps are visibly incorrect then im sure it would be reflected in GOE. But if judges can hardly see if that one edge on the turn/steps in one key point of the whole pattern was deep enough to satisfy levels, why that should be reflected in their GOE? Does that one detail should weight that much comparing to all the other things skaters do? I think the poster was trying to say (and what was the judges feedback) that how the whole pattern is skated is more important than one key point which is just a part of it...

Because according to the GOE assessment, judges need to take the accuracy steps into account for pattern dances. I totally agree that it's a matter of degree, and judges should be given some leeway when it comes to something that can be hard to tell like edges. But we're not talking about one key point here. We've had some strange incidences already this year where pattern steps were judged N for ALL four key points but still received +2 GOE.

Soucisse/Firus - All Key Points N - 4 judges gave +2
Fear/Gibson - All Key Points N - 2 judges gave +2
Kaliszek/Spodyriev - All Key Points N - 1 judge gave +3

I wonder if the judges should be made aware of what the technical panel has assessed about key points, much like ! and e calls on jumps in single skating.
 

Baron Vladimir

Record Breaker
Joined
Dec 18, 2014
^^^
If you look at the scoresheets you can see a lot of (juniors) skaters skated the first pattern with more Yeses and second pattern with more N/T. But when you watch their performances it doesnt appear that first pattern is skated 'right' and the second is skated 'wrong', the skaters looked as equally skilled skaters in both of the patterns (cause the same skaters skated them). So, the other judges dont look on the edges/turns of the key points, but on the whole pattern, the view which i think is also more closer to the view of the general audience.
 

Lester

Piper and Paul are made of magic dust and unicorns
Final Flight
Joined
Dec 7, 2014
Because according to the GOE assessment, judges need to take the accuracy steps into account for pattern dances. I totally agree that it's a matter of degree, and judges should be given some leeway when it comes to something that can be hard to tell like edges. But we're not talking about one key point here. We've had some strange incidences already this year where pattern steps were judged N for ALL four key points but still received +2 GOE.

Soucisse/Firus - All Key Points N - 4 judges gave +2
Fear/Gibson - All Key Points N - 2 judges gave +2
Kaliszek/Spodyriev - All Key Points N - 1 judge gave +3

I wonder if the judges should be made aware of what the technical panel has assessed about key points, much like ! and e calls on jumps in single skating.

I thought the element is called by the tech panel before they can give it GOE?
 

draqq

FigureSkatingPhenom
Record Breaker
Joined
May 10, 2010
I thought the element is called by the tech panel before they can give it GOE?

Someone can correct me if I'm wrong, but I think the technical panel only calls the actual element but not the level and key points it receives.
 

draqq

FigureSkatingPhenom
Record Breaker
Joined
May 10, 2010
^^^
If you look at the scoresheets you can see a lot of (juniors) skaters skated the first pattern with more Yeses and second pattern with more N/T. But when you watch their performances it doesnt appear that first pattern is skated 'right' and the second is skated 'wrong', the skaters looked as equally skilled skaters in both of the patterns (cause the same skaters skated them). So, the other judges dont look on the edges/turns of the key points, but on the whole pattern, the view which i think is also more closer to the view of the general audience.

Exactly. So what the judges are looking at is the overall picture and not necessarily the accuracy of the steps, even if technically each incidence of an "incorrect" step is supposed to result in -1 in GOE. It's not ideal but that's how it's done in practice.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
Someone can correct me if I'm wrong, but I think the technical panel only calls the actual element but not the level and key points it receives.

My understanding is that the tech panel calls the levels and the key points, but these are not shown to the judges in their computers.

I'm not sure how it works in dance, but I know that in singles and pairs the tech panel calls the levels but the judges don't see that part of the code.

E.g., in singles, if the tech specialist says "combination spin with change of foot, level 4," or "step sequence, level 3," the data operator inputs CCoSp4 or StSq3, and that's what shows up on the tech panel's and accountants' computers. But the judges screens only show CCoSp or StSq.

Do we have any reason to believe it does not work the same way in dance?
 

markovai

Rinkside
Joined
Jan 16, 2015
My understanding is that the tech panel calls the levels and the key points, but these are not shown to the judges in their computers.

I'm not sure how it works in dance, but I know that in singles and pairs the tech panel calls the levels but the judges don't see that part of the code.

E.g., in singles, if the tech specialist says "combination spin with change of foot, level 4," or "step sequence, level 3," the data operator inputs CCoSp4 or StSq3, and that's what shows up on the tech panel's and accountants' computers. But the judges screens only show CCoSp or StSq.

Do we have any reason to believe it does not work the same way in dance?

I think the judges do see what the technical panel gives. For example if the jump is under-rotated, or downgraded, or there is an edge call, the judges are required to give negative GOEs, so they should know what is the call (in case they didn't notice it themselves, which may sometimes be the case).
 
Top