Do we need a score box entry for PCS?

[Flying Feijoa]

There are so many different topics in this thread... what bothers me most about ice dance is the general attitude towards its scoring, like, there isn't even a decent effort being made to actually objectively define the scores on a pure sportive level. All this "figure skating is always subjective" talks really puts me off. The terms that are used in figure skating in general are not really helpful there, like "artistry" which can mean everything. In ice dance all these problems are still worse. Actually I think it is not subjective and every effort should be made to make the scoring as transparent and coherent as possible. Everything else is Eurovision song contest. Sorry for becoming agitated about this, I don't want to offend people, but that's what I think.

This
I guess probably for many people the drawcard for ice dance is entertainment. Personally what I like about it is its technicality. It's a great sport and worth being treated seriously as such.

The tech panel have a lot on their plate for ice dance and I think they do a good job mostly. It's a pity that all that hard work they put in to be precise and objective is often outweighed by the handwaviness in other areas of scoring.

And yeah, showing a real-time scorebox for PCS doesn't have much point when PCS is supposed to evaluate the programme over its entire duration. I do think an automated PCS cap for falls would be feasible to engineer - on condition that we decide that 'any fall (called by TP as such) = serious error'. Which is another kettle of fish

Actually, ice dance PCS was less subjective in the 2011-2014 Quad. We had a thread about its intracacies:

Ice Dancing PCS Rules 2011-2012

After reading all the different comments in the FD and SD threads for GPF, and also the thread where Maxim Trankov, long time pairs competitor, has trouble understanding his score, I thought it would be interesting to see whether I could put together a thread that would help all of us get a...

www.goldenskate.com

There has been a general watering down of ice dance PCS scores since:

• Now we have 3 choreo stunts per FD, which are entirely subjective
• The near disappearance of timing and rhythm requirements, where to get a top score you used to need to have 100% timing
• Knee slides are good instead of bad
• Sitting on someone's head is ok as long as it is not for long
• And there are now 3 PCS components instead of 5

It is not surprising that we don't see young teams advance as fast as they used to since more subjective scores has meant that past reputation has become more important.

Yes I think you hit the nail on the head there. I need to revisit videos from that era!

With the general trend in ice dance now I feel like it's moving more towards show skating (which I like watching, but don't think it belongs in a competitive arena).

 

[Anna K.]

Here are the current program component criteria:



Most of these criteria are qualitative, to be scored on range (of 0 to 10 with 0.25 increments) rather than simple yes/no decisions like most of what the tech panel does. And the different criteria within the same component need to be balanced against each other.

Have these criteria been defined?
In example, what is the definition of "unity"? If it is supposed to be rated in a scale from 1 to 10, what is the definition of "unity worth of 1" versus "unity worth of 10"? All right, this one could be tricky.

But how about "pattern and ice coverage"? What patterns, how much ice coverage should be worth 1 and how much - 10? Or, is it supposed to be thoroughly subjective and, based on subjective holistic impression, like, same patterns can be rated as "1" by one set of judges and "10" by another set of judges? I don't think so. To me, it looks that these qualities have also a quantitative side. Shall the judges ignore it because it's not "holistic"?

In any case, I think that definitions are important. We can't demand from commentators much talking about PCS if they are not given any definitions of what it is.

 

[gkelly]

Have these criteria been defined?

I found this video on youtube that seems to be from a judge training seminar explaining last year's change from 5 to 3 components. It's a long video with a lot of talking to explain the concepts.

If you pause starting at 1:41:25, you can read a slide that gives a bit more written explanation of each of the criteria, with additional spoken comments by the trainer. I don't know whether this document itself is publicly available anywhere online as a written document.

I think a lot of this training is more of an oral than written tradition. The written documents set out the basic concepts, but explaining exactly what each of them means is more a back-and-forth conversation that takes place in seminars and in the judges' room at competitions.
And then in critiques that officials offer to skaters and coaches.

Judges can expand their own understanding of how to apply the criteria by discussing with their peers and with more experienced judges about how they each understand the written criteria. One judge might offer wording that another judge hadn't thought of before and finds useful for future judging.

Fans can also share interpretations based on the written documents and learn from each others' ideas. We just don't have access to the conversations that happen in the official training environments.

In example, what is the definition of "unity"? If it is supposed to be rated in a scale from 1 to 10, what is the definition of "unity worth of 1" versus "unity worth of 10"? All right, this one could be tricky.

The definition of "unity" given here is "A coherent entity with all parts appropriately combined, contributing to a sense of completeness. In pair skating, ice dance and synchronized skating, all skaters contribute to making the purpose of the composition clear."

I'd think "unity worth of 1" would be barely a program, but just a skater doing isolated moves on the ice for the requisite amount of time while some kind of music plays in the background but neither the music nor the skating between elements does anything to tie those elements together.

"Unity worth of 10" could be program where every movement (each stroke, each arm movement or head turn, etc., let alone named elements) relates directly to the music and to the moves before and after it and there are no breaks to obviously telegraph the next technical element. It could also include thematic unity where the types of movements relate to each other and to the music within a particular dance style or other movement style.

But how about "pattern and ice coverage"? What patterns, how much ice coverage should be worth 1 and how much - 10?

The definition in the document on the video is "The interesting and creative placement of units of movement on the ice.
The design of the program and layout covering the entire ice surface.
In pair skating, ice dance and synchronized skating, all skaters contribute to create an interesting pattern. For synchronized skating, this includes the ability to stay aligned and maintain symmetry."

I watched some Basic Skills skaters competing last month. They have very low skill levels, so they have trouble covering much ice, skating on curves, turning at all let alone both directions or on recognizable edges. The patterns at that level are mainly straight lines down the midline of the rink and maybe crossovers around the hockey circles.

Elite-level skaters have the skills to cover both the full length and full width of the ice, to make curves of various sizes both clockwise and counterclockwise, to turn both CW and CCW (and to use rockers, counters, and choctaws that change curves at the same time as changing from forward to backward or vice versa), to visit all corners of the ice surface as well as the middle and the sides and to make pathways in different shapes with some surprising changes of pathway, maybe surprising placements of elements rather than, e.g., putting all the toe jumps at the ends of the ice and all the edge jumps on the same circles.

Some elite skaters use all those skills and variety of patterns all the time throughout the program and make it look effortless. That could be worth 9 or 10.

Other elite skaters spend most of their time just setting up elements and don't bother to put in many turns or deep curves or changes of direction -- except maybe in a step sequence. So those programs would have "pattern" worth much less, and "ice coverage" worth somewhat less even if they're fast and easily fill the length and width with their predictable patterns.

Even international-level skaters who have no interest in doing anything but set up elements still cover more ice and use more turns and curves than the beginners, so they would deserve scores somewhere higher than 1 and lower than 9 or 10. As would skaters who are competitive at middle skill levels (better than beginners, not elite), especially those who do make an effort to include a variety of skating directions and pathway shapes within their skill level but don't have elite-level power.

Judges and others who spend a lot of time watching skating at all levels come to develop a sense of what's worth a 3 or a 5 or a 7 or 9.

JGP events are useful for getting sense of most of the range. It's very common for a JGP to have weaker juniors who earn scores in the 3s and stronger juniors who earn 6s and 7s. Most years there are a couple of entries who are not really what we'd consider junior level who earn scores below 3 (sometimes well below!), and some star juniors who earn scores in the 8s. (We'll probably see more of the latter in coming years as skaters need to remain in the junior ranks until they're 17.)
But you might not see that wide a range in any given event.
At Junior Worlds there would be more juniors toward the higher end and fewer in the lower ranks.

In any case, I think that definitions are important. We can't demand from commentators much talking about PCS if they are not given any definitions of what it is.

The commentators have access to all the documents that we can find online. They can probably access some of the judge training material -- and also coach training material -- that may be inaccessible to the general public. They also have experience from their own skating days, and they are able to talk with officials and coaches behind the scenes to learn more. Some commentators may make more of an effort than others at self-education. Some of them are or have trained as officials; many are coaches.

Then the question is, how much of what they know are they prepared to share with viewers? Do the producers of the broadcasts encourage commentators to keep their comments very basic to appeal to casual or new viewers who don't want to be overwhelmed by a lot of technical information? Or do they gear their commentary toward more committed fans who are already knowledgeable about the basics and want to learn more?

Most viewers probably don't want to do homework like reading rulebooks and ISU Communications and detailed protocols, or watching all levels of skating, or discussing at length on message boards like this one. So the commentary is generally geared to viewers with less knowledge than the typical Golden Skate poster.

There is commentary about things that are easy for casual viewers to notice, such as jumps and lifts and falls.

It would be nice if there were more commentary about aspects of skating and judging that viewers would have a hard time picking up on their own just by watching. Viewers with knowledge of music and dance apart from skating may do just as well as the judges at noticing most aspects of the Presentation component, but viewers who have never skated won't learn much about Skating Skills unless someone points out what to look for. And the same with Composition, since some of those criteria rely on knowledge of what's easy or difficult, what's common or creative, what's highly valued, within the skating world that might not be relevant to performing arts with different techniques.

I think before getting into the real nitty-gritty of each component and each separate criterion, a first step would be to educate the casual viewers as to what each of the (now) three components are.

And to point out skaters who are particular strong in important areas of skating skill or program construction, especially if they can make connections between these strengths and correspondingly high scores on the relevant component.

It's trickier to point out areas where skaters are lacking, to explain why they earn lower scores. Commentators who can do that informatively but also tactfully and constructively are very welcome IMO.

 

[Anna K.]

Thank you for the video link! I've been looking for a while for an article or a video that would explain in detail why the change from 5 components to 3 was made and how the "new" (they are not new, they are optimized old) components work.
So I watched the video and, although I'm not a judge, it gave me some thoughts.

The first thought: videos and texts that feature judges or are meant for judges should be made available on open sources more often. Then we, the spectators, could see that judges are bothered by same problems that bother us and we could get an idea how hard it could be for judges to solve such issue (additional thanks @gkelly for showing up in different threads to remind us what is and what is not physically possible for a judge to do in the given lapse of time).
In example, I'll quote the reasons for the change from 5 components to 3:

Rationale for the Changes from 5 to 3 Components
Data confirms:
The judging of the 5 components did not clearly reflect quality execution of criteria.
The judging reflected an “average” of the different skills of the skater, including a positive bias towards the technical ability of skaters performing elements of greater difficulty.
Components marks were strongly influenced by previous results and by the starting order of the skaters.

we can see that these are issues that bother fans and are often discussed in this forum. So judges were also bothered by these issues and invented changes. As well as, at the end of the video they specifically discussed the cognitive bias and how to fight it even if it is not possible to avoid it.

The other thought: I as a non-expert would prefer professionally done videos that explain components with more slow-mo. In example, when Mr Chan's skating was compared to the other Mr's skating, I could not notice that Mr 2 did more strokes between steps because in both cases skating was rather fast. I might have noticed though if video were played slow-mo (impossible for judges because they watch the competition in real time). All listed criteria are actually physical features that can be spotted by an eye, although it has to be a trained eye to spot them on real-time. So I think this is possible for ISU to make videos that explain components just like they made videos that explain elements and I am eagerly waiting for such to appear, ISU, please!

To continue a parallel, this part of your post gave me a strong vibe of discussions from the times of pre-score box commentary on the elements. Same arguments, same questions asked: is it too complicated, too specific for casual viewers?

The commentators have access to all the documents that we can find online. They can probably access some of the judge training material -- and also coach training material -- that may be inaccessible to the general public. They also have experience from their own skating days, and they are able to talk with officials and coaches behind the scenes to learn more. Some commentators may make more of an effort than others at self-education. Some of them are or have trained as officials; many are coaches.

Then the question is, how much of what they know are they prepared to share with viewers? Do the producers of the broadcasts encourage commentators to keep their comments very basic to appeal to casual or new viewers who don't want to be overwhelmed by a lot of technical information? Or do they gear their commentary toward more committed fans who are already knowledgeable about the basics and want to learn more?

This gives me a hope that we might get that score box entry for PCS at the end of the day if Components become even more optimized and more tied to what happens on ice in real time. If there are planned elements then there are also planned components. These can be added to the preview. Also, I do not think that judges need to watch whole program to decide what range of PCS it belongs to. So this could be the preliminary mark shown on the score box and that preliminary mark could be corrected after the end of the program if the skater gets tired or has a fall/falls; or, contrary, has an exclusive skate that deserves even better PCS. It could be executed by a separate panel for PCS. I think it's worth it since these are important figure skating skills and features.

All right, your turn now

 

[gkelly]

To continue a parallel, this part of your post gave me a strong vibe of discussions from the times of pre-score box commentary on the elements. Same arguments, same questions asked: is it too complicated, too specific for casual viewers?

I don't think the ISU makes its rules with casual viewers in mind, nor should it.

Making educational materials available to broadcasters to share with casual viewers, and making the officials training information publicly available not only to coaches and skaters but also to interested (i.e., non-casual) fans would be welcome.

But the rules should be designed to reward technical details that are important to skaters, coaches, officials, and others within the skating community who are familiar with skating technique. Dumbing down the rules or scoring so that very casual occasional fans could understand it is not fair to the skaters who put in all the work to develop high-level skills.

This gives me a hope that we might get that score box entry for PCS at the end of the day if Components become even more optimized and more tied to what happens on ice in real time.

Again, the components are NOT scored in real time. There are not separate scores for each second or each minute of a program, but one score for component that is decided at the end of the program.

While the program is in process, judges are busy inputting their scores for the individual elements, maybe writing notes about the elements and/or components, and thinking about what score they think each component deserves. Even if they did have time, during a step sequence for instance, to input a score for one or more components, they would be free to change the scores after the end of the program if the remainder of the program ends up changing their mind about what score to award (e.g., if a skater struggles visibly with the end of the step sequence or the final spin).

And even if some judges do enter provisional scores for some components while the program is in progress, most would not, so there would not be enough scores to calculate an average during the program.

There will never be a real-time tracking box for program components. Instead, I think the best we can hope for on broadcasts are slowing down the K&C displays enough to show the separate averages for each component separately from the total PCS.

If there are planned elements then there are also planned components. These can be added to the preview.

No, there are not planned components.

If judges have specific scores in mind for a skater before the skater has performed the program, that would be prejudging. Which is exactly what they are not supposed to do.

Skaters know what their planned choreography is. But the component scores are all about how the skater executes the program during the actual competitive performance.

The skaters of course plan to execute their programs as well as they possibly can. But they can't know in advance when they will struggle technically during the program, or when they will be especially inspired by the audience response or by the energizing effect of better technical execution than they expected.

It's not the skaters' job to assign 1-10 PCS numbers to the composition of their program, or to their own skating skills or to how well they will perform on any given occasion.

And of course they can't predict whether this particular panel of judges will be especially favorable or unfavorable to the way they perform.

The closest thing to "planned" PCS might be past personal best PCS (which could be broken down by individual component), or averages for each component from earlier in the season. This would be information that broadcasters could research and compile, to let viewers know before the program starts what this skater could be capable of in terms of component scores.

But there is no way to show in the middle of the program what the judges are thinking about this particular performance and how to translate it into numbers on the 0-10 scale.

At least not until you can start downloading thoughts from judges brains into the computer.

And even then, they may not translate those thoughts into numbers until after the program is over.


The broadcasters (not the ISU) could provide information about past PCS and show the averages for each component after the program has completed and been scored. It is not possible and not meaningful to show any updates on component scores mid-program.

In terms of educating viewers, I'd love it if broadcasters would include short little videos of a few seconds to a minute or two explaining aspects of the components and what judges are looking for. Yes, the ISU could create a library of such little videos. Or any broadcaster is free to do so themselves. That's not about showing scores during a performance, but about educating viewers about Skating Skills especially and also about what's being rewarded in Presentation and Composition, so viewers would have more knowledge about what these scores mean.

 

[gkelly]

A few more thoughts:

So I think this is possible for ISU to make videos that explain components just like they made videos that explain elements and I am eagerly waiting for such to appear, ISU, please!

They have made some for judges. Eventually some show up on youtube.

The ISU has not made videos aimed at fans.

For casual fans, I think it would be better for individual broadcasters to develop segments geared toward the audience they're cultivating, including using commentators and/or demonstrators who are well known and beloved in that country.

For serious fans who want to understand the rules and judging processes in detail, just making the official documents easier for people who are not officials to access would be good.

The ISU already makes the detailed protocols available. It would be helpful if broadcasters also alerted viewers of their existence and where to find them.

Also, I do not think that judges need to watch whole program to decide what range of PCS it belongs to. So this could be the preliminary mark shown on the score box and that preliminary mark could be corrected after the end of the program if the skater gets tired or has a fall/falls; or, contrary, has an exclusive skate that deserves even better PCS.

It is probably true, especially for Skating Skills, that judges may get a number in their head early on for one or more of the components and then adjust it up or down in their heads as the performance continues.

But if they're also busy scoring the element GOEs, they're not going to go to the separate input screen on the computer several times during the performance to document those thoughts.

If they did input component scores during the program, they wouldn't all input the same scores at the same time. There wouldn't be a specific point in the program when we could count on at least 3 judges having input a Composition score so that an average could be displayed, or another point during the program where it would be updated.

Unlike the element scores, a skater doesn't start out with 0 for each component and then gain a few points here and later a few points there that continue building up across the program to reach a total.

Rather, some judges might see the beginning of a performance and think "That looks 'good'/7.0 in my book" and then at different points in the program various judges might think "Hmm, not so good, maybe 6.5" or "Wow, that [bit of choreography or interpretation of the music or use of blade skills during a step sequence or to get from element F to element G] was impressive -- maybe the applicable component should be 7.5."

How useful would it be to know that at 1 minute into the program somewhere between 3 and 9 judges were averaging a thought of "7.0" for a specific component, and that 3 minutes into the program the average had gone up or down, but with no way of knowing how many if any judges had changed their minds, or it was just that one or more additional judges had found time to enter their thoughts?

The score entries wouldn't be directly tied to what had just happened in the program, because most of the judges would be busy recording their scores for the most recent element, not for any of the components.

What if the first few judges to enter Skating Skills scores all tend to be real sticklers for skating technique and tend to give lower scores for that component to all skaters, and the judges who enter their SS later tend to be more generous with those scores? Or vice versa. Same for other components if, for example, the first judges to enter Presentation scores are more interested in connection to the music and the later ones projection to the spectators, for example. In such cases, the change in the averages wouldn't be because any individual judge changed their mind, but rather because judges with different priorities first entered their scores at different times during the program.

But viewers wouldn't know what was driving the change in scores from earlier to later in the program.

The only way I could imagine something like a real-time adjustment of component scores during the performance would be if judges could somehow use a lever/joystick-like device that moves a cursor along a visual analog scale as their opinion of the performance gets slight stronger or weaker. That could show moment-by-moment variations in one judge's opinion of one component. But it would be difficult to concentrate on manipulating that input device while also trying to judge the other components and/or the element GOEs.

It could be executed by a separate panel for PCS.

That would solve some of the issues above, while either adding to the costs of running a competition by increasing the total number of judges, or else reducing the statistical robustness of the scoring if fewer judges are used for components and for GOEs.

What matters in the end is the final score that is finalized after the program is over. I just don't see how watching the scores fluctuate during the performance actually would add to viewers' understanding of the judges' thought processes. If viewer understanding is the goal, much better to educate the viewers in general terms about the criteria that are being evaluated.

 

[Anna K.]

I'm glad that we agree (do we? in any case it is interesting to read your thoughts) that a separate judging panel for PCS would be helpful. When I watched the video,I jumped from my seat and shouted "yes" 2 times: when Paolo said that he wanted a separate judging panel for PCS and when he said that PCS judges should be able to compare skaters. (Needless to say that "comparing skaters" would only be possible if there is a separate judging panel for PCS and improved technical means.)

Rather, some judges might see the beginning of a performance and think "That looks 'good'/7.0 in my book" and then at different points in the program various judges might think "Hmm, not so good, maybe 6.5" or "Wow, that [bit of choreography or interpretation of the music or use of blade skills during a step sequence or to get from element F to element G] was impressive -- maybe the applicable component should be 7.5."

If we could find the way how to do it technically, it would be extremely useful for skaters to know it (to know what judges think about each moment of their performance exactly). That way, they would know what are good things to keep in their performance and what they should work on. Also, for fans there would be an insight into what matters and what does not.

The only way I could imagine something like a real-time adjustment of component scores during the performance would be if judges could somehow use a lever/joystick-like device that moves a cursor along a visual analog scale as their opinion of the performance gets slight stronger or weaker. That could show moment-by-moment variations in one judge's opinion of one component.

I imagined it differently.
When Paolo described what should be in judges mind when watching the performance he used distinctive phrases ("interesting position", "multi-dimensional movement", "great connection with music" etc.). These phrases can be offered as a menu where judges would merely click on them when they watch. Qualities that last ("great speed") should be in one menu where the key remains active until it is unclicked and then it turns into "lost speed". In other menu would be keys that highlight moments ("interesting position"). As a result we would get a transcript of judge's real-time notes that would be very useful for skaters, for commentators, for fans, and for judges themselves.

Of course, a separate panel for PCS and improved technical means is a must before we can get such transcript.

 

[gkelly]

I'm glad that we agree (do we?

in any case it is interesting to read your thoughts) that a separate judging panel for PCS would be helpful. When I watched the video,I jumped from my seat and shouted "yes" 2 times: when Paolo said that he wanted a separate judging panel for PCS and when he said that PCS judges should be able to compare skaters. (Needless to say that "comparing skaters" would only be possible if there is a separate judging panel for PCS and improved technical means.)

Well, there would be some advantages in terms of the quality of the PCS judging. Whether there would be enough improvement to be worth the extra costs is the question.

If we could find the way how to do it technically, it would be extremely useful for skaters to know it (to know what judges think about each moment of their performance exactly). That way, they would know what are good things to keep in their performance and what they should work on. Also, for fans there would be an insight into what matters and what does not.

I'm not sure that a real-time counter would accomplish that.

Skaters can get critiques of their programs outside of competition (e.g., at camps organized by their federations), or in some cases of their performances at less important competitions. Then one or more judges can give them detailed feedback on what worked well or not so well in that performance. The judges can take notes about specific moments and comment on those when talking to the skater. Other feedback might be more global.

For instance, "I liked the way you used the circle and triangle motifs throughout the program" would be more meaningful than "there's a circle," "there's a triangle," etc.

I imagined it differently.
When Paolo described what should be in judges mind when watching the performance he used distinctive phrases ("interesting position", "multi-dimensional movement", "great connection with music" etc.). These phrases can be offered as a menu where judges would merely click on them when they watch.

So there would be a limited number of options to fit on the screen. If a skater did something worth commenting on that didn't fit into one of the preexisting categories, there would be nothing to click.

Qualities that last ("great speed") should be in one menu where the key remains active until it is unclicked and then it turns into "lost speed".

But of course many of these things are qualitative rather than quantitative, or at least on a continuum rather than just either/or.

A skater who achieves "great" speed and then loses some might still be skating faster than another skater who is only able to achieve (and maintain) moderately good speed throughout the program. E.g., one skater might start out very fast and a judge thinks "Wow, that looks like skating skills worthy of 9s," and then they slow down enough that the judge is thinking low 8s instead.

Another skater might start out at 7-worthy speed and stay there throughout the program. So who would deserve higher scores (all other SS criteria being equal)? And how would this be documented in the menus you offer to the judges?

Also, why is the skater losing speed? Are they slowing down because they're out of energy toward the end of the program, or resting up before putting on a new burst of energy for the second-half jumps? Because they're stalking their axel? Because they're trying difficult steps in the step sequence and being cautious in their execution to make sure the turns are clean enough to be credited by the tech panel? Because the music has slowed down and the skater is using some contemplative edgework, maybe including moves like loops that don't cover much ice, to reflect the change in musical rhythm in that part of the choreography? Are they injured? Is there a problem with the ice quality? Did they get off balance and struggle to recover their momentum?

If we just want to know how fast each skater is going throughout the program, we can use some kind of technology to measure absolute speed, and maybe acceleration/deceleration.

If we want to know when slowing down looks unintentional or an intentional choice that's a tradeoff between speed and accuracy or an intentional choreographic choice, etc., then there would need to be a lot more options to check than just "lost speed."

How does this system distinguish among good, very good, and exceptional (or the opposite)?

In other menu would be keys that highlight moments ("interesting position"). As

How would you distinguish between "interesting position" vs. "the same interesting position reused several times in the program (maybe so often that it stops being interesting)" vs. "different variations on the same interesting position used throughout the program"?

Or between "nonstandard position that I've seen pretty often from different skaters over the years, but it is different than what the majority of skaters use on that move (e.g., arms overhead on a jump)" vs. "unusual position chosen specifically because it enhances the theme of the program" vs. "unusual/creative position that doesn't have anything to do with the concept of the program, but it sure is eye catching and shows off the skater's flexibility" vs. "Wow! I've never seen anything like that before! How did she do that?"

a result we would get a transcript of judge's real-time notes that would be very useful for skaters, for commentators, for fans, and for judges themselves.

Skaters can't get the feedback in real time while they're skating. At best they could go back and watch a video that had the judges' real-time scores superimposed. But there are better ways for them to get that kind of feedback after the fact. Even if there were to be some kind of breakdown of judges inputting reasons that factor into their component scores, there could be more detail presented in a detailed component protocol than could be flashed on the screen at a time when the skater can't access it anyway.

For technical scores, skaters don't need to see the GOEs that judges put in during the program and that show up on the score tracker box. Those scores first show up when three judges have input scores and then change as more judges input, until the box switches over to the next element. But the GOEs still change after the program if a judge realizes they made an input error, if video reviews show errors that weren't certain in real time (or show that the element was actually cleaner than it looked in real time), etc. It's not helpful to the skaters after the fact to know what the interim GOEs were -- the scores that really count will be available on the detailed protocols, in more detail than can be shown during the program.

It's useful to viewers to get a general sense during the program of which elements are scoring positive or very positive, negative or very negative, but even then wise fans need to be aware that not only the base values but also the GOEs for elements may change after the program is over. Or even just after that element is over and the scoretracker is showing the next element.

For a PCS tracker, are you imagining that all the available reasons for judges' mental pluses and minuses will be shown on the screen during the program? For each judge, or only if at least 3(?) judges select that particular menu item within X seconds of each other?
Or they will the menu items somehow be translated into numbers and averaged across all judges scoring the components, so that a viewer would see, e.g., a Presentation tracker fluctuating between low 8s to mid 7s to low 7s, up to high 7s, etc., as the program continues?

During a step sequence, for example, judges might be having thoughts about the Composition criteria and the Presentation criteria and the Skating Skills criteria all at the same time, or continually one right after the other. How are they going to input all those thoughts without taking their eyes off the skater?

How would a PCS tracker reflect the judges' thoughts during step sequence if some judges choose some options from the CO menu, others from the PR menu, others from SS, or one option from each, or none at all because they're so busy watching and thinking about the step sequence as a whole and don't look away to input scores until it's over? Would the scores of the judge who waits until the step sequence is over to input all their PCS-related thoughts about the step sequence even have their scores factored into the live tracker at all, if the other judges are already inputting scores inspired by the combination spin while she's just inputting her step sequence thoughts and the screen is now showing the (average?) spin-related thoughts?

Again, what would be more useful to the skaters themselves, as well as to fans who really want to analyze what all the PCS judges are thinking, where they differed and where they agreed, it would be more useful to generate some kind of detailed PCS protocol to be examined after the program. Lots of numbers or whatever popping up on the screen during the performance would be distracting and wouldn't reflect the final scores anyway.

If you want detailed PCS protocols after the program, or especially if you want the thoughts somehow timed to a video of the performance, then the judges will need to spend a lot of time making lots of inputs. The more inputs a judge needs to make while the program is in progress, the more they'll have to take their eyes away from the skater. So they will each undoubtedly miss some details -- hopefully different judges would miss different details.

And if you have a performance where a skater is just seamlessly exquisite throughout the program, a judge who is constantly documenting multiple thoughts per minute won't have time to just watch and enjoy the excellence of the program as a whole.

Another way to give fans real-time feedback about program components would be to have a judge not on the panel (preferably one who is retired from judging or otherwise would never be judging those skaters) give commentary on a separate stream, specifically sharing their thoughts as a judge throughout the program. Wouldn't it be great if fans could choose to view video with only the sounds of the music and the arena, or to choose commentators explaining very basics and offering human interest information to casual fans, or to choose commentary by a technical specialist or by a judge (or possibly by a judge discussing GOEs and another with a different judge discussing PCS)? Of course there would be limitations to that approach as well, but I think fans might find it easier to listen while watching than to keep looking back and forth between the skater and multiple scoring boxes.

 

[Anna K.]

Now, this is a lot
All right, let's go.

Well, there would be some advantages in terms of the quality of the PCS judging. Whether there would be enough improvement to be worth the extra costs is the question.

How about splitting the existing panel? I.e., some would do GOE and some would do PCS. It's not my vision about how it should be, it's just a variant that I'd like to know your thoughts about

So there would be a limited number of options to fit on the screen. If a skater did something worth commenting on that didn't fit into one of the preexisting categories, there would be nothing to click.

This one is interesting. What is this worth commenting that wouldn't fit into the table of listed Criteria? (Honestly want to know.)
Why am I talking about a table?
Imagine that we have a judging panel for PCS. So, they don't have their GOE buttons on their screens. What would they have on their screens, then? Boxes to type in their notes manually? Probably not. It would be time-consuming. Most likely, they would have buttons. Most likely, these buttons will not invent anything new, only display what we have already. What do we have already? We have 15 Criteria within 3 Components (this gives us the horizontal bar) and the colored 0-10 scale (this gives us the vertical bar). Combined, this gives us a table. Each cell of this table apparently is a button that a judge can press to enter his/her evaluation. If we will ever have a panel of PCS judges, such table will come to life. I don't even want to discuss it, I just see it in my crystal ball

There are lots of other things that can be discussed though. When and how often those buttons should be pressed, during the performance or after it? Should they act just as judges' notes or should they be counted by the computer instantly and constitute the mark? Maybe both: judges use them to make notes which they review after performance and approve/enter their final mark then? Should these buttons be just abbreviations and numbers or should there be well-defined phrases in the final protocol (I am still stubbornly convinced that the later would be a good idea )? Should such phrases or numbers be flashed on a score box for TV audience to see? And, above everything... Do we really need that PCS panel?

Skaters can get critiques of their programs outside of competition (e.g., at camps organized by their federations), or in some cases of their performances at less important competitions. Then one or more judges can give them detailed feedback on what worked well or not so well in that performance. The judges can take notes about specific moments and comment on those when talking to the skater. Other feedback might be more global.

Of course they can and I hope they will. However, this will remain an expensive (traveling to camps), time consuming (talking individually to each skater) and selective (not all skaters can go to camps, not all performances can be discussed) way. A protocol printout or viewing the protocol online would be accessible for everyone any time with no extra costs. The technology can even tie the judges notes/marks to the frame-to-frame performance breakdown. It would be beneficial for skaters/coaches who want to analyze performances and also for those who want submit an appeal.

How would you distinguish between "interesting position" vs. "the same interesting position reused several times in the program (maybe so often that it stops being interesting)" vs. "different variations on the same interesting position used throughout the program"?

This question I will redirect to Paolo. It was not my idea to use the phrase "interesting position", I simply quoted him. He meant the starting pose though.

Or between "nonstandard position that I've seen pretty often from different skaters over the years, but it is different than what the majority of skaters use on that move (e.g., arms overhead on a jump)" vs. "unusual position chosen specifically because it enhances the theme of the program" vs. "unusual/creative position that doesn't have anything to do with the concept of the program, but it sure is eye catching and shows off the skater's flexibility" vs. "Wow! I've never seen anything like that before! How did she do that?"

This is another good question to Paolo. Sans irony, because he gave me impression that he had a developed theory about how judges should spot interesting/boring and I'd gladly learn more about it.

During a step sequence, for example, judges might be having thoughts about the Composition criteria and the Presentation criteria and the Skating Skills criteria all at the same time, or continually one right after the other. How are they going to input all those thoughts without taking their eyes off the skater?

Back to the button table, since there are elements that hit multiple criteria at the same time, then apparently several buttons can be hit at the same (or almost) time. In example, during a spiral, there can be great glide, multi-dimensional position, connection to music, and great expressiveness at the same time and all this can be registered. Yes, tapping those buttons would distract judges from watching performance... Although our (Latvian) judge said about pressing buttons under JIS "we need to develop our peripheral vision", so maybe they have the skill already.

Wouldn't it be great if fans could choose to view video with only the sounds of the music and the arena, or to choose commentators explaining very basics and offering human interest information to casual fans, or to choose commentary by a technical specialist or by a judge (or possibly by a judge discussing GOEs and another with a different judge discussing PCS)?

It would be great indeed.
There is some danger that judges might turn it into a discussion between judges rather than analysis for skaters but I think the benefits outweigh the danger
Also for casual viewers, I would recommend a combo of a qualified judge and a journalist who has no previous knowledge about this sport but can formulate questions very well. We had it in Latvia for a while during the first years of JIS when there was no score box. It worked very well. And this is also the reason why my non-skater brain did not brow up after watching Paolo's video: I as a casual viewer have been already introduced to most of these criteria some while ago

 

[gkelly]

How about splitting the existing panel? I.e., some would do GOE and some would do PCS. It's not my vision about how it should be, it's just a variant that I'd like to know your thoughts about

It would be more cost effective than having 9 judges for PCS and a separate 9 for GOEs.

Using 5 judges for each could be feasible.

But not practical for smaller competitions that might only use 5 judges total.

So probably only used at the big important competition, while less experienced judges at less important competitions would still have to do both and use the existing component screen after the program.

There could be a whole separate thread about how splitting panels might work.

This one is interesting. What is this worth commenting that wouldn't fit into the table of listed Criteria? (Honestly want to know.)
Why am I talking about a table?
Imagine that we have a judging panel for PCS. So, they don't have their GOE buttons on their screens. What would they have on their screens, then? Boxes to type in their notes manually? Probably not. It would be time-consuming. Most likely, they would have buttons. Most likely, these buttons will not invent anything new, only display what we have already. What do we have already? We have 15 Criteria within 3 Components (this gives us the horizontal bar) and the colored 0-10 scale (this gives us the vertical bar). Combined, this gives us a table. Each cell of this table apparently is a button that a judge can press to enter his/her evaluation.

That's interesting.

Would there be 11 buttons/cells in each row, for each integer 0 through 10? Could judges indicate distinctions between, e.g., 7.5 vs. 7.75 by pressing different parts of the cell? Or should they stop thinking in those gradations for individual criteria and just call each one either good or very good? Then let decimal places for each judge come from the averages across criteria within the same component? (And then averages for the panel for each of the 15 criteria, or just for each of the 3 overall components?)

Nevertheless, most of the 15 criteria within the components cover several different things, so just checking a box for, e.g., "Clarity of edges, steps, turns, movements and body control" doesn't tell anyone whether the judge checked it because of especially clear edges or especially clear body movements, which might not go together in the same skater or at the same time.

There are lots of other things that can be discussed though. When and how often those buttons should be pressed, during the performance or after it? Should they act just as judges' notes or should they be counted by the computer instantly and constitute the mark?

How would pressing a button be translated into a mark?

Again, how does one button for, e.g., "Power and speed" translate into all the differences between 1 mph or 20 mph? "Good" speed for a tall 25-year-old man might be a different absolute speed than what would be considered "good" for a small 13-year-old girl. Which means if the small girl achieves the same absolute speed as the tall man, she would deserve more credit for achieving it with good technique rather than just muscle power.

How can pressing that button several times during a program translate into a score of 5 or 6 or 7 or 8 for that criterion, or contribute to a numerical score for Skating Skills combined with all the other SS criteria? That would be the other (colored) direction across the screen, different colored buttons for each numerical value (or for average, above average, good, very good)?

Maybe both: judges use them to make notes which they review after performance and approve/enter their final mark then?

Many judges do take notes on paper during the performance. They each have different shorthand that they use, which may include drawing shapes like ~ or / or more complicated. It would be hard to input those kinds of notes into a computer at all, let alone in real time while watching the skater, and they might not mean anything to anyone except that specific judge. But if asked to explain their thoughts after the fact they could translate their notes into words.

Or they might write words, including notes about surprising things they see during a given program that there's no preset button for.

If they have to be constantly pressing buttons during the program, they wouldn't also be able to take notes about specific thoughts about each specific program.

Should these buttons be just abbreviations and numbers or should there be well-defined phrases in the final protocol (I am still stubbornly convinced that the later would be a good idea )?

There could be abbreviations on the screen to save space but they could be expanded on the protocols, or at least in an ancillary document so everyone would know what the abbreviations stand for.

Should such phrases or numbers be flashed on a score box for TV audience to see?

I know that's what you're envisioning, but I can only imagine it would be distracting and confusing to me as a viewer, let alone to a new or casual viewer.

And, above everything... Do we really need that PCS panel?

What do you mean?

Of course they can and I hope they will. However, this will remain an expensive (traveling to camps), time consuming (talking individually to each skater) and selective (not all skaters can go to camps, not all performances can be discussed) way. A protocol printout or viewing the protocol online would be accessible for everyone any time with no extra costs.

Well, there would be the extra costs of using extra officials if you split the panel. And extra computing power. And all that paper if you print them out.

Would there be a separate sheet of paper for each judge for each program, showing all the criteria across time?

Or just an average of the panel? Or both?

The technology can even tie the judges notes/marks to the frame-to-frame performance breakdown. It would be beneficial for skaters/coaches who want to analyze performances and also for those who want submit an appeal.

No appeals are permitted except for incorrect element identification by the tech panel.

Judges' decisions especially for PCS are not "right" vs. "wrong" so there is no mechanism for skaters/coaches to say that judges were "wrong."

I really don't see how knowing when one or all judges pressed specific buttons would be sufficiently informative
There will be lags between when a judge has a thought and when they actually press the associated button. Especially if they also have to be pressing buttons related to other components/criteria at approximately the same time.

As with GOEs, would numbers come up for a given criteria only when at least 3 judges have pressed a button for that criterion?

How will skaters (or fans) know whether the number showing on the screen showing the video reflects what happened 1 second earlier or 10 seconds earlier?

This question I will redirect to Paolo. It was not my idea to use the phrase "interesting position", I simply quoted him. He meant the starting pose though.

Which criterion (button) would that fit under? It would probably go under "Originality" in the old Composition component, but the slimmed down list of components and criteria doesn't really have an explicit place to reward that.

You'd have to have many more than 15 buttons to capture every possible thought that a judge might have that a skater or fan might want to know about.

This is another good question to Paolo. Sans irony, because he gave me impression that he had a developed theory about how judges should spot interesting/boring and I'd gladly learn more about it.

I would too. But I doubt it would translate well to pressing buttons continually during a performance.

Back to the button table, since there are elements that hit multiple criteria at the same time, then apparently several buttons can be hit at the same (or almost) time. In example, during a spiral, there can be great glide, multi-dimensional position, connection to music, and great expressiveness at the same time and all this can be registered. Yes, tapping those buttons would distract judges from watching performance... Although our (Latvian) judge said about pressing buttons under JIS "we need to develop our peripheral vision", so maybe they have the skill already.

Yes, but even in a long program, they only have to input a maximum of 12 GOEs across 4 minutes, one at a time although occasionally one element will occur very quickly after the previous one ends. They can spend the rest of the time focusing on the skater not the input screen.

That's very different from being asked to make continual input across 15 or more categories every time during a performance they have a thought that fits one of the criteria.

It would be great indeed.
There is some danger that judges might turn it into a discussion between judges rather than analysis for skaters but I think the benefits outweigh the danger

I was thinking one commentator per stream, so no cross-talk between them.

Also for casual viewers, I would recommend a combo of a qualified judge and a journalist who has no previous knowledge about this sport but can formulate questions very well.

Yes, that would be good commentary for viewers.

But would be unrelated to how the judges on the panel input their scores during and after the program.

 

[Anna K.]

But not practical for smaller competitions that might only use 5 judges total.

Why not? To me, it simply looks like a smaller number of people doing their thing each.

There could be a whole separate thread about how splitting panels might work.

Agree. That's also what I meant by saying in the previous post

And, above everything... Do we really need that PCS panel?

that the need for such panel of judges for PCS is disputable/a matter for a discussion.If we discuss how such panels might work, then we can weight pluses and minuses and if it's even worth it.

Of course, I personally would prefer split panel. But this is my personal preference, not an objective need. And I am not even a skater or a judge who knows all aspects that form objective needs in this case. It's simply because I like PCS, even more than I like elements, and I would like to see more attention paid to PCS.

Would there be 11 buttons/cells in each row, for each integer 0 through 10? Could judges indicate distinctions between, e.g., 7.5 vs. 7.75 by pressing different parts of the cell? Or should they stop thinking in those gradations for individual criteria and just call each one either good or very good? Then let decimal places for each judge come from the averages across criteria within the same component? (And then averages for the panel for each of the 15 criteria, or just for each of the 3 overall components?)

Technically, any of these can be done. So it would be up to ISU's choosing what to implement.

Nevertheless, most of the 15 criteria within the components cover several different things, so just checking a box for, e.g., "Clarity of edges, steps, turns, movements and body control" doesn't tell anyone whether the judge checked it because of especially clear edges or especially clear body movements, which might not go together in the same skater or at the same time.

Can they press different parts of the cell?

How would pressing a button be translated into a mark?

That would be the other (colored) direction across the screen, different colored buttons for each numerical value (or for average, above average, good, very good)?

Apparently.

You'd have to have many more than 15 buttons to capture every possible thought that a judge might have that a skater or fan might want to know about.

Why?
I understand that judges spot compliance with Criteria and put their Component mark based on that. Why would judges need any extra notes/thoughts?
I would appreciate if you would explain that to me.

Again, how does one button for, e.g., "Power and speed" translate into all the differences between 1 mph or 20 mph?

Why should it?

"Good" speed for a tall 25-year-old man might be a different absolute speed than what would be considered "good" for a small 13-year-old girl. Which means if the small girl achieves the same absolute speed as the tall man, she would deserve more credit for achieving it with good technique rather than just muscle power.

Do 25 y.o. men compete against 13 y.o. girls? If not, I don't see how this is relevant.

There will be lags between when a judge has a thought and when they actually press the associated button.

How much thinking time a professional judge needs to identify compliance with a criteria, i.e. clean edge?

I know that's what you're envisioning, but I can only imagine it would be distracting and confusing to me as a viewer, let alone to a new or casual viewer.

Well, many members of this forum have said this about the existing TES score box and they are not casual viewers. At the same time others find the score box helpful. I believe that this is a matter of personal preference.
It would be great if broadcasters offered a choice between versions with and without a score box though.

No appeals are permitted except for incorrect element identification by the tech panel.

Judges' decisions especially for PCS are not "right" vs. "wrong" so there is no mechanism for skaters/coaches to say that judges were "wrong."

Shall it mean that judges are never wrong about PCS?

 

[gkelly]

Why not? To me, it simply looks like a smaller number of people doing their thing each.

If a small competition has 5 judges total, if you split that panel into some judges scoring GOEs and some judges scoring PCS, you'd only have 2 or 3 judges on each panel. There would be no dropping of the high and low mark, and any mistake by a judge would have a much bigger effect on the final score when averaged with only one or two others than if it were averaged with 4 or 8 other judges.

Is @Mathman around to weigh in on why such tiny panels would be statistically inadvisable?

At more important competitions, if you have a total of 10 judges with 5 judging GOEs and 5 judging components, then there would be enough to drop high and low scores.

Any more than that and you're adding to the costs to hold the competition.

For ISU championships it might be worth it. But for Grand Prixs? Or junior internationals or senior B events?

Of course, I personally would prefer split panel. But this is my personal preference, not an objective need. And I am not even a skater or a judge who knows all aspects that form objective needs in this case. It's simply because I like PCS, even more than I like elements, and I would like to see more attention paid to PCS.

I love PCS also.

I just don't think spending the whole program pushing buttons would be the best way to judge them.

I understand that judges spot compliance with Criteria and put their Component mark based on that. Why would judges need any extra notes/thoughts?
I would appreciate if you would explain that to me.

There are hundreds of possible aspects to a program that a judge might notice: everything that they have learned about skating in their years of judging (and their years of skating if they had skated themselves).

The rules are written to summarize all these possible details into categories.

The rules more or less break down the scoring from the top down:
*Total program
*Technical elements - Program components
*Each component's 3-5 criteria/bullet points (plus the summary explanations of each component above the list of criteria on the general program component chart)
*Each of those concepts could be explained in greater detail -- e.g., in the documents late in the video I I linked to earlier in this thread
*Judge training might verbally discuss nuances of some of the explanations in even more detail, with reference to specific examples

But judges experience each program more from the bottom up, in real time. What is the skater doing, what (if anything) specifically is notable about what they're doing? Only after they've perceived what's happening on the ice, and perhaps made notes related to specific notable moments, can they figure out which standard button(s) apply to those specific observations.

Suppose a judge sees a spiral and thinks "Beautiful body line, well timed to a musical highlight, shallow edge that could be steadier, resulting in not much ice coverage and unimpressive pattern"
Suppose they have a few abbreviations or shorthand symbols that they use to write down those thoughts in very few characters.
E.g., maybe "Spir P+ M+ e~"
That would actually take less time for the judge to write and would be a better reminder to them afterward (if critiquing the skater or justifying their marks to the referee) than something like Pattern and ice coverage 3; Musical sensitivity and timing 8; Clarity of edges, steps, turns, movements and body control 5
(Because the edges and body control are part of the same criterion in the rules, but the most notable parts of that compound criterion included one strong aspect and one weak aspect that canceled each other out numerically)
They wouldn't have to write/abbreviate the names of the criteria if they're pressing buttons, but they would need to find those three particular buttons on the screen and make sure not to press adjacent ones

Some of the criteria already combine several different but related concepts in the same criterion. If there's an "and" or "&" in the description, there's more than one concept already in the wording. So a single number (like the 5 for "Clarity of edges, steps, turns, movements and body control" in my spiral example above) doesn't tell us much about what the judge really thought about those aspects of the spiral, compared to, say, one that was very average in both edge quality and position

Some criteria are only one word, but if you want to know what "Unity" means you'd need expanded explanation, as we discussed in earlier posts. And if you want to know what judges thought about the unity of a program, you'd need to know what aspects of unity they were focusing on.
Plus, since that criterion is very much about how the different parts of a program relate to each other, it's not something that can really be evaluated until the program is over.

How much thinking time a professional judge needs to identify compliance with a criteria, i.e. clean edge?

For any specific edge, only as long as the edge itself lasts. Which may be anywhere from 0.1 of a second to 10 seconds or more.

In a step sequence, there may be dozens of edges within as many seconds. If they're all pretty similar in quality, then a judge might have just one overall thought about the edge quality during the sequence. But if some are better than others, for various reasons, then the judge's thoughts about the edges would be constantly changing during those few seconds.

Not to mention all the other criteria they can be thinking of during the sequence (for both GOE and PCS if they're judging both).

It's not just a question of "clean" vs. "not clean." There are multiple aspects of the "Clarity of edges, steps, turns, movements and body control" criterion (as well as the other Skating Skills criteria) that can be happening at the same time, with different mixes of positive, neutral, or negative qualities.

E.g., this section had really steady edges that didn't lose speed, and the turns were really clean, but the curves were small and the overall speed across the ice remained only moderate. A different section (maybe of a different performance, or by a different skater) had effortless acceleration and flow across the ice at high speed with very deep curves, but some of the turns were scrapy, would-be counters changed edge on entrances or rockers on the exits, or an intended loop didn't make a loop shape at all.

The judge can see all that in real time, they can write shorthand notes to capture some of those thoughts, but how do they translate those thoughts into numbers for most or all of the different SS criteria and make sure to push the correct buttons as close as possible to the moment when that section of edges is happening? While also considering the relevant CO and PR criteria. By the time they've pressed all the buttons for a given passage, the skater has already gone onto perform another section of the program/sequence, which might inspire completely different thoughts.

Well, many members of this forum have said this about the existing TES score box and they are not casual viewers. At the same time others find the score box helpful. I believe that this is a matter of personal preference.
It would be great if broadcasters offered a choice between versions with and without a score box though.

A small scoring box that shows scores for one element at a time plus total TES is one thing.

What are you imagining a PCS box would look like on the screen? Just 3 numbers (one for each component) showing the current average for each component made up of inputs from all the judges for all that component's criteria? That could at least be compact and out of the way in a corner. But it wouldn't give you much information. More about how "the panel's" opinions of each component evolve over the duration of the program, but less than the post-event protocols about how individual judges differed from each other.

If the box would show separate scores for every separate criterion and/or from each individual judge, it would cover up most of the screen! Or else be too small to read. That would give more detail but would probably better be presented after the performance to be examined at leisure.

Shall it mean that judges are never wrong about PCS?

I like to think of it more as "better" or "less good" rather than right or wrong.

There are so many different details that figure into each criterion, each component, that you're never going to get all experts to have the exact same thoughts, or to translate their thoughts into numbers in the exact same way.

I think the "better" scores are those that were arrived at by judges who see the most detail and are most consistent in their own practice of turning their observations into numerical scores. But different "better" judges may tend to focus on different relevant details.

Worse judging would be personal inconsistency and not considering all the criteria in as much depth.

Or, of course, being overly affected by sources of bias.

And even the best judges might occasionally not see something important because of where they're seated, when they happened to glance away from the skater, etc., or make a data entry error.

 

[CanadianSkaterGuy]

The thing here is that the deduction should be applied automatically. As judges mark PCS at the end of the skate, the system could just warn them that there was a major mistake involving PCS capping and the max PCS should be started lower.

The computer should physically prevent them from scoring any higher than the maximums. I'm not sure why that's so hard to cap PCS if a fall(s) is recorded.

The same should eventually apply to GOE. It's stupid that a judge could give a +5 for a fall if they wanted to.

It doesn't really matter though in the long run - these judges will still cap their closest challenger at 8.75 for a clean skate, so the popular/favoured skater can fall multiple times and be awarded "only" 9.0 and still come out on top, but then the judge can "justify" staying within the PCS parameters.

It is still absolutely insane that Chock and Bates got seven 9.75s with a fall, and 7/9 judges awarded them higher overall PCS than their nearest competitors. And so unnecessary too - they could have been awarded 9.00/9.25 max and still come out on top overall. On that note, what happened to the -2 deduction for a fall in ice dance? Did they feel that -2 was too harsh and brought it back to -1?

 

[Anna K.]

What are you imagining a PCS box would look like on the screen?

If you take a look at the title of this thread, it says "a score box entry for PCS", not "a score box for PCS".
So, this is precisely how I imagined it to look like: one extra line in the existing score box that shows either seasons best or seasons average of skater's PCS to provide viewers with a better idea of the expected total score. That's all.

Everything else (split panels, judging protocols for PCS, large score boxes for PCS alone, user friendliness of fascia panels for PCS etc.) is, strictly speaking, an off-topic conversation and I'd like to use this opportunity and thank you for joining me in this off-topic conversation. We certainly touched many interesting discussion points to some of which I'll try to return later, either in this thread or I might take some to a new thread as soon as I have spare time.

 

[zebobes]

I was skimming through your long conversation, thinking that the idea of a PCS score box entry would be a silly idea as judges give the scores based on the totality of the performance, so giving scores in the middle would be a waste of time and prevent them from seeing the performance. Then I thought of another idea, a way to also emphasize the importance of PCS in a meaningful way. We only see estimated TES during the performance, and nothing during the replays. What if we showed PCS scores during replays? It could show up as three gauges in the corner, and it would start out with all three gauges at zero. As judges started entering in their scores, you could see the gauge fluctuate up and down until it centers in on the final PCS score. Lower scores could appear more red, higher scores would be green. This way, it draws further attention to the PCS, and viewers can have time to react and debate PCS scores as they appear on screen.

Oh, and the current highest PCS scores could show up as a light gray background, so you can see how judges think a skater underperforms or outperforms the previous highest skater. And the commentators would have additional time to give meaningful comparisons during the replays, in addition to only talking about the TES.

 

[gkelly]

If you take a look at the title of this thread, it says "a score box entry for PCS", not "a score box for PCS".
So, this is precisely how I imagined it to look like: one extra line in the existing score box that shows either seasons best or seasons average of skater's PCS to provide viewers with a better idea of the expected total score. That's all.

That would fit on the screen for sure.
It wouldn't change during the program.
It would make sense to show personal best/season's best TES and PCS separately before the program starts, instead of or in addition the PB/SB total segment scores.

The current scoretracker box during the program shows the TES as it increases with each element, in relation to the TES earned by the current leader of the skaters who skated earlier, not in relation to the

As discussed earlier, it wouldn't really be feasible to show PCS scores increasing (or fluctuating up and down) during the performance. So the PCS score, as a line in a scoring box, wouldn't really be relevant for the score "tracker" that's shown during the program.

Before (previous best score by that skater) and afterward (as the PCS are input) would make sense.

What if we showed PCS scores during replays? It could show up as three gauges in the corner, and it would start out with all three gauges at zero. As judges started entering in their scores, you could see the gauge fluctuate up and down until it centers in on the final PCS score. Lower scores could appear more red, higher scores would be green. This way, it draws further attention to the PCS, and viewers can have time to react and debate PCS scores as they appear on screen.

I definitely like the idea of the three separate PCS being shown, in appropriate colors, at the end of the program, to emphasize the three separate scores and to draw viewers' attention to the fact that they exist at all.

The colors used by the ISU in their official component chart (https://www.isu.org/figure-skating/rules/id-handbooks-faq/26043-program-component-chart-id/file) show the full range between 0.25 and 10.0.

But I'm not so sure how useful those color definitions would be for viewers who only watch elite skating.

Should broadcasters develop a different color scale that doesn't match the colors judges associate with the various skill levels in their training?

Oh, and the current highest PCS scores could show up as a light gray background, so you can see how judges think a skater underperforms or outperforms the previous highest skater.

That would be similar to the current comparison between the skater who was leading and the current skater in the TES tracker.

And the commentators would have additional time to give meaningful comparisons during the replays, in addition to only talking about the TES.

That would be welcome.

 

[zebobes]

I definitely like the idea of the three separate PCS being shown, in appropriate colors, at the end of the program, to emphasize the three separate scores and to draw viewers' attention to the fact that they exist at all.

The colors used by the ISU in their official component chart (https://www.isu.org/figure-skating/rules/id-handbooks-faq/26043-program-component-chart-id/file) show the full range between 0.25 and 10.0.

But I'm not so sure how useful those color definitions would be for viewers who only watch elite skating.

Should broadcasters develop a different color scale that doesn't match the colors judges associate with the various skill levels in their training?

Speaking of colors, I find the colors for the "wordle boxes" can be improved. Right now, if the GOE is 0, then the box will show up as a dark gray color, and I find that most people find that color confusing. Also, another problem is that for an element's GOE to be exactly 0 is incredibly rare, it almost never happens. I think it would make more sense for any element that is between -.1 and 1 of raw GOE to be yellow, as any element that is below 1 is likely to be not ideal.

The same colors could be used for PCS scores, roughly based on the results of prior worlds. Red would mean likely to not make the free program, which would be roughly under 5.5 for pairs, 6.0 for women, 6.5 for men, and 7.0 for dance. Yellow would be in between, Green would be top two group worthy, which would be above 7.0 for pairs, 7.5 for women, and 8.0 for men and dance.

I didn't realize this before, but the judges seem to all think that the pairs and women are much weaker on PCS then the men and dance, and fans I think would also agree that the men and dance programs are currently inherently much stronger. Perhaps this should be a separate discussion, but what is causing the difference? Is there something about figure skating culture or rules that is making it more difficult for pairs and women to develop on the PCS side, and if so, what is it?

 

[gkelly]

I didn't realize this before, but the judges seem to all think that the pairs and women are much weaker on PCS then the men and dance, and fans I think would also agree that the men and dance programs are currently inherently much stronger. Perhaps this should be a separate discussion, but what is causing the difference? Is there something about figure skating culture or rules that is making it more difficult for pairs and women to develop on the PCS side, and if so, what is it?

Are you talking about the raw scores that they receive as shown on the detailed protocols available after the event, or the average total PCS shown during the Kiss and Cry?

The averaged totals are factored (in theory to make the PCS more comparable to the TES within each discipline), and the factors are lower for other disciplines than for men's singles. So, e.g., if a man earns 80 total PCS for a freeskate, and a woman earns 64, they actually received the same average scores from the judges (8.0 on average).

When there were 5 components, the factors were nice neat 1.0 and 2.0 for men's SP and FS, and 0.8 and 1.6 for women and pairs. With the change to 3 components, the factors involve more decimal places, but if you want to estimate what a total PCS of 80 or 64 translates into per component, you can use the old factors to get a general idea.

And I do think the displays shown on the broadcasts in the K&C (and commentary) could do a better job not only of showing the now 3 separate component scores but also explaining how they translate into the specific marks awarded by the judges.


If you are already know this and are referring to the raw scores -- e.g., you've seen on the protocols that individual judges are awarding more 8s and 9s in the men's event than the women's or pairs, and more 6s and 7s in elite women's and pairs' events than in men's, then that's a different discussion, probably for a different thread. I think that is probably true to some extent. It would not be nearly as big a discrepancy as it might seem by looking at the factored totals.

We could only speculate culturally about why men might tend to score higher on these measures.

 

[Andrea82]

It would be more cost effective than having 9 judges for PCS and a separate 9 for GOEs.

Using 5 judges for each could be feasible.

But not practical for smaller competitions that might only use 5 judges total.

So probably only used at the big important competition, while less experienced judges at less important competitions would still have to do both and use the existing component screen after the program.

There could be a whole separate thread about how splitting panels might work.

Indeed, at the last ISU Congress, the 2 proposals (one by Canada/Italy and one by Netherlands) about split panels left out B events as it was not feasible.
The Canada/Italy's proposal was to use split panels at ISU Events (ISU Championships, Grand Prixs) and Olympics. For Junior and Senior GPs, they proposed 2 sub panels of 5 members; for ISU Championships 2 sub panels of 7 members.
If a sub panel has less than 5 members (always at least 3 members), low and high marks aren't dropped.

Dutch proposal was also for ISU events and Olympics. Sub-panels size would range from 5 to 7 depending on the number of judges on site.

In terms of extra costs:
Senior Worlds, Junior Worlds and Europeans have 13 judges (because some judge only one segment) on site. So it would mean to pay for 4 extra judges (one per discipline) to have two 7 members strong sub-panels.
Same for Olympics.
Four Continents have only 10 judges per discipline on site. In the Dutch proposals, they would not meet extra costs (because they can do two sub panels of 5 judges), while they would need 16 more judges for Canadian and Italian proposal (and judging by the table of extra costs estimated, neither Canadian and Italian Federation seem to have realized there aren't 13 judges on site at 4CCs).

In Grand Prixs, there are usually at least 10 judges used anyway for Single and Pair disciplines. So they can split in 5 and 5. This season the number of Single and Pair judges used in GPs were
Skate America: 16
Skate Canada: 15
Int de France: 17
NHK: 15
Finland: 17
Sheffield: 17

JGP Courchevel: 17
JGP Ostrava: 23
JGP Riga: 28
JGP Gdańsk 1: 28
JGP Gdańsk 2: 23
JGP Egna: 24

They would need extra judges in Ice Dance though (if at least 5 members for each subpanel are needed).
At Senior GP level, Skate America, Skate Canada, Finland and Internationaux de France would need an extra judge. NHK and Finland had enough people on site for two panels of 5 judges.
At JGP level the situation was:
JGP Courchevel: 7 judges serving in the ID panel
JGP Ostrava: 6
JGP Riga: 10
JGP Gdańsk 1: 11
JGP Gdańsk 2: 9 (but 2 single judges present there are qualified to judge ID and so they could have been used in case of split panels)
JGP Egna: 11

So Courchevel and Ostrava were problematic.