Serious Question about Patrick Chan's skating ability compared to other skaters

[gkelly]

If there were two program scores, roughly one score for SS and TR and one score for P/E, CH and INT, these two scores could be weighted any way we liked to satisfy whatever notion of "balance" we wished, either between sport and art or between elements and program.

So what do you recommend?

You've thrown an interesting idea out here for consideration, inspired by general philosophical grounds.

OK, so we decide it's worth pursuing. Next comes the nitty-gritty work of figuring out the details. Care to get us started?

[Mathman]

^ Actually do some work? Instead of philosophizing and pontificating?

OK, here are my preliminary thoughts.

First, the thing about trying to make the program components add up to the same amount as the elements scores is kind of a red herring. If we suddenly decided to add 50 points to everyone’s PCSs across the board that would not change any of the placements nor the margins of victory, and it would not be giving greater relative weight to PCSs – it would just look funny in the protocols.

Still, I do like this aspect of the CoP, the separation of scores into elements scores and program scores. The old separation into technical and presentation was somewhat ill-defined, IMHO. A triple Axel turned with precise technique can also be a thing of beauty.

Likewise I have no quarrel with the idea of GOEs on technical elements, understanding that GOEs themselves can reflect both technical mastery and artistic expression.

So the idea that I threw out is not very radical at all. Leave the elements side as is and have the judges give two program components scores, one to evaluate the overall technical skills that the skater presented throughout the program, and the other (even more subjective, perhaps) to evaluate the effectiveness of the program as performance art.

Weigh the two program scores equally and weight them, as currently, to add to about the same expected value as the TES. I have become used to the quarter-point gradations in PCSs. This seems to work OK to give the judges room enough to place the skaters where they want them. At the same time, these gradations are so close together that they do not allow us the illusion that there is anything objective about saying that this performance is worth exactly 6.25 points, while that is worth 6.50. This, I think, is a good thing. Why pretend?

This is not really much of a change and does not address the problems under discussion in this thread, like reputation scoring and how many times we will let Patrick Chan fall down before we do something about it. The main advantage, in my mind, is that this system would be a little more honest as a reflection of how figure skating judging actually works.

[Jaana]

Based on the history of the way ISU judges score, I think reputation scoring is a plausible explanation for the results at SC. Take the case of Lysacek at the Olympics. One of the reasons he defeated Plushenko was because of his comparable PCS, and he received the score most likely because he was reigning world champion that year since many people were arguing that he lacked superior skating skills and didn’t deserve those scores.

Hmm, if one believes that Lysacek got reputation scores in PCS as the reigning World champion, shouldn´t one also remember that Plushenko surely must have gotten more reputation points as the reigning Olympic gold medalist (plus for all his other impressive past merits)? In Vancouver Lysacek won the freeskate based on technical elements, their PCS was the same. Who was definetely very much held up in PCS? Plushenko, compared to him Lysacek should have gotten way higher (or Plushenko way lower) PCS in some areas (at least in transitions and choreography), in my opinion.

[seniorita]

If we use two component marks instead of five, on a scale of 0 to 6.0

I got all ...until this Why would the pcs have the scale of 1 to 6 instead of 10? I mean I didnt get what it serves to change the scale

[ciocio]

Hmm, if one believes that Lysacek got reputation scores in PCS as the reigning World champion, shouldn´t one also remember that Plushenko surely must have gotten more reputation points as the reigning Olympic gold medalist (plus for all his other impressive past merits)? In Vancouver Lysacek won the freeskate based on technical elements, their PCS was the same. Who was definetely very much held up in PCS? Plushenko, compared to him Lysacek should have gotten way higher (or Plushenko way lower) PCS in some areas (at least in transitions and choreography), in my opinion.

They were both held up, each one in different areas but there are other skaters who were also very held up in PCS, specially Lambiel IMO.

[Mathman]

I got all ...until this Why would the pcs have the scale of 1 to 6 instead of 10? I mean I didnt get what it serves to change the scale

It was kind of a joke, meaning, "let's go back to the old system."

However, there is a sound psychological reason why scoring from 0 to 6 makes sense. Lots of studies have shown that most people can distinguish about 6 gradations, and with training and practice, maybe 7. You can try this experiment at home. make 6 sticks of six different lengths, then hold up one at random and see if you can gues whether that is, say, the third longest or the fourth longest. You can!

But if you do it with ten sticks, you can't. Just the way our brains work. (Of course holding up two sticks together and deciding which of the two is longer can always be done, even if they are very close together -- that's a different experiment.)

As for scoring from 1 to 10, there is nothing special about the number 10 except that we primates have 10 fingers, so we decided to count by tens.

That is also why I have to laugh at the CoP's claim to objectivity. In the PCSs there are 41 possible scores, 0.00, 0.25, 0.50,...9.75, 10.00. It is quite out of the question to be alble to define what features a program should have to be valued at 3.50, rather than 3.75 in choreography. However, that is not how these scores are used. Instead, it goes like this. The competition is at the level of intermediate. The range of scores for programs at this level are, say, from 3.00 to 4.50. The skaters that the judges like best are given 4.00 and 4.25, and the programs that are of lesser quality might get 3.25 or 3.50.

In other words, just like ordinal 6.0 judging. My point is, why pretend we are doing anything different from what we have always done?

[Joesitz]

I think we can all agree that reputation and past results should not play a role in how a competition is scored. But I haven't heard any suggestions about what we can do to bring about better judging, except -- "make the judges do a better job."

Well, we are all for that, of course.

I think if we allowed the Judges' names and countries they represent to be known, the reputation of the judges would be at stake.
The only reason we have secret judging is due to The Scandal. That scandal, regardless of the competition results, led to the creation of secret judging and saved Cinquanta's a$$.

We have no way of making the judges do a better job if we do not know their names and what country they represent on the Panel.

[janetfan]

^ Actually do some work? Instead of philosophizing and pontificating?

Weight the two program scores equally and weight them, as currently, to add to about the same expected value as the TES. I have become used to the quarter-point gradations in PCSs. This seems to work OK to give the judges room enough to place the skaters where they want them.

At the same time, these gradations are so close together that they do not allow us the illusion that there is anything objective about saying that this performance is worth exactly 6.25 points, while that is worth 6.50. This, I think, is a good thing. Why pretend?

.

Yes, you are right it is easier to pontificate ............ but this is a difficult question.

On your first point I have to wonder about weighting your suggested PCS format to being equal with the TES. I get the feeling that ISU does not want them equal. Am I wrong to wonder if it is part of ISU's current philosphical thinking (if such a thing exists) that wants skating to be more of a "real sport" and less about "performance art."

Hard to explain what I mean - but as an example I would suggest Patrick's very busy skating between recognized elements seems to be worth alot more than the way Alissa's spins can feel to be not just a spin - but part of the choreographic concept of her program.

Why is it that steps seem to be more about athleticism than relating to the music? Would Kwan's steps cut it today? They were often the highlight of her programs and seemed to be about expression and relating to the music more than what we see today.

Or the way a clean Joubert can get very high pcs even though casual skating fans can see Jeremy's CH, IN and TR is more substancial and intricate. Of course I am offering thoughts that are subjective.

This leads to your second comment which I don't understand. Could you please elaborate on it?

I have thought about remarks you have made in the past - when you referred to "musical judges" like Inman and Hoffman. Why were they considered "musical judges" ?

Was it perhaps that they not only valued artistry more than other judges but demonstrated an expertise in this area? What happens when a judging panel has several "musical judges" and several "tech judges" ? Is this ideal - or in a more perfect world would all of the judges be on the same page? Such differences in what judges value along with federation loyalties does not sound like a formula for the best results and ISU was self-serving but not stupid when they made the judges scores anonymous.

[gkelly]

It was kind of a joke, meaning, "let's go back to the old system."

However, there is a sound psychological reason why scoring from 0 to 6 makes sense. Lots of studies have shown that most people can distinguish about 6 gradations, and with training and practice, maybe 7. You can try this experiment at home. make 6 sticks of six diffewrent lengths, then hold up one at random and see if you can gues whether that is, say, the third longest or the fourth longest. You can!

Well, that's why the GOEs go from -3 to +3 instead of, say, -5 to +5 (if judges are also rating difficulty instead of having a tech panel assign levels), or -5 to +2 (if they're not assessing difficulty at all), even though those scales may more accurately reflect the range of possible errors and enhancements one might expect in an individual element.

That's also one reason why school figures were judged on a scale of 0 ("not skated") to 6 ("perfect and flawless").

The difficulty of the element or the figure is established in its base mark or scoring factor, and then the judges evaluate the quality on that 7-point scale.

With the PCS, I don't think that's what they're doing.

In very rough terms, I'd say that whole number before the decimal point is not comparing the skaters to each other at all. It's assigning a general skill level to the skater, especially in the Skating Skills mark -- the "Stroking, Edgework, and Transitions" mark if you want to combine and rename. The scale is something like 0 = beginner stumbling around on two feet, 6 or 10 = perfect/best in the world now and among the best ever.

Different judges might have different mental benchmarks for all the levels in between, and different benchmarks under 6.0 vs. 10.0 scales.

Under 6.0, the numbers applied to content as well as quality in programs -- e.g., I've heard of 3.5 = double axel as hardest jump offered as a guideline.
Some federations have tests for skating skills only outside the context of a program, to replace school figures. In the US we have Moves in the Field tests that measure power, edge quality, etc., on increasingly difficult stroking exercises and turns, with defined passing averages for each level. The passing average for senior level is 4.5 on a scale of 6.0.

So, on that scale, if you were judging quality and difficulty of skating only, aside from elements and artistry, then 4.5 would represent approximately minimum skill level expected to pass the US senior tests, or the equivalent in other countries. All the scores below 4.5 would represent skaters who don't look good enough for seniors, or so-so senior skaters having a bad performance.

The scale is calibrated differently if it goes from 0 to 10 instead of 0 to 6. But it seems that the way it's being used is to give more room at the top to distinguish among the skaters who have achieved well beyond minimum senior standard, to separate the good from the great. Scores in the 4s still seem to represent approximately average junior/weak senior standard, but of course every judge has a slightly different internal mental scale.

So for Skating Skills in a competition below senior level, the scores might represent approximately which test level the skater shows appropriate skills for. You're never going to see a competition where skaters who deserve scores in the low 1s for skating skills will be competing against skaters who deserve high 5s . . . unless those 1s are for an injured skater who goes out there and doesn't do much of anything.

Then the decimal places allow the judges to indicate finer distinctions between skaters. Officially they're not supposed to be comparing one skater to another directly, but to make sure they're being consistent within that competition they probably often do.

Judges had to recalibrate their mental scales when the scoring changed from 6.0 to 10.0. Going back to a 6.0 scale would mean another major mental shift. It could be done, but I'm not sure there would be any benefit to doing so. And it might result in narrowing down the room for distinction between good and great at the top.

For Performance/Execution/Choreography/Interpretation, whether scored with one number or three, there isn't a similar standard benchmark for different skill levels. Judges have to make up their own expectations of what they see as a 2.0 or a 4.0 or a 6.0 or whatever, on either scale. The easiest way to set those expectations is probably to visualize what they've seen in those areas on average from skaters with 4.0 skating skills and set that mental image as a 4.0 benchmark, and then score skaters higher or lower in relation to that image. Which is why they tend to give scores for those components that go up or down from the Skating Skills score by a few 0.25 increments, and we don't often see very different marks for different components.

So probably what we need is guidelines and training to help judges agree on what 4.0 or 5.0 presentation might look like and recognize it and then adjust with decimal places without even worrying about the quality of the skating.

Is this something that performing artists with no skating background could learn to apply equally well, after being instructed about the basic possibilities and limitations of the skating medium?

What might be some good guidelines to help define those components better?

[gkelly]

That is also why I have to laugh at the CoP's claim to objectivity. In the PCSs there are 41 possible scores, 0.00, 0.25, 0.50,...9.75, 10.00. It is quite out of the question to be alble to define what features a program should have to be valued at 3.50, rather than 3.75 in choreography. However, that is not how these scores are used. Instead, it goes like this. The competition is at the level of intermediate. The range of scores for programs at this level are, say, from 3.00 to 4.50. The skaters that the judges like best are given 4.00 and 4.25, and the programs that are of lesser quality might get 3.25 or 3.50.

Continuing on to address this more directly.

First of all, there's no rule that says the scores for intermediate competitions have to range from 3.00 to 4.50. In any given competition they might, although that would be a strong event at that level. Many intermediates get scores in the 2s. And occasionally a very exceptional one might break 5.0.

But say a judge has in her mind that 3.0 is average for an intermediate, 3.5 is good, 4.0 is very good at that level, and 4.5 or higher is outstanding.

So she sees a skater whose choreography is pretty good for intermediate level. Does she immediately think 3.0 or 3.5? Does she then look at the program (in memory and notes after it's over) to consider the fine points and decide whether to use the decimals to go up or down a little? Maybe mentally compare that skater to a previous one she already gave 3.0 or 3.5, to see if this one deserves to be higher or lower or is close enough to deserve the same score?

If there were official guidelines or standards as to what constitutes 3.0, the judge could compare this performance to that benchmark and decide whether it just meets that standard or deserves a little extra.

There might be time only to consider the general impression and the most salient points about the choreography. What's salient to another judge or another observer might get left out of that judge's decision. But the decision shouldn't be and when judges are judging correctly is not based on which skater the judge "likes best." It might be accurate to say it's based on whose choreography, as presented in those performances, the judge likes best, according to that judge's understanding of the guidelines and that judge's perceptions of the overall impression and salient choreographic aspects of the performance.

However, if an observer doesn't know what's going on in that judge's head (how can we?) or what the official guidelines are (hasn't read the published PCS guidelines or thinks they're too vague), and doesn't agree with the judge's estimation, then it's easy to summarize the process as "Oh, that judge liked the skater she gave 3.5 better than the skater she gave 3.0." But that trivializes and, I think, distorts what the judges are actually doing.

[seniorita]

Or the way a clean Joubert can get very high pcs even though casual skating fans can see Jeremy's CH, IN and TR is more substancial and intricate. Of course I am offering thoughts that are subjective.

I wanted to say fs audience is a bit snobish (i speak for myself not you jntfan :D). We tend to like at the end to pick and appreciate the more subtle artistic skater because this means quality but if it wasnt for people like Joubert who have huge presentation and draw attention the first minute (and I think he has as much quality by the way), I wouldnt have the luxury to know Abott... neither GS probably.It has gone for many fans this way, especially when you get zero info about fs around you. The recent years is the trend but how many people were thinking superirly about Savoie 5 years ago? (he is of my favs to watch on youtube)

MAthman you mean the brain cant compare more than 6-7 things at the same time? I havent thought about it. But why to do this simulteneously, and not pick an obvious relatively longer one and compare if this stick is longer than its next one or the previous one and so on? Ok I m not sure I make sense but isnt this what judges tend to do in both systems?Why do commentators keep saying that one top skater who happened to skate first in the group sets the tone?

[janetfan]

Is this something that performing artists with no skating background could learn to apply equally well, after being instructed about the basic possibilities and limitations of the skating medium?

What might be some good guidelines to help define those components better?

The whole post was interesting and informative.

I am not sure about the idea of educating professional dancers and musicians with the purpose of serving as judges.
The possibilty of serving on an additional panel - the "artistic panel" might seem more plausible.

You ask if professionals outside of skating could do a good job judging. Isn't it possible to also wonder how judges, many of whom were never very accomplished skaters - meaning they were never performing on TV at the Olympics or Worlds - and thus hardly have professional level performance experience are so well fit to judge IN, CH and PE?

For instance Tara or Michelle might know more about "performing" at the higest level because they did it. You seem extremely knowledgeable about the rules and technical elements.

Let's make you a judge for a GP event. Assumimg you were able to do a respectable job and keep within the range of the other judges does that make everything OK?

I might ask how much music and Dance training have you ever had? I don't mean it personally and for all I know you may have had alot. But are you or the typical skater who never made it to the elite level of skating really so qualified to tell me how well Joubert is interpreting his music? Or if Jeremy is expressing and showing some outstandind dance movements?

The question is why back away from expertise? You explained very well that tech panel members are picked because they have more knowledge and experience as skaters than the run of the mill judge.

So we are left with these second tier judges deciding Joubert's pelvic thrust counts for as much and sometimes more than a beautiful MIF done by Jeremy after he missed a quad.

If the tech specialists provide a useful service to juding a skating competition why not try and include some expertise for the components. Or as Joesitz has said many times, get rid of the music in the SP, let a tech panel judge required elements - and then let the LP really be free. And with a freer LP - YES, I think Dance and Music experts could add something that is missing - a professional evaluation of a skaters expression to go along with opinions of technical experts.

[Mathman]

I think if we allowed the Judges' names and countries they represent to be known, the reputation of the judges would be at stake.
The only reason we have secret judging is due to The Scandal. That scandal, regardless of the competition results, led to the creation of secret judging and saved Cinquanta's a$$.

We have no way of making the judges do a better job if we do not know their names and what country they represent on the Panel.

I agree 100%.

[janetfan]

I wanted to say fs audience is a bit snobish (i speak for myself not you jntfan :D). We tend to like at the end to pick and appreciate the more subtle artistic skater because this means quality but if it wasnt for people like Joubert who have huge presentation and draw attention the first minute (and I think he has as much quality by the way), I wouldnt have the luxury to know Abott... neither GS probably.It has gone for many fans this way, especially when you get zero info about fs around you. The recent years is the trend but how many people were thinking superirly about Savoie 5 years ago? (he is of my favs to watch on youtube)

?

You make a good point. Maybe I will surprise you but I think of Joubert as an iconic figure in the skating world. I thought Sasha earned that distinction as well. It doesn't matter if they never won the OGM - they made a mark on skating and were/are very popuar performers.

I just wonder about judging Joubert's best qualities, especially his non-jump elements against a less charasmatic skater like Jeremy. Or judging Sasha against Irina.

I think it is difficult and don't want "snobbery" to be the deciding factor. Nor do I want screaming teenage girls to be the deciding factor either.

[Joesitz]

My best suggestion on Scoring is to drop the GoEs and use the scale from 0-10 as it is in real judging sports. No need for GoEs and no need for a Tech Panel. However, I think the Referee should be given more to do than just wait for a skater to come to him with a problem.