Ladies Free Skate

The CONCEPT of CoP seems like a winner but APPLYING that concept is where it becomes blindsighted. The Technical score is cut and dry. Each jump has a face value but the problem lies is in the PCS just like the presentation score in the 6.0 system. It is very subjective. Even the GOE can be suspicious and troublesome.

I don't think the casual fan need to study CoP in order to be able to tell a sloppy skating from a clean, flowy one. The difference with this new system is that the casual fan is able to question the 6.0 system more and yet agree with it easily, with CoP their left with a blank look with their face and saying "What the *@%#?!?!

I agree it is up to the fans to study the system and most casual fans are only concern with the rankings. However, CoP sure take the joy out of these casual fans when they could scream like hooligans when they see those 5.8s, 5.9s and 6.0s. Instead, of jumping up from my couch and pumping my fist when I see the scores, I kind of just have little tweety birds flying around my head and asking myself - "Am I suppose to be cheering when I see the mark or should I be disappointed?"

Notice, at the GPs and Euros, the fans are quiet during the scores. This to me take out the excitement when you're in the arena or just watching the TV and you hear the screaming fans. I love seeing fans clapping when they see the scores, now fans are just subdued with this look

Crizzy brings up another good point -- it's rather hard to get up the same type of excitement over a 73.41 than it is over a 5.9. I think the old system led to more enthusiastic audience response, which can create some great energy in the arena and lead to better performances if skaters are able to feed off that energy. I am reminded of last season's Skate America, when USA TODAY ran an article about the compulsury dances and mentioned the scores given to a couple who were officially the first skaters to be scored by COP. The last line in the article (more of a blurb, actually), said "The crowd seemed unable to figure out whether to cheer or boo." Personally, one of the theories I have about this system is to keep the booing of the judges down to a minimum.

It would be extremely helpful if the marks were shown in the manner of the "detailed classification" you see on the results page of the ISU website. For example, for Plushy, this shows the SS 8.14, TR 7.89, PE 8.14, etc; this would at least give some indication of how these numbers were arrived at, instead of just throwing up the two scores and the total leaving people wondering how they came up with it. This is a "thing" with me -- if you're going to slap a number up there, I want to know how you got it. In gymnastics, I understand a 9.683 is an average; this makes sense. In diving, it's the marks (minus the highs and lows) and I think that's multiplied by degree of difficulty, isn't it? This makes sense. But if you're just going to show a couple of raw numbers.... ??????? (It's kind of like why I won't do my taxes by phone -- if you're going to tell me I owe such and such amount, I want to know exactly how you arrived at that conclusion.)

The reason why I made the point about having a system that should be more friendly to the casual viewer is, I was under the impression that at one point this was actually a concern of the ISU; it's one of the reasons why a short program was put into play (because viewers of the 72 Olys couldn't figure out how on earth Beatrix Schuba won that gold medal when her free skating was subpar compared to Lynn and Magnussen), and one of the reasons why the system was overhauled in 1980 (and I couldn't explain how the system in play between 73 and 80 worked if I tried my hardest, because to this day I still don't understand it.) Now we have something that requires "COP class" (thanks, doggygirl, that's the perfect word for it) to understand. And quite honestly, the judges have my sympathies. Here they've been spending decades learning how to judge under the old system, and now they're expected to learn something that operates on a completely different concept than what they've been using all these years. If there are new types of "judging irregularities", I can certainly understand why; there have got to be more than a few longtime judges who are completely mystified.

British Eurosport did show the PCS numbers.

Mathman said:

The main generic argument against the whole idea of the CoP is that its numerical scores are based on parametric analysis (estimates of means, standard deviations, etc.), while a judged sport such as figure skating, by it's nature, ought to use the techniques of non-parametric statistics (e.g., rankings, ordinals and OBO judging).

Click to expand...

Because humans are so good at making those non-parametric judgements? Because humans are so capable of ignoring cognitive dissonance?

What I find most revealing is the ability to calculate standard deviation, which can show how insignifcant the differences are between performances and placements, and how any "definitive" ranking system can be arbitrary. But of course in sport, we need Da Winner.

hockeyfan228 said:

Because humans are so good at making those non-parametric judgements? Because humans are so capable of ignoring cognitive dissonance?

Click to expand...

I don't know what the rationale is for this point of view. I have read enough about it to believe that it really is a question about statistical methodology rather than merely about psychology. However, I have to say that I remain unconvinced by the arguments of Dr. Rossano and other CoP detractors.

What would be most interesting would be a large data base of events that were judged by both systems. The ISU does has some figures of this sort, but they're not going to share them with the public, for obvious reasons.

The obvious reason (in case it wasn't so obvious after all, LOL), is this. If the results turned out to be pretty much the same, then it doesn't matter what system you use -- why bother to change? But if there are lots of differences in final placements, then that really opens a Pandora's box of which skaters are being robbed and whether the changes in the sport brought about by the CoP judging are for good or ill.

The ISU does not want such a debate. They want everyone to accept the CoP as a system that is more fair, harder to cheat at -- a system that does better than the old one towards insuring that the skater who skates the best wins.

Personally, I do not have a strong feeling one way or the other. Figure skating judging, and the public perception of figure staking judging, have bigger fish to fry than bickering over point systems.

Mathman

PS. Yeah, that standard deviation thing is one of the talking points of the anti-CoP crowd. This shows that, just like in the old 6.0 system, the part of the scoring that is completely up to the whim of the judges totally swamps all other variation in the numerical scores -- so why bother?

I think the casual fan knew what a 5.0-6.0 meant from each of the judges whom they knew by name and nationality of. That no longer exists. Instead they do not know who the judge is and the scores at the end of the competition total 195-230 are a bit bewildering because two skaters didn't seem to be that different. Speedy is right on to keep the casual fan from objecting to something they do not understand and no one to blame. No scandal!

Most of us have not seen the LPs of Irina and Susanna. If you can tape them and give all the scoring for each skater in your opinion, then compare them with the details given on the results. See if you agree and see if you concur with other members of GS. this would be a good exercise on whether the CoP is working. Since there are really no fans of Susanna and no one is anti Irina in Golden Skate, our results should be without bias. I would also like your opinions if this detailed scoring is easy on a one time showing.

Joe

Mathman said:

PS. Yeah, that standard deviation thing is one of the talking points of the anti-CoP crowd. This shows that, just like in the old 6.0 system, the part of the scoring that is completely up to the whim of the judges totally swamps all other variation in the numerical scores -- so why bother?

Click to expand...

Rather than dismissing standard deviation, I think it shows how close the skaters actually are much of the time. Statistics can show that the top three or five skaters, for example, could just as easily have earned gold, based on different combinations of strengths and weaknesses. It's ordinals that I find arbitrary, because there can be no ties, virtual or actual.

Joesitz said:

Most of us have not seen the LPs of Irina and Susanna. If you can tape them and give all the scoring for each skater in your opinion, then compare them with the details given on the results. See if you agree and see if you concur with other members of GS. this would be a good exercise on whether the CoP is working. Since there are really no fans of Susanna and no one is anti Irina in Golden Skate, our results should be without bias. I would also like your opinions if this detailed scoring is easy on a one time showing.

Joe

Click to expand...

Joe, there is one poster at ISU board who was at arena during lady's LP. Sounds like Elena Liashinco was the one who has better speed, better ice-usage among all the three. scroll down looking for Andy. He is a general fan as far as I can tell.

http://forums.isu.org/viewtopic.php?t=675&start=90

mzheng said:

Joe, there is one poster at ISU board who was at arena during lady's LP. Sounds like Elena Liashinco was the one who has better speed, better ice-usage among all the three. scroll down looking for Andy. He is a general fan as far as I can tell.

http://forums.isu.org/viewtopic.php?t=675&start=90

Click to expand...

I know speed comes up a lot in these threads but I personally find it overrated. speed to me is relative to the music and the choreography. If there is no variety in the speed there is no real performance. Ice usage, as you call it, is important. There is an arena to fill. But I've seen Liashenko prepare for a lutz taking up the entire arena before she actually jumps. Not my taste no matter how perfect the jump was. Maybe she's improved. that would be nice.

Mzheng - Let's see what the TV shows. They are not American so we can not expect anything more than Irina. The TV will do a whole number on T&M because of the fall in Skate America.

Joe

Joesitz said:

I know speed comes up a lot in these threads but I personally find it overrated. speed to me is relative to the music and the choreography. If there is no variety in the speed there is no real performance. Ice usage, as you call it, is important. There is an arena to fill. But I've seen Liashenko prepare for a lutz taking up the entire arena before she actually jumps. Not my taste no matter how perfect the jump was. Maybe she's improved. that would be nice.

Mzheng - Let's see what the TV shows. They are not American so we can not expect anything more than Irina. The TV will do a whole number on T&M because of the fall in Skate America.

Joe

Click to expand...

Having downloaded Liaschenko, I can't believe anyone would put her over Poykio or Irina. ITA with Joe that the way she sets up for her jumps totally distracts from the program. At least Irina's jumps have something going into them. For that lutz, Liaschenko stays on the outside edge forever before finally jumping. Plus Liaschenko two footed most of her jumps anyway.

hockeyfan228 said:

Rather than dismissing standard deviation, I think it shows how close the skaters actually are much of the time. Statistics can show that the top three or five skaters, for example, could just as easily have earned gold, based on different combinations of strengths and weaknesses. It's ordinals that I find arbitrary, because there can be no ties, virtual or actual.

Click to expand...

Well, as I understand it, the argument goes something like this. (I am playiing devil's advocate a little bit here. I have no quarrel with the CoP, I am just trying to understand the reasoning of the people who do.)

(a) There is nothing wrong with a close contest. If my basketball team loses 112 to 111 in triple overtime, that was a close game. Still, to use the language of probability, I am 100% sure that you won and I lost.

Yes, I can say, if only I would have made that last three-pointer, or if only that stupid referee hadn't wrongfully called me for a foul on the last play, I would have won. But I didn't, and he did, and I didn't.

(b) I want to decide which of two steel rods is the longer. I take ten measurements of each. The average is 10.34 cm for the first one and 10.33 cm for the second.

But suppose that the actual measurements are all over the lot, so that the standard error is, say 2 cm. The sampling error of the measurements completely dominates over the thing that is being measured. All we can say is maybe we are 55% sure that that the first rod really is longer.

According to many of the statisticians that have analyzed the two judging systems, the ordinal system is like (a) and the CoP is like (b).

Under ordinal judging, in order to win you have to garner more first place ordinals from the judges than your opponent (just like a basketball team has to make more baskets). If you secure 5 first place ordinals and your opponent secures 4, then you win. Period. End of discussion. Close game.

CoP judging is like (b). You average up all the GOEs and Component scores, etc., and whoever has the higher average is declared the winner. But if the standard error over the judging panel is larger than the difference in total scores, you might as well decide the contest by flipping a coin.

For example, here are the component scores for two skaters and five judges (down from 7 after high and low have been throw out).

Michelle: 50 60 40 10 90

Sasha: 100 0 25 76 50

Who won?

Now suppose judge number 4 decides, well Michelle stunk, but I'll give her a 12 instead of a 10.

Mathman

Mathman said:

Under ordinal judging, in order to win you have to garner more first place ordinals from the judges than your opponent (just like a basketball team has to make more baskets). If you secure 5 first place ordinals and your opponent secures 4, then you win. Period. End of discussion. Close game.

Click to expand...

Well, it's not always that simple, because sometimes no one secures 5 firsts.

Sometimes the ordinals are more like

A: 1 1 1 2 2 2 3 3 3
B: 4 4 4 4 1 1 1 1 2

Who wins?

Under the majority system, A will definitely beat out B on the strength of a better majority of 2nds. Whether they end up 1st and 2nd, 2nd and 3rd, or 1st and 3rd will depend on how the rest of the ordinals fall out among the other skaters.

(Just ask Shishkova & Naumov, who had long program ordinals very similar to B at 1996 Worlds and finished 4th.)

Under OBO, we don't even know from this much information which of the two will come out ahead. We know that B has a "win" over A by 5 judges to 4, but we don't know how many wins they each have over C and D.

Mathman said:

According to many of the statisticians that have analyzed the two judging systems, the ordinal system is like (a) and the CoP is like (b).

Under ordinal judging, in order to win you have to garner more first place ordinals from the judges than your opponent (just like a basketball team has to make more baskets). If you secure 5 first place ordinals and your opponent secures 4, then you win. Period. End of discussion. Close game.

Click to expand...

Ordinals do two things:

1. Force the judge into an arbitrary decision between skaters, when there may be little significance in the difference between them.

2. Flatten the closeness or distance between performances. A 5/4 split looks closer than a 7/2 split, but that isn't necessarily the case: in the first case, the 5 judges voting for Skater A may feel Skater A is 25% better than Skater B, while the 4 judges who voted for Skater B may feel that Skater B is 10% better than Skater A. In the 7/2 split, all nine judges may have felt that there was a slight difference between the two skaters and ranked them because the judges' job is to make that decision.

Also, to compare a game in which quality is not judged, just quantity -- i.e., a rimless three-pointer scores fewer points than four, wobble-around-the-rim free throws -- to a judged sport is a rather disingenuous analogy. If that were the case, then the judges could go home, and the callers could fight it out -- whoever completes the most difficult elements wins, regardless of quality. Period. End of discussion. Close game. (Or not.)

Mathman said:

CoP judging is like (b). You average up all the GOEs and Component scores, etc., and whoever has the higher average is declared the winner. But if the standard error over the judging panel is larger than the difference in total scores, you might as well decide the contest by flipping a coin.

Click to expand...

If the standard error over the judging panel is larger than the difference in total scores, then

a. The judges are judging incorrectly when compared to the written standards, whether by lack of training, unclear standards, or ignoring them, and the data shows this OR
b. There is virtually no difference between the performances when all criteria is taken into consideration. OR
c. The critera has been applied properly and judged correctly, but the results the system has created are "unexpected"
d. Any combination of the above. (And I'm sure there are other possibilities.)

There are plenty of judged sports. While gymnastics changes its code of points every few years, and it's impossible to compare competitions from 10 years ago to this year -- 1995's 9.9 requires a different set of elements than 2005's 9.9, due to the requirements for the starting base score -- last year's Worlds can be compared to this year's Worlds, based on scores. While no two contests are judged identically, you have the same issue in "non-judged" sports like baseball, where a strike zone is different among umps, but it is considered a level playing field if the strike zone is applied consistently.

What other judged sport uses placements? In what other judged sports can't you compare one performance and one competition to another? If that's not a red flag, I'm not sure what is.

Mathman: I know you're stating others' arguments. This isn't aimed at you.

Thanks, GKelly. Yes, it can get a little more complicated with more than two competitors, especially in some of the lower placements.

Hockeyfan said:

What other judged sport uses placements? In what other judged sports can't you compare one performance and one competition to another?

Click to expand...

Dog shows? Blueberry pie baking contests at the county fair? (Now you might think that's a wussy sport, but I'm sure we all remember the Mayberry episode where Aunt Bee whacked Thelma Lou on the knee...)

No, seriously, I think the model would be the juried art show. The judges say, this work gets the blue ribbon, that one the red. If you don't want to be judged, don't submit your work.

Ordinals (1) force the judge into an arbitrary decision between skaters, when there may be little significance in the difference between them.

Click to expand...

Yes. This is the function of judges. Leave out the word "arbitrary" and this is the definition of what judges do.

(2) In the first case, the 5 judges voting for Skater A may feel Skater A is 25% better than Skater B, while the 4 judges who voted for Skater B may feel that Skater B is 10% better than Skater A. In the 7/2 split, all nine judges may have felt that there was a slight difference between the two skaters and ranked them because the judges' job is to make that decision.

Click to expand...

It is indeed. I do not see how that is an adverse criticism of the ordinal judging system. Yes, the judges must judge, even when they feel that the contest was close.

But I think the main point made by critics of the CoP, or of any system that attempts to assign numerical values to the judges preferences for one skater over another, is right there in your example. I think an experienced and knowledgable judge would have no trouble at all in deciding, of the two performances, I thought A was better than B, and of giving reasons for this choice. It is when you start giving point values to your choices that things start to get "arbitrary."

Do we really think that a judge can tell the difference between "skater A was 25% better than skater B" and "skater A was 26% better?" It is the assigning of numbers that is arbitrary, not the choice of who deserves the first place ordinal.

(Well, if anyone has read this far, LOL...)

There have been many psychological studies of the question, how many different grades of things can we distinguish among? The answer is very consistently: 7, for most people. A typical experiment goes like this. You show the subject 6 wooden poles of varying lengths. Then you take them away. Then you show the subject one of them, and ask: Is this the first, second, third, fourth, fifth or sixth longest. Most people will get it right almost every time.

But if you do it with eight poles, very few people can do it. With seven, most people can do it often but sometimes make mistakes. (You can train yourself to do a little better.)

BTW, this justifies scoring figure skating performances from 0 to 6. If you try to score from 0 to 10, say, a typical judge will not be able to give a consistent meaning to a grade of 4, for instance. It also says that in ordinal judging, having the skaters compete in groups of 6 is just the right size. A judge can remember the performances of 6 skaters, and the ordinal ranks that he/she wants to assign to them, but in a group of 8, this is too hard. (Thus a skater in the second group almost never moves up to the top six after the LP, even if there is a lot of movement within the top group and the within the second group separately.)

So that is supposedly an argument in favor of the old judging system. Even in a close contest, it is quite possible for a judge to see six performances and rank them first to sixth, and to do this consistently and in accordance with accepted conventions of what constitues a superior performance.

But it is not possible for that same judge to give component marks, let's say a 7.2 for choreography in a particular performance -- and I mean that performance deserves a 7.2, not a 7.1 or a 7.3 -- and do that with any sort of consistency from one performance to another, or from one event to another. The same judge will give different marks to exactly the same performance, on two different viewings. In fact, super critics of the CoP such as Rossano, claim this kind of true arbitrariness is responsible for most of the variation in scores that we are seeing under CoP judging.

Bottom line, it is hard to look at the component scores that the judges are giving out and have any confidence that they know what they are doing.

BTW, although the average person can usually distinguish among no more that 7 objects, if we take any two of them and compare, people can tell which one is longer 100% of the time, even if they are very, very close in length. I suppose this is the rational for OBO (one-by-one) analysis of ordinals, where each skater is compared "one by one" with the others until someone has the majority of highest ranking ordinals.

Mathman (I wish we still had that little devil smiley :evil: )

Mathman said:

No, seriously, I think the model would be the juried art show. The judges say, this work gets the blue ribbon, that one the red. If you don't want to be judged, don't submit your work.Yes. This is the function of judges. Leave out the word "arbitrary" and this is the definition of what judges do.

Click to expand...

On the whole, juries in the arts work like juries in a courtroom: they get together and decide the winner after long and often heated discussions, and very often, after a deal has been made. They don't leave the jury room until a decision has been made, and most of the time, they are allowed to declare joint winners, and they are also allowed to declare no winner, if the standards haven't been met. Not exactly the same in figure skating.

Mathman said:

It is indeed. I do not see how that is an adverse criticism of the ordinal judging system. Yes, the judges must judge, even when they feel that the contest was close.

Click to expand...

The adverse criticism of the system is when things are very close -- assuming that they can make the appropriate and constant readjustments to placement, which I question -- is that the difference between two or more skaters may be insignificant, yet the judges choose anyway. At that point, they may as well be tossing the same coin as under CoP, but because they choose, the results are "definitive."

Mathman said:

But I think the main point made by critics of the CoP, or of any system that attempts to assign numerical values to the judges preferences for one skater over another, is right there in your example. I think an experienced and knowledgable judge would have no trouble at all in deciding, of the two performances, I thought A was better than B, and of giving reasons for this choice. It is when you start giving point values to your choices that things start to get "arbitrary."

Do we really think that a judge can tell the difference between "skater A was 25% better than skater B" and "skater A was 26% better?" It is the assigning of numbers that is arbitrary, not the choice of who deserves the first place ordinal.

Click to expand...

If there were no standards for assigning the values, I might agree, but what is the difference between assigning a numeric values to two skaters, one of which is slightly higher, and having a judge try to decide that Skater C was 26% better than Skater A and Skater B was 25% better than Skater A, so the ordinals should be:

1-Skater C
2-Skater B
3-Skater A

Mathman said:

It also says that in ordinal judging, having the skaters compete in groups of 6 is just the right size. A judge can remember the performances of 6 skaters, and the ordinal ranks that he/she wants to assign to them, but in a group of 8, this is too hard. (Thus a skater in the second group almost never moves up to the top six after the LP, even if there is a lot of movement within the top group and the within the second group separately.)

Click to expand...

Exactly--if there were six skaters in a competition, then perhaps the outcome would be "right," but there aren't; there may be as low as nine in a GP event, but for Worlds, the top 24 from the SP skate in the LP.

Mathman said:

But it is not possible for that same judge to give component marks, let's say a 7.2 for choreography in a particular performance -- and I mean that performance deserves a 7.2, not a 7.1 or a 7.3 -- and do that with any sort of consistency from one performance to another, or from one event to another. The same judge will give different marks to exactly the same performance, on two different viewings. In fact, super critics of the CoP such as Rossano, claim this kind of true arbitrariness is responsible for most of the variation in scores that we are seeing under CoP judging.

Click to expand...

That's why the scoring is in chunks of .25, not chunks of .1, which translate into just meets requirements for level, exceeds level in some sub-component, exceeds requirements for the level in at least half the sub-components,and exceeds the requirements for the level in most sub-components, but doesn't meet requirements for the next level. There are enough sub-components listed in the descriptions to make this decision for each component score.

If you really look at the PCS marks, there is no more than a 6-level range in any competition, unless there a country sends the occasional "Eddie the Eagle." (This happened in DC, in which Armenia sent a coach who was asked by his Federation to go at the last minute, and he had no programs to speak of. He may have merited in the 2'-3'ss for choreography and performance/execution.) Realistically, anyone who makes it to Worlds, unless they bomb completely, could qualify in the 4-9 range. (In a regional competition, this might be closer to 1-5.) So it's a matter of determining which of six levels to start with, which really means choosing between a set of two, and then determining how many of the subcomponents the skater qualified for. The criteria and sub-criteria are there on the screen. It's a two step process.

Mathman said:

Bottom line, it is hard to look at the component scores that the judges are giving out and have any confidence that they know what they are doing.

Click to expand...

I would argue the same about ordinals, but in the case of component scores, there is a lot of data to back up whether the results match the code, which is impossible under ordinals, with the exception of the technical score in the SP, in which one could extrapolate whether the required deductions were taken.

Having read through most of the posts now and understood some of them, my opinions waver with regards to the CoP

My over-riding impression is that it doesn't matter how good the scoring system, the corupt will always find a way to beat it. CoP is suffering a triple whammy at the moment 1) as potentially good as the system is, there are inherent flaws that as with any statistical model cannot be ironed out or recognised until after it has been applied to a real situation, not exactly fair on the skaters who may suffer from this but what else can you do?
2)It is a system that was rushed into place, sorry but I do not believe Speedy and his claims that it has been in the offing for a number of seasons, and the people using it are not well enough trained in it to make adequate use of it.
3)The corruption and politics are still rife and because this was not properly dealt with, the CoP was on a loser before it started.