What Do You All Think About CoP Now?

[Mathman]

I have to say, I never did understand ordinals, they completely confused me,...

Me, too.

The reason that ordinal judging is so tricky is that you cannot do arithmetic with ordinal placements, and hence you cannot subject them to statistical treatment like you can do with regular numbers. (You cannot apply all those formulas that have sigma over the square root of n, for instance. )

Ordinal placements cannot be added together.

1+2=3. Yes.

1st place + 2nd place = 3rd place. No.

You cannot average ordinal placements.

The average of 1 and 2 is 1.5. Yes.

The average of 1st place and 2nd place is one-and-a-halfths place. No.

Also, the ordinals given by each judge are not independent. So you can only compare the entire list of placements given by one judge against another. This is very complicated when there are more than two judges.

If fact, there are only two mathematical operations that you can do with ordinals. You can count them. This leads to the "majority of ordinals" system. You count up how many first place ordinals someone has. If a skater has a majority of first place ordinals, that skater wins. If no one has a majority of first place ordinals, then you count up the number of first and second place ordinals together for each skater, and so on.

The other thing you can do with ordinals is compare them. Fourth place is worse than third. If you do this for each judge and each skater one by one you get the "OBO" method. Both methods are open to strange anomalies like "flip-flops." This occurs when two skaters who have already competed have their ordinals suddenly reversed when some third skater beats yet a fourth skater, both of whom are below the original two. There is no ordinal system that can avoid flip-flops (and have a certain set of other desirable features) except in the case of a dictatorship (only one judge).

The CoP avoids all that (making it less interesting, IMHO )

And as for the actual scores, it was fun to see if the judges scores matched what you thought they'd get, but it was all pretty predictable and almost funny since when I'd think someone would get 5.7, low and behold, most of the scores were 5.7's.

Yeah, I agree, that was fun. That was fun.

Or if you're used to sports like speedskating where total points (times) from a first race carry over to the second race, so that it makes no difference who wins the second race, just whether they beat the "time to beat"

It's no coincidence that some of the specifics touted in the change of scoring system were championed by a speed skater.

That is interesting, because on television broadcast now they do exactly that. When the skater is in the Kiss and Cry for his marks the announcers say, "He needs 102.34 points in the LP to move into first place overall. And here are his marks! Oh, only 100.99 -- he's into second."

That's actually pretty cool.

 

[jettasian]

Yu-Na Kim has won SPs by some pretty big margins - over 15 points at the 2009 TEB and 2009 Skate America (she did not win the LP at SA).

Brian Joubert built up some pretty big leads back in the day. 2007 Skate Canada and 2008 CoR come to mind (he didn't win the LP at either of these events).

Davis and White had a 10 point lead out of the SD at this season's Skate America.

Shen and Zhao's lead in their comeback event, 2009 CoC, was almost 10 points. Their lead at that year's Skate America was even bigger.

So now that we have non-Chan examples for all disciplines, can we move on?

I don't mind skaters winning it in the SP. It is much harder to build up a huge SP lead with fewer elements and smaller PCS factors, so more power to those who are dominant enough to win it in that segment.

Well, it depends on the definition of "big lead". Is 10 point considered big? It's just one jump away. but 5 points of Ice Dance can be very big.

 

[jettasian]

^ IMHO that's CoP-think. Add up some points, then add on some more points. Yes, if you have already decided that this is the way figure skating should be scored, then there is nothing further to discuss.

But I like better the model of the semi-finals, then the finals. If you do well in the semi-finals, your reward is that you make the finals. You don't get to carry your score over, however impressive it may be.

As for "artificially" creating excitement, it is artificial only if you have already bought into the CoP model. I don't see anything artificial or disrespectful to tell the athletes that in order to win the championship they must skate two excellent programs.

I think the unit to be judged should be the program. If you win the short program over your opponent, then you are ahead going into the long by a score of 1 to 0. (I do realize that I am a voice crying in the wilderness in this regard. )

But if one semi finalist is way better than the other semi finalist, and s/he deserves to be way ahead.

 

[Mathman]

^ Again, yes if you think a priori that the CoP is the right type of scoring system for figure skating. No if you think otherwise.

Actually, I like the the IJS just fine. What frustrates me is arguments that go like this:

Premise: The CoP is good.
Conclusion: Therefore the CoP is good. Q.E.D.

Also -- I apologize for posting in a testy fashion earlier. Sorry.

 

[Violet Bliss]

I abandoned a half written post earlier when I realized I didn't want to spend time and energy to understand 6.0 better just for the sake of discussion. Studying Latin would be a better investment if the only use is scholastic or to impress the lesser unlearned.

 

[mskater93]

The one thing that is bothersome about IJS is that the general premise is sum of parts = whole program when actually sum of part =/= whole program. Just because you put up a lot of tech and fill the ice with a bunch of meaningless turns, it doesn't make it a good PROGRAM. The concept of 6.0 was to judge the program as a WHOLE and in relativistic fashion (skater A's tech was a little better than skater B, but skater B's PROGRAM was a lot better than skater A).

 

[brightphoton]

I've never seen a ski jumping contest where commentators offered conspiracy theories about the results based on nationality of the officials

One of the things I was sick of from 6.0 and am glad to see disappear forever. Good riddance.

 

[ImaginaryPogue]

Mathman, I don't think you're being fair.

 

[Dragonlady]

Which is most meaningful when there's a large field with many skaters at similar or overlapping levels of general skill, so that expert comparisons of how many competitors I was better than today is reasonably correlated with how well I skated today

Under 6.0 the judges had to rank the skaters correctly from top to bottom. In reality, 5.7, 5.8, 6.0 mean nothing. Only the ordinals mattered. Skaters who skated early in the competition were handicapped because the judges “had to leave room” in case someone skated better. What’s worse, scores tended to rise as the competition wore on and after 20 or so skaters, someone who skated brilliantly often got left behind and didn’t finish as high as they might have had they skated later in the competition.

Usually the best skaters did come out on top, but for skaters in the middle rankings in a field of 20+ skaters, there were a lot of questionable placements. These were performances which had errors, but sorting out whether Sally’s fall on her salchow was worse than Suzy’s fall on her flip when Suzy had better spins, and Sally was faster, and where does Erica fit in – jumped well but was slow as molasses. Just the whole idea of ranking 20+ skaters from top to bottom on first viewing, boggles the mind.

Every time someone says they “understood” 6.0, it turns out they really don’t understand 6.0, ordinals or factored placement, but it was familiar and they were comfortable with it. CoP has taken people out of their comfort zones but the idea that get X number of points for doing a salcow, GoE if you do it well, you total up all your points and the skater with the highest points wins. That’s, a lot less confusing than it depends on how many people you beat on the grid, and how many judges placed you first versus how many judges placed Sally or Erica first.

 

[Mathman]

I abandoned a half written post earlier when I realized I didn't want to spend time and energy to understand 6.0 better just for the sake of discussion. Studying Latin would be a better investment if the only use is scholastic or to impress the lesser unlearned.

If you study Latin you can read neat stuff like Caesar's conquest of Gaul. What makes figure skating judging worth thinking about is its application to voting systems and social choice. A good number of these ideas where developed (and vigorously debated) by French mathematicians in the aftermath of the French Revolution in the 1780s and 90s, as political scientists struggled with questions about how best to configure the new republic.

(In the event, Napoleon came along and that was the end of it.)

Mathman, I don't think you're being fair.

I understand what you mean, and I agree, I'm not. I will stop.

 

[Spun Silver]

(Shorthand for all appropriate disclaimers re: my lack of expertise in the intricacies of the COP and for that matter skating technique... I present this purely as a layperson's experiment.)

I've been wondering about the dilemma Mathman brings up of the skater who does a great SP and then a dreary FP with falls that seriously break up the program but that don't not seem to affect the outcome. You can think of me as a typical enthusiastic but uninformed audience member who goes, "Huh? That's nuts. Skater X was way better tonight!" The proliferation of difficulty without competence under the present IJS seems, IMHO, to require a counterbalance in the direction of competence.

The solution I keep wondering about is a clean-program bonus. I'm very rushed right now (getting ready to go to a funeral out of state) so I can only give a brief example of how it would have affected actual scores.

I invented a 5-point clean-program bonus for the FP only because (sticking with MM's problem case) I would like to see audiences going from the final part of the event with a sense that justice was done, so to speak. Why 5 points? I thought the bonus should be round about the level of an easier triple jump, in order to be substantial, but not as much as a hard one, but maybe it should equal a flip or lutz. I am awarding it to anyone who got no deductions in the FP, as I'm defining clean as no falls. (That could certainly be debated as a lot of stepouts are ugly as heck and certainly detract from the overall effect, but like I said, I'm very rushed.)

Applying this bonus to the Ladies event in US Nats (which seemed rather dreary this year), I found that it would change the overall outcome slightly. The podium, instead of Wagner-Czisny-Zawadski-Zhang, would be Wagner-Czisny-Zhang-Zawadski. Of course it has more of an impact in the FP where the bonus is actually applied. Caroline and Rachael would go 2-3 instead of 3-4 and Alissa would drop to 4th. Going down the line of the top 10, there is no further difference from the actual placements until you get to Lam and Baga, currently 9 and 10, who would trade places.

So, at least at 5 points, a clean-program bonus would not radically change the scores, but it would give a little boost that could change the podium in some cases. When I get back I may play with this a bit, maybe making it 6 or 7 points to provide a bigger incentive to do clean programs. Any thoughts?

 

[Mathman]

^ I think that the same effect could be achieved by giving 0 credit to an element ending in a fall. The designation on the protocol would be "element not completed." (Again the problem would be half-falls, hand down, etc.)

Without changing the rules, I also think that the judges can, and maybe do, give a "clean program bonus" in the program components. Choreography and musical interpretation are marred by falls, certainly.

 

[mskater93]

Giving zero points for a fall would encourage safe programs (think 2008-2010 Men's events). For me:
Each fall should be a -3 GOE mandatory, 2% off the total technical element score (this equates to 1 point for a TES of 50 points), and 0.25 lost on the PE mark and 0.25 off the SS mark for each fall (basically have the IJScalc take these points off from the judges' final score when a tech panel marks an element as a fall). I don't care whether the skater got up quickly, it still detracts from the overall presentation of the program and a fall is a direct reflection of a skaters' ability to skate - if they fall, they did something "wrong" on the ice.

 

[Mathman]

Every time someone says they “understood” 6.0, it turns out they really don’t understand 6.0, ordinals or factored placement, but it was familiar and they were comfortable with it. CoP has taken people out of their comfort zones but the idea that get X number of points for doing a salcow, GoE if you do it well, you total up all your points and the skater with the highest points wins. That’s, a lot less confusing than it depends on how many people you beat on the grid, and how many judges placed you first versus how many judges placed Sally or Erica first.

To me, it is a different kind of confusing.

Under ordinal judging the job of the judge was quite simple. This skater was best, this skater second best, that one third. The complicated part – well, that was done by computer and usually was of little interest.

Under the CoP it really is impossible for the average fan to know just what it is that the technical panels and judges do. This spin was rated level 3 because it had such and such features, but lacked thus and so. This footwork sequence got +1 GOE because it satisfied three bullets but not four. This skater deserved a 7.25 -- not a 7.00 and not a 7.50 -- in choreography because...um...

Under CoP the job of the computer is easy to understand. Add up the points. But to try to understand what the judges do, IMHO, is way more challenging.

 

[jenaj]

Under 6.0 the judges had to rank the skaters correctly from top to bottom. In reality, 5.7, 5.8, 6.0 mean nothing. Only the ordinals mattered. Skaters who skated early in the competition were handicapped because the judges “had to leave room” in case someone skated better. What’s worse, scores tended to rise as the competition wore on and after 20 or so skaters, someone who skated brilliantly often got left behind and didn’t finish as high as they might have had they skated later in the competition.

I think judges still do some ranking in the PCS. I think they view some skaters, like Patrick,as 9s and others, like Rachael, as 6s.

 

[ImaginaryPogue]

For me, COP is easier to understand. However, 6.0 is easier to enjoy without understanding.

Bluebonnet, I love high value elements, but I don't think a high value element should be the "Be-all-and-end-all" of skating. Take this as a for instance....

The quad toe base value: 10.3
A quad toe w/ -3 GOE: 10.3 - 3 = 7.3
A solid but not outstanding triple lutz = 6.0

Now, the fall deduction is applied to the program as a whole, not to the element itself I don't think this is an accurate representation of the sport. I would prefer if a fall on an element was equal to that element landed with one less rotation, or approximately 50% (whichever is higher). I'd have to give it more though, truthfully.

Make no mistake, I'm not advocating an all-or-nothing here. I don't believe that was ever the case, regardless of the judging system. I think the unintended consequences of an all-or-nothing system outweigh the benefits. I think that the ISU panicked a little when they saw a quadless guy win the Olympics over a quaded guy who was "clean" (whereas when Buttle and Lysacek won their world titles, Joubert had fallen, so we could rationalize) and made far too many changes that promoted the quad. [sarcasm]The ISU not showing foresight will shock everyone who saw the rule changes to the GP this season or Plushenko at Worlds [/sarcasm].

 

[jettasian]

Under ordinal judging the job of the judge was quite simple. This skater was best, this skater second best, that one third. The complicated part – well, that was done by computer and usually was of little interest.

It's easier said than done. The judges watch...what, 24 skaters? How would they remember skater #12 is better than #6, but not as good as #10. Skater #22 is better than 12, but almost the same as #8, yet worse than #4.....yeah, like it makes any sense. Like Dragonlady said, 6.0 doesn't make sense.

 

[drivingmissdaisy]

It's easier said than done. The judges watch...what, 24 skaters? How would they remember skater #12 is better than #6, but not as good as #10. Skater #22 is better than 12, but almost the same as #8, yet worse than #4.....yeah, like it makes any sense. Like Dragonlady said, 6.0 doesn't make sense.

In practice I don't think it's quite as complicated, as you are only having to evaluate one skater at a time. (i.e. they don't have to remember anything because their placements for previous skaters are right in front of them; they only have to put the current skater above one other and/or below one other. Decisions are made on a real time basis for each athlete.)

Probably the hardest thing for me to wrap my head around is how programs with such low technical content in the jumps are doing well. I'm speaking in particular of the ladies events, in which programs that would have likely not medaled in any major competition from 1992-2006 place quite well now. For the casual fan who has been following for years and doesn't understand transitions or skating skills, seeing jump content that Kristi Yamaguchi or Oksana Baiul could easily surpass might make them scrath their heads.

 

[Mathman]

For me, COP is easier to understand. However, 6.0 is easier to enjoy without understanding

Maybe that is what I have been struggling to say all this time, without success. 6.0 is easier to enjoy.

 

[gkelly]

Sorry for the long post -- I was away for a day and I'm trying to catch up.

I invented a 5-point clean-program bonus for the FP only because (sticking with MM's problem case) I would like to see audiences going from the final part of the event with a sense that justice was done, so to speak. Why 5 points? I thought the bonus should be round about the level of an easier triple jump, in order to be substantial, but not as much as a hard one, but maybe it should equal a flip or lutz. I am awarding it to anyone who got no deductions in the FP, as I'm defining clean as no falls. (That could certainly be debated as a lot of stepouts are ugly as heck and certainly detract from the overall effect, but like I said, I'm very rushed.)

By "deductions" do you mean no fall deductions? Does it apply to music or costume or long lift deductions as well?
Or do you also mean no negative GOE for any reason?

Would the 5-point bonus apply to the freeskating only? So a skater who fell twice in the SP but didn't fall in the LP would get the bonus, but a skater who skated the only clean program in the short and had one fall in the long would not?

How about smaller bonus for a clean short and a larger bonus for a clean long (2 and 3 points, respectively, or 3 and 5 points . . . might need to depend on the discipline and level of competition).

5 points seems like a reasonable size bonus for the current middle-to-high senior ladies' field. But the effects might be different at other levels.

E.g., among the elite senior men, maybe skater A does two clean quads and two clean triple axels but falls on two lesser triples and all level 4 elements. Skater B skates "clean" with one triple axel and one underrotated quad (< sign and average -1GOE), and easier spins and steps. Does B get the bonus? Even if he does, it might not be enough to make up for A's lead in base mark. Not even taking PCS into account.

On the other hand, for girls trying to get past the qual round at Junior Worlds, maybe Skater C lands one of each kind of triple up to lutz but falls on a second triple lutz: no bonus. Skater D does two triple sals, two triple toes, and a bunch of doubles but doesn't fall. If C is better than D in other ways besides jump content, the bonus might not make that much difference. But it does seem weird to give D a bonus equal in value to a jump she's not even capable of attempting.
Unless you think that the difference between falling once and not falling at all should be equal to the equivalent 0.5 in all five each program component for men and close to 0.75 for women (multiplied by the long program compnent factors of 2.0 and 1.6 respectively).

^ I think that the same effect could be achieved by giving 0 credit to an element ending in a fall. The designation on the protocol would be "element not completed." (Again the problem would be half-falls, hand down, etc.)

And what about falls that don't occur on jumps? Especially with step sequences (especially dance or pair step sequences where the fall occurs during a dance hold and the partner can quickly pull the fallen partner to his feet), the element doesn't always end with the fall and there may be some good stuff in the post-fall part of the element.

But there are already certain kinds of mistakes that invalidate complete elements, so we could decide that falls should be one of them. As is, single and double jumps and most non-jump elements end up with less than 1 point for the element if they earn -3 GOE (as is likely with a fall), so after the fall deduction they lose points. Just making the element worth nothing and not taking the 1.0 fall deduction would actually be more forgiving for these elements.

What about falls between elements? I think the whole point of fall deductions being subtracted from the total score rather than from elements was to penalize those as well. By putting the penalties back into the element scores only, that would leave only PCS as a place for judges to penalize these often "silly" types of falls.

I could live with that.

Without changing the rules, I also think that the judges can, and maybe do, give a "clean program bonus" in the program components. Choreography and musical interpretation are marred by falls, certainly.

But by how much often varies from one incident to another, let alone from one observer to another. So we'd have to accept that sometimes P's fall was more disruptive than Q's and deserved to be penalized more in the PCS, and that sometimes we thought Q's fall was disruptive but some of the judges thought that Q did a great job of maintaining connection to the music and audience and program theme while falling and recovering and didn't penalize Q at all in those components.

Under ordinal judging the job of the judge was quite simple. This skater was best, this skater second best, that one third. The complicated part – well, that was done by computer and usually was of little interest.

As others have mentioned (see below), the process wasn't really that simple and was subject to a lot of noise and variation depending in part on skate order, especially in the middle ranks of large fields.

The numbers presented -- one mark for technical merit and one for presentation -- look simple, and the single ordinal placement that they really represent look even simpler. So it's easy for outside observers to think that the process was simple. But really judges had to consider almost everything that both the judging panel and the tech panel now take into account and do their own weightings. Much of it was by overall gut feeling rather than adding up numbers.

I invite you to choose a competition for which you can find video of 18-24 long programs and score them all yourself according to everything you know about what should be rewarded as good skating and what should be penalized. Difficulty of the spins and steps does count. Jump rotations and takeoff edges and one-foot back outside landings do count. Transitions or in-betweens do count. Speed and edge quality and multidirectional skating count. Keep good notes, knowing that the referee might ask you why you put Skater X 5th when the panel as a whole put her 15th, or vice versa.

Not so simple after all.

Under the CoP it really is impossible for the average fan to know just what it is that the technical panels and judges do.

Under 6.0 it was really impossible for the average fan to know just what it was the judges do (i.e., all of the above and then some). But because the numbers looked simple, it was possible for fans to believe the process was a lot simpler than it really was. And the real process was all hidden.

Most fans don't really want to go to the trouble of learning what makes a level 3 spin or a +1 step sequence, but if they do want to the rules and guidelines are right there in black and white for them to see, as well as the actual decisions (results, not reasons) for each element. For those who want to become educated fans, it's a lot easier to do so under IJS.

This skater deserved a 7.25 -- not a 7.00 and not a 7.50 -- in choreography because...um...

At that level of distinction, it comes down to individual judges' sense of the standard and there could be confounding effects such as skate order. If you think the skater deserves 7.0 or 7.5 but all the judges give 7.25, you're not wrong.

But if you want to know why this skater deserved approximately 7.25 and not 6.25 or 8.25, if you watch enough above-average, good, and very good performances you can get a sense for the differences between those different levels of accomplishment, same as you could get a sense under 6.0 of which skaters started out with presentation worthy of high, middle or low 5s or lower.

I think judges still do some ranking in the PCS. I think they view some skaters, like Patrick,as 9s and others, like Rachael, as 6s.

But when they give those scores, they're not saying Patrick is 3 points better than Rachael in each of these components. (Especially since Patrick and Rachael would never compete against each other.) They're saying Patrick meets or exceeds their standard for "superior" skills in those areas and Rachael is somewhere in the range of "above average" to "good."

If the overall skill level of two different skaters in the same event is that far apart, there's no need to compare them directly to each other, any more than under 6.0 a judge would look back at a skater he's awarded 5.6 or 5.3 for presentation and ask whether another skater was better or worse before awarding 5.9.

If the skaters were close in ability, then the judge needed to make a direct comparison under 6.0. Under IJS they don't need to, but it's probably good to make sure they're consistent in their use of the scale throughout the event (e.g., if they started marking high for the first few skaters, to stay high for the rest). And when marking Skater K, it might be meaningful to remember the marks already given to Skater J and think "slightly weaker on skating skills, so 0.25 lower than I gave J there, about the same on transitions, but significantly stronger on performance, choreo, and interp, so 0.5, 0.75, 1.0 higher on those."
These decisions aren't actually rankings, though, because the judge can't keep track of what the TES added up to -- they can make an effort to rank the PCS (in this case, making a clear decision to give K higher overall PCS than J) -- but they can't guarantee that base mark plus GOEs plus PCS might not put J ahead overall if you just added up that judge's column of scores.

It's easier said than done. The judges watch...what, 24 skaters? How would they remember skater #12 is better than #6, but not as good as #10. Skater #22 is better than 12, but almost the same as #8, yet worse than #4.....yeah, like it makes any sense. Like Dragonlady said, 6.0 doesn't make sense.

In practice I don't think it's quite as complicated, as you are only having to evaluate one skater at a time. (i.e. they don't have to remember anything because their placements for previous skaters are right in front of them; they only have to put the current skater above one other and/or below one other. Decisions are made on a real time basis for each athlete.)

Again, I invite you to try it. The first six skaters will be easy. The second six might offer some challenges in deciding which skaters in the first group to compare the current skater to, but it shouldn't be too hard. By the third and fourth groups, it can become all too easy to forget about earlier skaters who were close in general ability for different reasons. This will be even more true in an unseeded large group like a short program or qualifying round (or juvenile freeskate, since they only skate one program).

You need to make sure not to tie any of the skaters. Especially not by mistake because you forgot you'd already given those scores to someone else, or because you boxed yourself in with the numbers and it's too late to change the scores for earlier skaters to give yourself room.
Using the numbers properly to keep track of the rankings isn't really a skating-evaluation skill per se, but it was a skill that judges needed under 6.0 and that fans who only watch six skaters at a time don't need to worry about.

Probably the hardest thing for me to wrap my head around is how programs with such low technical content in the jumps are doing well. I'm speaking in particular of the ladies events, in which programs that would have likely not medaled in any major competition from 1992-2006 place quite well now. For the casual fan who has been following for years and doesn't understand transitions or skating skills, seeing jump content that Kristi Yamaguchi or Oksana Baiul could easily surpass might make them scrath their heads.

Well, they're doing well in relation to the field they're competing against. If they have to compete against skaters with stronger jump content and also stronger everything else, they wouldn't do so well.

I'm sure Yamaguchi with her 1992 skills packaged into a program designed to meet the 2012 rules could place quite well in the current field. (Not sure she could have medaled at 2010 Olympics, but probably a final group contender, comparable to Flatt's and Nagasu's results. The field is usually strongest in Olympic years.)

Baiul is probably not a good comparison. Yes, she could do five different triples but she had trouble with landing on one foot and on executing combinations, and she didn't really show (or have?) the skills to earn high levels in steps or high GOEs in spins. I think her PCS would have been all over the place, probably highest in Interp, or P/E for a good performance.