You Be the Judge | Golden Skate

You Be the Judge

skatinginbc

Medalist
Joined
Aug 26, 2010
Forget about 6.0. Forget about CoP. Forget about any scoring system or criteria that the authorities have prescribed. Watch the following SPs and LPs and rank them based on actual performances, not on a skater's reputation or potentials, or on the published official results, or on what your dream podium would be. Remember the Long weighs twice as much as the Short:

Competition #1 (Japanese Nationals):
Yuzuru Hanyu: Long: http://www.youtube.com/watch?v=RXX7mfeKJ_M Short: http://www.youtube.com/watch?v=MSfxAYKqJNc
Takahiko Kozuka: Long: http://www.youtube.com/watch?v=azKlGGypwT8&feature=related Short: http://www.youtube.com/watch?v=bQqDLc_BkKY.
Daisuke Takahashi: Long: http://www.youtube.com/watch?v=dpWWi_6G2xo&feature=related Short: http://www.youtube.com/watch?v=ziQ253eUi2I.

The CoP had the following placements:
1. Takahashi (96.05 + 158.55 = 254.60)
2. Kozuka (85.60 + 165.37 = 250.97)
3. Hanyu (74.64 + 167.59 = 241.91)

The 6.0 would have had this order:
1. Hanyu (4/2 + 1 = 3)
2. Kozuka (2/2 + 2 = 3)
3. Takahashi (1/2 + 3 = 3.5)

A "no-credit-for-a-fallen-jump" system would have had this order:
1. Kozuka (250.97-4.48 = 246.49)
2. Hanyu (241.91)
3. Takahashi (254.60-7.3-4.68 -1.97 = 240.65)

Competition #2 (Cup of China):
Nan SONG: Long: http://www.youtube.com/watch?v=ORWZUPXIam8 Short: http://www.youtube.com/watch?v=uZQtz52Dq74
Nobunari ODA: Long: http://www.youtube.com/watch?v=Tw0jGphMdRE Short: http://www.youtube.com/watch?v=WXPXhuhGva8
Jeremy ABBOTT: Long: http://www.youtube.com/watch?v=7G_5igglTsc Short: http://www.youtube.com/watch?v=svMXuvFwCuo

The CoP had the following placements:
1. Abbott (79.32 + 149.17 = 228.49)
2. Oda (77.65 + 149.46 = 227.11)
3. Song (72.72 + 154.03 = 226.75)

The 6.0 would have had this order:
1. Song (5/2 + 1 = 3.5).
2. Oda (4/2 + 2 = 4).
3. Abbott (3/2 + 3 = 4.5).

A "no-credit-for-a-fallen-jump" system would have had this order:
1. Song (226.75)
2. Oda (227.11 – 1.60 = 225.51)
3. Abbott (228.49 – 7.30 = 221.19)

Which system matches best your judgment on those performances? Or which system gets the final placement right in your opinion?
 

let`s talk

Match Penalty
Joined
Sep 10, 2009
The 6.0 would have had this order:
1. Hanyu (4/2 + 1 = 3)
2. Kozuka (2/2 + 2 = 3)
3. Takahashi (1/2 + 3 = 3.5)

The 6.0 would have had this order:
1. Song (5/2 + 1 = 3.5).
2. Oda (4/2 + 2 = 4).
3. Abbott (3/2 + 3 = 4.5).
That's why I find the 6.0. more meaningful.
 

skatinginbc

Medalist
Joined
Aug 26, 2010
That's why I find the 6.0. more meaningful.
If the 6.0 results match best your judgment, then I think you might agree that one of the problems associated with the CoP is that someone can win the whole thing with a big lead in the Short despite a disastrous performance in the Long. The competition is basically over after the Short. The drawbacks of 6.0 mainly concern its broad categories (i.e, Techniques vs Presentation), so broad that a category involves too many different skills for the judges to apply the same criterion. We can combine the advantages from both systems: Use CoP's detailed approaches to score Technical elements and PCSs and yet 6.0 logic to determine the final placement. Problems solved.
 
Last edited:

skatinginbc

Medalist
Joined
Aug 26, 2010
A "no-credit-for-a-fallen-jump" system would have had this order:
1. Kozuka (250.97-4.48 = 246.49)
2. Hanyu (241.91)
3. Takahashi (254.60-7.3-4.68 -1.97 = 240.65)

A "no-credit-for-a-fallen-jump" system would have had this order:
1. Song (226.75)
2. Oda (227.11 – 1.60 = 225.51)
3. Abbott (228.49 – 7.30 = 221.19)

This system happens to yield the results closest to my judgement. Hanyu had a poor skate in the Short but 6.0, which concerns only the ranking order, would have negated the huge difference between his and the other two competitors. The 6.0 system is norm-referenced (i.e., comparing one's performance against the norm). When there is not much "norm" to compare with in a small competition, the results can be rather skewed. For instance, if there were only three competitors, the 6.0 outcome would have been: 1. Hanyu (3/2 + 1 = 2.5). 2. Kozuka (2/2 + 2 = 3). 3. Takahashi (1/2 + 3 = 3.5). But if more elite skaters had been there (e.g., Oda) and outskated him in the SP, Hanyu would have gone farther down in the ladder, and the result could have easily been: 1. Kozuka (2/2 + 2 = 3). 2. Hanyu (5/2 + 1 = 3.5). 3. Takahashi (1/2 + 3 = 3.5). In other words, the 6.0 results are rather unstable unless there is a large pool of competitors.

The similarity in results between 6.0 and a "no-credit-for-a-fallen-jump" system is rather striking. The no-credit system seems to incorporate the advantages from both 6.0 and CoP. It penalizes disastrous performances (e.g, Hanyu's Short and Takahashi's Long) and rewards the most consistent skater (Kozuka :clap:).
 

ivy

On the Ice
Joined
Feb 6, 2005
I haven't had a chance to watch the vids (it's a hassle on my system), though I have seen the performances back when they happened.

Looking at the lists, one advantage that 6.0 had was that the winner of the long was most likely the winner of the whole event. It felt good when you watched it, some one delivers the goods and then wins! It's also good for casual fans who likely haven't seen the SP and don't really care to watch it. Of course it didn't always happen, and ordinals had their own problems.

My favorite part of CoP is protocol sheets. I like being able to understand how a skater got enough points to win.


I'm a Kozuka fan and would love to have him win, but still believe in partial credit.
 

Violet Bliss

Record Breaker
Joined
Nov 19, 2010
Looking at the lists, one advantage that 6.0 had was that the winner of the long was most likely the winner of the whole event.

Under 6.0, the top 3 from the Technical Program are equal finalists, so that whoever wins the Free Program wins the event. The examples given happen to have the same top 3 for both programs. It gets complicated when someone off the top 3 TP wins the FP.

The filp side of not having someone win with the SP is that it's close to impossible for someone off the top 3 from the TP win or even medal. Stars have to be aligned exactly for it to happen, with convoluted calculations. Essentially, the event is most often lost with a poor TP. Kurt Browning would probably win an Olympic medal with his Casa Blanca FP under COP. With 6.0, he had no hope whatsoever and his FP was underscored as well due to his low standing from the TP.

I'm not completely sure how it would work out but under 6.0 in GPF, Daisuke being in 5th from the SP would render it impossible to win and extremely difficult to finish second as he did.

It is not because of COP that someone could win with the SP. It is because a big margin in SP is achieved by the top competitor so that the rest have a difficult time catching up to him in the LP even if he messes up somewhat. In the unlikely event of a lesser skater winning the SP by a big margin due to top skaters faltering, he would not be as certain to win the event because the better skaters can easily catch up in the LP which accounts for twice as much as the SP.
 
Last edited:

Blades of Passion

Skating is Art, if you let it be
Record Breaker
Joined
Sep 14, 2008
Country
France
Competition #2 (Cup of China):
Nan SONG: Long: http://www.youtube.com/watch?v=ORWZUPXIam8 Short: http://www.youtube.com/watch?v=uZQtz52Dq74
Nobunari ODA: Long: http://www.youtube.com/watch?v=Tw0jGphMdRE Short: http://www.youtube.com/watch?v=WXPXhuhGva8
Jeremy ABBOTT: Long: http://www.youtube.com/watch?v=7G_5igglTsc Short: http://www.youtube.com/watch?v=svMXuvFwCuo

The CoP had the following placements:
1. Abbott (79.32 + 149.17 = 228.49)
2. Oda (77.65 + 149.46 = 227.11)
3. Song (72.72 + 154.03 = 226.75)

The 6.0 would have had this order:
1. Song (5/2 + 1 = 3.5).
2. Oda (4/2 + 2 = 4).
3. Abbott (3/2 + 3 = 4.5).

A "no-credit-for-a-fallen-jump" system would have had this order:
1. Song (226.75)
2. Oda (227.11 – 1.60 = 225.51)
3. Abbott (228.49 – 7.30 = 221.19)

Which system matches best your judgment on those performances? Or which system gets the final placement right in your opinion?

Yuzuru Hanyu, who placed 4th, should have won this competition. Here are his performances:

Short Program - http://www.youtube.com/watch?v=6nSihTWuRMw
Long Program - http://www.youtube.com/watch?v=0A7XW9AR4TE

My rankings would be the same under both CoP and 6.0 for this competition:

1. Yuzuru Hanyu (1st SP, 2nd LP)
2. Nan Song (4th SP, 1st LP)
3. Jeremy Abbott (3rd SP, 3rd LP)
4. Artur Gachinski (2nd SP, 6th LP)
5. Nobunari Oda (5th SP, 5th LP)
6. Richard Dornbush (8th SP, 4th LP)
 
Joined
Jun 21, 2003
OK, here is my suggestion. This is called the Trophy Dash option, by analogy with old-time stock car racing.

1. Replace the Short Program by a stand alone Trophy Dash. Two-and-a-half minutes, anything goes. 6.0 type judging with heavy emphasis on audience appeal, innovative moves, and the WOW factor.

Your prize for winning the Trophy Dash -- a big trophy (not those dinky small medals that no-one cares about). The trophy would be engraved in flowery letters, "Nobonari Oda, winner of the 2012 NHK Trophy Dash. Ta da!" Together, of course, with a big cash prize.

2. Now the Technical Skate. This will be essentially the same as the free skate now. Basically a marathon of how many jumps you can do before you run out of gas. IJS rules as they are currently, so blade work and program components will still count, as in the present system. Only the Technical Skate will count toward ISU rankings, Grand Prix points, etc.

Advantages. This eliminates once and for all the issue of someone winning the SP by a big margin and then loafing through the long program and winning anyway. At the same time it allows for someone to finish last in the first stage (no Trophy Dash trophy for you, bub), or even decline to skate it, but still have a fresh start when they skate for the championship in the real deal.

Plus, it satisfies the need to cater to the paying customer, yet does not compromise the convictions of skating experts and ISU officials as to what should be rewarded and how. The trophy dash competition might even be shown on TV.

And a skater like Michelle Kwan could finish her career and say, I may have lost the Olympics but look at all the Trophy Dash trophies I have in my case! (That will be one dollar, please, to view them.) :yes:
 
Last edited:

skatinginbc

Medalist
Joined
Aug 26, 2010
Yuzuru Hanyu, who placed 4th, should have won this competition. Here are his performances:

Short Program - http://www.youtube.com/watch?v=6nSihTWuRMw
Long Program - http://www.youtube.com/watch?v=0A7XW9AR4TE

My rankings would be the same under both CoP and 6.0 for this competition:

1. Yuzuru Hanyu (1st SP, 2nd LP)
2. Nan Song (4th SP, 1st LP)
3. Jeremy Abbott (3rd SP, 3rd LP)
4. Artur Gachinski (2nd SP, 6th LP)
5. Nobunari Oda (5th SP, 5th LP)
6. Richard Dornbush (8th SP, 4th LP)

Interesting! You can certainly argue that Hanyu should have won the Short since it was merely 0.27 difference between Gachinisky (81.64) and him (81.37), and as a result he could have won a medal under the CoP given his total score was only 0.58 less the silver's. That alone, however, would not have changed the final placement under 6.0 unless his LP moved up as well. To change his LP standing is harder to justify, especially his less than stellar performance during his second-half of the program (two falls + doubling the Sal + running out of the steam). Say, the judges were really impressed with his LP choreography and interpretation and gave him a whole point more in each category (multiplying by factor 2 = 4 extra points). It would have given him the Gold under CoP or the Bronze under "no-credit" rules, but whether it could pull him up in the LP ranking under the 6.0 system is hard to say.

Conclusion: You are more likely a CoP supporter. :biggrin: Or we may say the scenario is more likely to happen under CoP than the other two systems.
If you were the only figure skating expert in the world, I would have proclaimed that CoP is the most valid system among the three systems in question, and that CoP's drawbacks concern its reliability (because your rankings were different from the official result), not validity.

I hope more experts will participate in this "Be the Judge" exercise and tell us their placement decisions. It is a method for me to get a sense of validity and reliability with regard to each scoring method.
 
Last edited:

skateluvr

Record Breaker
Joined
Oct 23, 2011
If the 6.0 results match best your judgment, then I think you might agree that one of the problems associated with the CoP is that someone can win the whole thing with a big lead in the Short despite a disastrous performance in the Long. The competition is basically over after the Short. The drawbacks of 6.0 mainly concern its broad categories (i.e, Techniques vs Presentation), so broad that a category involves too many different skills for the judges to apply the same criterion. We can combine the advantages from both systems: Use CoP's detailed approaches to score Technical elements and PCSs and yet 6.0 logic to determine the final placement. Problems solved.

This is what I have thought would be more fair, a better system.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
Forget about 6.0. Forget about CoP. Forget about any scoring system or criteria that the authorities have prescribed. Watch the following SPs and LPs and rank them based on actual performances, not on a skater's reputation or potentials, or on the published official results, or on what your dream podium would be.

From this, and the thread title, I thought you wanted us to watch the programs and arrive at our own assessments of how we would rank each of the programs. I like to do that, so I was disappointed that that wasn't what you really meant.

I've been busy with other things this season, so I hadn't seen any of these performances before and was only vaguely aware of the results at the time, didn't remember exactly. I thought I'd try judging them by 6.0 . . . and I found it very hard to do so now that I'm so used to IJS.

Since the long program results determine overall results under 6.0 if the skaters are within two places of each other after the short, I just watched the long programs. Thank you for the links and inspiring me to watch some good performances.

But it turns out you weren't really asking how I would rank these programs, how we would judge the actual skating. Instead you're asking, if we accept that the results of each phase of these competitions would have been the same as the published official results for that phase, which accounting method of combining the long and short program results would give the most intuitive or emotionally satisfying overall result.

I don't have time to analyze the short programs as well and come up with an opinion on these particular results.

In general, my opinion is that I like the fact that under IJS margin of victory based on the quality of the skating carries over from short to long program rather than the number of other skaters in the competition who placed in between. However, I think it would be better if there were some way to make it easier to achieve big leads in the long program than the short program, so that if two skaters each have a good performance in one phase of the event and poor in the other, only in opposite phases, the one who does well in the long will be more likely to place ahead. I think that's an area where the IJS as it's currently structured doesn't do as good a job as it could, but I'm not certain what could be changed to make that work better.

But I don't think there will ever be a system that accounts perfectly for all possible differences in quality between the first and second phase of the competition from my own point of view. I.e., I don't think it's possible to please one person all of the time. Much less to please everyone all the time.

The 6.0 would have had this order:
1. Hanyu (4/2 + 1 = 3)
2. Kozuka (2/2 + 2 = 3)
3. Takahashi (1/2 + 3 = 3.5)

Again, you're assuming that the placements in each phase would have been the same under 6.0 judging as they were under IJS -- you're not asking us how we would have ranked those short programs or those long programs. And since we don't get to see the skater who placed 3rd in the SP, we don't know whether we believe Hanyu should have "controlled his own destiny" vs. Takahashi or not.

Watch the following SPs and LPs and rank them based on actual performances, not on a skater's reputation or potentials,

Watching the LP videos and looking only at those actual performances -- the whole performances, the spins and the skating content and quality in the steps and transitions, and all of the 6.0 presentation criteria, not just the jump counts -- I would have had Kozuka first in the free skate at Japanese Nationals.

I can even see possible reasons for putting Takahashi second in the free skate there, or Abbott first in the free at Cup of China. I think we likely would have seen mixed ordinals among the groups of skaters you chose, depending how many judges on the panels gave more weight to number of jumps completed or number/severity of jump mistakes, and how many gave more weight to other qualities.

That's not based on "reputation" or "potential" -- it's based on my impression of all the skills that each of these skaters showed in these specific performances. Your mileage may vary. (That's why there are 9 judges. Or whatever number)
 

drivingmissdaisy

Record Breaker
Joined
Feb 17, 2010
Looking at the lists, one advantage that 6.0 had was that the winner of the long was most likely the winner of the whole event.

I think this aspect of 6.0 also meant you have to really go for your jumps in the long program. Sometimes I feel like when I'm watching a skater with a big lead, it seems like they are skating to hold on to that lead, rather than skating to win.
 
Joined
Jun 21, 2003
I think this aspect of 6.0 also meant you have to really go for your jumps in the long program. Sometimes I feel like when I'm watching a skater with a big lead, it seems like they are skating to hold on to that lead, rather than skating to win.

I agree.

In CoP there is, in fact, no reason to have two different programs at all. Hence my suggestion above, to have an SP that is a self-contained event of a different type, the results of which are carry their own reward and are not combined with the long.
 

Violet Bliss

Record Breaker
Joined
Nov 19, 2010
I think this aspect of 6.0 also meant you have to really go for your jumps in the long program. Sometimes I feel like when I'm watching a skater with a big lead, it seems like they are skating to hold on to that lead, rather than skating to win.

We have currently a champion who never holds back but goes for broke even after a huge lead from the SP, but people complain constantly about the undeserved win and/or too big the winning margin due to favoritism. What gives?
 

skatinginbc

Medalist
Joined
Aug 26, 2010
From this, and the thread title, I thought you wanted us to watch the programs and arrive at our own assessments of how we would rank each of the programs. I like to do that, so I was disappointed that that wasn't what you really meant.
It wasn't? I thought that was what I meant.
Instead you're asking, if we accept that the results of each phase of these competitions would have been the same as the published official results for that phase, which accounting method of combining the long and short program results would give the most intuitive or emotionally satisfying overall result...
No, that's not what I intended.

OK, please allow me to explain myself. If it were a real life experiment, I would have just asked experts to watch those performances and to rank them with their own criterion. All I want from them is their ranking decisions, just like what Passion of Blades had provided. I, not them, will then compare the results against those from the three accounting methods in question. Because it is a discussion forum, I thought if I simply said "rank them" and nothing else, people would ask: "Why those performances?" "Why do we need to watch them again?" So, I put in the extra stuff.....Sorry, it becomes confusing.

As I said in an earlier post, if Passion of Blades were the only expert in the world, his rankings gave out a lot of information: for instance, falls in his mind are forgivable (inferred from his willingness to forgive Hanyu's falls and rank him higher). That logic challenges the validity of the "no-creidt-for-a-fallen-jump" system.

Watching the LP videos and looking only at those actual performances...I would have had Kozuka first in the free skate at Japanese Nationals....putting Takahashi second in the free skate there, or Abbott first in the free at Cup of China...
OK, the Presidential Election results are coming in:
LP (Japanese Nationals): 1. Kozuka. 2. Takahashi. 3. Hanyu
LP (Cup of China): 1. Abbott. 2. Yet to be announced. 3. Yet to be announced.
We are still waiting for the SP results and the final placement based on both stages (SP and LP) of competition. :biggrin:
 
Last edited:

gkelly

Record Breaker
Joined
Jul 26, 2003
I'm not sure exactly what you're trying to measure.

It might be more useful to choose one phase of one competition, give us several performances (some with falls, some without), and ask how we would rank them and why.

If it was a real competition held under IJS, we should probably rank them holistically by 6.0-style ordinals, and if it was an older 6.0 competition we should probably score them by some version of IJS and then add up our scores.

Or choose performances that were not all from the same competition, so we can use whatever method we like without being influenced by knowledge of the real-life results.

What I think you'll find is that we don't all agree on how to rank the performances because we each have different priorities in what we look for and what we give the most weight.

If you can get enough people to participate, maybe you'll find a consensus on rankings that might give some answer to the question of whether number of falls is or should be directly correlated with results for that program.


Or, if you're more interested in the question of how short and long program results get combined by the scoring system, choose a competition in which each skater only had one clean performance across two programs and let us try to rank each skater's pair of performances.
 

skatinginbc

Medalist
Joined
Aug 26, 2010
I'm not sure exactly what you're trying to measure.
I'm just exploring, to "get a sense" of their validity and reliability, as I said. I'm not doing quantitative analysis at this moment. Through this early stage, I've pretty much ruled out 6.0 system, whose demise was for a good reason. The "no-credit" proposal is an extreme case to test the water. It can be easily relabeled as "more penalty for a fall". But from the ratings Blades (a choreography expert) and you (a skating expert) have provided so far, I got the impression that experts seem to have a greater tolerance for falls. Given that your ratings for Japanese Nationals LPs are significantly different from the official results, it leads to two areas for research focus: (1) Inter-rater reliability and (2) Weights for each criteria (a validity issue). Apparently you weighed certain skills in a manner different from that of the judges who gave the official results.

So, through this preliminary exploration, one can save the cost and time of doing studies that would eventually prove irrelevant.
 
Last edited:
Joined
Jun 21, 2003
We have currently a champion who never holds back but goes for broke even after a huge lead from the SP, but people complain constantly about the undeserved win and/or too big the winning margin due to favoritism. What gives?

I think the complaint is that a skater can build up such a big lead in the short program that when it comes to the long program it doesn't matter whether he goes for broke or not.
 

Violet Bliss

Record Breaker
Joined
Nov 19, 2010
I think the complaint is that a skater can build up such a big lead in the short program that when it comes to the long program it doesn't matter whether he goes for broke or not.


so why such comment as

drivingmissdaisy said:
I think this aspect of 6.0 also meant you have to really go for your jumps in the long program. Sometimes I feel like when I'm watching a skater with a big lead, it seems like they are skating to hold on to that lead, rather than skating to win.

with which you agreed?
 

ImaginaryPogue

Record Breaker
Joined
Jun 3, 2009
I think the complaint is that a skater can build up such a big lead in the short program that when it comes to the long program it doesn't matter whether he goes for broke or not.

Does that happen often enough at major competitions for it to be a fair complaint?
 
Top