Hamilton Shares His Thoughts on Judging System

[antmanb]

I agree. I would assume that most falls on jumps occur because you don't get that landing edge down exactly right. So is it really completely rotated?

The only way that it would be clear is the case where the jump is landed properly and then ... oh! I just can't quite hold on after all! Even then, a huge burden on the caller, considering how many points are at stake for that one element.

Totally, but in this respect Joubert is also well loved by the caller because he has had many quads land between a half and quarter turn short but he's very quickly switched all his weight to the free foot and back again on the landing and been given credit for a faulty quad when it should hav ebeen called a triple. I do, however, think that the quad must be so frustrating for buttle because he always (until the landing!) has a perfect jump, and i can't see why it goes wrong for him.

Ant

 

[antmanb]

What would be wrong with a jump that ends in a fall gets zero points for TCS and a minus on the PCS to be decided at the time of the fall and how bad the fall was.

There just aren't too many falls that do not disrupt the end result of the program.

In the sport of running - if you fall, they don't subtract yards from the total of the race because you may have looked cute while falling and would have been further along in the race if you hadn't fallen.

Ladybug

But in the sport of running, you aren't given a grade of execution for how you ran the race, nor are you given marks for how you got from teh start line to the finish line and how good you looked while you were doing it. The comparisons with running isn't even apples and oranges, its apples and concrete blocks!

Ant

 

[Mathman]

So Midori Ito never landed a triple axel because we can't find it in a protocol???? Come on guys - there is a world of skating pre CoP!!! I think Buttle's quad landing came under 6.0.

Right you are. Buttle's quad came in the 2003 Four Continents short program -- see post #45 above.

 

[Joesitz (RIP)]

They have replay, which shows where the blade took off and where it landed.

Now where will that take the fall if the fall happens on the derriere and not the edge. I think I am getting at that there are different kinds of falls (Zhang and Zhang, e.g.) or a skater landing on toe pick then falling, etc. How much credit for those kinds of falls that couldln't be landed properly but the air turns were fine?

Joe

 

[Mathman]

You may be losing your memory on us Mathman!

True But Jeff did skate at 2003 Worlds (finishing 15th), but not at 2004 Worlds (at the end of the 2003-2004 season).

With all the discusion of the take-off edge of a lutz/flip being only one part of the jump and you can't just call what jump it is based on that, I would think that the landing is only one part of the jump. If the guys (or ladies) can get it up in the air, rotate it, and then just can't quite do it, then they should be rewarded for at least trying and getting it partly right.

I guess (to use some math) It's like getting partial credit on a question. If I have to do a problem and I do it exactly right until, oops! I accidentally used the wrong number in part of the problem, should I get 0 points? I didn't get the answer exactly right, but my process was there so I should get some points for pulling that off, right? Shouldn't it probably be more points than I would have gotten for getting an easier question entirely correct?

I agree with that point of view. What I think is out of whack is this. If your mistake is coming up a little short on the rotation, the penalty is 5 points. (You get 4.0 for a double instead of 9.0 for a quad).

If your mistake is that you fall on the landing, the penalty is only 4 points (-3.00 GOE and -1 fall deduction).

I think that should be adjusted in the "partial credit" criteria.

MM

 

[Mathman]

... a Quad just makes it harder, hence more risk,...

We must be using a different defintion of "risk." Just because something is hard does not mean that any risk is associated with attempting it. A "risk" means that you lose a lot if you fail. This does not seem to be the case with the quad, under the current rules. (?)

Yes, a quad quite properly gets a lot of points. It gets a lot of points because it is hard to do, not because it is risky.

I guess I was on once and now I am off...

Me, too. I go back and forth about this. If I had my druthers I guess I would favor some kind of sliding scale on the severity of the fall combined with lessing the mandatory penalty for short rotation. Maybe the judges should have the right to judge these things on their own without being preemptively overridden by the caller. Twelve heads are better than one? More grade and less code?

MM

 

[Joesitz (RIP)]

OK. But in other sports I watch these instant replays on television over and over and I still can't tell. The exact position of the blade at the exact split second that it touches the ice? Does one part of the blade touch first?

I have been watching those computerized reinactments for line calls at Wimbledon this week. Hmm.

I do not think we will ever be able -- even if it were desirable to do so -- to make judging free of judgment.

ITA. Those Tennis instant replays work because all the camera has to do is focus on the boundary line. Easy to see where the ball bounced.

In figure skating the boundary line is a variable, each jump has its own line formed by the take off and the landing. I think it would take three differently aimed cameras to accurately photo the take off (although I've read here that the take off is not all that important), and the angle of the landing.

What is it? A 2.75 landing is ok for a triple but a 2.74 downgrades the jump. I don't think instant replay will catch that but if the Caller has eagle eyes he just might. All praise to the mighty one.

I can't help but fall back on my belief that there is too much to look at and too little time to do it. There is for a me a Rush to Judgement in figure skating.

Joe

 

[hockeyfan228]

Twelve heads are better than one? More grade and less code?

I think you mean twelve heads are better than three.

 

[Mathman]

Yes, that's what I mean. I ask it as a question, because I am not sure whether 12 heads really are better than three or not, in this instance. The judges have many other things to be looking at simultaneously, whereas on a jump landing the tech specialist and his team can focus on that landing edge.

Still, I'm not ready to sign up for the tuba section in Speedy's band just yet. Judges = crooks, technical specialists = saints, judges = incompetents, tech specialists = experts, judges = biased partisans, tech specialists = neutral observers. It's a pretty tune, but we'll see.

MM

 

[hockeyfan228]

Still, I'm not ready to sign up for the tuba section in Speedy's band just yet. Judges = crooks, technical specialists = saints, judges = incompetents, tech specialists = experts, judges = biased partisans, tech specialists = neutral observers. It's a pretty tune, but we'll see.

I don't think these formulas work, either. My rule of thumb is that people will work towards their own agendas. The international judges are at the behest of the Federations and the agenda of the Federations and are accountable to the Federations, because the Federations give them the plum assignments. The agenda of the Federations is to have their skaters do as well as possible under the circumstances, which can include trade-offs and paybacks for past and future. The tech specialists are accountable to the ISU, and if they want to be hired by the ISU, presumably must follow the ISU agenda, which seems to be 1. to have a judging system that at least has the appearance of solid sports officiating and 1b/2. to wrest control from the Federations.

I don't think it's reasonable in any officiated sport to say that the officials are saints or uncorruptable or always competent or correct. (Look at the brewing scandal in Italian football for game-fixing, which included officials.) What they are is transparent -- every decision is on the printout -- and accountable to the central governing body of the sport.

The main flaw I see is that there is a formula against which all judges are marked that compares their scores to other judges scores, rather than to the standards of CoP. It is inevitable that all of the PCS scores are close together and to each other -- that is self-preservation. In one of Christine Brennan's books, she tells the story of the Czech judge who voted against the bloc for John Curry in the 1976 Europeans. In her telling, the judge was a hero. In Biachetti's telling, he was so sure he'd be cited by the ISU for being out of line if he put Curry in second that he scored Curry first, and only after he saw the split panel did he realize his mistake, and what that meant to his future. (Obviously, no one sent him the memo.)

The ISU doesn't have the power to make a clean sweep of entire panels who flaunt the code. The ISU might think that consensus scoring, even if contrary to the standards of the code, gives the impression of competence to the outside world -- "see, all of our judges agree!"

 

[Joesitz (RIP)]

With those examples we still have to think the judges know what they are doing. Actually, I don't mind the judges who do know the difference between a lutz and a flip, they're ok if there on the ball that night. It's the inept ones that drive me mad.

When one examinse GoE scores and see such grades as -2 through +1 for the same skater, something has to be off.

Joe

 

[SeaniBu]

JAT - can't the judges tell to some degree by the position of the leg, body, where the weight is shifted to? It only SEEMS to me that if a knee is bent angled over the foot, butt slightly centered above the ankle leaning to the outside, and the skaters chest and head are centered above or in front of the toe / arch, ................. bla bla - like I know

IOW, can't they tell by body position?

 

[Mathman]

I don't think these formulas work, either. My rule of thumb is that people will work towards their own agendas. The international judges are at the behest of the Federations and the agenda of the Federations and are accountable to the Federations, because the Federations give them the plum assignments. The agenda of the Federations is to have their skaters do as well as possible under the circumstances, which can include trade-offs and paybacks for past and future. The tech specialists are accountable to the ISU, and if they want to be hired by the ISU, presumably must follow the ISU agenda, which seems to be 1. to have a judging system that at least has the appearance of solid sports officiating and 1b/2. to wrest control from the Federations.

That's the theory (well stated!), and an interesting one it is, too. So, is Cinquanta the master Machiavelli of all time, pulling the wool over the eyes of the wicked but gullible National Federations, "wresting control" from them even while they re-elect him over and over by acclamation?

The main flaw I see is that there is a formula against which all judges are marked that compares their scores to other judges scores, rather than to the standards of CoP. It is inevitable that all of the PCS scores are close together and to each other -- that is self-preservation...

When one examines GoE scores and see such grades as -2 through +1 for the same skater, something has to be off.

It's a catch 22. If the judges are all the same, we say, "hey, what's going on here?!"

If they are different, we say, "hey, what's going on here?!"

Actually, the criteria for how much leeway a judge is given before his/her scores are marked as an "anomaly" and reported to the Officials' Assessment Committee (OAC) for review, is pretty generous. There has to be a fairly consistent discrepancy accoss all categories for a specific skater.

For instance, if you have it in for Johnny Weir in the short program, and if every other judge gives Johnny straight +1s for GOE on all eight elements, you can give him straight 0s and not get caught. (Or, more sneaky yet, give him +1s on his low-point elements and -1's on his high-point elements.) Of course you need a couple of partners in crime to make sure that all of your marks aren't thrown out in the trimming of the mean.

For Program Component Scores, you have a "corridore" of 7.5 points altogether. So, for instance, if the average of the judging panel gives Fumie 8.25 across the board, you can give her 6.75 in every category without setting off any alarms.

Interestingly, the OAC reports problems with judges to the ISU Technical Committee (made up of representatives of national federations), while problems with the performances of technical specialists and technical controllers are reported direct to the ISU Council/General Secretary -- in other words, directly to Cinquanta's posse.

MM

 

[Joesitz (RIP)]

Mathman - You defender of the CoP and the consensus of the judges, lol. If you could just post your views before you see the details of the judges scores, I would be happy.

I don't see anything wrong about disagreeing with scores including the details of those scores. The only reason I can think of as fans saying 'what's going on there' is that their thinking conspiracy which doesn't exist anymore. The big question remains: why can't the experts be closer in their decisions? They are experts and should judge as experts.

I contend, if the judges are in agreement, that is the end of any debate. Plushenko won, Shizuka won, T&M won, N&K won. over and out. All obvious. The details of the scoring were all in agreement without looking.

I can't imagine Johnny getting different scores from judges who know what they are judging. They are not favoring him or holding him back. There is no reason to do that. Any one judge giving him low scores with the others giving him more than adequate or high scores shows that low scoring judge to be inept.

Nothing wrong with the 'corridore' if it is met by the judges. What if there are 2 or 3 who differ widely and they are in the mix for judging? (I know you will have a soft answer to that question. lol.)

All this business about reporting problems. Do you really believe that happens? I think it would happen if the media got a hold of it.

I'm not at all unhapppy with the CoP. I've even gotten over the cheating (well, almost) but for me the question is how is it being practiced with so much to judge in so little time. The placements in recent competitions would be the same if the 6.0 system was used. The Olys, the 2006 Worlds' results would be the same. But that would take the fun out of playing with the numbers and saying Lambiel is no longer a great spinner. Yeah.

Joe

 

[hockeyfan228]

That's the theory (well stated!), and an interesting one it is, too. So, is Cinquanta the master Machiavelli of all time, pulling the wool over the eyes of the wicked but gullible National Federations, "wresting control" from them even while they re-elect him over and over by acclamation?

I'm not sure what their choices are. The committees are run by Federation representatives, which I think is where they retain their power.

It's a catch 22. If the judges are all the same, we say, "hey, what's going on here?!"

If they are different, we say, "hey, what's going on here?!"

For me, it isn't an issue of all the same or all over the map. The issues are:

1. Are they similar because they've been pre-determined?

2. That PCS are within the same corridor, for most judges within, at most, a point of each other, when there are far greater discrepancies in transitions and choreography, to give one example, in the majority of programs.

3. That the corridors make the final "placements" more controllable by leaving only a small amount of room between skaters. (i.e., I give skater A 6.50-7. I give skater B 6.25-6.5. I give skater C 6.5-6.75. etc.)

4. That the number of relatively high scores (0-+1) for skaters with monster flutzes, lips, and telegraphs and combinations where the second jump in combo is relatively weak/tacked on suggests that the credit given to good entry or landing and height and rotation (in the sole or first jump) is higher for the flawed elements -- and compensates for those flaws -- than credit given for very well done elements that score +1 or +2 in total. Particularly among the best skaters.

 

[Mathman]

Nothing wrong with the 'corridore' if it is met by the judges. What if there are 2 or 3 who differ widely and they are in the mix for judging? (I know you will have a soft answer to that question. lol.)

Well, of course I know the answer to that question. I haven't spent the last 3 hours studying ISU documement 1327 for nothing.

http://www.isu.org/vsite/vfile/page/fileurl/0,11040,4844-168605-185823-91843-0-file,00.pdf

If two or three differ widely on just one or two elements, nothing happens. If the total amount of all this differing (from the trimmed mean) adds up to more than plus or minus 1.00 GOE score per element for a particular skater, then that judge is "outside the corridor" and flagged for an "anomaly." These anomalies are reported to the Officials' Assessment Commission which recommends action, such as reprimand or suspension, if they think it is appropriate.

If it is only two or three that are off, they will not be included in the trimmed mean for evaluation purposes (they throw out the top two and the bottom two for each element for each skater). They may be included in the random draw for the event, and some of their marks may even survive the trimming process used in determining the winner of the event. This is a separate procedure.

BTW, if the OAC disagrees with the judgment of the majority of the judging panel, they can expand the "corridor" as they deem appropriate.

About "opinions," I have to admit I don't trust them very much. Facts are better, IMHO.

Actually, I do think that the ISU follows through on its evaluative procedures. It would be better if they published the results (maybe they do, somewhere in the labyrinthe of all these "Communications"). It is to the clear and certain the advantage of the ISU to weed out incompetent and biased judging if they can. Why wouldn't they follow the prcedures that they labored so hard to come up with?

MM

 

[Mathman]

I contend, if the judges are in agreement, that is the end of any debate. Plushenko won, Shizuka won, T&M won, N&K won. over and out. All obvious. The details of the scoring were all in agreement without looking.

But it's so interesting to look. Take Worlds, for instance. Here are the total GOEs given by the 12 judges for Lambiel's long program:

8 3 5 14 3 10 6 6 6 5 6 3

That is, Judges #2, #5, and #12 thought that the total Stephane deserved on all of his 14 elements combined, was 3 points, or an average of 0.2 per element.

In contrast, judge number 4 gave him 14 points of GOE -- an average of a full 1.0 on each element.

For Joubert, it was

9 5 9 14 0 12 6 1 5 4 7 6

One judge gave him a big fat nothing, and another gave him 14. (There was also a 1 and a 12.)

So who won? Judges #1, 2, 3, 6,11 and 12 favored Joubert in this part of the program, while judges #5, 8, 9, and 10 went with Lambiel, with two ties (judges #4 and 7).

Who won? Lambiel, 156.58 to 156.47. (Over and out.)

 

[Mathman]

Hockeyfan, your point #4 is quite subtle. I will watch for that in the future.

Point #3, I'm not so sure about. The rating scale has to accommodate skaters from beginners to champions. If someone placing 9th in the Midwest Sectional gets 4.25 in skating skills, and Shizuka Arakawa gets 8.25, I don't see where else you are going to put Irina and Sasha except 8.00 and 7.75.

I quite agree that this is the judges' way of controling the placements (along with point #2). Although this is more 6.0 than CoP, I am not really so terribly up in arms over this sort of "ordinal-on-the-sly" judging. New judging system or old, at the end of the day the judges have to say who they thought was best.

MM

 

[Theatregirl1122]

True But Jeff did skate at 2003 Worlds (finishing 15th), but not at 2004 Worlds (at the end of the 2003-2004 season).I

Apparently I'm the one getting old here! (I'm going to be 20... in 3 years)
Alright, I work 14 hours a day weekdays and 5 hours a day weekends right now, so you'll have to forgive me for my lack of reading skills.

I guess I tend to stand in the middle. I don't think that having a clean take-off edge, rotating a quad, and falling on the landing is the same as slipping off the edge and not even doing a jump. If you do a quad and fall, you haven't done nothing, you've done something and it was probably pretty hard. But then I don't think that a fall on a quad should be worth more than your average tripple. I don't think that falling on a quad toe should be the same as successfully doing a tripple toe. I think that maybe it might be fair to make a fall on a quad toe equal to a tripple with 0 GOE.

The other thing that I have a problem with is that, if you have a clean take-off, great air possition, rotate a quad, and then fall, you get 5.0 points. If you have a sloppy take off, bad air position, manage to rotate a quad and then fall, you still get 5.0 points. Is it wierd to say that there should be some way to make a fall on a poorly executed jump worth less than a fall on an otherwise well executed jump?

 

[hockeyfan228]

Point #3, I'm not so sure about. The rating scale has to accommodate skaters from beginners to champions. If someone placing 9th in the Midwest Sectional gets 4.25 in skating skills, and Shizuka Arakawa gets 8.25, I don't see where else you are going to put Irina and Sasha except 8.00 and 7.75.

Skating Skills consist of:
*Balance and rhythmic knee action and precision of foot placement
*Flow and effortless glide
*Cleanness and sureness of deep edges, stops and turns
*Power/energy and acceleration
*Mastery of multi-directional skating
*Mastery of one-foot skating

In my opinion, Arakawa excels at all of these. Slutskaya is not particularly strong in balance and rhythmic knee action nor effortless glide; she gets speed at the expense of stillness of upper body and posture and a rather choppy technique. (This is from observing her in person. I actually think her skating skills improved, except for power/energy, in Dortmund, when she was slower and more careful.) Cohen is weaker than either of the other two overall in skating skills; where she excels is in interpretation, performance/execution, and interpretation. I think it's perfectly reasonable to give Arakawa 8.25, Slutskaya 7.5, and Cohen 6 in skating skills.

But I wasn't clear enough, because my point was that each skater is given a limited range for all five of the PCS, instead of, for example:

Cohen:
SS 6.0
TR 6.5 (Wilson added a lot to her LP for Torino)
PE 8.75
CH 7.75
IN 9

instead of 7.75-8.25. The outcome wouldn't be so certain if the numbers were all over the place; it's a lot to remember and fix, particularly when the judges are also distracted by ranking the technical elements. It would be easier with a split panel, but, hopefully (just call me Don Quixote) the PCS judges will be held to the code.