The judges strike back?

Last year critics and scholars of the ISU judging system had a lot to say about the role of the Technical Specialist and his/her team. It seemed like the outcome of every contest was determined by who got downgraded and who didn’t, who got an edge call and who got away with it, and who held their spin positions for enough counts to get a high level call.

After each competition the buzz (or non-buzz) in discussion groups was, well, we really can’t have an opinion about what we thought we just saw until the protocols come up and/or we can go to You Tube, protractors in hand, and start drawing diagrams of how many degrees short someone’s landing may or may not have been.

Lately the judging panels seem to be staging a countre-coup. In quite a few competitions recently the judges have taken matters into their own hands and given out GOEs and PCSs of sufficient weight to make sure that the “right” skater – that is, the one that the judges thought skated best -- ended up on top.

This is a lot like ordinal judging of old. The judges decided in their minds who they thought did best, second best, etc., then gave out scores – the 5.8’s and 5.9’s – required to make it so. This, after all, is the job of a judge – to exercise judgement.

Are we seeing a deliberate re-assertion of authority on the part of the judging panels?

In that case we may as well return to 6.0.

The biggest issue with the Cop system is it tries to be objective when it ISN'T. It CAN'T BE. (Man, now I sound like PM.) The technical mark is one thing, because what the skater did can't be disputed (however, I do think the downgrade issue needs to be examined closely and fixed). But the PCS? meh

The current system can be fixed. But it's no use beating this subject to the ground because I don't see the ISU making the changes necessary to make the judging more transparent.

I have seen this playing with the numbers in ice dance almost incessantly, but I have to say most of the singles and pairs events seemed reasonably fair until nationals and now 4CC. I don't really have a beef with Abbot winning, dispite his flawed LP at nationals. However I feel that McLaughlin and Brubaker as well as Alissa were given "all the extras" like +GOE's and high PCS just to insure that they landed on top the podium.
At 4CC Joannie was the cleanest skater there (no flutz, no lip...no obvious) underroattion) but still finished a distant second to a less than stellar yu-na. (i guess the judges felt that they had to reward her for skating such a brilliant short with the title). How many times have to the "older" more "polished" dance team taken gold over the up-and-comer, such as Belbin and Agosto winning the national title last year and Delobel and Shonfelder being handed gold after a sloppy freedance at worlds 2008.
By 2011 we will have reverted right back to the "under the table" judging of the 6.0 system, only with added drawbacks.

Tinymavy15 said:

By 2011 we will have reverted right back to the "under the table" judging of the 6.0 system, only with added drawbacks.

Click to expand...

Sadly, we do seem to be heading that way.

I like to believe that the COP, though 4 years old, is still a work in progress. There is room for improvement.

Tinymavy15 said:

By 2011 we will have reverted right back to the "under the table" judging of the 6.0 system, only with added drawbacks.

Click to expand...

2011? We've already been there for a while. Arguably we always were. CoP was always added drawbacks and never fixed judging. Its only benefit (ideally) is more sensible technical element scoring.

There have been obvious errors by the Tech Panelists as well as the Judges since the CoP came into existence. They are never corrected once the scores have been announced. As to discussion of them later, who knows? Certanly, any tech or judge errors will never be brought out to the fans. We just have to swallow it.

as long as there are human judges there will be human error - both by mistake and deliberate. That's just how things are. No juding system will ever "work" 100% of the time.

^^^
It should be working 90-95% of the time. I don't think it is. And old maxims are not going to change my mind.

Without passing judgement about whther it is good or bad, the question that I am really most curious about is this. Since the beginning of the CoP the judges -- so it seems to me -- have been content to take a back seat to the tech panel.

Now all of a sudden they are asserting themselves more. Is this really a trend, or just an anomaly in one or two recent competitions?

Mathman said:

Without passing judgement about whther it is good or bad, the question that I am really most curious about is this. Since the beginning of the CoP the judges -- so it seems to me -- have been content to take a back seat to the tech panel.

Now all of a sudden they are asserting themselves more. Is this really a trend, or just an anomaly in one or two recent competitions?

Click to expand...

I believe serious non political judges may have a gripe with the Tech Panel. The TP's job is to cite levels and errors without question. The TP sits in the same row as the judges or slightly above it. The Judges see exactly what the TP sees, and they are not dumb about figure skating having judged for many years including what the reasons are for the TP.

Levels, of course, are subjective for the most part, but errors must be distinguished as poor skating skills. I can understand the judge's being upset with some of the Tech Panel's calls. However, the power of Tech Panel can not be overturned.

My own suggestion would be to do away with the Tech Panel. All those judges can see levels and errors. They can click a key on their computers as to levels and errors and a much better result would come up. A majority of 9 is better than a majority of 3.

Joesitz said:

My own suggestion would be to do away with the Tech Panel. All those judges can see levels and errors. They can click a key on their computers as to levels and errors and a much better result would come up. A majority of 9 is better than a majority of 3.

Click to expand...

I do agree that nine pairs of eyes are better than 3 but when the rules for e.g. step sequences are so convoluted how can one pair of eyes truly spot everything that is required for the levels. Didn't someone post something about how the tech panel splits up things to watch for in a step sequence to catch it all so that one person watches upper body movement to determine how complicated the movemenets are for the purposes of the levels, another watches out for rotation in the step sequences to see whether it is evenly spread across both directions and the final person spots the turns and steps to see if enough different and cmoplciated turns are racked up?

I dare say that one person cannot possibly watch out for all of those things.

Also how would it work if the judges did no unanimously decide on which level to assign? Would there be an median or mode average taken?

Ant

Exactly as Ant says.

I think the point of the tech panel is to agree on what was done. Occasionally judges will miss something significant, including how many revolutions were performed in a given jump.

This is especially true in pairs when there are two skaters to watch -- each judge with one pair of eyes is bound to miss some details.

If we were to get rid of the tech panel and have judges identify elements and mark each one, then probably the concept of features and levels (for non-jump elements) would have to be ditched. Instead, judges could add bonus points for what they individually consider difficult features of an element (according to guidelines less stringent than those followed by the tech panel) and could add additional bonus points for good execution (= positive GOEs) and they could also subtract points for mistakes and weaknesses of execution.

This would simplify the scoring somewhat. But it would also mean all the decisions now made by the tech panel about whether or not to reward a feature with a higher level would be much more subjective than they currently are. It may be debatable whether more subjectivity = more room for disagreement between judges in this area would be a good thing or a bad thing.

Currently, when the execution of an element falls into what they call a "gray area" there is room for disagreement about what exactly was performed.

This includes added difficulty in spins, spiral sequences, etc., that just barely meet or just barely miss the requirements to receive credit for a given feature. Getting rid of the concept of "levels" and just letting judges subjectively reward extra difficulty along with rewarding or penalizing good or bad quality would eliminate the issue of borderline features.

It includes jumps on the borderline of deserving to be downgraded (and even if the cutoff point were changed from 90 degrees to 135 or 180 degrees, there would still always be some instances that fall very close to the borderline, whatever it is). If there's a base mark for, say, a double loop and a higher base mark for a triple loop, what would the base mark be for a cheated triple loop that half the judges call triple and half call double, even if they all rate it at a -2 grade of execution?

It includes fluke mistakes that might change the nature of the element being performed to something different from what was intended or what is required (esp. in short programs) or allowed, which can have a huge effect on whether the skater should receive credit for this element or for a later element.

If some judges see a solo triple jump with a slight turning hop on the landing and other judges see a triple jump-single loop combination, how might that affect the scoring not only of this element but of another jump combination in the program? That's going to be a problem for the skater under any judging system if it happens in a short program where only one jump combination is allowed (and must include at least triple and double jump at senior level), or in a long program if it results in more than the maximum of three allowed combinations being performed. It will also be a problem for whoever, or whatever computer program, needs to combine the element scores from the full panel of judges if half of them interpret the element one way and half another way.

If some judges see a combination spin with an accidental extra step at the change of foot and other judges see two separate spins, how can that element/those elements be consistently scored? If there's a maximum number of spins allowed (always the case in short programs, and they always need to be specific kinds of spins as well), what happens if some judges score too many spins and call them by several different names? Who decides which spins the skater should actually get credit for?

In the 6.0 system, if those kinds of errors happened in short programs especially, the judges who interpreted the errors as just simple errors on attempted legal elements might take small deductions, and those who interpreted them as resulting in the wrong kind of element being performed might take several severe deductions, which could result in vastly different required element scores from different judges for the same program. No standardization for the skaters, but freedom for each judge to mark what he or she saw however he or she interpreted it.

In IJS using a technical panel, the three members of the panel get to review the videotape, consult each other about what they saw and how to interpret it, and inform the judges of the final determination. There are clear rules about how to handle most kinds of situations and the review process allows the tech panel to develop a consensus on how any given example of a borderline element fits into those guidelines. There isn't really an efficient way for nine or more judges to consult and come to a consensus about what the elements were before finalizing their marks, so with no panel that task would have to fall to the referee or some computer operator position or to an equivalent of the technical controller operating after the fact and it would take that much longer to finalize the scores. Might as well forget about announcing scores after each skater, because the review and data entry process would take even longer than it did the first year of IJS when the tech panels were still getting the hang of it.

We seem to be talking about 3 pairs of eyes v. 9 pairs of eyes.I've never heard of the Tech Panel being split up to watch particilar movements before. I'm certainly not against that, but then you have one pair of eyes looking at step sequences. - hardly a mjority result. I'd rather split up the 9 judges.

Judges have been doing all that the Tech Panels do for years. They could continue with it without any change in the results of competitions. I'm sure Paul Wylie could and he isn't even a judge.

However, there will be no changes in the makeup of the CoP so those who consider it a perfect document need not worry about changes, and one panelist will decide the level of the step sequences. So be it.

Red Dog said:

In that case we may as well return to 6.0.

The biggest issue with the Cop system is it tries to be objective when it ISN'T. It CAN'T BE.

Click to expand...

It's such a shame only fans seem to notice this and not the ISU officials!

Joesitz said:

We seem to be talking about 3 pairs of eyes v. 9 pairs of eyes.I've never heard of the Tech Panel being split up to watch particilar movements before.

Click to expand...

They don't "split up to watch particular movements" but specifically in step sequences, they split up in looking for particular features. So for example, the TC may look for variety/complexity of turns, the TS may be looking for change of direction/rotations and the ATS may be looking for modest/full body movement. In this way, THREE pairs of eye can discern whether multiple features have or has not been met. If one judge attempted to discern up to 4 different features in a step sequence, they'd be reviewing a lot longer than they currently do, even if all nine judges were the eyes!

As for spins, it's not subjective what constitutes a feature, what is subjective is the view that the tech panel gets to count the starting revs of that feature (it's also subjective where the camera is placed and what the review looks like for the tech panel). This is why most coaches are teaching skaters to get ABOVE the minimum required for a feature (always get at least 3).

Joesitz said:

My own suggestion would be to do away with the Tech Panel. All those judges can see levels and errors. They can click a key on their computers as to levels and errors and a much better result would come up. A majority of 9 is better than a majority of 3.

Click to expand...

Absolutely NOT TRUE. Most judges cannot tell you the level of the spin that they just saw, only how well it was executed in terms of centeredness, speed, positions and *maybe* number of revolutions. And since it takes 3 trained technical specialists to call a step sequence, there's no absolutely no way judges could do it.

If figure skating is 50% sport, 50% art, then I think it is absolutely fair to have half of the score (TES) calculated objectively like a sport and the other half (PCS) judged like an art. No judging system will ever be perfect because judging is, by definition, subjective. This one does pretty well considering the complexities of judging this sport.

mskater93 said:

As for spins, it's not subjective what constitutes a feature,

Click to expand...

For the most part. There are a few areas where the tech panel has some discretion.

E.g., for the clear increase of speed feature, if there's a slight visible increase, some observers might consider that "clear" enough to deserve the feature and some might not.

A difficult variation is defined as "a movement of a body part, leg, arm, hand or head, which requires more physical strength or
flexibility and that, has an affect on the balance of the main body core."

If a skater does a new or uncommon variation that isn't already established as a variation that qualifies for this feature and that requires some physical strength or flexibility but not obviously an extreme amount of either, then the tech panel will have to make a judgment call as to whether it qualifies or not.

I personally really like that the judges are being very strict about under-rotating, wrong edges and such....

It helps to critique the skaters more and judge them more accurately.

But i do wish the technical specialists were a little more even across the board on what the calls are.

GKelly, do you have a sense that judges are giving out more generous plus GOEs this year than they have in the past?