Understanding the +5/-5 system

[NymphyNymphy]

With the new +5/-5 system, I think it would be of ISU's interest to sit down all judges after an event for overview session where:

1) a random element is selected e.g. a specific jump or spin
2) compare that element performed by the top three skaters
3) have each judge provide a reason for why they gave the GOE they gave

Having this feedback after an event should help audiences better understand the system and lessen any frustrations that people may have. It will also help prevent judging bias: Knowing that a specific element will be selected and require explanation for their GOE can deter a judge from giving absurd GOE's. And lastly, we get to know the judges better. Giving judges more public appearance and connection with the audience is 1) appreciation of their hard work 2) make judges aware that their actions are being watched

I feel like a short video like this could have benefits for viewing and judging purposes. Look at soccer, they've got playbacks and hosts that discuss technique after the game. We rarely get that with figure skating. With youtube being the main platform for free media consumption, ISU has no excuse for not creating such videos.

 

[Baron Vladimir]

You are neglecting simple fact that every single judge (or better to say their cognitive system) could not be that precise, not because the bias, but because the nature of the job, and the fact that job need to be done in required time. So they dont make mistakes because they dont know, but because they didnt or coudn't see/count/process correct. Thats why not all of the judges marks are part of the final score, system itself expect that will be at least 2 mistakes in the system, so lowest and highest marks dont count... Of course, its possible that some of the judges will have different opinions while evaluating some elements. Thats why average opinion of those 2 judges which is something between their original opinions count, so again the system talking care of itself (by its mathematical procedures)... And also we can read how some judges, who made multiple/bigger mistakes, were excluded from judging for some time

 

[karne]

You mean, basically, the round table discussion that already happens, just with video? And why just the top three? You're actually more likely to find the weird stuff further down.

 

[NymphyNymphy]

You mean, basically, the round table discussion that already happens, just with video? And why just the top three? You're actually more likely to find the weird stuff further down.

Yeah now that I think about it.. maybe they should randomly choose a skater? So we have a mix of top, mid, lower tier skaters.

 

[NymphyNymphy]

You are neglecting simple fact that every single judge (or better to say their cognitive system) could not be that precise, not because the bias, but because the nature of the job, and the fact that job need to be done in required time. So they dont make mistakes because they dont know, but because they didnt or coudn't see/count/process correct. Thats why not all of the judges marks are part of the final score, system itself expect that will be at least 2 mistakes in the system, so lowest and highest marks dont count... Of course, its possible that some of the judges will have different opinions while evaluating some elements. Thats why average opinion of those 2 judges which is something between their original opinions count, so again the system talking care of itself (by its mathematical procedures)... And also we can read how some judges, who made multiple/bigger mistakes, were excluded from judging for some time

Yes I understand that judges are people and they have a lot on their hands. But after the competition I think it would be fun and helpful to provide us an interactive short video showing us their mindset when they are giving GOE's. It will make it easier to understand how they give GOE's and lead to less frustration.

 

[Baron Vladimir]

^^^
They already have those discusssions. Its not in a way you are suggesting because its kind of pointless - They can 'correctly' score (cause they are already educated) based on the after competition slowmo video. They simply didn't have that possibility at the moment of judging... You have some ISU videos how the new GOE should be aplied already...

 

[Mathman]

I think the heart of NymphyNymphy's suggestion is that the video-discussion meeting would be broadcast to the viewing audience and to the live audience in the arena. As it is now, it is behind closed doors with just the judges and ISU officials.

As a step in that direction, the television commentators could offer their expert opinions, with selected video as to why a particular skater's triple Lutz deserved a +3 while another's only deserved a +1. This is what happens in other sports, as NymphyNymphy mentions in the OP.

Look at soccer, they've got playbacks and hosts that discuss technique after the game.

The audience gets review from the hosts, but not from the actual officials that made the call. In American football on TV, they have a "rules expert" on call for real-time video consultation with the in-house commentators as to what exactly the rule is and why the referee's call is right or wrong.

 

[gkelly]

A "rules expert" would be more useful for discussing level calls on non-jump elements, or explaining why certain elements didn't get any credit at all. They could also educate new viewers about how underrotations/downgrades and edge calls are made. But they would need to use the exact same camera angles as the tech panel to give a good sense of why this call was made on this occasion. And even so, in borderline cases an expert who wasn't on the panel might reach a different decision than the panel. A good outside expert commentating on specific calls would point out what the panel would be looking at but would not second guess their decision.

And there's nothing stopping broadcasters from bringing in outside experts of that sort. That wouldn't be the ISU's responsibility. (Except perhaps allowing broadcasters to use the same camera feed that the tech panel uses.)

For GOEs, especially with the wider range now, there is almost never going to be a single correct answer. That's not the way the system is designed, nor ever has been. The only time all judges should have exactly the same GOE would be if the GOE needs to be -5, either because of a short program jump combo with only one jump that counts (mandatory -5 in the rules) or because of multiple errors whose reductions add up (subtract down) to at least -5.

For positive GOEs, for negative GOEs for errors with varying severity and a range of recommended reductions, and for balances of positives and negatives, there will be valid differences of opinion on the judging panel. Of the 11 possible GOE scores, some options will be obviously wrong for a given element, but there will probably be 2-5 possible right answers.

It certainly would be instructive to hear judges who were actually on the panel explain their reasoning, so fans could hear how different judges come up with different scores for the same element, or the same score for different reasons, and get a sense of how GOE scoring is a mix of ticking boxes and exercising value judgments, not purely ticking boxes.

One reason the ISU might not want to do this would be that sharing part their discussion in public after the end of the event would be extra work for the officials. Another reason would be that it would expose individual judges not only to being verbally attacked by disgruntled fans, but also potentially to displeasure from their federations if they didn't support the home country skaters.

A possible way around those objections would be to have the referee lead a roundtable discussion with the judges in private, and take notes on interesting points of disagreement or unusual situations that occasioned interesting thought processes, and let the referee give a press conference/discussion with broadcasters to summarize the highlights of the roundtable without naming individual judges. Of course that would be more work for the referee.

 

[Baron Vladimir]

^^^
Yeah, as a judge you can make one desicion thinking its correct during live competition or a sport match and have a different 'correct' oppinion on the same topic while watching the game (and slowmos) from a TV studio... That is what i was trying to point out...

 

[Mathman]

One reason the ISU might not want to do this would be that sharing part their discussion in public after the end of the event would be extra work for the officials. Another reason would be that it would expose individual judges not only to being verbally attacked by disgruntled fans, but also potentially to displeasure from their federations if they didn't support the home country skaters.

I think the main objection would be that it would affect the independence and objectivity of the judges if they knew that they would be called on the carpet and held up to public humiliation after each event.

Inquisitor: "So, how can you possibly justify giving PCS to Adelina Sotnikova comparable to Yuna Kim?"

Judge: "Well, ..."

Half the audience: "Booooo. Off with his head."

 

[chopinskate]

I think the main objection would be that it would affect the independence and objectivity of the judges if they knew that they would be called on the carpet and held up to public humiliation after each event.

Inquisitor: "So, how can you possibly justify giving PCS to Adelina Sotnikova comparable to Yuna Kim?"

Judge: "Well, ..."

Half the audience: "Booooo. Off with his head."

lol

I think somehow eliminating the feds from the picture of judges is the best solution. Idk how though.

 

[chopinskate]

I do think an American Idol type competition would get the skaters better, or at least more fun, personable programs, and the judging would be fun (any nominations for Cowell? ), but the audience would just always vote the most popular skater than the best.

 

[gkelly]

I think somehow eliminating the feds from the picture of judges is the best solution.

Well, that would solve the problem of judges being influenced by loyalty to their federations and fear of losing status/opportunities.

But it wouldn't give insight to fans about the honest judging process.

Nor keep fans from booing etc. judges whose honest educated opinions differ from their own.

 

[NymphyNymphy]

I think the main objection would be that it would affect the independence and objectivity of the judges if they knew that they would be called on the carpet and held up to public humiliation after each event.

Inquisitor: "So, how can you possibly justify giving PCS to Adelina Sotnikova comparable to Yuna Kim?"

Judge: "Well, ..."

Half the audience: "Booooo. Off with his head."

I think this pressure will ensure that judges stay on track.

 

[chopinskate]

Nor keep fans from booing etc. judges whose honest educated opinions differ from their own.

The main issues people scream about are reputation judging, PCS boosts for high tech, and bad tech calls, which are reflected in the overall judging, instead of just for one judge. Actually peer-reviewing judges to a good standard of knowledge of TES and PCS should also help -- often skaters get high scores for small jumps, and high IN scores for skates that have very little interpretation going on in them by all objective standards. There MUST exist objective standards before we move into the realm of personal opinion.

Training TPs better is a given. Maybe AI solutions can come soon.

 

[gkelly]

I do think an American Idol type competition would get the skaters better, or at least more fun, personable programs, and the judging would be fun (any nominations for Cowell? ), but the audience would just always vote the most popular skater than the best.

Well, you could have the technical competition with scoring by trained judges, and then have an American Idol-type competition featuring the top placing technical skaters, plus maybe some fan favorites who would be determined by some kind of fan vote after the freeskates. And the host country skater(s).

Essentially, let fans decide who wins the exhibition, and award some kind of meaningful prize for doing so. Those who are so inclined could consider that the most important prize and the technical medals and scores just a means of determining who gets to compete in the fan-friendly event.

I think this pressure will ensure that judges stay on track.

What track should they stay on?

Who nominates the inquisitor(s)? How are their own biases, and those of the fans they represent, taken into account? I imagine a Russian inquisitor would have a very different agenda than a Korean one with regard to the 2014 Olympic ladies results, etc.

 

[Mathman]

I think this pressure will ensure that judges stay on track.

It could go either way. It might put pressure on the judges to give out scores that will please the audience, rather than their actual assessment of the quality of the skating.

 

[Mathman]

I think somehow eliminating the feds from the picture of judges is the best solution. Idk how though.

I don't see how this could be done, either, with the current structure of the ISU as a confederation of national federations. For one thing they would have to generate enough revenue to maintain, train and oversee an independent association of professional figure skating judges.

Although they did take a step in that direction with the creation of the technical panel.

As for American Idol-type formats, I think the ISU would not want to go in the direction of entertainment at the expense of serious athletic competition.

 

[chopinskate]

I don't see how this could be done, either, with the current structure of the ISU as a confederation of national federations. For one thing they would have to generate enough revenue to maintain, train and oversee an independent association of professional figure skating judges.

Although they did take a step in that direction with the creation of the technical panel.

As for American Idol-type formats, I think the ISU would not want to go in the direction of entertainment at the expense of serious athletic competition.

If enough of us storm the Bastille, we could bring it down!

What I mean is... The sport is barely a sport with the way it's judged, and it's barely an art with the way the programs are going. The audience gets more uninterested by the year. Maybe the ISU has a threshold to how little revenue they generate before they start fixing stuff.

Until then, well

 

[karne]

I think this pressure will ensure that judges stay on track.

On track for what? Have the judges too terrified to mark a skater down because their fanbase will shower them in death threats if they do, even if the skater actually deserves it?