ISU evaluation of questionable judging

[Mathman]

Actually, though, what I really think about anonymous judging is this.

As a matter of principle, open and aboveboard is better than sly and furtive. When you try to hide something, that makes it look like you have something to hide. The ISU can ill afford a reputation of always having something to hide.

For this reason, the default should always be, put it out there for public scrutiny. If you want to classify something as secret, you need a darn good reason.

The reason offered by the ISU seems, to me, both unconvincing and untruthful. I think the real reason why the ISU came up with anonymous judging was to placate the IOC and to head off future public outcry like what happened at Salt Lake City.

As my grandmother used to say (bless her sainted soul), “Tell the truth and shame the devil.”

 

[sk8rdad]

Come on seriously?!! "Dear Mr Cinquanta, I notice that judge number 4 on skater X's sheet relating to the 3Lz must be smoking crack because skater X did a great triple lutz for which the rest of the panel gave +2 GOE yet crack smoking judge in position 4 for the element in the protocol sheet gave a -2GOE. What are you going to do about it? Love and kisses antmanb xxx"

Hey why not as long as you don't seriously expect him to answer.
I was thinking more along the lines of the discussions in these forums not in correspondence to Speedy.

No ther sport makes this level od detail available to the public because no other sport is this damned complicated in it's scoring or has annoymous people scoring it.

Actually there are more complicated sports, baseballs rule book in something like 11 chapters long. There have been occasions were an umpire has had to pull it out in the middle of a game to refer to it.

Yes that's exactly what i'm advocating

Of course not but I thinmk we have a right to know who the cheating judges are. If not we have to believe that the ISU is doing it's job properly and (a) catching cheatnig judges and (b) sanctioning them correctly. We can look at pre-annonymous judging and note that the ISU was particularly bad at even looking for, let alone finding cheating judges, and even when they did, the likes of Danilenko and Balkov and back on judging panels having been reluctantly sanctioned to what, 3 season bans? And now with annoymous judging we're expected to believe that they cleaned up their act and are judging correctly despite the greater public not knowing how they judged?

I'm simply not that naive.

You were the one who suggested showering them with the derrision they deserve and soccer fans do tend to be a rowdy bunch. Maybe we should start throwing stuffed animals at the most hated judges.

I would say a 3 year suspension is pretty severe. Certainly on a par with what a cheating athlete gets. Your very nicely written letter to Speedy highlights exactly what I have an issue with. Any time a judge does not mark a skater the way the skaters uberfans think he should be marked that judge is immediately and vociferously accused of cheating. This is hardly fair to the judge who may not have cheated but saw something he didn't like or even made a "gasp" human error. If a judge is found guilty then absolutely post their name on every tree and off with the head!

There is a very good reason - the one i've listed above.

And i don't think attracting new fans has any place in an argument about the integrity of the sport. The judging system and the judges should be beyond reproach. History tells us the judges are not beyond reproach and history shows us that the ISU does not clean up it's messes correctly (like invesitgating the exchange of votes for ice dance at SLC). That being the case the public should be able to know who is responsible for what.

And if we do entertain arguments about attracting fans or participants to the sport then i would argue that a clean governing body with tarnsparent judging and accountability in place is always going to be more favoured by fans and participants than one with annonymity.

Ant

I think attracting fan is important to the discussion. No integrity, no fans, no sport. Where I differ in view is that I don't think that removing the judges annonymity will make a tinkers damn bit of difference to the root cause of the problem which is as you said "the ISU does not clean up it's messes correctly". What it does do is reduces the tendancy for judges to be personally publicly castigated for every perceived slight against <insert your favorite skater here>

 

[Joesitz (RIP)]

Problem with cheating judges is that they are really judging the country and not the contestant. That's very sad for a contestant who works so hard leading up to the contest only to realize it's my country that is being judged - not me!

 

[sk8rdad]

I would think that keeping larger, more powerful federations who control more $$ would provide plenty of reasons for competitions that are not fair. I agree that if the judging were ALWAYS, as in EVERY SINGLE TIME, blatantly unfair, then the sport would not last. But a certain amount of scandal probably drives more interest in the sport, such as Harding's cohorts' attack on Kerrigan, the 2002 pairs scandal, etc. Part of the reason likely being the behavioral principle that random results actually produce behavior that is very resistant to extinction, whereas consistent negative results, being predictable, suppress behavior quickly. So some people keep tuning in to see what will happen THIS TIME.

But obviously the needed thresholds for staying interested in and/or believing in the system will differ for different individuals. Some of us find the rate of reinforcement ("correct" or "fair" results) to still be enough to keep us watching our TVs or internet feeds or buying tickets, especially if we have other factors interesting us, like being a big fan or family member of someone competing. If we have these other vested interests, we're also going to have added motivation to believe in the system -- because if we didn't believe in it, we'd just quit. That motivation's gotta be incredibly strong for the athletes, too -- some part of them has to believe in the system, or they must have enough other reasons to do it (like they love skating enough despite results or it provides income or they are independently wealthy and just find it fun, etc.), or they wouldn't compete at all.

Others, such as my DH, can still admire the skating but deride any attempts to view it as a competition. He can't believe I still get emotionally invested in the results as he sees it all as being completely rigged and bogus.

Hmmm... perhaps Speedys ultimate goal with anonymous judging is to ultimately replace the actual results with randomized results thus creating more scandal that we can blame on the judges and ultimately draw the whole world into watching what will unfold next week. It just might work in a weird twisted sort of way.

This seems like a small number of sanctions compared to competitions, but I really don't know the total number of judged events and the total number of judgements involved so it's hard to make much out of it. But there was only one judge with 3 flags, but we don't know what if any sanction was imposed, right?

Interesting on the difference between dance and the other disciplines re: where the outliers occurred.


This really does seem to be saying that 2006 was better because potential scandals were hidden better, not because there were legitimately fewer reasons for there to BE a scandal. Is this really what is being argued here, that problems are being hidden better, and that's a good thing? Or am I misunderstanding what's being said?

They also show some summaries of previous years. There seem to have been fewer before COP, a large jump during the transition years (to be expected as everyone learns the new system) and then fairly consistent number after COP with the post COP number being slightly higher than pre COP.

This may or may not be an indication of the system working better. The ISU should be more public about the type and nature of the assessments for more transparency.

Those here who want to eliminate the anonymous judge would likely say that nothing has changed and that problems are being better hidden. My take is more it doesn't matter if the public knows or not as the root cause of the problems is that the ISU doesn't discipline judges who cheat so all removing the anonymity of the judge would accomplish is the general populace can grip about specific people rather than a faceless entity.

 

[Mathman]

...so all removing the anonymity of the judge would accomplish is...

That is the problem I have with this whole discussion. People in favor of anonymous judging say, why remove it?

I think the default should always be open judging. Then the correct question is, why impose anonymous judging?

 

[sk8rdad]

That is the problem I have with this whole discussion. People in favor of anonymous judging say, why remove it?

I think the default should always be open judging. Then the correct question is, why impose anonymous judging?

Because any time a judge does not mark a skater the way the skaters uberfans or the skaters national media think he/she should be marked that judge is immediately and vociferously accused of cheating. It's guilty until proven innocent and then guilty anyway because the ISU is corrupt.

 

[Mathman]

^ That is the best answer I have heard so far. Judges already are never acknowledged in a positive way, always blamed for everything, plus they don't even get paid for their labors of love.

On top of all the other qualifications, a figure skating judge must develop a thick enough skin to stand up in front of a hostile crowd and say, "Oh, yeah? Boo THIS!" :thumbsup:

 

[Mathman]

She also loved that in the COP "You get booed less".

I think that's because the fans are so bewildered by the scoring system that they don't know whether to cheer or boo.

.... Hmmm...OK, I said that as a joke, but now that I think about it, there may be something to it. Under 6.0 judging, if the Ukranian judge gives my skater a number four ordinal when I thought she was better than the number three girl, I can let out a lusty boo in all good conscience.

But under CoP, the score comes up as 163.89. This includes base scores for 12 different elements of varying difficulty, together with negative GOEs for errors like edge calls and underrotaions, plus levels achieved by accumulating various "features," plus positive GOEs for satisfying a check-list of bullets, plus five component scores based on a graduated quarter-point scale -- who am I to have an opinion about whether I liked the skate or not?

 

[sk8rdad]

Over the last 12 years I have developed a huge respect for the judges in this sport. Judges are primarily volunteers and make little to no money for what is a essentially a thankless job. They spend untold hours in rinks not only judging but providing feedback to skaters to help them improve. While I don't always agree with how a judge might mark something and I have see judges make mistakes it does not lessen my respect for what they do for pure love of the sport.

I is truely unfortunate that a few bad apples have so severely damaged the reputations of all judges.

 

[antmanb]

Ok then... lets pursue this antmanb

Lets say the ISU makes the Judges names available with the marks again.

What is the criteria to decide if a judge cheated?

Who makes the determination that an investigation called for?

Who decides if a sanction is necessary?

Who decides what the sanction should be?

For all of your questions the answer is the ISU following the proper rules of its constituation, with a good a proper appeals procedure that ultimately can go up as high as the Court of Arbitration for Sport. The difference is that we as the fans and particpants looking in have nothing hidden from us. If we are to believe that the ISU is doing its job properly then it should all be completely transparent. If the judges aren't brave enough to have their names showing by their marks (like they had for about 100 years before annonymity was brought in) then get out of the kitchen!

I will offer a few observations:

In hockey video review is done by officials involved that you or I do not know the names of. This may be true in other sports. In the NFL do you know who made the actual call on any given play. In soccer the sideline officials typically are not know by name AFAIK. It is also rare for these sports governing bodies to discuss sanctions against their officials or even ban them outright.

As far as i was aware in all Premier league games we get over here in the UK when all the names of the players etc are displayed, the names of the referees and all officials are also listed. In cases where questionable decisions/calls have been made by officials, if official complaints are put in my the manager of a club, then it is all over the news. I forget the exact example but Manchester United were involved in one recently. I'm prtty certain that the controversy surrounding Thiery Henry's handball that knocked Ireland out of the world cup qualifiers was widely reported and the referee named, but i don't follow football so closely so i'm sure someone more in the know will correct me if i'm wrong.

With the amount of discussion we have here over officiating I would guess that we would have no officials left after a single grand prix season.

In reality though, the judges mostly get it right. Sometimes it seems wrong, and not just because my favourite didn't win. Sometimes when it feels wrong, the protocols show that there is an anomily and right now I can't identify who gave the score and try to see why. Maybe it would placate people to see all of the scores by judge so that they can say at least see who marked in what way and explain the result. The implication is, that if the public know this then an outcry like SLC might ensue. This is what rubs me up the wrong way. Annoymous judging looks suspicious even if it might not be. The reasno for this is that is has been put in place to stop the public from being able to react to competitions like they did in SLC, not to make the judging better.

Cheating athelets are usually given a second chance so why not officials.

Well all athletes are subject to random drug tests. The embarassment of cheating athletes usually means they do very little after their bans are over and if they do win again, the win is viewed with suspicion. But you know what? Those athlete's are not hidden from us, the viewing public so we can come to our own conclusions. We can't with the now annonymous judges.

Maybe convicted judges who judge again should be subject to random phone taps/bugging without their knowledge, just to see if they've really cleaned their act up

I will grant that the ISU is notorious for slaps on the wrist rather than more severe penalties and this should be fixed. SLC was a case in point while the offending judge received a severe penalty the two accused federation heads got little or nothing for their involvement. Did knowing the names help? Not much!

In terms of sanction and the like no. But in terms of discovering that is happened and kicking up a ****-storm because there was cheating? Absolutely! Look at how the whole judging system changed because we knew the players and we could prin headlines based on facts. The IOC was embarassed.

I take the opint that the sanction was not enough and nothing has changed in that repsect but equally i think nothing changed because it didn't have to - they brought in annonymity so nothing needed to change.

Ant

 

[antmanb]

Hey why not as long as you don't seriously expect him to answer.
I was thinking more along the lines of the discussions in these forums not in correspondence to Speedy.

Obviously i was joking but you seemed to suggest that because of the protocols we had some recourse by complaining officially, and the point i was making(albeit in a joking manner) was that we really don't. The power we might have is to whip up a media frenzy, however, that doesn't work if you can't identify the person who's marks you are questioning.

Actually there are more complicated sports, baseballs rule book in something like 11 chapters long. There have been occasions were an umpire has had to pull it out in the middle of a game to refer to it.

As does cricket, but regardless of how complicated the rules are, the result and the details behind the results is not complicated - who ran the most, who knocked down the wickets. It's not hard to understand. A particular call or particular rule may need explaining or referal to the big book of rules but the result does not require an encyclopedic knowledge of the sport like FS seemingly does.

You were the one who suggested showering them with the derrision they deserve and soccer fans do tend to be a rowdy bunch. Maybe we should start throwing stuffed animals at the most hated judges.

I was becuase you know what, if I am ever in attendance at a competition where a cheating judge appears on the panel, I will not clap politely I will boo them and if needs be explain to the people around me why i'm booing.

I would say a 3 year suspension is pretty severe. Certainly on a par with what a cheating athlete gets.

But I would like to see more investigation take place by the ISU. In SLC Le Gougne first said that deal in the pairs was a direct swap for a result in the dance event. While she did come up with a lot of stories, the explanation for the fixing of the pairs really did warrant further investigation. Why did the ISU simply say "nothing to see here" sweep it under the carpet, scape goat the judge that confessed and not investigate a single bit further. In fact, let's sanction those from within our own organisation who are telling me to look into this further. That was when it was all eyes on the ISU. Now we wouldn't even be able to identify who judged which way. If we had annonymous judging and the same situation what would have happened if Le Gougne broke down to Sally Stalpeford now? It would all have been done "internally" the stroy may have leaked ot the press but what would the press have done? Look at the protocols? And this tells them what? Cinquanta goes into overdrive denying what's happened, issuing sanctions for any judge that speaks out about the judging. The whole thing is covered up and nothing happens. This is the point of annonymous judging, so that a scandal like SLC would never again be discovered by the public.

Your very nicely written letter to Speedy highlights exactly what I have an issue with. Any time a judge does not mark a skater the way the skaters uberfans think he should be marked that judge is immediately and vociferously accused of cheating. This is hardly fair to the judge who may not have cheated but saw something he didn't like or even made a "gasp" human error. If a judge is found guilty then absolutely post their name on every tree and off with the head!

What on earth does this have to do with uber fans? My example was one where every judge bar one on the panel gave a +2 except one that gave -2. That is completely out of whack. In all likelihood that would be human error. No drama queen gasp required. Fix it. But the ISU says sorry this cannot be fixed after the event. What if the judge says i think the -2 is justified, well unless it can be i think the judge doesn't have a grasp on the rules properly and should at least be demoted in terms of abiliy to judge the level of event s/he was judging and go down a few steps to hone their judging skills. But what harm does it do to know that judges name?

I think attracting fan is important to the discussion. No integrity, no fans, no sport. Where I differ in view is that I don't think that removing the judges annonymity will make a tinkers damn bit of difference to the root cause of the problem which is as you said "the ISU does not clean up it's messes correctly". What it does do is reduces the tendancy for judges to be personally publicly castigated for every perceived slight against <insert your favorite skater here>

I and others have said it over and over again. If you hide something from the public, the perception is that you are trying to be sneaky or have something to hide. If you are open with the public then it shows that you are being above board.

The skaters put themselves out there for public criticism and castigation, if the skaters (who are often minors) can do this then why shouldn't the (adult) judges? Skaters know what they're getting themselves into, so do judges. Why do those that agree with annoymous judging expect children to be able to take the comments that are made about their skating, but don't think judges can handle it?

Ant

 

[inside edge]

I think that's because the fans are so bewildered by the scoring system that they don't know whether to cheer or boo.

.... Hmmm...OK, I said that as a joke, but now that I think about it, there may be something to it. Under 6.0 judging, if the Ukranian judge gives my skater a number four ordinal when I thought she was better than the number three girl, I can let out a lusty boo in all good conscience.

But under CoP, the score comes up as 163.89. This includes base scores for 12 different elements of varying difficulty, together with negative GOEs for errors like edge calls and underrotaions, plus levels achieved by accumulating various "features," plus positive GOEs for satisfying a check-list of bullets, plus five component scores based on a graduated quarter-point scale -- who am I to have an opinion about whether I liked the skate or not?

:yes: :yes:

 

[gkelly]

I think that's because the fans are so bewildered by the scoring system that they don't know whether to cheer or boo.

But under CoP, the score comes up as 163.89. This includes base scores for 12 different elements of varying difficulty, together with negative GOEs for errors like edge calls and underrotaions, plus levels achieved by accumulating various "features," plus positive GOEs for satisfying a check-list of bullets, plus five component scores based on a graduated quarter-point scale -- who am I to have an opinion about whether I liked the skate or not?

Do you only like skates that deserve the highest marks?

I think some fans who have an interest in both numbers and the details of skating technique can get pleasure out of analyzing the performances in similar ways that the judges do.

And/or passing judgment on the judges, since that seems to be a favorite pastime of many fans, but to do so with any authority they would have to actually understand the judging process and the technique that is being judged.

Not a casual pleasure -- it takes effort to acquire the necessary knowledge -- but rewarding nonetheless for those who are interested. And it's not so much about liking the performances as analyzing and evaluating them.

As I started attending and watching videos of more and more competitions at different levels and studying the rules, under the old system ~15 years ago, I found that the more I learned, the more I understood and often agreed with the results. That's still true for me in the new system.

If that's not where your interest lies, why not just enjoy the skates you enjoy for whatever reason you enjoy them without needing validation from the results. Heck, the judges themselves might enjoy performances more that they don't give the highest marks.

Maybe some skaters just get to you emotionally. Maybe you especially like they way they relate to music, their body lines, the way they move in general, their evident joy in skating, the way they interact with the audience and make eye contact with you specifically. Maybe you know something about their history and can appreciate their just being at this competition, getting through the program, landing a specific jump, skating "clean," etc., as a personal triumph.

None of those pleasures have anything to do with the scores. Just enjoy.

 

[gkelly]

What on earth does this have to do with uber fans? My example was one where every judge bar one on the panel gave a +2 except one that gave -2. That is completely out of whack. In all likelihood that would be human error. No drama queen gasp required. Fix it. But the ISU says sorry this cannot be fixed after the event. What if the judge says i think the -2 is justified, well unless it can be i think the judge doesn't have a grasp on the rules properly and should at least be demoted in terms of abiliy to judge the level of event s/he was judging and go down a few steps to hone their judging skills. But what harm does it do to know that judges name?

If we're just talking about one isolated GOE that's out of whack, it's a mistake of some sort.

Data entry error, didn't see something that everyone else saw, forgot the rules applying to that particular element (e.g., 0 or +1 for a nice double jump in a senior ladies' SP), etc.

Or less likely the judge did see something everyone else missed. E.g., that lutz was in the corner on her end of the panel, she could see a severe edge change, the tech panel at the other end didn't have a good view so gave the skater the benefit of the doubt and didn't call e or even !, the other judges rewarded good qualities of the jump.

Judges do make mistakes and do see elements differently. That's exactly why there is a panel and not a single person making the judgments and why high and low scores are dropped for each element.

How much time (and time is money) would it take to research every discrepancy and go back afterward to change the marks and in very close cases possibly the results if a simple error were involved?

Is the public really interested in every detail of every skater's marks? Probably not -- probably only when it affects the medalists or their favorites. And they all have different favorites.

The real problem is if a judge has a pattern of out-of-whack marks. That's what needs to be tracked and followed up on.

If there are just a lot of randomly out-of-line numbers, then the judge is likely incompetent and either needs to be educated or no longer used.

If there's a pattern of consistently favoring certain skaters or certain nationality of skaters and consistently lowballing others, that would suggest cheating.

As I understand it, the system does currently statistically check for national bias patterns and flags those judges for explanations or assessments.

If two or more judges show the same patterns, that might suggest collusion and be reason to investigate. I don't know if there's any means in place for tracking this. If not, there probably should be.

I and others have said it over and over again. If you hide something from the public, the perception is that you are trying to be sneaky or have something to hide. If you are open with the public then it shows that you are being above board.

Which is why I think it's more a question of bad PR than bad policy per se.

It might be interesting once to research and break down the discrepancies for the public to explain each out-of-line mark, just to show how innocuous most of them are and give insight into the judging process, the places where there's room for honest disagreement, and the places where innocent errors can all too easily creep in. Could make a good educational feature for the ISU to put out. Do it at an event that doesn't have a lot on the line and therefore little incentive for cheating, so we could have confidence that the judges identified and interviewed would be telling the truth about the reasons for their out-of-line marks .

Hey, ISU, how about doing this at Nebelhorn some year and distributing the resulting written and video report to the media?

But I don't think the public wants resources to be spent on that kind of public nitpicking for every event. Once you understand the process, most of the details are not all that interesting.

When they make a difference in who wins Olympic medals, of course, the public does care. But those are exactly the events where the judges are under the most pressure to deliver results for their federations.

It would be great if judges were allowed and even encouraged to go public in the media after a high-profile event like Worlds or Olympics to identify themselves and explain their judging. But for those judges who choose not to, should we assume that they're trying to hide from us the fact that they cheated or trying to hide from someone else the fact that they didn't?

 

[Mathman]

(Responding to post #74 above.)

Well said. But still, it's a sport. A fan likes to cheer his team on to victory, as well as to appreciate a fine effort. It is not only, oh look how pretty, but also, haha, she just kicked the other girl's butt!

I think I probably have as good an understanding of the scoring system as any fan who is not actually involved in the sport in any way, but just enjoys watching the performances. I also enjoy poring over the protocols after the fact and playing with the numbers. I greatly enjoy reading posts on Golden Skate from people who know vastly more than I do about skating.

Yet, I do not expect ever to get good enough at it that I can score along with the judges, know whether a spin deserves a level three or a level four, or even tell a loop from a Salchow in real time.

Maybe it was just a psychological trick that I played on myself, but under 6.0 scoring I got a thrill watching the 5.7s and 5.8s come up. It was part of the performance. What?!, a 5.4 for that beautiful skate? -- Booooo. The judging -- and agreeing or disagreeing with it --was part of the fun.

With the CoP it seems like the whole process has been taken over by an anonymous robot, chugging along mechanically counting the clicks. To me, the immediacy of the connection among performance and fan has been diminished.

 

[gkelly]

Maybe it was just a psychological trick that I played on myself, but under 6.0 scoring I got a thrill watching the 5.7s and 5.8s come up. It was part of the performance. What?!, a 5.4 for that beautiful skate? -- Booooo. The judging -- and agreeing or disagreeing with it --was part of the fun.

See, I think this is a problem for a technical sport.

If you want disagreeing with the results based on gut feeling and singling out individual officials to disagree with to be part of the fun, then you want bad judging. You want the sport to build in opportunities for you to boo or to get outraged.

Many pro competitions seemed to actively encourage that kind of audience engagement. But those weren't serious sporting contests with serious consequences for the competitors. And I think they did a big disservice to expectations of what real skating competitions are about.

The sport of figure skating needs above all to be as fair as possible for the athletes. Fun for the audience has to take a backseat to that consideration where the two goals are at odds.

Yet at the same time some fans who enjoy booing also want to clean up corruption (perceived as well as real) in judging.

Of course that needs to be done.

But you can't have it both ways. If the judging is totally clean, is it appropriate to be booing honest judges just because it's fun to boo?

If corruption or subjectivity can be better eliminated and/or identified by itemizing the scores in ways that are tedious to announce in arenas or for less knowledgeable fans to follow, is it better to sacrifice fan fun for technical accountability?


To ask another question,

If you could have IJS without anonymity (as is the case on the JGP, at US domestic competitions, etc.), or 6.0 with anonymity (the "interim system" from 2003 and 2004), which would you prefer?

 

[antmanb]

If we're just talking about one isolated GOE that's out of whack, it's a mistake of some sort.

Data entry error, didn't see something that everyone else saw, forgot the rules applying to that particular element (e.g., 0 or +1 for a nice double jump in a senior ladies' SP), etc.

Or less likely the judge did see something everyone else missed. E.g., that lutz was in the corner on her end of the panel, she could see a severe edge change, the tech panel at the other end didn't have a good view so gave the skater the benefit of the doubt and didn't call e or even !, the other judges rewarded good qualities of the jump.

Judges do make mistakes and do see elements differently. That's exactly why there is a panel and not a single person making the judgments and why high and low scores are dropped for each element.

How much time (and time is money) would it take to research every discrepancy and go back afterward to change the marks and in very close cases possibly the results if a simple error were involved?

Is the public really interested in every detail of every skater's marks? Probably not -- probably only when it affects the medalists or their favorites. And they all have different favorites.

The real problem is if a judge has a pattern of out-of-whack marks. That's what needs to be tracked and followed up on.

If there are just a lot of randomly out-of-line numbers, then the judge is likely incompetent and either needs to be educated or no longer used.

If there's a pattern of consistently favoring certain skaters or certain nationality of skaters and consistently lowballing others, that would suggest cheating.

As I understand it, the system does currently statistically check for national bias patterns and flags those judges for explanations or assessments.

If two or more judges show the same patterns, that might suggest collusion and be reason to investigate. I don't know if there's any means in place for tracking this. If not, there probably should be.



Which is why I think it's more a question of bad PR than bad policy per se.

It might be interesting once to research and break down the discrepancies for the public to explain each out-of-line mark, just to show how innocuous most of them are and give insight into the judging process, the places where there's room for honest disagreement, and the places where innocent errors can all too easily creep in. Could make a good educational feature for the ISU to put out. Do it at an event that doesn't have a lot on the line and therefore little incentive for cheating, so we could have confidence that the judges identified and interviewed would be telling the truth about the reasons for their out-of-line marks .

Hey, ISU, how about doing this at Nebelhorn some year and distributing the resulting written and video report to the media?

But I don't think the public wants resources to be spent on that kind of public nitpicking for every event. Once you understand the process, most of the details are not all that interesting.

When they make a difference in who wins Olympic medals, of course, the public does care. But those are exactly the events where the judges are under the most pressure to deliver results for their federations.

It would be great if judges were allowed and even encouraged to go public in the media after a high-profile event like Worlds or Olympics to identify themselves and explain their judging. But for those judges who choose not to, should we assume that they're trying to hide from us the fact that they cheated or trying to hide from someone else the fact that they didn't?

Trust gkelly to come along with sensible debate and suggestions...how am i supposed to use hyperbole and an innate drama queen-ness with this ?!

In reality I think the flagging and checking to see how often judges are outside the "corridor" probably does the trick. I think that having open judging with names by the numbers would enhance the fact that we as fans can do our own "corridor assessment" and do our own version of the judges review meeting.

It would be a "nice to have" if we did get some report back on serious judging disciplinaries but i don't think it would be necessary.

I think in all likelihood if the judging was no longer annonymous, a fair bit of grousing from the fans would go.

Ant

 

[sk8rdad]

Trust gkelly to come along with sensible debate and suggestions...how am i supposed to use hyperbole and an innate drama queen-ness with this ?!

In reality I think the flagging and checking to see how often judges are outside the "corridor" probably does the trick. I think that having open judging with names by the numbers would enhance the fact that we as fans can do our own "corridor assessment" and do our own version of the judges review meeting.

It would be a "nice to have" if we did get some report back on serious judging disciplinaries but i don't think it would be necessary.

I think in all likelihood if the judging was no longer annonymous, a fair bit of grousing from the fans would go.

Ant

ITA with gkelly, he is obviously better at debating the points than I am. It would also go so far as to say it should be required that judges be named when disciplined at least for serious or repeat cases. Part of the assessment process is to identify judges who need remedial training to improve their judging. And while I am being somewhat of a devils advocate here by taking the anonymous judging side (yes I would prefer not to have non-anonymous judging) but I feel that many judges are wrongly accused of cheating regardless of the facts (guilty until proven innocent).

Even in your letter scenario you assumed that the one out of kilter mark was the cheating judge but if you were a believer in block judging the reverse could be true where the -2 GOE was the judge not involved in holding up a skater. Not the most realistic scenario but I hope you get my point.

If the ISU is more transparent with the investigative process then yes the judges don't need to be anonymous because the general fan base would be accusing all judges of being cheats. It is unfortunate that a few have given the whole a bad name.

 

[chloepoco]

ITA with gkelly, he is obviously better at debating the points than I am. It would also go so far as to say it should be required that judges be named when disciplined at least for serious or repeat cases. Part of the assessment process is to identify judges who need remedial training to improve their judging. And while I am being somewhat of a devils advocate here by taking the anonymous judging side (yes I would prefer not to have non-anonymous judging) but I feel that many judges are wrongly accused of cheating regardless of the facts (guilty until proven innocent).

Even in your letter scenario you assumed that the one out of kilter mark was the cheating judge but if you were a believer in block judging the reverse could be true where the -2 GOE was the judge not involved in holding up a skater. Not the most realistic scenario but I hope you get my point.

If the ISU is more transparent with the investigative process then yes the judges don't need to be anonymous because the general fan base would be accusing all judges of being cheats. It is unfortunate that a few have given the whole a bad name.

:agree:

 

[Hsuhs]

In reality I think the flagging and checking to see how often judges are outside the "corridor" probably does the trick.

There's a chance it doesn't. The scores need to almost make no sense to end up outside the "corridor". Like, for instance, those fixed, perfectly identical sets of pcs in CoR from 2 judges - they were not outside the "corridor", nevertheless that was some shameless blatant cheating. Has that situation been addressed? I don't know.