ISU evaluation of questionable judging

[sk8rdad]

I think it is laughably naive to think a judge would answer truthfully or at all if a Federation where pressuring them to vote one way and they didn't. From the documentation available on the ISU site the ISU knows which set of marks belong to which judge nowhere is there any indication that the ISU tells federations which judge had which marks. The ISU has no reason to do this and while I am sure a determine federation could figure it out there is still no guarantee for a federation that the mark would be included or that it would have any significant influence to the average.

Anonymous judging only makes the ISU look like a den of thieves to those who are more interested in conspiracy theories than the actual sport. Unless someone can provide a reasonable proof that the ISU not an National federation has any motive for controlling the results then any argument presented is simply baseless conjecture or whining from a fan whose skater did place they way they wanted.

 

[Ptolemy]

How does it allow the opportunity? Please explain.

I seems to me that the premise of those who are against the anonymous judging of COP base their complaint on the assumption that the ISU as a whole can and will control which skaters will win medals. I find this argument completely spurious since the ISU has no reason to want anything but fair competition. If the judging were really blatently fixed the sport would not last and the ISU know this. Anonymous judge protects the judges from National Federation pressure to vote a specific way as the Federations will not know if a judge voted the way the federation wants.

I agree that anonymous judging does take pressure off judges so there are benefits. However, I see more risks for spurious behavior than benefits.

I don't know enough about the structure of the ISU but I will assume it is structured like most companies where the member countries vote in the leaders much like the board members of companies vote in the CEO and other senior executives.

So let's assume Cinquanta wanted to stay in power or make some extra cash. Member countries could agree to support him if their skaters were favored over other skaters. Cinquanta could ask judges to favor skaters from those countries and offer favors to those judges. With anonymous judges this could easily happen quite transparently. It is more difficult when the judges and their marks are public.

 

[Hsuhs]

Thank you for digging that up, Mathman.

A total of 17 warning were given. Two judges each received two warnings. No one received three warnings.

Not much, but at least something is done.

Of these warnings, 10 were on program components

But of course. After the Olympics that number will prob grow.

 

[bigsisjiejie]

Just wanted to point out that from Mathman's link, the statistics quoted previously on judging Assessments applied to Singles and Pairs only. Just below that (in the link) are the Ice Dance stats, which showed a total of 11 Assessments (one judge getting three), but a different split: only 2 were for Program Components and 9 were for Technical. Interesting difference between the disciplines, though with such a small sample size, not sure how statistically significant this is.

And we don't know if these Assessments represent the actual total number of computer flagged wonky judgments, or just a subset that include only the ones that were computer flagged and then officially investigated and reported by the ISU.

I do agree that anonymous judging gives the appearance that the ISU could be hiding something, whether it really is or not. In this respect, I don't think it's a positive development for the sport.

 

[antmanb]

I really don't understand all the fuss over annoymous judging. The governing bodies know who the judges are, we as fans aren't entitled to that information - just as we aren't entitled to reviews of officials in other sports. In 6.0 we had judges that were visible to everyone. In the last three Olympics that used 6.0 - 2002, 1998, 1994 - all had some sort of scandal, controversy, public outrage over some of the results. The one Olympics with annoymous judging (2006) that we have had so far, was controversy free! Let's see how things go in Vancouver

If the governing body (ie ISU) and federations know which judge is which then the fundamental principle of annoynmous judging put forward by $peedy is wrong. The claim was if judging was annonymous then judges wouldn't be pressured to mark in certain ways because NO ONE would know how any judge scored or even if that judges marks counted. If that isn't the case then the only reason for annonymous judging is to dupe the public so that they don't know who gave which marks.

Also, are you seriously saying - when the public could see which judges gave which marks we had three skating scandals, now that the public cannot see which judges gave which marks there was no scandal therefore the system is a success? Sounds to me like the ISU got their wish, if the marking is transparent their lies and tricks are spotted, when it isn't they aren't. That is not a success and it is not a system that works.

ITA. Just how often do you actually see an official in any sport disciplined publicly. It isn't the fans job to decide this it is the sports governing bodies job no matter what sport.

Football (soccer) in the UK - referees in the sport are disciplined and the fans are that rabid that they want their pound of flesh. It is widely reported what the outcomes of those disciplinaries are, and because the referees are not annoymous the fans know who they are and shower them with the derrision they deserve if they referree an important game again.

Ant

 

[i love to skate]

If the governing body (ie ISU) and federations know which judge is which then the fundamental principle of annoynmous judging put forward by $peedy is wrong. The claim was if judging was annonymous then judges wouldn't be pressured to mark in certain ways because NO ONE would know how any judge scored or even if that judges marks counted. If that isn't the case then the only reason for annonymous judging is to dupe the public so that they don't know who gave which marks.

Also, are you seriously saying - when the public could see which judges gave which marks we had three skating scandals, now that the public cannot see which judges gave which marks there was no scandal therefore the system is a success? Sounds to me like the ISU got their wish, if the marking is transparent their lies and tricks are spotted, when it isn't they aren't. That is not a success and it is not a system that works.

The individual federations do not know who the judges are - only the ISU. In basis of past Olympics - I would say that annoymous judging is working. 6.0 had many Olympics and so far the COP has had one - let's give it another shot, that's all I'm saying.

 

[Joesitz (RIP)]

Isn't there a method of changing the judges from the SP to the LP. I don't think that helps an iota in Cinquantas scheme. The replacement judges do know the scores of the SP and can booster up a favorite skater's LP score but also can bear down on the score of an unfavored skater. So how good is changing judges in midstream to prevent favoritism? And maybe there is time enough for collusion, There are so many ways to tamper with the CoP system.

The solution, imo, seems to be to end the Secret Judging.

Judges should have the power of their convictions, and the Feds should allow them. Losing a judgeship should only be for incompetence, but for illicit scoring, the penalty should be more than a few years of non judging.

 

[sk8rdad]

If the governing body (ie ISU) and federations know which judge is which then the fundamental principle of annoynmous judging put forward by $peedy is wrong. The claim was if judging was annonymous then judges wouldn't be pressured to mark in certain ways because NO ONE would know how any judge scored or even if that judges marks counted. If that isn't the case then the only reason for annonymous judging is to dupe the public so that they don't know who gave which marks.

Also, are you seriously saying - when the public could see which judges gave which marks we had three skating scandals, now that the public cannot see which judges gave which marks there was no scandal therefore the system is a success? Sounds to me like the ISU got their wish, if the marking is transparent their lies and tricks are spotted, when it isn't they aren't. That is not a success and it is not a system that works.



Football (soccer) in the UK - referees in the sport are disciplined and the fans are that rabid that they want their pound of flesh. It is widely reported what the outcomes of those disciplinaries are, and because the referees are not annoymous the fans know who they are and shower them with the derrision they deserve if they referree an important game again.

Ant

Currently the public gets a huge amount of detail on the judges score sheets. I dont know of any other sport that makes this level of detail available to the general public. If someone has an issue with a set of marks then you can refer to the judge by position on score sheet. The ISU knows which judge is which otherwise they would not be able to discipline. Are you suggesting that figure skating fans should act like soccer fans. Maybe a couple of drunken brawls in the stands or the fans charging the judges and attacking them.

There is no reason for the general public to know the name of a judge other than to berate them or their federation publicly for cheating. That certainly wont help draw new fans to the sport. I personally feel that COP has significantly reduced the judging problems that the 6.0 system was notorious for. Under 6.0 we would never have seen the potential for a North American sweep of the Dance podium. While it may or may not happen the fact that it can even be discussed speaks to not only the strength of Dance in NA but the improvement that COP has made to the fairness of the judging.

And yes I do know that COP isn't perfect there are a lot of little things that will likely continue to be reviewed and change over the years. This forum may even drive some of those changes but it is and always will be a subjective sport and as such will never be perfect.

 

[janetfan]

There is no reason for the general public to know the name of a judge other than to berate them or their federation publicly for cheating. .

And what would be so wrong about that

 

[sk8rdad]

Isn't there a method of changing the judges from the SP to the LP. I don't think that helps an iota in Cinquantas scheme. The replacement judges do know the scores of the SP and can booster up a favorite skater's LP score but also can bear down on the score of an unfavored skater. So how good is changing judges in midstream to prevent favoritism? And maybe there is time enough for collusion, There are so many ways to tamper with the CoP system.

The solution, imo, seems to be to end the Secret Judging.

Judges should have the power of their convictions, and the Feds should allow them. Losing a judgeship should only be for incompetence, but for illicit scoring, the penalty should be more than a few years of non judging.

So you think its the public who should decide if a judge should be sanctioned? What would your criteria be? There are no more ways to tamper with COP than 6.0 or judging in any other sport for that matter. Under COP there have been more "Assessments" issued that there were prior to COP and far few questionable results that would seem to indicate the system is better, to me anyway.

 

[Joesitz (RIP)]

Currently the public gets a huge amount of detail on the judges score sheets. I dont know of any other sport that makes this level of detail available to the general public. If someone has an issue with a set of marks then you can refer to the judge by position on score sheet. The ISU knows which judge is which otherwise they would not be able to discipline. Are you suggesting that figure skating fans should act like soccer fans. Maybe a couple of drunken brawls in the stands or the fans charging the judges and attacking them.

There is no reason for the general public to know the name of a judge other than to berate them or their federation publicly for cheating. That certainly wont help draw new fans to the sport. I personally feel that COP has significantly reduced the judging problems that the 6.0 system was notorious for. Under 6.0 we would never have seen the potential for a North American sweep of the Dance podium. While it may or may not happen the fact that it can even be discussed speaks to not only the strength of Dance in NA but the improvement that COP has made to the fairness of the judging.

And yes I do know that COP isn't perfect there are a lot of little things that will likely continue to be reviewed and change over the years. This forum may even drive some of those changes but it is and always will be a subjective sport and as such will never be perfect.

Those kids work hard on their craft and are entitled to a fair judgement. Cheating officials should be weeded out and exposed, and not ever to be permitted to officiate again.

 

[i love to skate]

Currently the public gets a huge amount of detail on the judges score sheets. I dont know of any other sport that makes this level of detail available to the general public. If someone has an issue with a set of marks then you can refer to the judge by position on score sheet. The ISU knows which judge is which otherwise they would not be able to discipline. Are you suggesting that figure skating fans should act like soccer fans. Maybe a couple of drunken brawls in the stands or the fans charging the judges and attacking them.

There is no reason for the general public to know the name of a judge other than to berate them or their federation publicly for cheating. That certainly wont help draw new fans to the sport. I personally feel that COP has significantly reduced the judging problems that the 6.0 system was notorious for. Under 6.0 we would never have seen the potential for a North American sweep of the Dance podium. While it may or may not happen the fact that it can even be discussed speaks to not only the strength of Dance in NA but the improvement that COP has made to the fairness of the judging.

And yes I do know that COP isn't perfect there are a lot of little things that will likely continue to be reviewed and change over the years. This forum may even drive some of those changes but it is and always will be a subjective sport and as such will never be perfect.

Spot on!

Those kids work hard on their craft and are entitled to a fair judgement. Cheating officials should be weeded out and exposed, and not ever to be permitted to officiate again.

Yes, that's why the ISU has outlines that describe the steps that are taken for an official who judges outside of the "margin".

 

[sk8rdad]

Those kids work hard on their craft and are entitled to a fair judgement. Cheating officials should be weeded out and exposed, and not ever to be permitted to officiate again.

I fully understand that, I have a son who is a competitive skater and has skated at the national level. Sanctions against cheating officials should definitely be tougher but that should be the ISU's job not the court of public opinion. The ISU executive need to understand that they must investigate and take action when a controversy occurs. Once a sanction has been determine I have no problem with publishing the offenders names they would likely come out at this point anyway.

Would the following satisfy you:

Would the following satisfy those who dislike the anonymous judging:

1. The criteria for flagging a mark that will be reviewed for assessment be documented and a indication for a flagged mark be included in the detail report.
2. The review report be made public (sans names) so that the explaination of why a mark was given is known (I can see this info be useful to the athete)
3. More detail be provided by the ISU when an assessment is given. I would not be adverse to publication of names at this point since there was obviously something that was determined to be a real issue.

 

[antmanb]

The individual federations do not know who the judges are - only the ISU. In basis of past Olympics - I would say that annoymous judging is working. 6.0 had many Olympics and so far the COP has had one - let's give it another shot, that's all I'm saying.

COP is not annonymous judging, they do not go hand in hand. $peedy had to cover his a$$ so there was also the interim system under 6.0 - the one with the 6.0s being thrown out for anything and everything....because the judging was annonymous.

Just to be clear - the gripes I have with COP are genuine things that could be fixed with the system.

Annonymous judging is something I cannot swallow in any shape or form, regardless of the judging system used.

If you have people referees, judges, line judges anyone that can affect teh outcome of a sporting event I see absolutely no reason whatsoever why their calls, judging etc should not be open to the public and them be accountable for their actions.

There is no justification for not knowing the identity of the judges and how they scored.

Cheating judges should not be allowed to ever judge again. sadly the ISU thinks cheating judges should. If convited cheating judges are on a panel i want to see what marks they give.

Ant

 

[antmanb]

Currently the public gets a huge amount of detail on the judges score sheets. I dont know of any other sport that makes this level of detail available to the general public. If someone has an issue with a set of marks then you can refer to the judge by position on score sheet. The ISU knows which judge is which otherwise they would not be able to discipline.

Come on seriously?!! "Dear Mr Cinquanta, I notice that judge number 4 on skater X's sheet relating to the 3Lz must be smoking crack because skater X did a great triple lutz for which the rest of the panel gave +2 GOE yet crack smoking judge in position 4 for the element in the protocol sheet gave a -2GOE. What are you going to do about it? Love and kisses antmanb xxx"

No ther sport makes this level od detail available to the public because no other sport is this damned complicated in it's scoring or has annoymous people scoring it.

Are you suggesting that figure skating fans should act like soccer fans. Maybe a couple of drunken brawls in the stands or the fans charging the judges and attacking them.

Yes that's exactly what i'm advocating

Of course not but I thinmk we have a right to know who the cheating judges are. If not we have to believe that the ISU is doing it's job properly and (a) catching cheatnig judges and (b) sanctioning them correctly. We can look at pre-annonymous judging and note that the ISU was particularly bad at even looking for, let alone finding cheating judges, and even when they did, the likes of Danilenko and Balkov and back on judging panels having been reluctantly sanctioned to what, 3 season bans? And now with annoymous judging we're expected to believe that they cleaned up their act and are judging correctly despite the greater public not knowing how they judged?

I'm simply not that naive.

There is no reason for the general public to know the name of a judge other than to berate them or their federation publicly for cheating. That certainly wont help draw new fans to the sport.

There is a very good reason - the one i've listed above.

And i don't think attracting new fans has any place in an argument about the integrity of the sport. The judging system and the judges should be beyond reproach. History tells us the judges are not beyond reproach and history shows us that the ISU does not clean up it's messes correctly (like invesitgating the exchange of votes for ice dance at SLC). That being the case the public should be able to know who is responsible for what.

And if we do entertain arguments about attracting fans or participants to the sport then i would argue that a clean governing body with tarnsparent judging and accountability in place is always going to be more favoured by fans and participants than one with annonymity.

Ant

 

[ManyCairns]

I seems to me that the premise of those who are against the anonymous judging of COP base their complaint on the assumption that the ISU as a whole can and will control which skaters will win medals. I find this argument completely spurious since the ISU has no reason to want anything but fair competition. If the judging were really blatently fixed the sport would not last and the ISU know this..

I would think that keeping larger, more powerful federations who control more $$ would provide plenty of reasons for competitions that are not fair. I agree that if the judging were ALWAYS, as in EVERY SINGLE TIME, blatantly unfair, then the sport would not last. But a certain amount of scandal probably drives more interest in the sport, such as Harding's cohorts' attack on Kerrigan, the 2002 pairs scandal, etc. Part of the reason likely being the behavioral principle that random results actually produce behavior that is very resistant to extinction, whereas consistent negative results, being predictable, suppress behavior quickly. So some people keep tuning in to see what will happen THIS TIME.

But obviously the needed thresholds for staying interested in and/or believing in the system will differ for different individuals. Some of us find the rate of reinforcement ("correct" or "fair" results) to still be enough to keep us watching our TVs or internet feeds or buying tickets, especially if we have other factors interesting us, like being a big fan or family member of someone competing. If we have these other vested interests, we're also going to have added motivation to believe in the system -- because if we didn't believe in it, we'd just quit. That motivation's gotta be incredibly strong for the athletes, too -- some part of them has to believe in the system, or they must have enough other reasons to do it (like they love skating enough despite results or it provides income or they are independently wealthy and just find it fun, etc.), or they wouldn't compete at all.

Others, such as my DH, can still admire the skating but deride any attempts to view it as a competition. He can't believe I still get emotionally invested in the results as he sees it all as being completely rigged and bogus.

Just wanted to point out that from Mathman's link, the statistics quoted previously on judging Assessments applied to Singles and Pairs only. Just below that (in the link) are the Ice Dance stats, which showed a total of 11 Assessments (one judge getting three), but a different split: only 2 were for Program Components and 9 were for Technical. Interesting difference between the disciplines, though with such a small sample size, not sure how statistically significant this is.

And we don't know if these Assessments represent the actual total number of computer flagged wonky judgments, or just a subset that include only the ones that were computer flagged and then officially investigated and reported by the ISU.

A total of 17 warning were given. Two judges each received two warnings. No one received three warnings.

Of these warnings, 10 were on program components, 4 were on technical elements, 2 were for both, and one was for a poor referee's report.

By the way, section 10 of this same document specifies the procedure ny which members can submit proposals for future rules changes.

This seems like a small number of sanctions compared to competitions, but I really don't know the total number of judged events and the total number of judgements involved so it's hard to make much out of it. But there was only one judge with 3 flags, but we don't know what if any sanction was imposed, right?

Interesting on the difference between dance and the other disciplines re: where the outliers occurred.

The individual federations do not know who the judges are - only the ISU. In basis of past Olympics - I would say that annoymous judging is working. 6.0 had many Olympics and so far the COP has had one - let's give it another shot, that's all I'm saying.

This really does seem to be saying that 2006 was better because potential scandals were hidden better, not because there were legitimately fewer reasons for there to BE a scandal. Is this really what is being argued here, that problems are being hidden better, and that's a good thing? Or am I misunderstanding what's being said?

 

[janetfan]

This really does seem to be saying that 2006 was better because potential scandals were hidden better, not because there were legitimately fewer reasons for there to BE a scandal. Is this really what is being argued here, that problems are being hidden better, and that's a good thing? Or am I misunderstanding what's being said?

Looking at 2008 Worlds - there might not have been scandals but I think many questioned both the podiums, and the order of the podiums for the Ladies and Mens events.

Many fans still think the Dance at 2009 was rigged and if not that the podium was wrong. Not being able to see how the judging went down helps ISU hide alot.

 

[sk8rdad]

Ok then... lets pursue this antmanb

Lets say the ISU makes the Judges names available with the marks again.

What is the criteria to decide if a judge cheated?

Who makes the determination that an investigation called for?

Who decides if a sanction is necessary?

Who decides what the sanction should be?

I will offer a few observations:

In hockey video review is done by officials involved that you or I do not know the names of. This may be true in other sports. In the NFL do you know who made the actual call on any given play. In soccer the sideline officials typically are not know by name AFAIK. It is also rare for these sports governing bodies to discuss sanctions against their officials or even ban them outright.

With the amount of discussion we have here over officiating I would guess that we would have no officials left after a single grand prix season.

Cheating athelets are usually given a second chance so why not officials. I will grant that the ISU is notorious for slaps on the wrist rather than more severe penalties and this should be fixed. SLC was a case in point while the offending judge received a severe penalty the two accused federation heads got little or nothing for their involvement. Did knowing the names help? Not much!

 

[Mathman]

I think it is laughably naive to think a judge would answer truthfully or at all if a Federation where pressuring them to vote one way and they didn't.

Let's look at a real-world example.

The President of the Russian skating federation, Valentin Piseev, is expected to appoint Alla Shekhovtsova to judge ice dancing in Vancouver. Ms. Shekhovtsova is Mr. Piseev's wife.

Do you think that anonymous judging at Vancouver will prevent Ms. Shekhovtsova from scoring Domnina and Shabalin above Virtue and Moir, or from reporting back to her husband that she had done so?

An extreme case, granted. Still, I think the fallacy here lies in assuming that national federation officers and the judges they appoint are adversaries. In fact, they are colleagues and teammates.

 

[Ptichka]

Ms. Shekhovtspva is Mr. Piseev's wife. She is also the Russian judge that Didier Gailhauget promised Marie Le Gougne would support Anisinna and Peiserat at Salt Lake City in exchange for Le Gougne's support of Berezhnaya and Sikharudlidze in pairs.

You see, the reason I could never believe this story is that I cannot believe Shekhovtseva would ever do anything but support Lobacheva & Averbukh, the students of her good friend Linichuk (sure, Anissina used to be Linichuk's student, too, but then the latter made it impossible for Marina to skate in Russia). Nor do I buy that anyone would believe that she would do such a thing. I don't know if there was any backroom deals regarding B&S, but I just don't see how it could involve dancing judging in this way.