An algorithm that flags situations where a tech panel call requires a GOE reduction and therefore the GOE cannot be higher than X would work. This would ensure that judges do not finalize their marks without being aware that there was a call that was supposed to result in a GOE reduction.
There could also be a flag if a judge enters a component score with 0 as the leading digit, since that can also happen accidentally, albeit rarely.
Obviously if a judge represents the same country as the best skaters, their skaters are going to get the highest scores across the board.
I guess the question is whether judges score their own skaters consistently higher than the rest of the judging panel. But then you also have to control for whether that judge is marking everyone higher than the other judges on the panel.
There are three general reasons why a judge would be likely to score their own country's skaters higher:
*Cultural preferences.
If, for example, there is a strong tradition in that country that the most important thing about flying camel spins is a beautiful camel position, then skaters from that country are more likely to achieve beautiful positions and judges from that country are more likely to reward them. A different country might consider the takeoff and air position of the flying entry more important than the position in the spin; skaters from that country would be more likely to have a strong fly and judges from that country more likely to reward it.
There is nothing wrong with those differences of emphasis. They skating community may agree that both areas are important and should be reflected in GOEs, but it's impossible to write into the rules just how important one such quality is compared to another.
That's exactly why there are multiple judges on the panel.
*Feelings of patriotism and familiarity.
Judges are more likely to have warm feelings for skaters representing their own country and specifically for skaters they have judged often over the years since well before junior or senior level. The judges may make a strong conscious effort to avoid their scores being influenced by those feelings, but there may still be an influence at an unconscious level.
And sometimes the judges make such a strong effort to avoid bias that they end up overcompensating and underscoring their compatriots.
*Intentional score manipulation, i.e., cheating.
Judges may consciously go into a competition with the intention of helping their country's skaters to place as high as possible. Thus they may systematically overmark their own skaters, and undermark their skaters' most likely close rivals. If they're smart about math and strategy they'll do it in subtle ways that are unlikely to get flagged just by eyeballing the protocols.
If they're really nefarious they'll make deals with other judges to help out each other's skaters and together slightly lowball the rivals.
Interesting. I remember having an e-mail exchange 20+ years ago with a skating fan who was also a statistician, who recommended a similar solution: every time a skater performed who had a compatriot judge on the panel, that judge's scores would be dropped and the substitute judge's or referee's scores would be substitute in their place.
I argued that that would be nonsense with the scoring system at the time, because the whole point of the scores under ordinal judging was to rank the skaters but the actual numbers used could vary from one judge to another. Just because the Fredonian judge gave the Fredonian skater 5.8/5.8 and the substitute judge gave her 5.7/5.7 doesn't mean the substitute judge ranked her lower -- the substitute could have been using a lower score range in general and actually thought this skater should rank higher than the compatriot judge. Or vice versa.
However, with IJS, where judges are not ranking skaters but just contributing evaluations of each element and program component to an average of the panel, and an odd number of judges is not required, it could make sense to replace or omit individual judges' scores for individual skaters.
If overmarking by compatriot judges is a consistent problem, especially of the second and third types mentioned above, then this solution would be worth considering.
I would not support that. Any given individual score might be out of line because of a data input error (some of which could be caught by algorithms and flags to the judges in the computer) or because the judge had a different view of an element than the rest of the panel, or other reasons that are not nefarious and not appropriate to laugh at.
Sometimes their scores might even be more correct than the majority's.
So I just send that to them and say "here's a brief summary, watch for five minutes, then get back to me later". Otherwise I'd be talking at them for hours about skating and they tune me out ... LOL
