Scoring bias at the national level

[Mathman]

There was a national championship recently with a 10+ PCS gap between 1st and 2nd. The correct person won, and was well ahead of the competition. Still, it seemed quite extraordinary to me.

Has this ever happened before?

These kinds of gaps in PCS for 1st and 2nd might happen in a tiny skating nation where 4-5 people take part in the competition with one pro against four amateurs, but in one of the bigger skating nations I'm not so sure.

I think that there is a tendency for federations to shower points on their national champion before sending her/him/them off to worlds. If the skater won anyway, let's start the hype early to give her a reputation boost (or at least to keep pace with the national champions of other countries that are doing the same thing.)

 

[Magill]

It's written that way in case the rule excludes all qualified candidates to fill the role. In a sport like figure skating that is so dependent on the national federations to train, and develop the sport it's always going to be difficult to develop officials who aren't strongly connected to the national federation.

But does anyone even try? What is the procedure for recruitment of the right trainees to start with? Is it public and open to anyone interested in becoming a judge, tech spec etc, with publicly announced requirements you need to meet and a proper documentation and external supervision? This is a genuine question as I have no idea how it works and to get judges and tech specs separated from feds for better objectivity you should basically start with the recruitment and training procedure designed to reach people who are outsiders and not involved with the fed to start with. Otherwise, it is all "me and my sister and her best friend and no one else available" kind of situation (which, frankly it seems it is in most cases, the more you read about it).

 

[Andrea82]

To me, this is not a good rule at all. Can a federation official who aspires to serve as a tech specialist challenge the ISU's opinion as to who can reasonably be expected to finish 1-5 in a competition that has not taken place yet? Me, I would hire a lawyer to put forth the asrgument, "You think that our crappy skaters can finish in the top five?"

Referees, technical controllers and technical specialists in ISU Championships and Olympics are named directly by ISU (by the President based on the advice of Vice-President for Figure Skating after consultation with the respective Technical Committee). Federation organizing the Championship can recommend 2 people for Single and Pairs and 1 for Ice Dance for those roles.
So even if somebody challenge the likelihood to be top 5, ISU can name someone else anyway because it is not based on a draw.

Anyway, this recommendation has not been strictly followed last season.
2023 Worlds Pairs:
Referee: Italy
Technical specialist 2: Canada
2023 Euro Pairs:
Technical controller: Italy
2023 Euro Men:
Referee: France (and in this case, they can' even say it was not possible....it was enough to switch the referee of Women and Men events. Women had a Spanish referee)

 

[Mathman]

But does anyone even try?

I think the reality is that "all figure skating is local." People begin by becoming involved with their local club, most likely as skaters and coaches themselves. If they are sufficiently enthusiastic in their commitment, later they may take on responsibilities at the national level, usually starting as volunteers. I think that this is what forms the "bpttom up" pool that the ISU has to work with.

 

[Andrea82]

But does anyone even try? What is the procedure for recruitment of the right trainees to start with? Is it public and open to anyone interested in becoming a judge, tech spec etc, with publicly announced requirements you need to meet and a proper documentation and external supervision? This is a genuine question as I have no idea how it works and to get judges and tech specs separated from feds for better objectivity you should basically start with the recruitment and training procedure designed to reach people who are outsiders and not involved with the fed to start with. Otherwise, it is all "me and my sister and her best friend and no one else available" kind of situation (which, frankly it seems it is in most cases, the more you read about it).

At the moment, there is an examination to become an "International" official and then another examination to be promoted from "International" to "ISU" official.
"International" and "ISU" qualifications allow to serve in different tiers of competitions.
To sum up (by memory):
Judges: ISU qualification is required for Olympics, ISU Championships and Grand Prix Final. "International" is enough for other international competitions.
Technical controllers and specialists: ISU qualification is required for Olympics, ISU Championships, Grand Prix Final, Senior and Junior GP stages. Challenger Series events should have 2 members of the technical panel with ISU qualification out of 3.
Referees: Same for technical panel, except that "international" referee can serve in Junior GP stages.

Generally, Feds propose the candidates to take the exams.

To undergo the exam for first appointment, you need some services at national level

Judges
have served, during the 36 months preceding July 31st of the calendar year of the nomination as a Judge:
- for Single & Pair Skating: in at least three National Competitions (as per Rule 411, paragraph 9.a),
- for Ice Dance: in at least two National Competitions (as per Rule 411, paragraph 9.a)

Technical specialists
have served, during the twenty-four months preceding July 31st of the calendar year of the nomination, as a Technical Specialist in two National Competitions (as per Rule 411, paragraph 9.a).

Technical controllers:
have served, during the twenty-four months preceding July 31st of the calendar year of the nomination, as a Technical Controller in two National Competitions (as per Rule 411, paragraph 9.a).

To take the exam for promotion from International to ISU level, you need

Judges:
have served, during the 36 months preceding July 31st of the calendar year of the nomination, as a Judge:
- for Single & Pair Skating: in at least four International Competitions (as per Rule 411, paragraph 9.b) including one Senior and one Junior, one Pair and one ISU Event. The ISU Event must have taken place in the last twelve months preceding the nomination.
If the judging service in Pair Skating is insufficient, the Official must have participated in an ISU Seminar for Judges dedicated to Pair Skating during the thirty-six months preceding July 31st of the calendar year of the nomination.
- for Ice Dance: in at least three International Competitions (as per Rule 411, paragraph 9.b) including one Senior, one Junior and one ISU Event. The ISU Event must have taken place in the last twelve months preceding the nomination.

Technical controllers:
have served, during the forty-eight months preceding July 31st of the calendar year of the nomination, as a Technical Controller in a minimum of three competitions at least one of which must be an International Competition (as per Rule 411, paragraph 9.b), or for ISU Technical Specialists to become ISU Technical Controller have served, during the forty-eight months preceding July 31st of the calendar year of the nomination as a Technical Controller in a minimum of two National Competitions.

Technical specialists:
have served, during the forty-eight months preceding July 31st of the calendar year of the nomination, as a Technical Specialist in minimum three Competitions at least one of which must be an International Competition as per Rule 411, paragraph 9.b.

 

[gkelly]

There was a national championship recently with a 10+ PCS gap between 1st and 2nd. The correct person won, and was well ahead of the competition. Still, it seemed quite extraordinary to me.

Has this ever happened before?

I don't follow/analyze the scoring at various events that closely, so I couldn't say.

But it wouldn't surprise me.

Even in bigger skating nations, there are some disciplines that are stronger than others, consistently over the years or varying between disciplines from one year to the next.

There might well be a number of skaters who are all competitive with each other at a level worthy of international assignments, even Worlds and Grand Prix assignments, but who are not medal contenders at that level, and one skater from their country who is already a world medalist or on track to become one.

If you're looking just at men's free skates, for example (chosen because the PCS factors are more convenient to work with and also more likely to produce larger gaps), if one skater averages 8.5 for all components and the next-best skater averages 7.5, the total PCS difference for the free program will be 10.0.

That would be completely normal if one skater is approximately that much better at all the PCS skills than the next best skater, and especially if the stronger skater happens to have a good day and the next-best skaters don't skate their best (perhaps from nerves over battling with each other for the second Worlds slot, for example).

These kinds of gaps in PCS for 1st and 2nd might happen in a tiny skating nation where 4-5 people take part in the competition with one pro against four amateurs, but in one of the bigger skating nations I'm not so sure.

Pros vs. amateurs is not really correct terminology. They all need to be eligible skaters, and whether they earn more money from skating than they spend on their training depends on a lot of factors. Success at the international level would be the primary one, since that will come with prize money and with greater likelihood of more funding from national federations, greater likelihood of invitations to skate in well-paid ice shows, and greater likelihood of being offered corporate sponsorship.

But does anyone even try? What is the procedure for recruitment of the right trainees to start with?

At the international level or domestically?

I don't think individuals can apply to become international officials without a background of serving in that capacity domestically. (See @Andrea82's post above)

Different federations will have different rules about who is eligible to train as a national official (or lower than national first, in larger federations). Some require different levels of personal skating experience for different roles (e.g., judge vs. TS).

Smaller federations are more likely to support any good national officials they have in becoming international officials. But they may not be able to support them financially, in which case the would-be official's personal financial situation might make a difference.

And of course their availability to travel for training and for actually officiating.

Is it public and open to anyone interested in becoming a judge, tech spec etc, with publicly announced requirements you need to meet and a proper documentation and external supervision? This is a genuine question as I have no idea how it works and to get judges and tech specs separated from feds for better objectivity you should basically start with the recruitment and training procedure designed to reach people who are outsiders and not involved with the fed to start with.

As stated, the requirements for some roles will include personal skating background. So you won't get fans (or non-fans) off the street who are eligible even to be considered to train as officials in those roles. Which means they will have some connection with at least one federation that they dealt with when they were skating, although it's possible they live in a different country now.

And you won't get people who have never volunteered as officials domestically accepted as applicants to train for international appointments.

If international appointments, and later assignments, were to be completely separated from the federations, what would be good ways for the ISU to determine who to accept into training as an international official?

Maybe some combination of domestic experience in that role (national judges in largish federations would be expected to have lots of experience; applicants without that experience would be less attractive candidates, but there could be active programs to encourage potential officials from other parts of the world); past personal skating experience; and entry exams testing knowledge of skating technique and scoring rules.

And then who pays for getting the candidates to the seminars/trial judging opportunities, if not the federations? The ISU might have some development funds to support candidates from underrepresented regions to participate in training, but otherwise without federation support the candidates' personal wealth will likely weed out who gets the opportunities to train internationally.

 

[gkelly]

I think that the reali

I think the reality is that "all figure skating is local." People begin by becoming involved with their local club, most likely as skaters and coaches themselves. If they are sufficiently enthusiastic in their commitment, later they may take on responsibilities at the national level, usually starting as volunteers. I think that this is what forms the "bpttom up" pool that the ISU has to work with.

Yes.

They're all volunteers.

They get all their expenses paid, maybe a token stipend, and at higher level competitions they get nicer perks -- nice hotels, nice meals at the rink and hospitality at the hotel or elsewhere, maybe some nice gifts.

But there's no salary involved in being a judge or tech specialist. It's not a job you are hired to do.

 

[4everchan]

I think that there is a tendency for federations to shower points on their national champion before sending her/him/them off to worlds. If the skater won anyway, let's start the hype early to give her a reputation boost (or at least to keep pace with the national champions of other countries that are doing the same thing.)

depends where... not in Japan usually. definitely not in Canada. Seeing the scores from Russian Nationals, yes, big scores... usually, the USA judges are very generous as well at their National events.

 

[4everchan]

So, whether you like it or not, all judges come from deep involvement within their national federation... so one can be mad at Leanna Caron as she was the acting president of Skate Canada back then, but it doesn't change the fact that president or not, she'd been involved with Skate Canada for years and years and years and it's the same with ALL judges from pretty much ALL federations.

 

[Magill]

Thank you all for your responses.
I think it all comes back to the volunteering issue that makes it all very difficult to solve. You either need to rely on some people making their money elsewhere and yet having enough time for extensive travelling not just over weekends but also including normal weekdays and sufficiently knowledgeable about FS to become officials (not outright impossible but cannot possibly serve as a normal standard to be implemented everywhere) , or you need to find people whose employers are willing to be flexible enough to allow it, or just wealthy rentiers with appropriate expertise. Seems a bit excessive and no wonder that in the end of the day the positions end up being filled with employees of the feds or clubs, or their family members, etc.
Still, from an outside perspective, it all seems like a clique and just very opposite of what transparent and unbiased judging should be expected to be.
Professional judges drawn (not nominated) from a well-defined pool, in accordance with some well defined rules, and being paid for the work they do seems like the only solution in the long run. Apart from AI, software and computers, of course
No money? Then you need to find some. It is not like ISU has no money in the bank...
OTOH, I find it difficult to believe that so big feds like Canada or Russia (the case of Lakernik on the Sochi panel) really did not have anyone else qualified enough to be sent to perform these roles during Olympics. Sorry, I do not buy it.

 

[Andrea82]

Different federations will have different rules about who is eligible to train as a national official (or lower than national first, in larger federations). Some require different levels of personal skating experience for different roles (e.g., judge vs. TS).

Candidates for TS are required to have past skating experiences also by ISU rules

Rule 415 -Specific requirements for nomination and appointment of Technical Specialists


1. To be eligible for first appointment as an International Technical Specialist, the Official must fulfill the following requirements:
a) Age: have reached the age of twenty-four but not the age of seventy in the calendar year of the nomination.
b) Background:
i) be recruited from the group of Coaches, former competitive Skaters, or ISU/International Judges or Referees;
ii) be involved on at least a weekly basis for the discipline concerned on site;
iii) have been a former high level Skater (as a minimum at national level);
iv) have the highest knowledge of the discipline concerned with regards to the technical aspects;
v) possess good communication skills;
vi) be able to take directions and work within a team environment.

 

[4everchan]

Candidates for TS are required to have past skating experiences also by ISU rules

Rule 415 -Specific requirements for nomination and appointment of Technical Specialists


1. To be eligible for first appointment as an International Technical Specialist, the Official must fulfill the following requirements:
a) Age: have reached the age of twenty-four but not the age of seventy in the calendar year of the nomination.
b) Background:
i) be recruited from the group of Coaches, former competitive Skaters, or ISU/International Judges or Referees;
ii) be involved on at least a weekly basis for the discipline concerned on site;
iii) have been a former high level Skater (as a minimum at national level);
iv) have the highest knowledge of the discipline concerned with regards to the technical aspects;
v) possess good communication skills;
vi) be able to take directions and work within a team environment.

We are stuck
I, of course, understand the need to have judges who are highly trained skaters at the national level. How could one judge the quality of complex turns without ever having performed them? I have never figure skated but have been following the sport and educating myself for decades, yet, a lot of it remains mystery for me exactly because I didn't skate. I am okay with jumps, throws and most lifts and spins... but turns and steps??? Also, I have a hard time imagining non-skaters judging skating, as I make a parallel in my own field... a non-pianist (or a non-musician) judging a piano competition would be really random

So yeah, this requirement means that figure skating officials will always have been involved quite deeply within their own federation.
Think about Kevin Reynolds who is climbing the ladder now as a technical specialist... All his skating life, he was funded by Skate Canada... and represented Canada internationally. Kevin, so far, is very good, very demanding and very even with all skaters. I think he can become a great Tech Specialist. However, there is absolutely NO WAY to eliminate any sense of the appearance of conflict of interest because of the nature of the sport and the selection process/training of officials.

It is just impossible.

 

[Andrea82]

For instance, taking the list of current international and ISU technical specialists for Ice Dance (single is too long), with a rapid check, the vast majority of them competed in international competitions during their career

Spoiler: Full list and details

Borounov Maria: competed at 4 Continents

Lyapina Svetlana: competed at GP level, won medals at Junior Worlds

MacDonald Monica: competed at Olympics and Wolrds

Chalmers Robi : competed in international competitions

Firkin Katherine : competed in international competitions

Fraser-Lukanin Kristen: competed at Olympics

Komleva Julia

Bowness Marie : competed at Worlds

Lemay Rock : medallist at Junior Canadian Nationals

Senft Lauren Helena: competed at 4 Continents, Junior Worlds and GPs

Balkwill Kristy: did Canadian Nationals various times

Gravino Marisa: competed at Junior Worlds (with Patrice Lauzon)

Gustafsson Pia-Maria: competed at Worlds

Petetin Roxane: competed at Worlds, Euros, GPs

de la Asuncion Marien: competed at 4 Continents, GPs

Druesne Amaury

Pernet Cedric: competed at Junior Worlds

Blum Benjamin : competed at Junior Worlds

Caruso Stefano Mattia: competed at Olympics Worlds, Euros

Prior Helen

Towler-Green Candice: competed at Junior Worlds

Whyte Jamie : competed at Junior Worlds

Elek György: competed at Worlds, Euros

Szijgyarto Bianka: competed at Worlds, Euros

Chait Moracci Galit : competed at Olympics Worlds, Euros, GPs

Fermi Francesca: competed at Worlds, Euros

Margaglio Maurizio : competed at Olympics Worlds, Euros, GPs

Pedrazzini Fabrizio : competed at Worlds, Euros

Ballabio Matteo

Cappellini Anna : competed at Olympics Worlds, Euros, GPs

Zanni Matteo : competed at Junior Worlds

Higashino Ayako : skated in GPs

Tsuzuki Nakako : competed at Worlds, 4 Continents and GPs

Tanaka Hiroshi : competed at Olympics Worlds, Euros, GPs

Gijtenbeek Marie-Louise : she won Dutch Nationals (as the sole entry teamed with her brother) for almost a decade

Budner-Szuman Joanna : competed at Worlds, Euros and GPs

Dostatni Andrzej : competed at Olympics, Worlds, Euros and GPs

Krytskaya Anna

Nowak-Trebacka Sylwia : competed at Olympics, Worlds, Euros and GPs

Durnev Petr

Samokhin Denis : competed at Junior Worlds

Volobuev Ivan : competed at Junior GP level

Grincova Slavka

Kika Tomas

Vanova Jana

Yang Tae-Hwa : competed at Olympics, Worlds, 4 Continents

Stergiadu Aliki : competed at Olympics, Worlds, GPs

Ucar Alper : competed at Olympics, Worlds, Euros, GPs

Akimova Olga : competed at Olympics, Worlds, 4 Continents, GPs

Baranov Sergei : competed at Junior Worlds and in GPs

Tumanovskaya Maria

Blumberg Judy : competed at Olympics, Worlds, GPs

Rey Julia : competed at US Nationals various times with Philipp Rey

Rey Philipp: competed at US Nationals various times with Julia Rey

Jenkins Stephanie

 

[Jumping_Bean]

definitely not in Canada.

In 3 out of 4 disciplines I guess I would agree, but in Dance, Canada's National Championship scores are no more "realistic" than the scores given at some other big (Dance) fed Nationals (Not small fed level though, I don't think many could be beating British or Italian Nationals (or even German Nationals, lol) in that aspect).

I say this without wanting to give a moral judgement for which scores are "right", just that there's definitely a gap between National scoring and international scoring. For example, LaLa's score from last season's Nationals is still their highest score by 5+ points (9+ point compared to their SB 2022/23). Or Lauriault/le Gac scoring ~196 at their two domestic competitions last season and only breaking 180 one time internationally (at Skate Canada).

As long as everyone is aware of that, and skaters still get scored in proportion to each other, it's not terribly bad imo - Kind of unnecessary, and a bit stupid (because it goes completely against the general purpose of even having judging guidelines), but it generally doesn't cause much harm. It's when skaters (especially juniors) are only used to domestic scoring, and then are in for a rude awakening when they get their first international assignments when I start having a serious problem with it - Something we've definitely with for example the Canadian Jr Ice Dancers, or the US singles skaters.
The only ones who tend to have positive upward movement of scores internationally throughout the bench for comparable skates are indeed the Japanese singles skaters (not necessarily the Ice Dancers though).

We are stuck
I, of course, understand the need to have judges who are highly trained skaters at the national level. How could one judge the quality of complex turns without ever having performed them? I have never figure skated but have been following the sport and educating myself for decades, yet, a lot of it remains mystery for me exactly because I didn't skate. I am okay with jumps, throws and most lifts and spins... but turns and steps??? Also, I have a hard time imagining non-skaters judging skating, as I make a parallel in my own field... a non-pianist (or a non-musician) judging a piano competition would be really random

So yeah, this requirement means that figure skating officials will always have been involved quite deeply within their own federation.
Think about Kevin Reynolds who is climbing the ladder now as a technical specialist... All his skating life, he was funded by Skate Canada... and represented Canada internationally. Kevin, so far, is very good, very demanding and very even with all skaters. I think he can become a great Tech Specialist. However, there is absolutely NO WAY to eliminate any sense of the appearance of conflict of interest because of the nature of the sport and the selection process/training of officials.

It is just impossible.

That requirement is just for the tech panel + data operators though, and the technical panel is generally not the panel that is being complained about the most when it comes to national bias anyway. To become a Judge there are no requirements to have been a skater yourself, though you still need fed support to even be allowed to become an international judge.

With the smaller number of people needed for the tech panel (3/4 vs 9 for the judging panel), it should also be more realistically possible to avoid using anyone from a federation expected to place top 3 or even top 5. Obviously, that seems to not be done consistently at the moment, and won't solve all problems, but at least it's possible, which might not be the case for the judging panel.

 

[4everchan]

In 3 out of 4 disciplines I guess I would agree, but in Dance, Canada's National Championship scores are no more "realistic" than the scores given at some other big (Dance) fed Nationals (Not small fed level though, I don't think many could be beating British or Italian Nationals (or even German Nationals, lol) in that aspect).

I am aware that in ice dance, the scores are high, especially for the top teams but as you point out yourself, everyone else and their little cousin does it too... I think the levels are not necessarily boosted in Canada (I may be wrong) but the GOEs and PCSs are generous.

I say this without wanting to give a moral judgement for which scores are "right", just that there's definitely a gap between National scoring and international scoring. For example, LaLa's score from last season's Nationals is still their highest score by 5+ points (9+ point compared to their SB 2022/23). Or Lauriault/le Gac scoring ~196 at their two domestic competitions last season and only breaking 180 one time internationally (at Skate Canada).

As long as everyone is aware of that, and skaters still get scored in proportion to each other, it's not terribly bad imo - Kind of unnecessary, and a bit stupid (because it goes completely against the general purpose of even having judging guidelines), but it generally doesn't cause much harm. It's when skaters (especially juniors) are only used to domestic scoring, and then are in for a rude awakening when they get their first international assignments when I start having a serious problem with it - Something we've definitely with for example the Canadian Jr Ice Dancers, or the US singles skaters.
The only ones who tend to have positive upward movement of scores internationally throughout the bench for comparable skates are indeed the Japanese singles skaters (not necessarily the Ice Dancers though).


That requirement is just for the tech panel + data operators though, and the technical panel is generally not the panel that is being complained about the most when it comes to national bias anyway. To become a Judge there are no requirements to have been a skater yourself, though you still need fed support to even be allowed to become an international judge.

With the smaller number of people needed for the tech panel (3/4 vs 9 for the judging panel), it should also be more realistically possible to avoid using anyone from a federation expected to place top 3 or even top 5. Obviously, that seems to not be done consistently at the moment, and won't solve all problems, but at least it's possible, which might not be the case for the judging panel.

I bet many of the judges are former skaters too ...

 

[Jumping_Bean]

I bet many of the judges are former skaters too ...

Certainly, most people don't just become so heavily invested and active in a sport later in life, and very rarely without having done the sport on some level, but it's not a requirement to have been at least a nationally-ranked skater, which changes the situation quite significantly.

 

[Magill]

I bet many of the judges are former skaters too ...

Even so, there is a big difference between being a former skater and a current employee of the federation or its president....

 

[4everchan]

Even so, there is a big difference between being a former skater and a current employee of the federation or its president....

Yes and no. A former skater may have received funding from their federation all their lives and of course, represented that federation. They may no longer be "hired" by them but they may have a sense of "gratitude" towards their federation.

 

[4everchan]

Certainly, most people don't just become so heavily invested and active in a sport later in life, and very rarely without having done the sport on some level, but it's not a requirement to have been at least a nationally-ranked skater, which changes the situation quite significantly.

I meant elite aka funded by the federation.. but you get the point.

 

[Diana Delafield]

I am aware that in ice dance, the scores are high, especially for the top teams but as you point out yourself, everyone else and their little cousin does it too... I think the levels are not necessarily boosted in Canada (I may be wrong) but the GOEs and PCSs are generous.

I bet many of the judges are former skaters too ...

And I know several who are the non-skating parents of former skaters, who just got in the habit of volunteering for tasks to keep busy while at the rink anyway, and ended up settling into training as competition judges.