Should the Technical Caller role be abolished?

[DancingCactus]

But speed and distance are easy to measure. Figure skating has more variables.

And while jump height and distance traveled is interesting information, how do you translate that into points? The higher the jump, the higher the GOE? But what about air position, running edges, position of the free leg etc? Hunched posture, wobbly entries etc?

Not to mention this is only the jumps, what about the rest of the program?

What we see with the so-called AI (scoff) at the moment is that the algorithm sucks at context and nuance. Not really suited to judging fs methinks.

 

[4everchan]

But speed and distance are easy to measure. Figure skating has more variables.

And while jump height and distance traveled is interesting information, how do you translate that into points? The higher the jump, the higher the GOE? But what about air position, running edges, position of the free leg etc? Hunched posture, wobbly entries etc?

Not to mention this is only the jumps, what about the rest of the program?

Of course there is more to it. Ice coverage is not about jumps btw... nor is speed necessarily.

What I am talking about is that right now, some of this easily to measure data could be helpful not to replace the human but to allow for an objective part of the scoring.
Judges can still give GOEs on what they see.
TPs can still call the elements.

There could be a smaller part of the scoring that would be calculated objectively with this information. It doesn't need to applied as GOE.

I didn't want to get into this in detail but let me give you an example.

Ice coverage : data measured by AI.

Basic ice coverage = skater covered less than X% of the ice surface Level 1 = BV = 4.0
Average ice coverage = skater covered between X and Y % of the ice surface Level 2 = BV = 4.5
Good ice coverage = skater covereed between Y and Z % of the ice surface Level 3 = BV = 5.0
Excellent ice coverage = skater covered more than Z% of the ice surface Level 4 = BV = 6.0 ( i am being cheeky using 6.0 here LOL)

Judges would still need to put GOE on top of that score.

There are tons of other ways to include measurable date into scoring. It can be as I put up there. but it could be done in a ton of ways.

Another example : Speed. There could be key areas where speed is measured. In and out of jumps. During the step sequences. Speed rotation in spins....
This could be rewarded in various ways.
For instance, for spins and steps, it could be either an automatic bullet given by AI if speed requirements are met. Or it could be a level even, leaving all GOE scoring to judges. It could even just be a bonus applied to specific elements.
BTW it doesn't have to be measured in absolute. It could also be measured on a curve. Top 5-10% of skaters get a X point bonus for speed or so.

Maybe I am just thinking way too creatively for this thread when including new judging components like AI, one has to think outside the box here. The judging system would need to be modified here.

The way I see it is not to replace tech controllers nor judges but to have some components of the score that are left to the machines. The sport then is judged by humans with various levels of subjectivity/objectivity AND also by the technology.

Yes, AI has faults too. Some data may work out better than other for scoring. In the worst case scenario, one could say that AI has some subjective components when it comes to judging figure skating... however, there is no denying that a lot of things are objectively and quantitatively measurable and that it could give the sport some added credibility and to the skaters, some added fairness.

 

[Warwick360]

It's not just one person. It's a panel of three people who work together to determine a lot more than jump rotations and edges. Learn more about what they actually do if you want to make meaningful suggestions for change.

Oh. Did not know that. Always thought it was single person making the calls. Good to know more.

Makes it even less excusable that three people can't call an edge.

 

[Anna K.]

Oh. Did not know that. Always thought it was single person making the calls. Good to know more.

Makes it even less excusable that three people can't call an edge.

May I ask you something. Do you know what an edge call is?

 

[ElSoteroLoco]

You would still need someone to be the final authority on what each element was.

For most jump elements naming the element for the data input would be sufficient if there is a technological process for determining takeoff edges and rotation.

But occasionally skaters execute jump elements in ways that require some interpretation of exactly what the skater did and how that fits into the definition of allowable jump elements. E.g., a skater does a jump, turns on the ice, followed by double toe. Someone has to determine whether it was just a double three without the free foot touching the ice (just call the combination, let the judges reduce the GOE according to their guidelines) or whether there was weight on the other foot during those turns, e.g., a step out, and which program this is (+COMBO for short program, +SEQ for free skate).

Even weirder things can happen that don't have exact precedents or exact rules. A computer wouldn't know what to do with those and a human would need to intervene to interpret how to match up what just happened with what the rules allow and what they say about how to handle various categories of errors.

Most non-jump elements have levels. There are numerous details that the tech panel looks at to determine the level of a spin or step sequence or lift. It would be much more difficult for a computer to perceive those details and apply the appropriate rules.

The technical panel doesn't determine whether/how much a spin traveled. That is left up to the judges in awarding grades of execution for quality.

In case you're not familiar with everything that the technical panel does, here is the handbook with the detailed guidelines.

If and when technology is ready to take over some of those decisions, that could lessen the load on the panel so they could focus their attention on the non-jump elements.

But you'd need to come up with other ways to determine the levels/base values for all the non-jump elements.

Do you anticipate that technology will be able to evaluate the difficulty of these elements as set out in these guidelines?

Or do you want to get rid of the concept of levels altogether and make everything "choreographic" elements and let the judges each reflect difficulty as well as quality in their grades of execution? That would lead to more subjectivity than the current system.

If you're aiming for less subjectivity, then it might make sense, when the technology is ready, to take the determination of jump takeoff edges and rotation out of the hands of the technical panel.
But not to abolish the technical panel entirely.

 

[Mathman]

There are still judging scandals. I can name you a few Some people have still not digested some of the Olympics results
2010 men
2014 women, dance
2018 dance, etc.

:Sorry, but none of these is a "scandal." In 2010 mens, Lysacek outpointed Plushenko when Plushenko omitted a small slice of his planned technical content. Unless one thinks that the scandal is that Takahashi by far exhibited the most accomplished skating and deserved to win .

2014 dance, Davis and White were on a roll, having beaten Virtue and Moir at the Grand Prix Final (by one point) as well as at the previous Worlds. At the Olympics they skated flawlessly. Where's the scandal?

2014 womens, yes Yuna Kim was an all-time great (as were Plushenko and Virtue & Moir) and Adelina Sotnikova wasn't. If Kim had skated like she had at 2013 Worlds she would have won. At the Olympics Sotnikova did seven triples tp Kim's six and ended up putpointing her. (Granted, the Russian judge was a little too happy with Sotnikova's win. )

2018 dance. Papadakis and Cizeron were favored, and they won the free dance as expected. But their misfortune in the short dance took them outof the running. Fans are not always pleased with the results when their favorite skaters are involved, but that does not constitue a judging scandal.

In the good old days, skaters (for instance world champions Gilbert Fuchs and Ulrich Salchow) refused to compete in each other's countries because of the judging. Sonja Henie won her first world championship (in Oslo) by a vote of 3 Norwegian judges to two German/Ausrian judges. Five-time and defending world champion Herma Szabo promplty retired never to skate again, and the ISU passed a rule that from then on no coungtry would be allowed to have more than one judge on the panel.

One time Henie's father arranged for the train carrying a rival skater to a compettion to be delayed at the border to prevent the rival from competing. Some time prior to the Berlin Olympics, Henie and her father dined with Hitler at the father's estate.

Just sayin'.

 

[Mathman]

Do you know what an edge call is?

What tickles me is when the technical panel solemnly proclaims: "We hereby rule that the edge was 'Unclear,' Score it whichever way you think it should go."

 

[4everchan]

:Sorry, but none of these is a "scandal."

I do not personally think they are scandals either... but many fans do. I don't think it's fair to dismiss their strong opinions about them So I acknowledge that some fans do believe that there were strong judging errors in many events since IJS

 

[4everchan]

What tickles me is when the technical panel solemnly proclaims: "We hereby rule that the edge was 'Unclear,' Score it whichever way you think it should go."

When the call is !, which means unclear edge, the BV of the jump is not affected, but judges are required to reduce their GOE by -1 to -2.

So no... it's not supposed to be whichever way you think it should go.

 

[Mathman]

When the call is !, which means unclear edge, the BV of the jump is not affected, but judges are required to reduce their GOE by -1 to -2.

Thank you! If I study the IJS for another 20 years I will begin to understand it's intricacies.

This is actually an excellent rule. If the edge is so straight up at the instant of take-off that it is not clear to the human eye which edge the skater is on, by a few degrees, then give the skater the benefit of the doubt on base value but acknowledge that the execution is not admirable because the skater did not hold and display a distinct outside edge throughout.

This raises the qiestion of how we would want AI to haldle the situation. For one thing, what is the precise instant of take-off, given that the blade is continuously changing and turning? If our eagle eye robot determines that the instant of take-off comes at precisely 1 munite and 5.329 seconds and that the blade is then at an angle of 1.874 degrees inside of vertical, what should we instruct the robot to do? What would be best for figure skating?

 

[4everchan]

Thank you! If I study the IJS for another 20 years I will begin to understand it's intricacies.

Just ask google when you don't know... takes 20 seconds

This raises the qiestion of how we would want AI to haldle the situation.

We do NOT need to ask AI to handle edge calls. This is not at all what I am suggesting

 

[Diana Delafield]

Thank you! If I study the IJS for another 20 years I will begin to understand it's intricacies.

This is actually an excellent rule. If the edge is so straight up at the instant of take-off that it is not clear to the human eye which edge the skater is on, by a few degrees, then give the skater the benefit of the doubt on base value but acknowledge that the execution is not admirable because the skater did not hold and display a distinct outside edge throughout.

This raises the qiestion of how we would want AI to haldle the situation. For one thing, what is the precise instant of take-off, given that the blade is continuously changing and turning? If our eagle eye robot determines that the instant of take-off comes at precisely 1 munite and 5.329 seconds and that the blade is then at an angle of 1.874 degrees inside of vertical, what should we instruct the robot to do? What would be best for figure skating?

My husband refused to watch any racing sport for which a computer determined the winner by .0000001 of a second. "So the runner with the biggest shoe size wins? Or the flattest curve at the front of his skis? Snort."

I was in my twenties before I realized I was probably not doing proper take-offs for toe jumps, when I saw a large teenage boy learning a triple Lutz puncture a pipe under the newly put-back-in ice at a summer school. I was probably really doing an edge jump and just tapping my toe pick on the ice as I jumped. Never got called for it by a judge. Would a computer have been able to measure how far into the ice my pick went, if at all? (Computer thinks busily: "Now let's see -- she's a small person, looks like she weighs less than 100 lbs, using MK Pro blades that have a shorter jumping pick than other brands, so.....um.......")

 

[Mathman]

Just ask google when you don't know... takes 20 seconds

Twenty seconds to Google it; twenty years to work the information into my conscious brain.

 

[Mathman]

My husband refused to watch any racing sport for which a computer determined the winner by .0000001 of a second. "So the runner with the biggest shoe size wins? Or the flattest curve at the front of his skis? Snort.

I remember the big kerfulle when they went to the new rule on the length of skis, that the ski can be no longer than 145% of the athlete's height. This was slammed as a political move to penalize Japanese skiers who were at that time making their move into prominence in the sport at the relative expense of the (generally taller) Norwegians.

 

[Weathergal]

My husband refused to watch any racing sport for which a computer determined the winner by .0000001 of a second. "So the runner with the biggest shoe size wins? Or the flattest curve at the front of his skis? Snort."

I remember the big kerfulle when they went to the new rule on the length of skis, that the ski can be no longer than 145% of the athlete's height. This was slammed as a political move to penalize Japanese skiers who were at that time making their move into prominence in the sport at the relative expense of the (generally taller) Norwegians.

I apologize as I'm veering OT a bit here.

I hope what I'm about to say doesn't get completely misconstrued because yes, skating is a sport, and we do want to have standards and apply them fairly, but... These two comments express a feeling that I've had with skating for a while now. When do things get too measured, too nit-picky, too "hair splitting" to the point where we've examined and overexamined so much we've taken the soul out of skating? I certainly don't have the answer to that, and I'm sure for each of us that's a different place. But I think there has to be some sort of balance in keeping the integrity of the sport without losing what makes it special. And I would argue that the the "tricks" whether they be a textbook Lutz, a sublime Ina Bauer, or a step sequence perfectly timed to the music, are only part of the equation. The best skates - and skaters - make you feel something, too. I hope skating considers that, whatever technology is used.

 

[gkelly]

The whole idea of the tech panel came about in response to the pairs judging controversy at the 2002 Olympics. The point was not that the technical specialists would necessarily be better qualified than the judges at spotting under-rotatons, etc., but rather that the judges' primary loyalties and obligations were to their own individual national feserations. The tech specialists would be recruited and appointed directly by the ISU and answerable only to the international organization. Therefore, in pronciple, they would be less susceptible to national bias and deal-,making.

Yes that too.

But primarily:

If the idea of the new system was to award point values for each individual element then someone needs to be responsible for telling the computer what each element is that was performed.

In the 6.0 system, judges awarded scores for two general areas of whole programs, not for individual elements. If some judges misinterpreted what they saw in a particular element, or different judges had different equally valid interpretations of an ambiguous element, it usually didn't make much difference if any to their overall scores to the whole program. And the skaters never learned what went into each judge's scores in any case.

In a system with points for elements, each element is identified in the protocol with its base value and with the judges' grades of execution.

So the idea was that the tech panel of three members would be responsible for identifying the elements (and an additional tech panel member, the data entry operator, enters the elements into the computer as called, without having any input into the decisions themselves).

Meanwhile the judges are responsible for awarding grades of execution and also for awarding scores for the program components (areas of evaluation for different characteristics of the program as a whole.

Could this work with judges responsible for identifying the elements as well as determining the quality of each element?
And the difficulty of the variations or features in spins, steps, lifts, etc.?

I.e., would judges be responsible for doing everything that the technical panel does now (which is quite a lot, maybe more work for non-jump elements than for jumps), in addition to evaluating the quality of the elements and of the various aspects of whole programs?

Well, what if the judges don't agree on what an element was? How does the computer know which element name/code and therefore which base value is correct?

The primary purpose of the tech panel is to identify the elements for the computer. That also involves identifying combinations, variations, features, repetitions, and errors that affect the base values.

I can go into a lot of detail about what they need to consider for each different kind of element.

Or you could just read the technical panel handbook (see https://www.isu.org/figure-skating/rules/sandp-handbooks-faq for Singles and Pairs handbooks -- ice dance is separate and even more complex).

Maybe this thread needs to be reframed (in a separate thread?) as 1) What does the technical panel do? and 2) Could any of those tasks be performed better by technology that currently exists or could be easily invented and implemented in the foreseeable future?

I don't think you can jump to discussing 2) until you already have a good understanding of 1).

 

[Mathman]

What we see with the so-called AI (scoff) at the moment is that the algorithm sucks at context and nuance. Not really suited to judging fs methinks.

I quite agree that we are using the term "Artificial Intelligence"in a fast and loose way here. What we should be talking about is using technology to make accurate measurements.

AI refers to mimicking human intelligence, especially by iterative algoritms that "improve themselves" with experience.

By the way, one area of figure skating where now-modern technology could have been employed is in judging figures. A computer can certainly calculate the degree of roundness of a circle and compare a theoretical picture of the ideal figure with the actual tracings on the ice. Smoothness, speed (and acceleration? -- but the third derivatives should ve 0 ) and depth of edges could also be measured. Posture, looking magnificent while you are doing your thing, ala Beatrix Schuba -- that might be more of a challenge.

I once had a student (in a math class) who claimed to be the champion free-hand circle drawer of Canada. He could draw a 99% perfect circle on a chalkboard in a split second.

 

[Anna K.]

Apparently this is off-topic, but I'll say this regarding that

jump height and distance traveled is interesting information

because this is where we objectively stand.
I love figure skating metrics and I wish we had more of it available in open sources because this is great material for journalists and fans to buzz about.

However, after having measurement tools we need a period of time to accumulate data. After that, we need to analyze this data to find out if there are correlations usable for improvement of the equipment that is currently used by judges and/or tech panel. This is the course of development. I love to come to technology threads in this forum and discuss it time by time. But the current discussion about AI replacing humans right tomorrow is ... No disrespect by I'm off.

 

[Mathman]

But primarily:

If the idea of the new system was to award point values for each individual element then someone needs to be responsible for telling the computer what each element is that was performed.

In the 6.0 system, judges awarded scores for two general areas of whole programs, not for individual elements...

I think we could say it this way: In the 6.0 system each judge was his own computer and data entry person. In the Battle of the Brians (1988 Olympics) Brian Boitano edged Brian Orser (5 judges to 4) because the judges could all "call" that Boitano did two riple Axels and Orser only did one. (Plus Orser had a slight bobble on his triple flip, even though Orser won the second mark 6 judges to 3.)

In the late 1990s the ladies were all upgrading thier solo jump in the short prgram from a triple toe to a triple flip. The judges were never in doubt as to what element was performed or that the triple flip was worth more towards the technical score.

Other aspects of what is now the technical panel's job, especially levels on spins and footwork, did not play much of a role, as far as I could tell. It was more important to look pretty on your layback spin (Angela Nikodinov) than to do a variety of changes of edges and positions. Same with spirals. (Brian Boitano did a gorgeous and lengthy spread eagle in 1988. I think that with every change in the scoring system something is lost,something is gained. The same will be true as we slowly move toward more precise measurements using technology.

 

[gkelly]

The judges were never in doubt as to what element was performed

Or if they were in doubt, or they missed one detail (perhaps an important one) that the rest of the judges saw, it usually didn't make an overall difference to their holistic score.

Occasionally it did. In which case, people might have asked themselves "What was that judge looking at?!!"

or that the triple flip was worth more towards the technical score.

But there was never any answer as to why one judge scored a specific skater much lower or higher than the others, relative to the rest of the field or the closest competitor(s).

Maybe one judge mistakenly saw a 3T or a 2F where everyone else saw a 3F. Or maybe they saw the 3F but saw it is underrotated and therefore awarded higher scores to a skater with an easier but cleaner solo jump.

Other aspects of what is now the technical panel's job, especially levels on spins and footwork, did not play much of a role, as far as I could tell. It was more important to look pretty on your layback spin (Angela Nikodinov) than to do a variety of changes of edges and positions. Same with spirals. (Brian Boitano did a gorgeous and lengthy spread eagle in 1988.

I think judges under 6.0 probably formed an overall impression of the quality and difficulty of the spins (and field moves, whether in spiral sequences required in women's short programs, or as transitions). Some judges might have been most impressed by beautiful positions, others by speed and centering, others by creativity and added difficulty (such as edge changes on camel spins, which is where they were most used before IJS).

But they didn't quantify those impressions. There might have been a thought of "weak spins" "average spins" "good spins" "excellent spins" or "beautiful spin positions" or "fast spins" or "creative spins" (in general, not necessarily distinguishing between the different spins in a program unless there was significant difference in quality) as a single data point in evaluating the program as a whole.

And in short programs, noticing whether the spins met the requirements under the rules and taking deductions if not.

I think that with every change in the scoring system something is lost,something is gained. The same will be true as we slowly move toward more precise measurements using technology.

Very true.

With 6.0, everything each judge evaluated regarding difficulty and quality of individual elements and the program as a whole, objective identification and subjective evaluation, all got summed up in two holistic scores, so we never knew exactly what the a judge had been focused on when making their decisions. The better observers (including fans and also the skaters themselves) understood all the rules and guidelines, the better we could guess what a judge might have been thinking, but there was no way to know for sure.

With IJS, the objective decisions about what the skater did are broken out element by element. The technical panel's job is to identify elements and features.

The judges' decisions are more subjective, more focused on how well the skater did what they did. Their job is to evaluate.

Where technology might come in would be to add additional kinds of information that can be measured by instrumentation rather than just estimated by the human eye. But that would not replace the basic tech panel function of identifying the elements in the first place.