Components from 5 to 3: has the change achieved its objectives?

[gkelly]

Oh, no, that would mean we feed the AI with all the human biases, favouritism and errors which are the very reason we want AI to replace human judges in the first place!
FS is very different from chess and tennis as no one is judging the gracefulness with which a chess player grabs a pawn or a tennis player is running to the net. In the end of the day, their score is based on hard facts which can be misjudged only as a result of human error - and this is where AI comes in more and more. To better assess the hard facts.
With FS the question is not just about human errors in miscalculating rotations or failing to notice a wrong edge, UR etc., not even about differences in subjective opinions about what constitutes a great jump, great composition or interpretation and which skater does it better. It is also about favouritisms and intentional misjudgements which are different from built-in subjectivity, but, alas, have been happening all the time nevertheless.
Yes, intentional. As all the facts like corridor scoring of PCS, awarding PCS in relation to TES, taking into account what federation, which coach, what history, etc etc. this is all intentional misjudgements which are built into the system by now but which should never happen and which might be simply eliminated by AI provided no one would feed this compromised notions into its software

So how would you design an AI for judging skating, without building in your own biases?

 

[Mathman]

Oh, no, that would mean we feed the AI with all the human biases, favouritism and errors which are the very reason we want AI to replace human judges in the first place! ...

That is the ironic thing about AI in general. What do we mean by "intelligence?" In AI it means acting as much as possible the way humans do. An AI program that writes poetry, for instance, is regarded as successful if it writes poems that are like the poems that human poets write.

As for using AI to prevent skulguggery and cheating, the anti-AI movement is more afraid that the robots themselves will turn evil and eliminate the human race altogether.

 

[Mathman]

It wasn't always as simple as that. Nor was the word "artistic" anywhere in the rules as of the early 2000s.

The Evolution Of Technical Merit And Artistic Impression

Everything you want to know about the history of figure skating in one place! A fascinating archive of over a century of figure skating history.

skateguard1.blogspot.com

Thanks so much for this article. Great read. (By the way, I notice that the very first thing I have to do to sign up for this figure skating newsletter is to check the box, "I am not a robot.")

Myself, I think that they should bring back the criterion "Grace" for Valsing competitions.

My takeaway is that figure skating has undergone a century of struggle to try to define "artistry," or to find alternative words like "harmonious composition" that don't sound quite so nebulously defined but mean the same thing. We cannot seem to do any better than, "I don't know what "art" is, but I recognize and admire it when I see it."

 

[skatesofgold]

Art is completely subjective, imo. One person can love Picasso's artwork for example, but another person may hate his work. Same goes for music, literature, film, etc.

 

[4everchan]

Art is completely subjective, imo. One person can love Picasso's artwork for example, but another person may hate his work. Same goes for music, literature, film, etc.

What you express is true for people who are not trained artists. For instance, if I play a concert for the general population, some people like me or not... depending if they enjoyed their evening, liked my outfit or my hair LOL.. j/k... or my personality... or if I played their favourite composers etc... or even, if they were just in a good mood, receptive to listening to a classical music concert... However, they have no idea what I really did out there... If there are audience members who are also classically trained musicians, peers, colleagues, even mentors, they will be able to tell me exactly what they liked or disliked and why... because they understand the music, the technique, the intent, the sound quality and the performance aspects of the concert.... So their evaluation is not just subjective... it's very much based into knowledge = objective.

 

[Mathman]

Still, whatever the sophistication of the viewer/auditor, the bottom line in the entire "arts and entertainment" industry, of which sports is a part, is "satisfy the customer however uninformed and philistine he may be. Shakespeare did not write plays to produce enduring works of great literature He wrote to collect the pennies from the groundlings. When my toilet is backed up, I am very appreciative of the art of plumbing. Especially if the plumber can match the vintage gold fixtures in my wufe's bathroom. (Secret to a happy marriage: separate bathrooms.)

Seriously, though, "art" in sports is a tricky proposition. There are only a few sports -- diving for instance -- in which "artistgry" plays a direct role in the scoring. A great chess move can be a thing of beauty and a work of art that sends shivvers down your spine, but if it doesn't lead to a wimming checkmate, no style points for you, buddy. My favorite distance runner is Meb Keflezighi -- his striding technique was just gorgeous. Did it help him to run faster? Of course it did! They interviewed him after he won the 2014 Boston Marathon and asked him,"What thoughts were going through your mind as you felt the Kenyans ans Ethiopeans closing th gap in the last 1000 meterss?" His answer: I was thinking, "technique, technique, techniique, technique, technique, technique, technique." Now that's art. Or is it tech?

 

[Mathman]

...depending if they enjoyed their evening, liked my outfit or my hair LOL.. j/k...

Ice dancer Susie Wynne, as a beginning skating commentator, once famously said that she welcomes constructive criticism that hel[ps her do her new job better. "I hate your hair" -- no, that doesn't count as constructive.

 

[4everchan]

Still, whatever the sophistication of the viewer/auditor, the bottom ;line in the entire "arts and entertainment" industry, of which sports is a part, is "satisfy the customer however uninformed and philistine he may be. Shakespeare did not write plays to produce enduring works of great literatur.e He wrote to collect the pennies from the groundlings. When my toilet is backed up, I am very appreciative of the art of plumbing. Especially if the plumber can match the vintage gold fixtures in my wufe's bathroom. (Secret to a happy marriage: separate bathrooms.)

Seriously, though, "art" in sports is a tricky proposition. There are only a few sports -- diving for instance -- in which "artistgry" plays a direct role in the scoring. A great chess move can be a thing of beauty and a work of art that sends shivvers down your spine, but if it doesn't lead to a wimming checkmate, no style points for you, buddy. My favorite distance runner is Meb Keflezighi -- his striding technique was just gorgeous. Did it help him to run faster? Of course it did! They interviewed him after he won the 2014 Boston Marathon and asked him,"What thoughts were going through your mind as you felt the Kanyans ans Ethiopeans closing th gap in the last 1000 meterss?" His answer: I was thinking, "technique, technique, techniique, technique, technique, technique, technique. Now that's art. Or is it tech?

Art cannot happen without technique.

 

[el henry]

Still, whatever the sophistication of the viewer/auditor, the bottom ;line in the entire "arts and entertainment" industry, of which sports is a part, is "satisfy the customer however uninformed and philistine he may be. Shakespeare did not write plays to produce enduring works of great literatur.e He wrote to collect the pennies from the groundlings. When my toilet is backed up, I am very appreciative of the art of plumbing. Especially if the plumber can match the vintage gold fixtures in my wufe's bathroom. (Secret to a happy marriage: separate bathrooms.)

Seriously, though, "art" in sports is a tricky proposition. There are only a few sports -- diving for instance -- in which "artistgry" plays a direct role in the scoring. A great chess move can be a thing of beauty and a work of art that sends shivvers down your spine, but if it doesn't lead to a wimming checkmate, no style points for you, buddy. My favorite distance runner is Meb Keflezighi -- his striding technique was just gorgeous. Did it help him to run faster? Of course it did! They interviewed him after he won the 2014 Boston Marathon and asked him,"What thoughts were going through your mind as you felt the Kanyans ans Ethiopeans closing th gap in the last 1000 meterss?" His answer: I was thinking, "technique, technique, techniique, technique, technique, technique, technique. Now that's art. Or is it tech?

Meb! what an inspiration he was...

Not only did Shakespeare write for the masses, what we see as "art" today is not what those who created it saw. The Parthenon, when it was constructed, looked like a French bordello. Garish colors to satisfy the masses, much like our favorite skating costumes. None of those classic white marble structures, including statues, were white when constructed.

Yet, although classicists know that, people gazing at the Parthenon or the white marble statues do not. Does that make their opinions invalid? Should we only accept those based on extensive learning?

I don't know, I really am just asking.

 

[Mathman]

I don't know, I really am just asking.

To me, the main thing is that experts in any field should not be contemptuous or dismissive of people who are not as well informed as they.

If a work of art touches your soul and buoys your spirits, if its details strike you (paradoxically) as “inevibable yet beyond expectation” – then good for you and good for the artist.

 

[4everchan]

To me, the main thing is that experts in any field should not be contemptuous or dismissive of people who are not as well informed as they.

If a work of art touches your soul and buoys your spirits, if its details strike you (paradoxically) as “inevibable yet beyond expectation” – then good for you and good for the artist.

you know... the experts in a field, speaking especially from my field, are very rarely dismissive of people who are not as well informed... For a few reasons : 1) because I can appreciate someone wanting to know more or showing genuine interest in music ( my passion!) but not having the tools to hear it the way I do, and 2) because it takes an entire life of hard work to get there, which brings in a lot of humility and thus, empathy..... and I can tell you that I am very well aware that while I was devoting myself to my art, there was no way I could become an expert in everything...

However, what irks some experts, including myself- especially in this era of social media where everyone is now becoming suddenly an expert on everything , is when people who are not knowledgeable state their opinions as facts. Everyone can have their opinion on music or art... but my "opinion" is informed, I am speaking as an "expert", and thus, one would expect some respect... BUT, it goes the other way around way more often than you think... non-experts are the ones who are dismissive of those who do have extensive knowledge and expertise. I admit, I am even guilty of that myself when I criticize a figure skating judge, because truly, they know better than i do

 

[streams4dreams]

Everyone can have their opinion on music or art... but my "opinion" is informed, I am speaking as an "expert", and thus, one would expect some respect.

This makes it sound like the experts' opinion is more valid, and like everyone that is sufficiently informed would have the same opinion. This is manifestly not true, for example, judges' opinion at the Chopin competition in 1980 disagreed so strongly that two ended up resigning from jury -- both informed experts. While obviously opinions should be respected, I don't think there is one correct opinion on art, and therefore it is always subjective, regardless of how well-informed.

 

[4everchan]

This makes it sound like the experts' opinion is more valid, and like everyone that is sufficiently informed would have the same opinion. This is manifestly not true, for example, judges' opinion at the Chopin competition in 1980 disagreed so strongly that two ended up resigning from jury -- both informed experts. While obviously opinions should be respected, I don't think there is one correct opinion on art, and therefore it is always subjective, regardless of how well-informed.

You didn't get what I am saying in my long post... focusing on that one sentence is not helping Yes, you are correct, different experts can have different opinions, on both objective and subjective content. It does happen that judges disagree... same with figure skating.. It does happen even that scientists disagree even with full data available. I never said that all experts would agree to the same results

What I am saying is that
1) there are things that can be looked at objectively, even in art, even in program components.
2) some people feel it's all subjective because they do not have the expertise to look at it objectively: and this is what is very annoying to those who actually do have the expertise.


Regarding Pogorelic, he is a bit before my time but if I recall correctly, he was eliminated because his interpretations were out of the box. Piano competitions tend to be based in tradition. So yes, most of the experts agreed that he perhaps was going too far in terms of the cherished Chopin style. (I always say that it's not in those competitions that one will find the most interesting musicians, because of the emphasis on tradition... INTERESTING = Definitely subjective ) Some of the experts, most notably one of the best pianists ever, Marta Argerich, resigned from the jury because truly, Pogorelic was a sensational pianist. I do not blame her

The result is that the scandal gave Pogorelic much more exposure and success than the actual winner Dang Thai Son. Funny enough, his own student (DTS's) Bruce Liu (a pianist born in France who grew up in Canada) won this year's Chopin competition...

 

[streams4dreams]

So how would you design an AI for judging skating, without building in your own biases?

I had the exact same thought as @Magill that training AI on old competitions would be the exact wrong thing to do. I guess it would be interesting to see what happens, but my understanding is that AI is great at picking up correlations, and since the TES and PCS are so strongly related in archival scores, my bet would be that's the thing AI would learn.

I am personally not sure AI is the direction figure skating should move into, as there is some romance in being able to complain about the judges when we disagree with them However, I do like data, so I wonder what could be measured quantitatively. I am not so familiar with the PCS rules, but I imagine that when people value a skater's glide, it means that a skater can traverse long distances on few strokes, or that deep edges mean that the angle between the skates and the ice is smaller. These things might be measureable from camera footage, so that's how I'd design AI for judging skating -- not for all of PCS, but to quantify some aspects of it that can be quantified.

 

[4everchan]

as there is some romance in being able to complain about the judges when we disagree with them . I am not so familiar with the PCS rules,

This is exactly what I am talking about
One fan, and don't think I am picking on you, because I do it too, complains about the judging, but then admits that they are not familiar with the PCS rules

Do you understand better what I am referring to in what an expert opinion can bring to the table, rather than anyone' s opinion ?

 

[Mathman]

... I imagine that when people value a skater's glide, it means that a skater can traverse long distances on few strokes, or that deep edges mean that the angle between the skates and the ice is smaller. These things might be measureable from camera footage, so that's how I'd design AI for judging skating -- not for all of PCS, but to quantify some aspects of it that can be quantified.

That, however, is not AI. It is using technology -- things like high-speed cameras and accurate protractors to measure things. Where does the "intelligence" come in? Even sorting, analyzing and correlating billions of bits pf data in a split second is not an "intelligemt" function.

That, I believe, is the big buggaboo in the whole AI field. No one has ever come up with a definition of "intelligent behavior" except, "Um, you know. like what humans do."

 

[Magill]

So how would you design an AI for judging skating, without building in your own biases?

I had the exact same thought as @Magill that training AI on old competitions would be the exact wrong thing to do. I guess it would be interesting to see what happens, but my understanding is that AI is great at picking up correlations, and since the TES and PCS are so strongly related in archival scores, my bet would be that's the thing AI would learn.

I am personally not sure AI is the direction figure skating should move into, as there is some romance in being able to complain about the judges when we disagree with them However, I do like data, so I wonder what could be measured quantitatively. I am not so familiar with the PCS rules, but I imagine that when people value a skater's glide, it means that a skater can traverse long distances on few strokes, or that deep edges mean that the angle between the skates and the ice is smaller. These things might be measureable from camera footage, so that's how I'd design AI for judging skating -- not for all of PCS, but to quantify some aspects of it that can be quantified.

Exactly.
In general, I believe much more features are measurable and quantifiable than we tend to think, and much more of them can be measured, and more precisely, by technology than by human eye. Computers are also obviously waaay quicker than humans, so no more complaining that doing this or that is too time consuming for judges so, for example, they'd rather judge by what they see in real time than use slow motion, no matter how imperfect it is.
There is already some research done into it so anyone who is really seriously interested can look it up for themselves. Yuzuru Hanyu, for instance, wrote his thesis on the ways to use AI for training and judging jumps in FS, and I do not think he was the only one who did research on this
Another aspect is the human tendency to be biased towards one's favourites which in the end of the day always comes up in discussions about FS. And yes 4everchan many experts, by no means all of them but many, use their expert knowledge and position to - knowingly or unknowingly - justify their otherwise biased judgments. It happens in FS just like in any other field, be it sports, art, science, or politics. Again, it is human. Being an expert does not make you any more unbiased or objective by default. It might make your opinion better rooted in facts, but it might also just allow you better choose facts to justify your biased opinion, or provide you with better sounding arguments, especially when discussing with those less expert than you are. This is a well known phenomenon and I do not see why we should pretend FS judging is free of it.
And last but not least. Unlike human judges, AI - or call it technology, I will not argue with that - would be able to actually score the "skate" itself regardless of who the skater is which is not possible for human judges. No matter how much they try, they simply KNOW who the skater is and it obviously affects not just their judgement but also their very perception. With software, if we do not feed it with skater's ID data such as federation, nationality, name of the skater, their past placements, their coach, their past scores etc, but instead give them anonymised data to score skater 1,2,3, based on measurements done on the spot on this very day, AI - or call it technology - will simply ignore all these other factors which in fact should not be taken into account, but, alas, we all know they are. Honestly, I believe this alone could revolutionize the results of many competitions, you'd be surprised

 

[4everchan]

well.. in competition, some of the issues come from politics... and in that case, anything goes.

But see, I do have to mark exams... piano exams... not my students.. don't know them... don't know who their teachers are... i am just hired to do it.. i sometimes am even provided with a scoresheet... and there is no ranking here.. no prize money, no competition.. just a year exam grade ... i believe that it's possible to judge art with the most objective criteria.

I am convinced that it is possible for judges in figure skating to do so (evaluate with the most minimal bias) as well... is it commonly happening? will it ever happen for all???

that, I don't know. The one good thing about AI is that it may remove that nationalistic aspect of judging. But until then, I think that this thread has really evolve into surprising topics and it's been fun... though now, it feels like work to me, because it's too close to home... and i am on holidays... so hasta la vista !

 

[Magill]

well.. in competition, some of the issues come from politics... and in that case, anything goes.

But see, I do have to mark exams... piano exams... not my students.. don't know them... don't know who their teachers are... i am just hired to do it.. i sometimes am even provided with a scoresheet... and there is no ranking here.. no prize money, no competition.. just a year exam grade ... i believe that it's possible to judge art with the most objective criteria.

I am convinced that it is possible for judges in figure skating to do so (evaluate with the most minimal bias) as well... is it commonly happening? will it ever happen for all???

that, I don't know. The one good thing about AI is that it may remove that nationalistic aspect of judging. But until then, I think that this thread has really evolve into surprising topics and it's been fun... though now, it feels like work to me, because it's too close to home... and i am on holidays... so hasta la vista !

And why do you think they hire you to mark the work of students you do not know instead of allowing their exams to be marked by teachers who do know them? Why do you think most credible exams are those which use coded and anonymised data and do not allow examiners to know whose work they are assessing (or simply leave the work to computers if possible)? Why do you think it happens if not to ensure objective assessment for everyone?
And if we think these measures to mark school exams normal and expectable, maybe we have a right to expect world and olympic champions of any sport to be decided in no less diligent and objective manner? And not by unpaid volunteers but by fully accountable professionals heavily supported by computers, or simply just supervising the work of AI?

 

[gkelly]

And if we think these measures to mark school exams normal and expectable, maybe we have a right to expect world and olympic champions of any sport to be decided in no less diligent and objective manner? And not by unpaid volunteers but by fully accountable professionals heavily supported by computers, or simply just supervising the work of AI?

So how much does it cost to develop fully accountable professionals heavily supported by computers, or to develop AI to do the work for them with professional human supervision?

Developing the heavy computer assistance and even moreso AI would be the work of many years. Once it works to everyone's satisfaction, would expenses still be high for implementation at any given competition, or would it also be affordable for lower profile events of little interest to paying audiences/sponsors?

Similarly, should the fully accountable professionals be available primarily for the high-profile events that bring income and prestige into the sport?

Or would it also be affordable to bring in well-paid, well-trained professional officials for every JGP and senior B and lesser international competition?

What about nationals of small federations, and domestic qualifying events of large federations? What about domestic/open nonqualifying events funded almost entirely by entry fees paid by the non-elite (some of whom may prove to be not-yet-elite) participants?

If we want the very best system possible for the Olympics and World Championships and other high-profile events, at what point do low-profile events have to use a completely different system? Or where would be the cutoff between using professional vs. volunteer officials?