Components from 5 to 3: has the change achieved its objectives?

[icewhite]

I wanted to bring AI and technology into play to make my point about numbers, but I also want to say that I think humans can do much better than they currently do and for that it wouldn't even need that extreme changes - more and more I am convinced that what we call politicking and corruption is often mostly a result of the psychological factors in judging not getting tackled but enhanced by the system, most of all the evaluation, because there is no real possibility to challenge the results even in case of obvious, blunt mistakes, and judges are supposed to judge like all the others if they want to be considered good judges while those who trust their eyes before the consensus have to justify their scoring.
For me that's now the biggest flaw of the system that needs to be adressed.

Edit: I'm convinced that would also be helpful in regards to PCS being so tied to the TES, but I am not against a higher limit of PCS scores.

Second are issues like prerotation that can be improved by a lot with a simple change of rules, which need to be stricter about correct technique.

Then it would be very beneficial if it was more clearly defined what some terms mean and if what the judges got told in meetings and education was all public and rules instead of some guidelines talked about in the backrooms.

 

[4everchan]

And why do you think they hire you to mark the work of students you do not know instead of allowing their exams to be marked by teachers who do know them?

Because of my credentials aka expertise. It's really the only reason. All my colleagues have similar experience and knowledge.

Why do you think most credible exams are those which use coded and anonymised data and do not allow examiners to know whose work they are assessing (or simply leave the work to computers if possible)? Why do you think it happens if not to ensure objective assessment for everyone?

The exams are not anonymised. When I say that I don't know the students nor their teachers, I am simply saying that I have no advantage to give a great mark or not to any student because i have no players in the game. Thank you for saying it's possible to ensure an objective assessment... that's exactly the point i have been making. it's possible to judge objectively, even art/figure skating

And if we think these measures to mark school exams normal and expectable, maybe we have a right to expect world and olympic champions of any sport to be decided in no less diligent and objective manner? And not by unpaid volunteers but by fully accountable professionals heavily supported by computers, or simply just supervising the work of AI?

I think I did bring up technology in here (not AI per se) ... so you don't need to convince me.

I believe that because of nationalistic and political agendas, it is much harder for figure skating judges to be absolutely objective about judging the sport... I am also saying though, that if some components may be more subjective, there is enough objective elements to have skaters marked properly.

 

[streams4dreams]

This is exactly what I am talking about
One fan, and don't think I am picking on you, because I do it too, complains about the judging, but then admits that they are not familiar with the PCS rules

Do you understand better what I am referring to in what an expert opinion can bring to the table, rather than anyone' s opinion ?

See, I do feel picked on, because I personally never do it. I don't post very often, so it is really easy to check that I never once complained about any score, TES, PCS, or total.

As for your second paragraph: I never discarded that the judges know the rules better than the public, I merely provided examples of how some of the PCS rules could be quantified by technology, and therefore applied uniformly to all skaters. There is a general feeling that this is currently not the case, and the reason we are having this discussion.

 

[4everchan]

See, I do feel picked on, because I personally never do it. I don't post very often, so it is really easy to check that I never once complained about any score, TES, PCS, or total.

As for your second paragraph: I never discarded that the judges know the rules better than the public, I merely provided examples of how some of the PCS rules could be quantified by technology, and therefore applied uniformly to all skaters. There is a general feeling that this is currently not the case, and the reason we are having this discussion.

Well, you shouldn't.

Technology : I think I was the one who brought it up in this specific thread.

We are having this discussion because some people believe that judging figure skating PCS is subjective. If that can be true for some parts of the PCS, there are still things that remain objectively measurable, not quantitatively but qualitatively . For me, it's clear and I brought up my field because if it is possible to do so in an artistic discipline, it is certainly possible to do so for a sport. If some disagree with that, then that's fine too... I explained my POV and that's the best I can do, really

 

[streams4dreams]

That, however, is not AI. It is using technology -- things like high-speed cameras and accurate protractors to measure things. Where does the "intelligence" come in? Even sorting, analyzing and correlating billions of bits pf data in a split second is not an "intelligemt" function.

That, I believe, is the big buggaboo in the whole AI field. No one has ever come up with a definition of "intelligent behavior" except, "Um, you know. like what humans do."

That's a really good question for what constitutes AI. In my field (astronomy), we often use machine learning as an interpolator. Even when we know how to calculate something explicitly, it is often too time-consuming. So instead, we train a neural network on a set of exact physical models to produce these calculations much faster when we need them on the fly.

So I was thinking that there may be a direct parallel in figure skating: sure, we could use high-speed cameras etc to measure the speeds, heights, angles and so on exactly, but that does come with a lot of extra resources. What I had in mind was in fact a neural network model (what is conventionally called AI) trained to evaluate these quantities from regular footage.

@gkelly brings a great point about how feasible all of that is, and I don't have a good idea. Looking from the outside, models like chatGPT seem extremely capable, which suggests that developing an AI model for figure skating may be possible. But it may not be profitable, unlike chatGPT and similar models, so who knows.

 

[kolyadafan2002]

In Japanese nationals they show the program maps sometime. Even providing viewers and judges with this map at ISU championships (including speed, ice coverage etc) can help them seperate the skaters.

 

[Magill]

Because of my credentials aka expertise. It's really the only reason. All my colleagues have similar experience and knowledge.

The exams are not anonymised. When I say that I don't know the students nor their teachers, I am simply saying that I have no advantage to give a great mark or not to any student because i have no players in the game. Thank you for saying it's possible to ensure an objective assessment... that's exactly the point i have been making. it's possible to judge objectively, even art/figure skating

I think I did bring up technology in here (not AI per se) ... so you don't need to convince me.

I believe that because of nationalistic and political agendas, it is much harder for figure skating judges to be absolutely objective about judging the sport... I am also saying though, that if some components may be more subjective, there is enough objective elements to have skaters marked properly.

Just to clarify, I understood they bring you in to assess students you do not know so that their names do not affect your assessment of their work but the work itself is not anonymised for you in your case. Yet, in many countries most important exams like school or university final or entry exams, and in some universities all of the written exams, are in fact fully anonymised. It is just taking the thing one step further in recognition of how important getting objective results is. The situation is similar with many competitions. Jury just reads / views and reviews the anonymised entries and appoints awards and only then these are matched with the contesters data to find who has won. That's pretty common procedure, isn't it?

 

[Magill]

In Japanese nationals they show the program maps sometime. Even providing viewers and judges with this map at ISU championships (including speed, ice coverage etc) can help them seperate the skaters.

This exactly. I do feel in this thread a sort of "all or nothing" attitude. If we cannot afford having a complete AI system to judge the whole competition and we are not able to develop it by tomorrow, then just forget it, let's stick to what we have and pretend any criticism comes just from haters and eternal naysayers or idealistic daydreamers.
There are many small things which can be introduced here and now and step by step to raise the objectivity of the whole thing.
And yes, judges accountability is one of them. And introducing technology to measure the most obviously measurable features is another one. As is the scoring transparency.
I wonder where this reluctancy to base scoring on the use of modern technology really comes from. Is it really about money, or is it rather generational, or related to the perceived loss of influence and power if you entrust scoring to machines which cannot be instructed informally backstage on the way to score?

 

[Magill]

So how much does it cost to develop fully accountable professionals heavily supported by computers, or to develop AI to do the work for them with professional human supervision?

Developing the heavy computer assistance and even moreso AI would be the work of many years. Once it works to everyone's satisfaction, would expenses still be high for implementation at any given competition, or would it also be affordable for lower profile events of little interest to paying audiences/sponsors?

Similarly, should the fully accountable professionals be available primarily for the high-profile events that bring income and prestige into the sport?

Or would it also be affordable to bring in well-paid, well-trained professional officials for every JGP and senior B and lesser international competition?

What about nationals of small federations, and domestic qualifying events of large federations? What about domestic/open nonqualifying events funded almost entirely by entry fees paid by the non-elite (some of whom may prove to be not-yet-elite) participants?

If we want the very best system possible for the Olympics and World Championships and other high-profile events, at what point do low-profile events have to use a completely different system? Or where would be the cutoff between using professional vs. volunteer officials?

Taking into account Yuzuru Hanyu was able to write an FS-focused thesis based on AI, and he did it a few years ago, I do not think it takes decades and costs unthinkable amounts of money to have it developed at least to some extent....

 

[Blades of Passion]

I am not sure what point you are trying to make... mine was simply in reaction to your comment that says falsely the Schindler's List soundtrack could be in any movie... and that's just not true.

I never said the Schindler's List soundtrack would be appropriate for any movie. You ignored everything I wrote and then blatantly misquoted me.

You were given a piece of music to try and decipher the exact story the music was created for, and completely failed. You also showed in your attempted Home Alone analysis that, indeed, the exact story of the movie absolutely can not be known simply by listening to the music. Obviously the music is "evocative" and has clear Christmas theme elements (in parts, not the entire score). Christmas, youth, playfulness, wistfulness, and urgency being detectable throughout various parts of the score still does not reveal the source material.

The point being, music is versatile and never by itself has any exact thing it must mean. Music only has a singular identity when people prescribe that identity to it. The Star Wars title theme is easily recognized by a billion people, but it doesn't have to mean "space battles and other sci-fi/fantasy happenings in an alternate universe." The main driving force behind that piece of music is it's rousing. It can be played at the opening of a presentation to hype people up, without people expecting to see a lightsaber duels. Because that's the context it's being used in, and the emotions within music have a life of their own. In a different context, like watching a movie that's not Star Wars and where that piece of music isn't being used ironically, people will be upset about another movie ripping off Star Wars...IF they are aware of that connection (yes, it's possible someone doesn't know about Star Wars and hasn't heard that music yet).

Ultimately, when listening to and creating music, the emotional resonances are what matter most. The Schindler's List music can be described as sad and haunting (but actually does have sections that aren't particularly depressing). It could have been created for a movie about a Jewish family struggling through some different kind of hardship than the holocaust. But it happened to be created for Schindler's List. Similar to how the Home Alone music would fit right in for some other Christmas misadventure tale. Music is itself not fully tangible, it's an attempt to express things in a different way than words or pictures allow, but it doesn't have to be deliberate either, sometimes musical ideas just pop in peoples' heads.

So, with pieces of music that have widely known associations, skaters can choose to try and portray a specific thing about the story the music is associated with. Or they can choose not to. Either way, it comes down to how well the chosen approach is executed, but the topic arose around the question of how much Julia Lipnitskaia's program had to do with Schindler's List. I argued she did more than just wear the famous red dress, that parts of her performance were very much a direct reflection of the story, but at the same time the majority of her program is not literal, and doesn't need to be. Tapping into the story with the exquisite expression Julia was able to achieve creates a heightened platform for the rest of the program, and the story doesn't need to be known in the first place to feel the emotion of Julia's expression. It should be recognized when skaters are doing something superficially as a crutch, and here it wasn't a crutch, it was pure.

Art cannot happen without technique.

Not true, or rather, art can exist without formal training. Some people are born amazing singers. Or can draw incredibly well. Or dance well. Or create stories. Or can just happen to take a stunningly beautiful picture.

 

[gkelly]

I do think technology could and eventually will be used to enhance the scoring of figure skating, especially the technical elements scores and also the more quantifiable aspects of the whole programs (program components).

I don't think we can expect/demand that it happen ASAP before workable technology and associated rule adjustments can be developed. At least, let's talk about what it would take to get from where we are now and where we would like to be. How could we get there from here?

And whether those changes will apply only at the highest elite levels where moderately serious money and public attention are involved (although nothing compared to big-time professional sports), or also at the pipeline from grassroots to elite competition, where skating is still very much and likely to remain an amateur sport.

Of course part of the issue regarding officials, as long as human officials are involved, is that internationally the competition structures are organized in terms of national federations. Are there international sports, with no higher levels of popularity and income than figure skating worldwide, in which athletes represent themselves rather than their federations/home nations? (Aside from the Olympics, which is nationalistic for all sports.) Is there a model for how skating could restructure itself? Or does any solution need to fit within the national representation structure?

There are many practical considerations that would have to be addressed to make a professionalized corps of officials feasible. Do we want to discuss what some of those issues might be, and how they might be addressed?

And then there is the question of the more subjective aspects of figure skating and how it is evaluated. The sport can train officials (or AI) even more precisely to evaluate specific criteria of composition and performance, that can be considered "objective" evaluations in the sense that @4everchan discusses. The ISU could also do more to train audiences to understand the details of those criteria.

But there will still be plenty of casual viewers who don't want to study the details, who just know which performances move them emotionally more than others, and that is where much of their enjoyment comes from. (With skating cleanly vs. making obvious mistakes often part of the narrative that resonates with audiences who couldn't know or care less about what an outside edge is.)

And much of what resonates with viewers is culturally based, or very personal to individuals' own histories and associations. If we remove them from the marking guidelines (e.g., if a completely culturally neutral AI could be developed -- unlikely given that human beings will be training it), then subjectivity of officials could be greatly reduced, but there would still be a disconnect between what speaks to casual audiences and what the sport defines as worthy of the highest scores.

I can easily imagine performances that both casual viewers and dedicated fans who pore over the detailed criteria and professional judges who have done the same would agree deserve top scores for Performance, but which the latter groups would also agree are relatively lacking in the Composition and Skating Skills criteria. The casual viewers don't care about those.

 

[skatesofgold]

Well, you shouldn't.

Technology : I think I was the one who brought it up in this specific thread.

We are having this discussion because some people believe that judging figure skating PCS is subjective. If that can be true for some parts of the PCS, there are still things that remain objectively measurable, not quantitatively but qualitatively . For me, it's clear and I brought up my field because if it is possible to do so in an artistic discipline, it is certainly possible to do so for a sport. If some disagree with that, then that's fine too... I explained my POV and that's the best I can do, really

This I do agree with. I think artistry is completely subjective, but PCS isn't just artistry. There are components of it that are objective.

I can tell you how much it sucks being uninformed sometimes. I watch drum corps, and I still have no idea what General Effect or Visual Effect really means or how those are judged, so I never understand why the Blue Devils win about 2 out of every 3 years because I don't enjoy them most of the time. They'll probably 3-peat sooner than later.

 

[icewhite]

Are there international sports, with no higher levels of popularity and income than figure skating worldwide, in which athletes represent themselves rather than their federations/home nations? (Aside from the Olympics, which is nationalistic for all sports.) Is there a model for how skating could restructure itself? Or does any solution need to fit within the national representation structure?

Depends on what you mean. Of course there are, actually the figure skating system in which not athletes, but federations have starting places which they then assign to certain skaters, is pretty rare. Usually athletes qualify for events and competitions personally. Their flag is shown etc., but it is them, not their federation that's the center.

 

[Mathman]

How could we get there from here

I don't think that figure skating by itself has the resources to take the lead in the development of useful te4chnology.

@streams4dreams mentions astronomy as a technology-intensive field with AI definitely in the picture. NASA has an annual budget of 22.6 billion dollars. I think the path for figure skating would be to partner with some sort of consortium of sports entities, or with people outside the sports world who have similar interests in making precission measurements of human movement. In the medical field the CT scan technology was funded by the Beatles record sales -- now that's real money!

There might possibly be opportunities to piggyback on general purpose technology that was created for some other purpose. There are high speed cameras that record motion up to 250,000 frames per second. The ISU could buy some, but it's not clear whether such precission wolud be of sufficient benefit to figure skating judging to be worth the cost.

I actually think that figure skating is moving (dlowly) in the right direction within its budget. The height and distance of a jump can be measured quite accurately and relatively cheaply, as coulld things like sureness and depth of edges. If they had possessed 21st century technology in the days of school figures, these could have been judged with great accuracy as to the roiundness of the circles that skaters were expected to trace, etc. Overseeing the development of cutting edge self-learning AI programs, though, I thin k that's a bit out of the ISU's lane.

As for PCS, I personally think that it would be cool as all get-out to investigate the question of how do design an artificial intelligence program that quantifies the exact extent to which a skater (or dancer) is able to match physical mivement with the requirements and expectations of the music -- that sort of thing. If someone will give me a billiuon dollars, I'll jump right on it.

 

[4everchan]

Just to clarify, I understood they bring you in to assess students you do not know so that their names do not affect your assessment of their work but the work itself is not anonymised for you in your case. Yet, in many countries most important exams like school or university final or entry exams, and in some universities all of the written exams, are in fact fully anonymised. It is just taking the thing one step further in recognition of how important getting objective results is. The situation is similar with many competitions. Jury just reads / views and reviews the anonymised entries and appoints awards and only then these are matched with the contesters data to find who has won. That's pretty common procedure, isn't it?

It's just called an external examiner. They bring someone who doesn't teach in the institution to evaluate the students of said institution. Very common practice.

 

[4everchan]

Long post

A lot of the things you mention are not against what I said, some are even exactly what I said, yet some are completely at the opposite..but I don't believe it is a debate necessary in this thread. However, I never said that music wasn't versatile, talk about misquoting ! I only mentioned that it was not true to say that Schindler's List could fit the bill for any movie or wasn't referring to the Holocaust when there are such strong references to Jewish music in it... and of course, the soundtrack of Home Alone will not give you the synopsis of the movie, line by line though it truly (thanks for acknowledging) gives what the movie is about... FIlm music provides sonorous context... and this is why music is used in movies and becomes a strong and important supporting role. This is also why film music is often used in figure skating as it can help the skater connect with it, and have convincing choreography which then helps connecting with the audience.

 

[gkelly]

@streams4dreams mentions astronomy as a technology-intensive field with AI definitely in the picture. NASA has an annual budget of 22.6 billion dollars. I think the path for figure skating would be to partner with some sort of consortium of sports entities, or with people outside the sports world who have similar interests in making precission measurements of human movement.

And also, in the case of figure skating, making precision measurements of the movements of the blades on the ice.

@streams4dreams mentions astronomy as a technology-intensive field with AI definitely in the picture. NASA has an annual budget of 22.6 billion dollars. I think the path for figure skating would be to partner with some sort of consortium of sports entities, or with people outside the sports world who have similar interests in making precission measurements of human movement.

And also, in the case of figure skating, making precision measurements of the movements of the blades on the ice.

Which would be at least as relevant to the Skating Skills component as the movements of the body.

That might actually be easier than measuring the body movements, or at least no harder. But it would require an extra layer of scrutiny that wouldn't be relevant to the partners you refer to.

As for PCS, I personally think that it would be cool as all get-out to investigate the question of how do design an artificial intelligence program that quantifies the exact extent to which a skater (or dancer) is able to match physical mivement with the requirements and expectations of the music -- that sort of thing. If someone will give me a billiuon dollars, I'll jump right on it.

Heh.

But phrased that way "exact extent to which a skater 9or dancer is able to match physical movement with the requirements and expectations of the music" implies that there is a correct way to match the movements to the music. Which may be true for compulsory dances, but could actually work against the encouragement of artistry in the sense of personal interpretation.

 

[Diana Delafield]

I don't think that figure skating by itself has the resources to take the lead in the development of useful te4chnology.

@streams4dreams mentions astronomy as a technology-intensive field with AI definitely in the picture. NASA has an annual budget of 22.6 billion dollars. I think the path for figure skating would be to partner with some sort of consortium of sports entities, or with people outside the sports world who have similar interests in making precission measurements of human movement. In the medical field the CT scan technology was funded by the Beatles record sales -- now that's real money!

There might possibly be opportunities to piggyback on general purpose technology that was created for some other purpose. There are high speed cameras that record motion up to 250,000 frames per second. The ISU could buy some, but it's not clear whether such precission wolud be of sufficient benefit to figure skating judging to be worth the cost.

I actually think that figure skating is moving (dlowly) in the right direction within its budget. The height and distance of a jump can be measured quite accurately and relatively cheaply, as coulld things like sureness and depth of edges. If they had possessed 21st century technology in the days of school figures, these could have been judged with great accuracy as to the roiundness of the circles that skaters were expected to trace, etc. Overseeing the development of cutting edge self-learning AI programs, though, I thin k that's a bit out of the ISU's lane.

As for PCS, I personally think that it would be cool as all get-out to investigate the question of how do design an artificial intelligence program that quantifies the exact extent to which a skater (or dancer) is able to match physical mivement with the requirements and expectations of the music -- that sort of thing. If someone will give me a billiuon dollars, I'll jump right on it.

Makes me think of the plot of "Ice Princess", with a computer analysis of skating movements jump-starting (pun intended) the story.

 

[Mathman]

And also, in the case of figure skating, making precision measurements of the movements of the blades on the ice.

And also, in the case of figure skating, making precision measurements of the movements of the blades on the ice.

Which would be at least as relevant to the Skating Skills component as the movements of the body.

That might actually be easier than measuring the body movements, or at least no harder. But it would require an extra layer of scrutiny that wouldn't be relevant to the partners you refer to.

I would suppose that a program written for astronomers to track meteor showers could be modified relatively inexpensively, may be, to track blades moving across a sheet of ice. That actually might be a do-able project that the ISU could undertake right now.

 

[Mathman]

Makes me think of the plot of "Ice Princess", with a computer analysis of skating movements jump-starting (pun intended) the story.

!!! Now I am inspired to sit down and write a screenplay for an ice skating science fiction movie. Robots are taught to be figure skating judges. In due course they get together and decide that they, the robots, know more about skating than puny-brained humans do. They develop their own scoring system which involves 100,000 program components to replace the curernt 3, and dismiss all the human judges and tech specialists.

I hope that when my work is evaluated for literary merit, posters to Golden Skate will be kind.