Judging the "Old-Fashioned" Way

[silverpond]

I applaud the posters who provided detailed suggestions to improve the process and quality of judging at figure skating competitions. Well done!

In my opinion, there have been times that a competitive skater’s scores have defied reason. I’m not referring to the dreaded “national bias”, which unfortunately will probably always be part of the equation, and hopefully a very miniscule part of the equation.

My angst focuses on a basic knowledge of figure skating technique – how axels, lutzes, flips, loops, camel spins, sit spins, scratch spins, etc – are supposed to be performed. Let’s stick to the basics, please. Judges have to be extremely knowledgeable on the elements if there is to be any level of expectation that they will award appropriate scores.

If I’m correct about this, national and international skating judges are required to pass competency tests and remain current on the process of judging. No doubt, some of them opted to jump ship when the tried and true 6.0 system was abolished. Those who chose to stay, and all new judges, absolutely MUST be up to speed on this process. Granted, it can be difficult to watch every single element in a four-minute routine, especially when they are delivered in rapid succession; however, that’s the way it is, and you have to keep your eyes focused like a laser beam on the programs if you’re a judge.

Personally, I would like to keep this as simple as possible. Most of us are not Einsteins, and the CoP is, most likely, very confusing for many folks and drives them away from following the sport.

I prefer a “KISS” (keep it simple, skating) approach to scoring. Have a basic point assignment for the jumps – perhaps a 6 for a quad, a 5 for a triple/triple combination, a 4 for any other triple, a 3 for a double, a 4 for combination spins, a 3 for “single” spins, 4 for serpentine footwork, etc. Right next to the base element, have a point score of 1 to 5 for presentation. A completed quad would receive a 6, and if it was executed perfectly, it would receive a 5, for a total of 11. If a completed quad was executed poorly, it would receive a 1, for a total of 7.

I would award scores only on what was performed. If a skater falls on a lutz, that jump would not receive a score, plain and simple.

The panel of judges would consist of a dozen people representing the world on a global basis. Six would serve as “technical” judges and six would serve as “presentation” judges. They would be paired in six teams, with two judges on each team.

The overall point totals would be calculated, as quickly as possible.

Overall total scores:
90 and above = 6.0
85-89 = 5.9
80-84 = 5.8
75-79 = 5.7
70-74 = 5.6
65-69 = 5.5
60-64 = 5.4
And so on…..

Let the computer calculate the scores, and let the scores be “easy” for us to see. This could be a great partnership between the marvels of the computer and the 6.0 system.

Just my two cents, of course.

[gkelly]

My angst focuses on a basic knowledge of figure skating technique – how axels, lutzes, flips, loops, camel spins, sit spins, scratch spins, etc – are supposed to be performed. Let’s stick to the basics, please. Judges have to be extremely knowledgeable on the elements if there is to be any level of expectation that they will award appropriate scores.

If I’m correct about this, national and international skating judges are required to pass competency tests and remain current on the process of judging. No doubt, some of them opted to jump ship when the tried and true 6.0 system was abolished. Those who chose to stay, and all new judges, absolutely MUST be up to speed on this process.

So where does your angst come in? Do you see a problem with the current knowledge level of the judges? the knowledge level fo the fans? the disconnect between them?

I prefer a “KISS” (keep it simple, skating) approach to scoring. Have a basic point assignment for the jumps – perhaps a 6 for a quad, a 5 for a triple/triple combination, a 4 for any other triple, a 3 for a double, a 4 for combination spins, a 3 for “single” spins, 4 for serpentine footwork, etc. Right next to the base element, have a point score of 1 to 5 for presentation. A completed quad would receive a 6, and if it was executed perfectly, it would receive a 5, for a total of 11. If a completed quad was executed poorly, it would receive a 1, for a total of 7.

I would award scores only on what was performed. If a skater falls on a lutz, that jump would not receive a score, plain and simple.

This has potential for a simplified version of elements scoring. A lot of the specifics would still need to be worked out, after which it might not be quite so simple. But if the idea is that every judge is allowed to make his or her own determination of the value of each element, according to the rules, then there will often be major disagreements on how to score specific elements but as long as each judge is working from their best possible knowledge of elements and rules and visual perception of the elements, then getting different points of view is why we have judging panels in the first place instead relying on just one person.

The panel of judges would consist of a dozen people representing the world on a global basis.

For major international events. Obviously this wouldn't be possible or necessary at domestic events, and it might be cost prohibitive at smaller internationals.

Six would serve as “technical” judges and six would serve as “presentation” judges. They would be paired in six teams, with two judges on each team.

Are you suggesting that one judge in a team would contribute a technical score, one would contribute a presentation score, and the scores would be combined into a single ordinal to be calculated by the majority system?

I remember hearing a similar idea floated by some figure skating officials back around 1998, but I foresee a lot of paradoxical unintended consequences with that approach. If that is what you were thinking of, I'll explain what I think the problem could be. If not . . . whoops, never mind.

Also, if you want to go back to the majority system, remember that you need an odd number of judges, or judge pairs as the case may be. So 10 or 14 judges, but not 12.

Overall total scores:
90 and above = 6.0
85-89 = 5.9
80-84 = 5.8
75-79 = 5.7
70-74 = 5.6
65-69 = 5.5
60-64 = 5.4
And so on…..

I know these are just rough examples. If you do want to work out the details, remember that the numbers have tocover a wide enough range to accommodate the weakest international skaters, who may struggle with double jumps etc. (or lower than that if it's to be used at even lower levels domestically), and with the envelope-pushing quad jumpers, and every level in between. And they need to be flexible enough to distinguish between multiple skaters who are close to exactly the same skill level, so when they all do the same elements the results will come down to very fine points of technique and, yes, individual judges' preferences.

The cutoff for a maximum score (6.0 or whatever you call it) would have to be well out of reach of the best jumpers of the current generation, and even so some fluky jumper would probably come along and break through the ceiling long before the powers that be anticipate that the scores need to be recalibrated.

How were you thinking that judges would determine their presentation scores?
Would breakdowns of the elements and different aspects of presentation be available after the fact, as in the current protocols, or only ever one mark for tech merit and one for presentation as under 6.0?

Just my two cents, of course.

Interesting thoughts! Want to play with the details?

[Mathman]

So where does your angst come in? Do you see a problem with the current knowledge level of the judges? the knowledge level of the fans? the disconnect between them?

I think that the disconnect is a big problem. Nobody likes to be stupid. Nobody likes to be told he is stupid. Nobody likes to pay good money to have his ignorance rubbed in his face.

The audience does not like to be told (even when it is true), "I'm the expert, you're not, now sit down and shut up."

When you go to a baseball game, you might be entirely ignorant of the finer points of the game, but you can tell when someone knocks it out of the park.

In figure skating, someone can knock it out of the park, but you are not allowed to jump up and cheer because you have to wait until after the game is over and the experts come out and tell you, "Oh you silly person, that performance wasn't a home run. She turned in the other direction less than one-third of the time and she took an extra step between positions in her combination spin. The other team won. Go home and don't come back until you have read chapter 29(a) of the CoP rule book!"

[gkelly]

Off topic, but how often does a team with no home runs beat an opposing team who knocked out at least one?

I.e., the difference between landing a quad and doing enough other things right to win without one. One home run does not equal a whole performance/game. But the game is slower so there's plenty of time to cheer for each great play separately, whereas with skating the most intense cheering comes after the whole performance is finished.

And how often does a ball go out of the park but get called as a foul?
(=downgraded quad)

I don't think that the disconnect between judges' priorities and fans' priorities was really any less under 6.0 -- it just seemed that it was, so whenever fans disagreed with the judges, they could blame it on the judges getting it wrong due to politics etc. and boo with glee.

[gkelly]

For example

http://www.youtube.com/watch?v=ss8T7aGr9ZI
vs.
http://www.youtube.com/watch?v=k8xjFTEpo7I

Watching in the stands at the time I was bewildered by the results. I thought Weiss had done enough to win even if the quad wasn't perfect. It wasn't until I went home and watched the video a few times that I came to understand the majority of judges' point of view, and learned something in the process.

And that took being interested and willing to go to the trouble. But it was a lot harder then than now to gain the necessary knowledge to understand the results.

Watching on TV, you'd have to hope the commentators cared to explain. Sometimes they did and sometimes they fanned the flames.

[Mathman]

^ By knocking it out of the park I was thinking more of the second mark.

,,,so whenever fans disagreed with the judges, they could blame it on the judges getting it wrong due to politics etc. and boo with glee.

I think you are underestimating the satisfaction that this brings to sports fans.

Edited to add: Just look at how much fun we have on this board doing that very thig.

[janetfan]

Off topic, but how often does a team with no home runs beat an opposing team who knocked out at least one?

I.e., the difference between landing a quad and doing enough other things right to win without one. One home run does not equal a whole performance/game. But the game is slower so there's plenty of time to cheer for each great play separately, whereas with skating the most intense cheering comes after the whole performance is finished.

And how often does a ball go out of the park but get called as a foul?
(=downgraded quad)

I don't think that the disconnect between judges' priorities and fans' priorities was really any less under 6.0 -- it just seemed that it was, so whenever fans disagreed with the judges, they could blame it on the judges getting it wrong due to politics etc. and boo with glee.

Did you forget how simple it was back in the day when skating still mattered in USA?
It went something like this:

"The second set of marks should be much higher. She made a mistake on her 3L but the overall effect of her program was beautiful."

If fans saw the presentation marks were higher all was right in their world. They understood it with minimal help from the broadcasters.

If the presentation marks were not higher than the tech marks then the fans knew something felt wrong to Button and to most of them as well.

But they were involved emotionally and not detached - or worse, forced to find out what happened the next day on the internet.



Many long foul balls occur in baseball, typicaly several in every game with little or no controversy. Your analogy does not sound like that of a baseball fan.

I would also say a long fly ball caught on the warning track is an out in basebal. If it counted for a hit, maybe a single or double then it would be comparable to a failed element in skating that gets credit.

Partial credits might be fine for pageants or TV game shows but not for most sports.
To make it more confusing failed or downgraded elements that get partial credits or negative GOE can be subjective and not readily understandable to the vast majority of people that might be watching, whether in the arena or at home on TV.



Even if you argue some of the rules are good they do not necessarily make skating more fun or easier to understand.

[Mathman]

And that took being interested and willing to go to the trouble.

That is exactly the point I am struggling to make. Fans are "not interested enough to go to all that trouble."

Fans pay their money and hollar themselves hoarse. Our guy wins or he loses, the umpires are eagle-eyed or blind (depending on whether our guy wins or loses.) Then we go back home and save up our money until next time.

What fans don't want is a scoring system that says, "You have no right to an opinion about whether you liked the performance you just saw or not. If you think you liked the performance, you are just showing your ignorance We will tell whether you liked it or not after we consult the computer."

[dorispulaski]

Cheap Seats by Alabama
celebrates this kind of fandom.

This town ain't big, this town ain't small
It's a little of both they say
Our ball club may be minor league but at least it's Triple-A

"We sit below the Marlboro man, above the right field wall
We do the wave all by ourself
Hang ump, a blind man could've made that call"

....

"We don't worry about the pennant much,
we just like to see the boys hit it deep.
There's nothing like the view from the cheap seats."

^ By knocking it out of the park I was thinking more of the second mark.



I think you are underestimating the satisfaction that this brings to sports fans.

Edited to add: Just look at how much fun we have on this board doing that very thig.

[Mathman]

Cheap Seats by Alabama
celebrates this kind of fandom.

That's me.

The view that I most disagree with is the one that says, all we need to do is educate the fans about the CoP.

I have memorized the scale of values. I know that a triple flip is worth 5.3 pouints. I know that if a jump is short by more than 90 degrees but less than 180 then it is discounted thirty per cent. I even know how to compute 70% of 5.3.

But I do not enjoy a figure skating contest the more because of this knowledge.

(I do like local club shows and competitions, though. It may not be the bigs, but at least it's triple-A....well, 2A. )

[janetfan]

That's me.

The view that I most disagree with is the one that says, all we need to do is educate the fans about the CoP.

I have memorized the scale of values. I know that a triple flip is worth 5.3 pouints. I know that if a jump is short by more than 90 degrees but less than 180 then it is discounted thirty per cent. I even know how to compute 70% of 5.3.

But I do not enjoy a figure skating contest the more because of this knowledge.

(I do like local club shows and competitions, though. It may not be the bigs, but at least it's triple-A....well, 2A. )

Are we approaching a situation similar to Janet and Trixe with fans so disgruntled by the scoring they change the channel or don't watch at all?

How is it possible for the majority of fans to think 2010 Natls was scored well when Mr Skating USA, Scott Hamilton didn't get it either.

Does anybody here seriously think that there is any way possible to educate enough fans so they know more about skating than Scott?

That seems to be the only possible way for CoP skating to have much of an upswing in USA.

[Tonichelle]

Are we approaching a situation similar to Janet and Trixe with fans so disgruntled by the scoring they change the channel or don't watch at all?

How is it possible for the majority of fans to think 2010 Natls was scored well when Mr Skating USA, Scott Hamilton didn't get it either.

Does anybody here seriously think that there is any way possible to educate enough fans so they know more about skating than Scott?

That seems to be the only possible way for CoP skating to have much of an upswing in USA.

In 2002, Scott and Sandy both called for a new judging system. Now they're sitting back and griping that they got their wish. As much as I love and admire Scott, his act of "how did this happen" annoys even me as one of his fans (I will argue to the death with someone who says he's not worthy of his accolades, but he was far from "innocent" in this whole judging "mess")

[silverpond]

So where does your angst come in? Do you see a problem with the current knowledge level of the judges? the knowledge level fo the fans? the disconnect between them?

This has potential for a simplified version of elements scoring. A lot of the specifics would still need to be worked out, after which it might not be quite so simple. But if the idea is that every judge is allowed to make his or her own determination of the value of each element, according to the rules, then there will often be major disagreements on how to score specific elements but as long as each judge is working from their best possible knowledge of elements and rules and visual perception of the elements, then getting different points of view is why we have judging panels in the first place instead relying on just one person.

For major international events. Obviously this wouldn't be possible or necessary at domestic events, and it might be cost prohibitive at smaller internationals.

Are you suggesting that one judge in a team would contribute a technical score, one would contribute a presentation score, and the scores would be combined into a single ordinal to be calculated by the majority system?

I remember hearing a similar idea floated by some figure skating officials back around 1998, but I foresee a lot of paradoxical unintended consequences with that approach. If that is what you were thinking of, I'll explain what I think the problem could be. If not . . . whoops, never mind.

Also, if you want to go back to the majority system, remember that you need an odd number of judges, or judge pairs as the case may be. So 10 or 14 judges, but not 12.

I know these are just rough examples. If you do want to work out the details, remember that the numbers have tocover a wide enough range to accommodate the weakest international skaters, who may struggle with double jumps etc. (or lower than that if it's to be used at even lower levels domestically), and with the envelope-pushing quad jumpers, and every level in between. And they need to be flexible enough to distinguish between multiple skaters who are close to exactly the same skill level, so when they all do the same elements the results will come down to very fine points of technique and, yes, individual judges' preferences.

The cutoff for a maximum score (6.0 or whatever you call it) would have to be well out of reach of the best jumpers of the current generation, and even so some fluky jumper would probably come along and break through the ceiling long before the powers that be anticipate that the scores need to be recalibrated.

How were you thinking that judges would determine their presentation scores?
Would breakdowns of the elements and different aspects of presentation be available after the fact, as in the current protocols, or only ever one mark for tech merit and one for presentation as under 6.0?

Interesting thoughts! Want to play with the details?

Well, I was just sitting at my PC (as I am now) trying to come up with some suggestions to improve judging in the sense of making the whole process more understandable to the general audiences - those who might be more inclined to turn on the television or go online to watch competitions, and in so doing, raise the popularity of figure skating.

OK, my "team" judging idea is not going to work, most likely. We'll stick with a panel of judges of 9 members, each giving technical and presentation marks. I'd like to continue on with my suggestion that the overall scores be mapped to the decimal score. The highest scores would probably be awarded only to senior level skaters, based on the base levels of the technical elements. Perhaps skaters at the junior level might earn scores as high as 5.5 or 5.6, but no higher, and for them, those would be great scores.

Perhaps this just isn't possible. OK, but then let's have some education available for the general public. Occasionally, on the television broadcasts, Tracy Wilson has presented some informative replays of programs, with her analysis on what was performed well and what was not. This was always interesting, and I'm sure it was welcomed by the average sports fan who may not be at all familiar with figure skating.

I'm still pondering this issue.....really enjoy reading what everyone has to say!!