Is the standard of Judging deteriorating?

[pangtongfan]

Wallylutz, I read your well-reasoned analysis of Suzuki's program on the NHK thread, and I appreciate the effort you went to by separating out each jump into its own link so we could see what led into it. I'm willing to agree that Suzuki's marks were correct. But that doesn't mean that Mao's marks were also correct.

I definitely agree. The real question is not Suzuki's score which was quite correct IMO, and while she does seem very popular on this forum some seem to go overtop on her abilities on the scores she deserves I believe, but Mao's highly inflated score which is what led to Mao's suspect victory over Suzuki. Even Mao's SP score was a bit too high IMO although she did obviously skate well there. In the LP phase she probably should have been behind Nagasu and that young Chinese girl too (still deserved 2nd overall probably only due to her short program).

 

[lavender]

"Miserable marking from the German judge. Pathetic attempt to give Woetzel/Steuer the gold."

From the Nagano Olympics.

speaking of Olympics and such... scoring has always been suspect. My eyes were closed to this and now I wish I didn't know as much so I could totally enjoy this sport like I use to.

 

[sillylionlove]

While I wouldn't go as far as saying that the judging is biased I will say this....the new judging system stinks! All the programs are the same so the most point can be had. There is no artistry, no originality and none if the stuff that made programs in the 80's and 90's so memorable. It's probably why for the last few years I haven't been exciting for skating as I used to be.

 

[mskater93]

This judging system is intended to take away (some? most? all?) the subjectivity by assigning points to everything and adding them all up to get a score. The problem is, fans have long loved FS because often for the best programs, the whole is >>> than the sum of parts (Kwan, Boitano, Hamil, Lynn...). In this judging system the sum of parts >>> the whole and this is where fans (casual, die hard, and actual skaters) sometimes scratch their heads because they can't believe the overall results. There needs to somehow be a mark that takes into consideration (and is spelled out clearly) the holistic value of the program in a relative scale to the other programs skated at that particular event and assigns a ranking and therefore some kind of points to be added to the program to take into account the impact of the whole. I am having a hard time explaining this, but this seems to be where the judging is turning the fans off.

 

[gkelly]

There needs to somehow be a mark that takes into consideration (and is spelled out clearly) the holistic value of the program in a relative scale to the other programs skated at that particular event and assigns a ranking and therefore some kind of points to be added to the program to take into account the impact of the whole.

What would be a good way to define such a holistic mark?

Should it be an expanded version of what's now the Performance/Execution component, with a higher factor than the other components (except maybe Skating Skills)?

Or should P/E remain as is and the intangible holistic qualities get wrapped into some other new mark?

 

[mskater93]

I would wrap it separately. It would need to be some kind of "impact" mark that gets weighted higher.

 

[gkelly]

So what should be the criteria for that mark?

 

[mskater93]

It's one of those nebulous things, but it's a "how did the program come together as a whole". Unfortunately it's not something you can just throw criteria at and say "that's the way it should be done" at least, not off the top of my head. It's why, for example, people go back and watch Kwan's old program - there's an impact of how it's all woven together and the overall impression it makes on the individual, and the judge would calculate that AND would calculate how a fall (or mutliple falls) factored into the overall impression of the program - did it take the skater 5-6 seconds to get back into the program? Did it break the spell the program was creating? This is the piece that IJS is missing and that cause fans to scratch their head.

 

[gkelly]

OK.

If there were such a mark, I think we would see a place for judges to officially reward cleaner programs. But it would also be a place to reward programs for "creating a spell" (whether broken or not), which can be very subjective.

I suspect that fans or judges who just don't like a certain style of skating or the way a particular skater moves on the ice -- or who have personal beefs with a skater's behavior off ice -- would be more resistant to feeling that skater's spell than someone who has a warm feeling toward that skater to begin with. So it would amplify the effects of national bias and stylistic preference (neither of which can ever be completely eliminated as long as judges are human).

I also suspect that judges tend to watch programs that they're judging with a more analytical mindset, so they're not as receptive to having spells cast over them as most fans, or off-duty judges. On the other hand, they're more likely to be seduced by qualities like beautiful stroking technique that may be invisible to newer or otherwise less technically-attuned fans. So if we just added one more global mark to the current judges' responsibilities, I suspect that there would still be many occasions where fans would end up scratching their heads. Although it would cut down on the number of such occasions.

One way to guard against that, at least at big important events that can afford to bring in more officials, would be to have one set of technical judges score GOEs and Skating Skills and Transitions and another set of artistic impression judges score Performance/Execution, Choreography, Interpretation, and Overall Impression. That way the latter set could sit back and experience the program as a whole without having to focus on analyzing edges or counting steps or spin revolutions.

 

[drivingmissdaisy]

I also suspect that judges tend to watch programs that they're judging with a more analytical mindset, so they're not as receptive to having spells cast over them as most fans, or off-duty judges. On the other hand, they're more likely to be seduced by qualities like beautiful stroking technique that may be invisible to newer or otherwise less technically-attuned fans. So if we just added one more global mark to the current judges' responsibilities, I suspect that there would still be many occasions where fans would end up scratching their heads. Although it would cut down on the number of such occasions.

I agree, but I think a big part of the problem is that these programs are far too "busy" to be inspiring. Skaters are trying to cram in too many elements in a program these days. COP has killed some of my favorite things about skating such as spiral sequences and footwork that flies across the ice. As a result, I rarely have a spell cast over me.

 

[gkelly]

I guess that comes back to my question in the sine qua non thread -- is casting a spell over the audience something that should be required/expected for winning programs at a certain level, or at least given enough weight to overcome comparative technical weaknesses?

E.g., a frequent criticism of the 1994 Olympic ladies results is that Kerrigan was so superior technically that she should have won, but Baiul was so successful at casting a spell over a majority of judges, among others, that she got the gold more on the strength of her charisma than her skating ability.

 

[mskater93]

They tried splitting the judging before, but there was much complaining by the judges that there wasn't enough to do by only marking PCS. With the cost constraints of lowered fan support (not enough money in the ISU coffers), this is probably a prohibitive solution.

By adding in a holistic mark (which judges used to be able to mark in 6.0, so I don't think it would be too far fetched that they could do it), it adds a little of the 6.0 presentation mark feeling back into skating, which is what many fans seem to be feeling is missing - how did that program make me feel? When taken together, was that better or worse than the last program I saw? Was it 0.1 better or 0.2 better? Let's use the NHK ladies result as an example. Did you really feel Mao deserved to win overall when comparing both programs from both skaters (not considering scoring method)? If so, why? If not, why not? I felt taken in pieces, her score was mostly correct with the typical margin of error. I really didn't see much that I could quibble about However, as a whole, I felt the mistakes really ruined the program effect for me and took the energy out of that performance so that I didn't enjoy it. I felt Suzuki's score was mostly correct when taken in pieces as well. The difference is that I felt when you looked at the entirety of Suzuki's program, it made it better than numbers crunched from ijscalc and in some respects "underscored", but I can't necessarily say WHERE it was underscored outside of margin of error. Does this make sense? Is this what many of the rest of you are trying to say?

 

[Mrs. P]

They tried splitting the judging before, but there was much complaining by the judges that there wasn't enough to do by only marking PCS. With the cost constraints of lowered fan support (not enough money in the ISU coffers), this is probably a prohibitive solution.

By adding in a holistic mark (which judges used to be able to mark in 6.0, so I don't think it would be too far fetched that they could do it), it adds a little of the 6.0 presentation mark feeling back into skating, which is what many fans seem to be feeling is missing - how did that program make me feel? When taken together, was that better or worse than the last program I saw? Was it 0.1 better or 0.2 better? Let's use the NHK ladies result as an example. Did you really feel Mao deserved to win overall when comparing both programs from both skaters (not considering scoring method)? If so, why? If not, why not? I felt taken in pieces, her score was mostly correct with the typical margin of error. I really didn't see much that I could quibble about However, as a whole, I felt the mistakes really ruined the program effect for me and took the energy out of that performance so that I didn't enjoy it. I felt Suzuki's score was mostly correct when taken in pieces as well. The difference is that I felt when you looked at the entirety of Suzuki's program, it made it better than numbers crunched from ijscalc and in some respects "underscored", but I can't necessarily say WHERE it was underscored outside of margin of error. Does this make sense? Is this what many of the rest of you are trying to say?

Yes, I'm having a hard time with this concept as well. After watching both performances several times and reading the protocol sheets, I really don't have a problem with how Akiko was scored nor did I have a problem that Mao had higher PCS in some areas (namely skating skills), but something is lacking. I've concluded that my problem is how Mao was scored relatively to how a really good performance from her is typically scored. For example, if you look at her 2010 Worlds FS, which was amazing (like the program or not), she only got 62 to Yuna's 65 (who had some gaps in performance, namely during the later part of her FS when she made some errors). And now we have her getting 64 for a relatively bad performance while someone who did quite well -- Akiko -- got 62. It's like the tables have turned.

I do think some sort of overall impression mark would help --- that perhaps could measure the following things:
** Total packaging of the program -- did the jumps and spins go seamlessly with the choreography/transitions?
** Expression/Connection -- how well did the skater connect to the audience and/or express themselves during the WHOLE performance?
** Technical/Artistic balance -- did the skater balance the technical components of the program with good artistry (you could use the PCS as a guide).
** Disruptions -- Did mistakes interrupt the flow and packaging of the program?

These things would not be scored individually but would be considered in looking at the program as the whole

Maybe we can score it from 1-10 or something like that....and it would count for half of the overall score?

 

[wallylutz]

One way to guard against that, at least at big important events that can afford to bring in more officials, would be to have one set of technical judges score GOEs and Skating Skills and Transitions and another set of artistic impression judges score Performance/Execution, Choreography, Interpretation, and Overall Impression. That way the latter set could sit back and experience the program as a whole without having to focus on analyzing edges or counting steps or spin revolutions.

In an ideal world, what you are saying makes a lot of sense. And such proposal has been floated before but not viable for cost and administrative reasons.

If it were up to me, I'd eliminate PE, CH and IN from scoring entirely, leaving only two of the five components, which are SS and TR. I will however specify that SS must take into account the overall sureness, clean and execution of the elements. TES should be approximately 60% of the total marks with the components, 40% overall.

 

[Mrs. P]

In an ideal world, what you are saying makes a lot of sense. And such proposal has been floated before but not viable for cost and administrative reasons.

If it were up to me, I'd eliminate PE, CH and IN from scoring entirely, leaving only two of the five components, which are SS and TR. I will however specify that SS must take into account the overall sureness, clean and execution of the elements. TES should be approximately 60% of the total marks with the components, 40% overall.

That makes sense. I like that idea. I see SS + TR as more of a technical component rather than a performance one. Then you can integrate the bullets for PE/CH/IN in the overall impression.

Question is how would you score SS+TR with the other technical elements?

 

[wallylutz]

That makes sense. I like that idea. I see SS + TR as more of a technical component rather than a performance one. Then you can integrate the bullets for PE/CH/IN in the overall impression.

Question is how would you score SS+TR with the other technical elements?

SS and TR really represent everything technical except the elements themselves. When you are not judging the elements, you are really evaluating the skater's SS or TR. How does a skater go from element A to B? That's what SS and TR measures. Some skater like Plushenko just stroke to the next element but he may do so with very good speed, clean edges and in very secure manner, which will net him good SS but poor TR. Other skaters add quite a bit of in-between from A to B, therefore, increase the overall difficulty of the executed elements and the program itself. The nice thing about judging only SS and TR is that it really doesn't distract your attention away from the element. You know when you are not evaluating a specific element, you are automatically looking at SS and/or TR.

I have always been uncomfortable with PE, CH and IN, like many analytical minded people are. Put this way, in the NHK ladies FS thread, when someone said components were all subjective, I could easily rebut that by showing exactly where the transitions were lacking by including specific footage cut from the youtube clip. In that sense, it's very hard to dispute that when clearly the clips show no moves that could be construed as transitions - pretty much black or white. SS is harder to show via TV or internet but in the rink, it's very easy to show - the difference is quite noticeable, even between skaters who seem to be at the same level. When you placed Chan and Kozuka SBS, while Kozuka is very good, Chan's SS just makes Kozuka look like a minor league baseball player.

We would avoid a lot of controversies by getting rid of CH and IN as components for single skating. You want to watch a show? That's what the SOI tours (and their equivalents) and exhibitions are for. There is no shortage of opportunity for fans and general public alike to do so. In fact, I know people who only watch skating shows, but not the competitions. I am usually the reverse - I'll watch competitions but skip the shows.

 

[gkelly]

They tried splitting the judging before, but there was much complaining by the judges that there wasn't enough to do by only marking PCS.

Which is why I suggested that the GOE judges also judge Skating Skills and Transitions.

With the cost constraints of lowered fan support (not enough money in the ISU coffers), this is probably a prohibitive solution.

But the cost of extra judges would indeed be prohibitive. Especially for non-elite, non-televised events that need to play by the same rules as the big kids (JGPs or senior Bs or domestic qualifying competitions).

Let's use the NHK ladies result as an example. Did you really feel Mao deserved to win overall when comparing both programs from both skaters (not considering scoring method)?

I haven't had a chance to watch this year's NHK yet, so I personally can't give an opinion on this question. Unfortunately now it's too late and when I do watch I won't be able to form an honest opinion about who I think "should" have won based on my own impressions of the performances because they will be slanted by knowledge of what so many other people think.

With 6.0 judging, decisions about placements of each skater relative to the others, I feel that once I know the results the competition is spoiled for me in terms of trying to mock judge, even more so if I've already read discussions and controversies.

Live or without prior knowledge, that's not a problem for me. I loved to mock judge under 6.0 if I had the opportunity to watch events without knowing the results.

But that's just me.

I do think some sort of overall impression mark would help --- that perhaps could measure the following things:
** Total packaging of the program -- did the jumps and spins go seamlessly with the choreography/transitions?
** Expression/Connection -- how well did the skater connect to the audience and/or express themselves during the WHOLE performance?
** Technical/Artistic balance -- did the skater balance the technical components of the program with good artistry (you could use the PCS as a guide).
** Disruptions -- Did mistakes interrupt the flow and packaging of the program?

Your "total packaging" criterion would already be covered, along with several other criteria, under the Choreography component, and your "Expression/Connection" under some of the Performance/Execution criteria and almost all the Intepretation criteria.

So should those criteria be moved out of those components into the Overall Impression score but those components should still exist, with only the remaining criteria in each?
Should those aspects of the program be rewarded in several different scores?
Should those components be eliminated entirely and replaced by the single broader Overall Impression score?

These things would not be scored individually but would be considered in looking at the program as the whole

Maybe we can score it from 1-10 or something like that....and it would count for half of the overall score?

Well, maybe if you want to get rid of three of the components and replace them with one score that's worth the same amount as all the technical elements plus the other two components. But not if you keep all the components as is and then have another score that rewards some of the same global qualities again with a much higher factor, making the technical elements ultimately worth only about a quarter or at most a third of the total score.

That might work for pro competitions where entertaining the audience is the most important goal, but not for Olympic-style sport.

Here's my proposal:

Combine the Skating Skills and the technical aspects of the Transitions components into one mark

Combine the Choreography and Interpretation components into one mark, which would also include the choreographic use of transitions to link the program together

Redefine the Performance/Execution component to explicitly include the criteria you mention

If each of these three marks is factored the same, then P/E including consideration of technical/artistic balance, disruptions, etc., would be worth twice as much relative to the other components as it is now

If we're keeping the fall deductions, then maybe factor those so that falls cost more in the men's event than the women's (components are factored differently, why not falls?) and more for seniors than novices, and also add an additional 50% to the cost of each subsequent fall after the first.

Or, hell. Just let the technical panel and judges score according to technical criteria including all the technical aspects of the program components, more or less as is, and give medals based on those results.

Then also provide audiences with scoring gizmos to mark one (overall impression), two (technical impression and artistic impression), or more holistic scores, which are sure to be more subjective and more of a popularity contest than even the most biased but knowledgeable judges, add the audience scores to the judges' scores, and give a separate set of medals based on the combination of officials' and fans' marks, or even based on just the fans' marks alone. Often there will be one clear winner for both sets of medals, but when the overall artistic impression is at odds with the analytical breakdown, then there can be separate winners and more skaters as well as more fans can be pleased by the results.

 

[Bluebonnet]

There are lots of contests where all the tenths of points add up correctly -- but the wrong skater won.

Which means that it's difficult for the judges to control the outcomes. Isn't this way more objective?

My take is if each individual part seems correct, then accept the overall outcome.

 

[drivingmissdaisy]

My take is if each individual part seems correct, then accept the overall outcome.

I guess it's a question of whether you want the judges to decide who they like best (6.0) or you want them strictly judging elements (COP). Obviously judges can overmark/undermark in COP but I've found generally that the evaluation of programs is generally accurate, maybe a little more favorably for the top competitors.

 

[Bluebonnet]

Ah, at last, I finally watched Mao and Akiko's LPs.

There is no question that Mao is in a league of her own when it comes to performance. Now, I'm confused. What was this fuzz all about? Akiko won LP by a large margin 9.30. It all came from TES. Mao's PCS is wayyyy above Akiko's if you ask me. Even though Mao has doubled many jumps, they didn't interrupt her flow and performance. It was such an exquisite program and execution, deserved P/E higher than Akiko's.:yes: There were several parts in her program Akiko's performance detached from music. Besides, her music is a little ...like Kozuka's music - difficult to interpret. It's hard to believe that Mao has doubled so many jumps but still got 117.32. However, the numbers don't lie. Her TES was 11.73 less than Akiko's TES. I understand that Akiko has come a long way, overcame huge personal illness and obstacles. So she has won the hearts of the skating fans. But sorry, there is no reputation marks for that. Mao got deserving PCS. Akiko got deserving win on LP. The overall, unfortunately maybe, Mao won by 0.05. So again, live with it.

If you ask overall impression mark, I assume it is about performing art impact and artistic impressions, it will actually go to Mao.