Does anyone miss the "old" judging system? | Page 6 | Golden Skate

Does anyone miss the "old" judging system?

Just my opinion:

I hate program construction under COP. I miss having the fourth spin, think footwork takes up WAY too much time in most programs, and I also think that skaters need to stop doing half-assed 'choreographic sequences,' which frankly, suck. It has always driven me nuts when skaters do two spins in a row and now it seems to be the norm in many COP programs. COP doesn't seem to at all value positions (think of Elizaveta's HORRID sideways leaning spin or Yu-na's god awful spiral), but punishes to the hilt if a jump happens to be slightly short of rotation. At Skate Canada, it drove me crazy that Elizaveta's ugly triple axel attempt got ANY credit, but as the Eurosport guys said, "it was around, so she should get credit for it." Ridiculous judging flaw in the system, IMO.

Also sick of the love fest SP scoring that makes it possible for a skater to skate poorly in the LP, but still win. I know it's nothing new, but in 6.0 days, it was MUCH less likely with factored placements, which I preferred. Judges seem so desperate to make sure their preferred skater wins the SP that they go overboard with the second mark (since it is harder to figure out than a 5.9 or 6.0 mark), whether deserved or not. The second mark is used as a placeholder and doesn't actually work correctly. Julia Lipnitskaya is pretty much Russian Caroline Zhang these days, yet still scores strong PCs based on being Russian, and nothing else.

Ugh. Obviously, 6.0 is no longer a system we can use, but COP is most definitely not the answer either.

My idea: Look back on the old programs and see what everyone can agree was good, hell even great about them, then combine it with what we like about now (transitions are certainly better) and then create a new system. One that can quantify the technical but recognize the artistic for what it is. Also, skating skills should really be part of the technical mark, IMO, since it is more about technique than anything else.

Agree about footwork, footwork back in 6.0 days was WAY more interesting.

Now, disagree about everything else
It is a sport, Faster - Higher - Stronger. I prefer a system that rewards skaters for at least trying the hard jumps. And definiely rewards the skaters for rotating them. If one wants pretty, there are ice shows, now the competitions are supposed to be sport.
Most people i know don't watch FS basically because "its just a bunch of folks skating pretty" image of the sport. Hope it changes slowly.
 
Agree about footwork, footwork back in 6.0 days was WAY more interesting.

I would actually disagree about that as well.

Of course, "interesting" is largely in the eye of the beholder; I can't argue if you find a given step sequence interesting and I don't, or vice versa.

It might be an interesting experiment to choose a single year, maybe one major competition, each from 6.0 era and IJS era and compare a similar number of step sequences from programs at similar levels and see how many from each year we find interesting, or even give them scores purely on interest level.

I predict disagreement between posters as to which year produced more interesting step sequences on average. I won't predict which year would "win" with more posters preferring it.

Anyone want to choose two years? Which competitions? Which disciplines? How many programs from each?

I might nominate 1994 and 2014 Olympics for comparable events a round 20 years apart.


We couldn't compare men's short programs from 1989-2004 era with those from 2011-2016, because during the former period they had 2 step sequences in the SP and now they only have 1. But we could compare 1970s or 80s 6.0 with 2010s IJS (1 step sequence), or 1990s/early 2000s 6.0 with later 2000s IJS (2 step sequences).
 
My favorite step sequences were Yagudin's "winter" and Brown's "Riverdance" - and not in competition but pretty much anything Kurt Browning ever did.
 
About the only thing I miss about it is that there was both room for and a great appreciation of the beautifully simple move--an extended spread eagle or two, a shuffle step in dance. Those don't garner points under COP so they've been jettisoned. And I think that's a shame. Sometimes simplicity can take your breath away.

Otherwise, faults and all, I'm all for the now-decade-plus-old-so-we-can't-really-call-it-new-can-we scoring system.
 
It might be an interesting experiment to choose a single year, maybe one major competition, each from 6.0 era and IJS era and compare a similar number of step sequences from programs at similar levels and see how many from each year we find interesting, or even give them scores purely on interest level.

CoP has changed so much over the years in terms of the footwork sequences. The rules for footwork sequences right now are the worst they've ever been. There's very little room for creativity; the rules demand that everyone must include EVERY difficult turn in the book, in both directions, clumped together, and on different feet. It's awful.

Footwork sequences in 2006 and 2007 were great. The rules were far less restrictive back then (particularly in 2006) and CoP was still fresh, so people were still skating with a very 6.0-influenced mindset.
 
What kind of rubs me the wrong way, if we say that Michelle won 162.33 to 161.84, is that this gives the illusion that we have measured the two performances to greater accuracy than in fact we have.

No biggie.

Well, this is sort of the explanation of why I find it hard to be interested in competitive skating any more -- I didn't care for the 6.0 system, but I really HATE this aspect of the new system. In a close finish, I end up wondering if the winner is even the person the judging panel, as a whole, really thought should have won. Honestly, as the judges are going along, marking TES (let's see, is that a -1, a 0 or a +1 ...) and then, what, a half dozen or more presentation categories (hmmm, is that a 7.25 or a 7.3 for skating skills, and should it be a 7.75 or a 7.85 for whatever and whatever and whatever), they are not supposed to COMPARE to others, are they? Isn't the idea that they are simply quantifying THIS PARTICULAR performance, not comparing it to others (so there's no "leaving room" or anything antiquated like that -- simply hey, Skater A had a 7.25 level of skating skills in this program, and a 7.15 level of transitions, etc. -- NOT that Skater A had a 7.25 level of skating skills in comparison to Skater B, who had a 7.15 level of skating skills).

So, as the judges frantically try to quantify THIS PARTICULAR performance with a dozen or more scores for this element and that skill, do they really know what their SCORE for Skater A in relation to their SCORE for Skaters B and C? Are they trying to give first place to the skater they feel deserves first place (which was what 6.0 did -- you give the skater you felt should win your first-place ordinal, through whatever machinations you have to, and second-place ordinals to the skater you feel should finish second, etc.) or just trying to quantify one performance without any type of comparing with other performances? If you ARE trying to do that, then you don't really know who you are giving first place to -- you are independently looking at Skater A and giving what you think they deserve, and when you get to Skater K, you do the same thing, and deliberately try not to remember if you gave Skater G at 7.25 or Skater H a 7.15 or Skater B a 7.35 -- you just decide that this skater deserves, according to my thoughts, a 7.25 for skating skills. Or, are you comparing Skating K's skating skills to all the skaters before him, and trying to frantically figure out what you gave Skater B, who was a touch better than Skater K, and what you gave Skater G, who was not quite as good as Skater K, but WAS better than Skater H .... Honestly, what are these judges trying to do with all those marks? And when they've handed out all those marks, do we know -- heck, do THEY know -- if they gave first-place points to the skater they felt should be first? It appears that if it is done correctly, the judges wouldn't have any idea who they put in first or second or fifth or sixth -- they are simply touting up pluses and minuses for this or that and handing out random 7.25s or 8.35s or 6.55s or whatever. So when Michelle wins 184.15 to 183.95 over Lu, is that what the panel of judges MEANT to do, or is it just some random thing that doesn't reflect anything except it was close and this number got rounded up and that one got rounded down, so Michelle won. At least under 6.0, we KNOW that the majority of the panel of judges, say, in 1988, MEANT to put Brian Boitano first -- you can agree or disagree, but five of the judges knowingly put Boitano first and four knowingly put Orser second, so the gold went to the skater the judging panel overall decided should be the winner. Tiebreaks and such forsooth -- five judges said Boitano first, four said Orser first. Boitano win by a narrow concensus of the judging panel. Period).

But that is not the case with IJS. I may not trust the judges all that much, but that judging panel IS how figure skating is decided, and I'm not at all sure in close situations (or any situation, really, but especially the close ones) that the people we entrust to decide who wins and loses even know who THEY are putting in first as they hand out the scores. Do they KNOW that they've given Skater A .75 more points thant Skater B, or are they just handing out random marks and seeing where they fall? It drives me crazy to see this and feel like the people we've put in charge of deciding winners likely don't even know who they HAVE put as the winner unless they went into the event already deciding who they were going to put in first so they can knowingly award scores to that one skater that are higher than anything they have awarded previously or will award after. When someone wins a gold medal by .085, or misses the podium by .10, or a country loses a second or third skater for the following world championship because one of their skaters finished ninth rather than eighth by .005, it just drives me crazy. I KNOW that happened at times under 6.0, too, but at least that wasn't hyped as some mathematical formula that was going to give us a "real" winner, because we've quantified it and because this is a non-cheatable format. I just don't believe that any judge can go through all these scores and pluses and minuses for 20 or more skaters and really know, in real time, who they have in what position and by how much. And I think they SHOULD know that, because in one way or another, you really have to compare one performance to another in each different competition to come away with a winner who deserves to win THAT even.

And then there's the whole "non-cheatable" thing -- the format itself, especially combined with anonymous judging, is SO easy to cheat. So, what, you drop the high and low scores. So, even ONE judge wanting to try to skew things only has to give scores a LITTLE on the low side of what they think others will give -- they won't be out of line, just a little lower -- and that means any other low score automatically counts -- this judge doesn't even have to be in cahoots with anyone, just lowballs, a little, the skater he wants to hold down and highballs -- just a little - the skater(s) he wants to promote -- and even if his scores are both eliminated, it insures that all the lower scores (or higher scores) for the skaters will count, which, especially in a close event, can easily mean that the skater he wants will end up with .75 or .05 or whatever than the other skater, and end up finishing higher. It works the same way if you drop two highs and lows -- just highballing or lowballing the skaters you're trying to help or hurt makes sure that those skaters will have to count at least one higher or lower score, and you are "in line" enough that you never even get called on the carpet for it (if that even happens -- again, with the anonymous judging, who knows if the ISU is really policing this and trying to catch dishonest judging. I surely don't trust them to be doing it).

And we are all supposed to accept it because we can look at the protocols and say, well, they lost a few points here (don't know why Skater A's skating skills were 7.15 and Skater B's were 7.25, but since they were, and we have no idea which judge gave what, and hey, if this one judge gave a little lower than the others, well, it didn't count anyway), so that's why Skater A won.

And that's what I end up looking at when I watch competitions. All this quantifying and nitpicking and 7.25-ing are presented as a real, mathematical, non-cheatable way to arrive at the REAL outcome -- not some fake outcome like 6.0. But I'm seeing these people winning and losing and I have no confidence that the resulting order of finish had anything to do with what the judging panel saw or scored, because I don't know if even THEY know who they're putting in what position by how much. So my interest level in competitions has dropped a bunch -- I was all for a new judging system, but this one, to me, stinks, and leaves me looking for skating to watch that doesn't include the IJS.

I probably shouldn't post this because I know it's rambling and overlong, but I will, anyway. It's how I feel about this new system -- quite apart from the rather generic-looking programs we see and the backloading and the ugly looking spins (just because you CAN do it doesn't mean you SHOULD -- except, of course, if it will get you a level 4, and if it will, do it no matter how ugly it is) and the tick-it-off jumping passes, etc.
 
^^^This!:hap93:

It's partially why I don't care if I understand the new system. Objective? Measuring faster, higher, stronger, cause gosh darn it, figure skating is a sport? So we all can count revolutions and look at butt spins?

Sorry, I ain't buying that particular brand:dumb:.
 
mememe, the judges aren't supposed to care about who ends up in what order. The system is designed to look at each skater or team individually, reward for work completed individually, and the results will take care of themselves. Theoretically, it shouldn't matter anymore which flight a skater is in. This is not about ordinals, or meaning to place skaters in a particular order. The judges are not there to decide winners. They are there to award points, and in many cases, they provide valuable feedback to skaters and teams about areas to improve, what's working or not working, and how to get more points the next time after the competition.

How to award points in PCS is pretty clearly outlined in the technical manual, with clear indication what is a 7 vs 8 vs 9 etc. GOE is pretty clearly outlined as well. Judges who are students of the sport and subject matter experts know the expectations of the rules very well. The judges don't have to worry about the levels of elements. That's covered solely by the technical specialist and assistant, with help from referee when required. If a judge knows the rules inside and out, which international level judges should, it's not such an onerous or confusing task. The tech specialist calls the level, and the judge clicks a button how well they thought it was done. For PCS, there are only 5 criteria, which again have clear criteria the judge should know. There are even information cards a judge can use that provides a quick reminder if the judge needs it. PCS also is measured in 0.25 increments, so you wouldn't see 7.15. And a judge doesn't need to know math. The computer adds everything up.

The issue I see is that too many judges are thinking exactly as you are, about ordinals and placements. Skaters design programs around the same IJS criteria, but then the judges ignore all of that and simply rank as if it were still 6.0. A number of athletes I know refer to PCS as the "cheater marks". It's where favouritism and reputation and nationalism can still reign. With anonymous judging, there is no transparency or accountability. And judges often hang out together at an event, which I think just encourages more collusion and politicking. Talk about scores after an event is fully done - they can all learn from each other. But before and during the event, it should not happen. There are definitely ways to game the system. I don't think we will ever have 100% agreement on results. But at least the average fan can learn the rules themselves, and can call out situations where scores do not match the criteria of the system.

Overall, I think the "score what you see for the individual" is the right approach. And there has to be common criteria for measurement in order to ensure fairness in the sport. And as stated, the marks allow for better variation than ordinals in helping a skater understand how well they are doing, and what they need to work on specifically. Wherever possible, the system should be designed to minimize "cheater marks" - reputation, politicking, etc. Anonymous judging definitely needs to go, and I think using non-affiliated judges under a central body, where independent judges are chosen and not revealed until the competition starts, is a much better way. And to fix the "boring" programs, the point assignment criteria can be changed to match what is generally accepted practice. Want to fix back-loaded programs? Take away the reward for 2nd half jumps, and do a better job of rewarding a well balanced program in the choreography mark. And how about ensuring that programs actually match the prescribed rhythm in ice dance by rewarding or punishing the right vs wrong rhythm. I think PCS does need an overhaul, and could be designed to address some of the most common concerns about IJS.
 
You've both made valid points and I believe the correct answer is inbetween. It is important to have an objective scale that can be applied in isolation. We can look at a performance and ascertain it had amazing skating skills or good or average or bad, and find further gradations. At the same time, however, there must be a comparison too. The only reason we know something is amazing or good or average or bad is because of what we have experienced and learned. In a competition, we ARE trying to ascertain the placement someone deserves based upon what they put on the ice. Judges do need to compare the skaters to each other and give marks that properly reflect the differences.

The judging panel needs to be split up, though. We need tech judges and PCS judges. That is the only way to score a program with the most accuracy when put under the time constraints of competition. No single person in a few minutes time can be expected to review all of the technical elements AND the entire program as a whole, broken down into 5 components, and give the most fair assessment. The very fact that judges are currently clicking on a computer screen to score an element, WHILE THE PERFORMANCE IS HAPPENING, means that they are not paying 100% attention to the performance. Which leads me to my next concern:

The judges don't have to worry about the levels of elements. That's covered solely by the technical specialist and assistant, with help from referee when required. If a judge knows the rules inside and out, which international level judges should, it's not such an onerous or confusing task. The tech specialist calls the level, and the judge clicks a button how well they thought it was done.

This isn't how it should be done. A jump being called underrotated, for example, creates a significant gap in points and that gap is created by a call being made which could go either way. These things are contentious and not usually 100% certain. There needs to a singular set of technical judges who give the levels AND score the quality. If a jump is called as underrotated, the GOE shouldn't inherently suffer. There needs to be further gradation in the process of it all. In the case of a close call on a "clean" jump which questionable rotation, the judge should be thinking to themself "I will either score this as underrotated with +1 GOE or I will give benefit of the doubt on the rotation, but 0 GOE".

Too often the penalties are inaccurate, a jump gets -GOE AND the underrotation call, when it visibly looked okay. For example, Ashley Wagner's 3Flip+3Toe in the SP at 2015 Skate Canada. I feel that should have been scored as Triple Flip + underrotated Triple Toe, but it still deserved +1 GOE because the qualities of the actual jump combination as a whole were good.
 
So, as the judges frantically try to quantify THIS PARTICULAR performance with a dozen or more scores for this element and that skill, do they really know what their SCORE for Skater A in relation to their SCORE for Skaters B and C? Are they trying to give first place to the skater they feel deserves first place (which was what 6.0 did -- you give the skater you felt should win your first-place ordinal, through whatever machinations you have to, and second-place ordinals to the skater you feel should finish second, etc.) or just trying to quantify one performance without any type of comparing with other performances?

They're supposed to just be trying to quantify the various aspects of the performances (GOEs for each element and 5 separate PCS), comparing to standards for those kinds of elements and for those component criteria across the whole range of skating they've ever seen -- including but not exclusively during the event that's currently taking place.

They should already have a mental image of what a +2 spin or 8.00 choreography looks like, based on the written criteria and all the spins and all the programs they've ever seen.
So they can come up with pretty much the same scores for the same performance whether it's the first or last one of the day, or in the same general skill range of the rest of the field in this event or much better or much worse.

They might need to make some small calibrations as they go along. E.g., say a judge evaluates skater A's choreography and thinks "Low 8s. OK, 8.00 or 8.25?" If goes with the 8.25, then if skater B has choreography that the judge initially evaluates as low to mid 8s, for different reasons, he might think something like "B's choreo was at least as good as A's. I remember giving A 8.25; does B deserve the same score, or was B's choreography enough better to deserve a higher score?" and then give 8.5 if he decides it was better.

But the judge shouldn't be thinking "A landed a harder jump, but I want B to win so I'll give B higher component scores." When marking B's choreography, he should only be thinking about the choreography criteria.

Of course, we're not inside the judges' heads, so we don't know if they sometimes do intentionally manipulate the scores to try to achieve certain final results. But that's not the way the system is supposed to work, and there's no reason to believe that any given judge has used an incorrect thought process just because we don't agree with the score. The nature of most of the many criteria figure skating tries to measure is that there is room for disagreement as to what was better, and how much better, under any judging system. That's why you need a whole panel of judges to get a consensus and can't just rely on one person's opinions.

I first got obsessed with evaluating skating a little over 20 years ago. I liked to watch skating performances -- on TV or videotape, or live when I attended elite and local events -- and try to score along with the panel using my best knowledge of skating and scoring rules at the time. I learned more about skating as time went on and found it easier to understand the official results even if they didn't always match my own evaluations. When the scoring system changed, there was a lot more to learn.

I found with my own thought processes, when I was scoring along at home or in the stands under 6.0, I would often be distracted by thoughts about who I wanted to win rather than just concentrating on the skating. With IJS, I can just focus on evaluating each element according to the rules and doing my best to evaluate each component. I have no idea how the math is going to work out.

There are too many different numbers to keep track of, with different multiplying factors, and judges don't even get to know what levels are called. Even if skater A does harder/more successful jumps and skater B is clearly better in some or all of the component areas, it wouldn't be possible to figure out exactly how much higher to score B's PCS to make sure that the skater the judge prefers overall comes out ahead. Even if they tried to manipulate their own scores in an ordinal ranking approach, they would probably miscalibrate as often as not.

I feel I can do a more honest job as an armchair judge using IJS.

I think some judges feel the same way, and others preferred the more holistic 6.0 approach to evaluating performances as a whole. I have no idea how pressures to support one's home country skaters would factor into preferences of judges at the international level.
 
I have no idea how pressures to support one's home country skaters would factor into preferences of judges at the international level.

I would suppose that judges show their patriotism by giving their skater 8.75 instead of 8.25 in choreography, +1 instead of a 0 for her triple Salchow, etc.
 
I meant, I don't know whether international judges tend to prefer ordinals or IJS.

Oh, sorry. Ignore what I said.

Of course I don't have any inside information, but I doubt very much that international judges are saying, "I sure miss 6.0 where I could cheat to my heart's content," or, conversely, "Oh boy I really like the opportunities that the IJS provides for me to conceal national bias."
 
I doubt that either.

But some of them actually might be more interested in judging the skating and have a preferred method of doing so.

After all, they've got to come up with scores for all those other skaters in the events as well, regardless of what they do with their home team.
 
Do I miss the old scoring system? Nope. I was hooked the time Emanuel Sandhu beat Plushenko.

Speaking of Sandhu. Did he design his own costumes? He always looked good even when he skated badly. He had a rough skate at Worlds in 2001 but he always looked really good in what he was wearing.
 
Well, this has rambled on for 8 pages or so, and I guess it's time for me to chime in with the definitive answer::)

In terms of quality programs - Dance and Singles have improved under CoP. Pairs has been damaged - I much prefer to watch older pairs programs on YouTube than current competitive programs.

Things I like about CoP: It's become somewhat routine for skaters in 5th place, even lower, to come back and medal... even win competitions. CoP keeps skaters in the competition when everyone skates well and scores are tight. Very rarely would someone from the next-to-last group make it to the podium under 6.0. It seems like we see it a lot now.

Things I don't like - PCS is being used unfairly to prop up favored skaters who have poor technical outings.
 
Things I don't like - PCS is being used unfairly to prop up favored skaters who have poor technical outings.

This happens more on the men's side because their PCS is factored more heavily and the top men like Chan and Yuzu do everything better between the technical elements than the next tier of men. On the ladies side, the PCS spread between the top 5 (or so) women is pretty close, whereas Chan or Yuzu can be 15 points ahead of the 2nd place finisher on PCS when one of them is competing. As long as those two men rotate everything, that amount of points can offset quite a few errors.
 
Last edited:
drivingmissdaisy:

I actually don't disagree with Chan and Hanyu getting higher PCS. Quite a bit more if the situation merits.

However, if they or anyone else are getting 8+ on P/E, for example, when they're falling and stumbling and flailing about and losing the audience, then something is wrong.

While audience reaction is not a PCS element, it doesn't make sense that a lackluster performance that receives tepid applause should get through-the-roof performance scores.
 
mememe, the judges aren't supposed to care about who ends up in what order.

To me, this is the whole difference right here. In a judged sport, the judges DO care about the order of finish (after they see the performances, of course). In a measured sport, the measurer (stop watch, yardstick) does not care. This is the whole point to my unease with the CoP. It takes a judged sport and tries to pretend that it is not.

Bottom line -- it's OK with me either way. But I think that there is a pleasing symmetry in 6.0 between the sport itself and the method of judging. The IJS knocks this symmetry off kilter a little bit.

Tonto K said:
Things I like about CoP: It's become somewhat routine for skaters in 5th place, even lower, to come back and medal... even win competitions. CoP keeps skaters in the competition when everyone skates well and scores are tight. Very rarely would someone from the next-to-last group make it to the podium under 6.0. It seems like we see it a lot now.

I am undecided about whether this is a virtue or a liability of the IJS. The idea of factored placements treats the contest, to some extent, like "first you have the semi-finals, then the finals." To get to the finals you have to win (or at least, not to lose) the semis. If you lose in the the semis -- bye, bye cowboy, your ride is over. (Except that there is a back door if you at least show something in the semis.)

The IJS is more like the first half of a football game, and then the second half of the same game.

So which is better, the first half then the second half or the semi-finals then the finals?
 
Back
Top