How do federations influence the results?

[sammyott]

It would be possible at small competitions like the Grand Prix. Not at Worlds where every country is allowed an entry (presuming they have skaters who have earned the minimum scores).

And for important events with the best skaters, you also want the best judges. Those from countries with smaller skating programs may be more impartial, but they're also likely to be less experienced.

I guess that makes sense. But even with a judge from a country with their best skater down in the 30s, they will likely not be biased of the top skaters. I feel like the ISU should have a system where if there is clear favoritism (like the Italian judge of ice dance at Skate Canada), they should be replaced or suspended for not complying with the wants or ideals of good PCS of the ISU. I'm just spit balling here.

 

[topaz emerald]

This might have been said already, but I don't understand why they don't just have judges from non-competing countries... With ice dance at Skate Canada, the only judges that seemed fair were those who didn't have a team from their country skating (i.e. South Korea and the Netherlands).

Is there a logical reason for why this just isn't a requirement?

I have thought about this too....I think it will make it less corrupt.

 

[moriel]

Anonymous judging is over.....the panel was announced for each discipline and the judges #s are public as of now.

But as I realized in the discussion on the Opinion of SC thread last night, I believe that not very many of us understand how to look at the Judges scores in the TES and run that through the SOV (statement of value) chart published by the ISU to get the number to multiply the base value or BV by to get a single judge's score for each individual element.

Just in case, spammed in here: http://www.goldenskate.com/forum/sh...Skate-Canada&p=1530368&viewfull=1#post1530368
http://www.goldenskate.com/forum/sh...Skate-Canada&p=1530410&viewfull=1#post1530410

Well, for Ice dance, what i see without doubt:

1. National bias. EVERY SINGLE judge that had a team on that competition gave their team or teams higher than avergae GOEs and higher than average PCs. Every single one.
2. Strong evidence of voting agains close competitors of their national teams. For example, the italian judge really underscored G/P in FD (i suppose in a try to put C/L in 3rd) - you can even see it in the protocols without any statistical methods or whatever - for example, for each PCs category, Judge 8 gave the lowest score out of 9, including a whopping 7.75 for Performance ^^ Other judges did it too

Looking for conspiracy, i´ll try analysing how the judges who had no skaters competing for their countries in a certain discipline scored other skaters, maybe there is something interesting.

One thing is clear though - as there is national bias, canada had advantage there, because a single judge could give higher than average score to whole 3 teams, and not just 1 as RUS and CHN.

 

[moriel]

I guess that makes sense. But even with a judge from a country with their best skater down in the 30s, they will likely not be biased of the top skaters. I feel like the ISU should have a system where if there is clear favoritism (like the Italian judge of ice dance at Skate Canada), they should be replaced or suspended for not complying with the wants or ideals of good PCS of the ISU. I'm just spit balling here.

On the other hand, the JPN judge seemed to support C/B in dance ^^ Not sure if having judges from non-competing countries would help.
Also, while doable in GP, in Worlds it would be tricky - as most skating countries will be represented (and yeah, the countries with top skaters usually have the highest avaliability of judges).

See Brazil, they have 1 lady competing, but i´m not even sure they have judges who are qualified enough for international events...

 

[moon]

It isn't as complicated as that. From the results chart click on the judging panel and note which number each judge is. Then look at the protocol sheets. In the column directly below Judge 1 are the marks they will have entered. Next to each element it shows the GOE they applied to that element and you can compare it with the GOEs the other judges gave for the same element by looking along the same row. Then in the PCS section you can see the score Judge 1 gave for each component. There is no need for multiplying etc. You just need to look at the differences in scoring between the judges. If there is more than a 2 point spread between the judges marks they are supposed to be called into question. However, less than that is left alone. What you are looking for is obvious things like a judge giving someone 2 on an element and another one given them -1, or vastly different PCS scores. Some knowledge of what makes an element well executed and what to look for in quality PCS is definitely beneficial but not necessary.

Anonymous judging is over.....the panel was announced for each discipline and the judges #s are public as of now.

But as I realized in the discussion on the Opinion of SC thread last night, I believe that not very many of us understand how to look at the Judges scores in the TES and run that through the SOV (statement of value) chart published by the ISU to get the number to multiply the base value or BV by to get a single judge's score for each individual element.

 

[moriel]

It isn't as complicated as that. From the results chart click on the judging panel and note which number each judge is. Then look at the protocol sheets. In the column directly below Judge 1 are the marks they will have entered. Next to each element it shows the GOE they applied to that element and you can compare it with the GOEs the other judges gave for the same element by looking along the same row. Then in the PCS section you can see the score Judge 1 gave for each component. There is no need for multiplying etc. You just need to look at the differences in scoring between the judges. If there is more than a 2 point spread between the judges marks they are supposed to be called into question. However, less than that is left alone. What you are looking for is obvious things like a judge giving someone 2 on an element and another one given them -1, or vastly different PCS scores. Some knowledge of what makes an element well executed and what to look for in quality PCS is definitely beneficial but not necessary.

yup, this is what i did...
More than enough to see that they all support the skaters from their countries, for example.

Although the difference is not thaaaaat big: what usually happens is lets say most judges give a skater 2, and 1 judge gives a skater 3 or 1.
The difference is usually subtle but consistent - for example, a judge gives slightely lower GOE on pretty much element, or lower the PCs by 0.25 or 0.5 or so on.

What makes it visible is that it keeps happening again and again. A fair judge will either score in a similar way all elements for all skaters (so if a judge gives very low GOE, that doesnt automatically means he is biased - maybe he is just critical and gives low GOE to everybody). But when a judge gives slightely higher scores to very same team and in all programs, for example...

They got some sort of control too, so they must be reasonably close to other judge´s marks.

 

[TGee]

It isn't as complicated as that. From the results chart click on the judging panel and note which number each judge is. Then look at the protocol sheets. In the column directly below Judge 1 are the marks they will have entered. Next to each element it shows the GOE they applied to that element and you can compare it with the GOEs the other judges gave for the same element by looking along the same row. Then in the PCS section you can see the score Judge 1 gave for each component. There is no need for multiplying etc. You just need to look at the differences in scoring between the judges. If there is more than a 2 point spread between the judges marks they are supposed to be called into question. However, less than that is left alone. What you are looking for is obvious things like a judge giving someone 2 on an element and another one given them -1, or vastly different PCS scores. Some knowledge of what makes an element well executed and what to look for in quality PCS is definitely beneficial but not necessary.

I'm not sure that I can agree with you on his one. I don't think that one can infer bias or gamesmanship with your approach to finding patterns.

If a judge were really determined to game in a sophisticated way as moriel suggests, then it would make sense to adopt a tactic of having the greatest bias on the technical elements on which it would have the greatest impact. If one is going to look for intentional bias and gaming, one needs to look at both "naive" always support my skater rules (which are likely to get the score dropped or the judge called in for a discussion of why his/her scores are out of range) or work out what would more sophisticated dominant strategies that would not be likely to be caught.....

Sustainable strategies in gaming are likely to be more subtle and difficult to detect. So I wouldn't expect moriel's approach to reveal them or to be able to distinguish between deliberate gaming, different ways of seeing or unconscious bias.

I'll also note in passing that there there is a large academic literature in game theory that indicates that reputation effects, long run equilibria vs short run etc really matter.

On the other hand, my view and that of others is that we can expect national differences in concepts of excellence and judging priorities that the ISU consensus process and seminars may not be sufficiently evening out....especially for large federations that aren't obliged to have their skaters and judges get out of country as much at the lower competitive levels.

If we are looking for genuine differences in perception of what is excellence, we could expect to see a pattern of the kind you and moriel describe.

But even with that, we might see it consistently on jumps vs spins, or SS or TR for one country or PER and CO for another. That is, if certain elements or components are more strictly judged generally in a country or more rewarded by judges in that country, we would expect to see the skaters and coaches from that country putting priority on greater skill in those areas....

 

[Osmond4gold]

Nobody should be discounted, but actually that's usually what happens unless there is specific judging favor in place for a skater. Many people do not feel Osmond deserved 2nd place at Skate Canada. If she was, say, a Chinese skater who gave those exact performances, she likely would not have been in 2nd at the event. Her LP was boring and had big mistakes. Miyahara skated very well and, despite the various imperfections of her technique/program, it was a more complete effort both technically and as a performance.

topic: but thanks BoP for your comments. As others have pointed out on the forum, Osmond LP score was higher than Miyahara because Miyahara received 0 for her step sequence as it did not fulfill the requirements and had other under rotation issues resulting in Osmond beating Miyahara by 6 points on TES. I will add that I have asked those in attendance and they have commented on the forum that Osmond's speed was much faster than Miyahara's.

That said, I believe that the judges got it right with the women's event.

 

[YesWay]

I will add that I have asked those in attendance and they have commented on the forum that Osmond's speed was much faster than Miyahara's

Was this a speed skating competition?!

Seriously though - while speed makes everything more difficult, and may contribute to GoE's and Skating Skills (even though "speed" is not actually listed among the component criteria)... we can't just cherry-pick a few individual aspects of the skaters' performances, and use those alone to justify (or refute) how the scoring went.

(I say this, only because you seem to suggest that Osmond scored higher than Miyahara, simply because she skated faster. I'm just saying it takes more than that...)

 

[Osmond4gold]

Was this a speed skating competition?!

Seriously though - while speed makes everything more difficult, and may contribute to GoE's and Skating Skills (even though "speed" is not actually listed among the component criteria)... we can't just cherry-pick a few individual aspects of the skaters' performances, and use those alone to justify (or refute) how the scoring went.

(I say this, only because you seem to suggest that Osmond scored higher than Miyahara, simply because she skated faster. I'm just saying it takes more than that...)

Speed helps with jumps completion and GOE as you mentioned, and assisted Kaetlyn in nailing her 3/3's and most of the jumping passes, thus contributing to her TES scores and fewer ur's, versus Satoko.

 

[gkelly]

Was this a speed skating competition?!

It isn't a race. It is a skating competition, in which skating quality including speed affects not only the Skating Skills component but also the impression of the other components and of many of the elements.

Seriously though - while speed makes everything more difficult, and may contribute to GoE's and Skating Skills (even though "speed" is not actually listed among the component criteria)...

Speed is listed in the Skating Skills component criteria:

SKATING SKILLS
Defined by overall cleanness and sureness, edge control and flow over the ice surface demonstrated by a command of the skating vocabulary (edges, steps, turns etc.), the clarity of technique and the use of effortless power to accelerate and vary speed.

Use of deep edges, steps and turns
Balance, rhythmic knee action and precision of foot placement
Flow and glide
Varied use of power, speed and acceleration
Use of multi directional skating
Use of one foot skating

Maximum or average speed throughout the program is not explicitly listed. However, acceleration is more effective if it results in a higher top speed, achieved more quickly/efficiently. Varied use of speed is better achieved if it includes some fast and very fast skating along with some medium and slower speeds. So a skater who never reaches high speeds will by definition be at a disadvantage on that particular criterion.

"Flow and glide" and "Deep edges" are often closely related to speed, but they're not exactly the same thing so it is possible to have one without the other.

we can't just cherry-pick a few individual aspects of the skaters' performances, and use those alone to justify (or refute) how the scoring went.

True. But if one person says "Skater X absolutely should have won because she was better at A, B, and C," it's certainly valid for someone else to point out "Well, Skater Y was better at D, E, F, and G . . . and many experts consider D to be a highly significant quality."

I.e., if the first person cherry picks some qualities, the second person is not necessarily cherry picking different qualities, but expanding the range of qualities to be considered.

There can still be disagreement as to which qualities should always be considered most important, or as to the size of X's advantages in her areas of strength vs. the size of Y's advantages in hers. Which is why it's often hard to say that any skater absolutely should have placed higher with no questions -- only that there is an argument, perhaps a strong one, for putting that skater winning. That doesn't preclude the possibility of equally strong arguments for a different result.

Which is why it takes a whole panel of judges -- and now technical specialists -- to determine results.

ETA:

"Good speed or acceleration" is an explicit positive GOE bullet point for step sequences and choreo sequences, and "Poor speed" is an explicit GOE reduction for jumps. (Also "Slow or reduction of speed" is a negative for spins, but that's rotational speed which is different from speed across the ice.)

 

[ancientpeas]

I think people fall into 3 camps:
People who could never see Miyahara losing to Osmond under any circumstances.
People who really don't like Miyahara's jumps and want to see her not be rewarded for pre and under rotations
and finally, people who are willing to judge each performance as it comes.

That's seems to be the camps. The first 2 have already made up their minds and the last camp seems pretty thin on the ground sometimes. I think on a day when Miyahara's technical score is the same as Osmonds Satoko will win. She really lost this herself, imo, she's the one who got an element score zero and she is the one who underrotated her jumps. I watched it live and I thought Osmond deserved to win. I've never felt that before in a head to head between them. I might never feel that way again. I'm actually a fan of Satoko's.

Corrected my spelling of Satoko Miyahara's name. Sorry for the error.

 

[drivingmissdaisy]

Was this a speed skating competition?!

Seriously though - while speed makes everything more difficult, and may contribute to GoE's and Skating Skills (even though "speed" is not actually listed among the component criteria)... we can't just cherry-pick a few individual aspects of the skaters' performances, and use those alone to justify (or refute) how the scoring went.

Good speed also permeates various aspects of the performance, especially live in the venue. It shows that the skater is attacking the program and not holding back, and makes everything feel "bigger". Kaetlyn does a lot of things well, makes eye contact with the audience, has good extension, nice spins and strong jumps. For me, she may be the most underrated skater in the field today, but it's hard for me to say for certain since I haven't seen her live yet.

 

[chloepoco]

I think people fall into 3 camps:
People who could never see Myaharya losing to Osmond under any circumstances.
People who really don't like Myaharya's jumps and want to see her not be rewarded for pre and under rotations
and finally, people who are willing to judge each performance as it comes.

That's seems to be the camps. The first 2 have already made up their minds and the last camp seems pretty thin on the ground sometimes. I think on a day when Myaharya's technical score is the same as Osmonds Satoko will win. She really lost this herself, imo, she's the one who got an element score zero and she is the one who underrotated her jumps. I watched it live and I thought Osmond deserved to win. I've never felt that before in a head to head between them. I might never feel that way again. I'm actually a fan of Satoko's.

If you are really a fan of hers, you might try spelling her name correctly. It's not just once, or twice, but three times you misspelling Satoko MIYAHARA's name.

 

[Blades of Passion]

Miyahara received 0 for her step sequence as it did not fulfill the requirements.

It did fulfill the requirements, that call was ridiculous and completely random. Her step sequence was better than Osmond's, who got a Level 4 call with tons of +GOE when I saw that one of her Loops in the sequence was not completed properly, her hops and reverse twizzles within the sequence were pretty weak, and overall it didn't actually have much content.

So really, Osmond deserved Level 3 with +1 GOE at most for her Step Sequence (3.8 points, instead of the 5.5 she got) and Satoko deserved a Level 4 with +1/+2 GOE's (let's say it comes out to 5 points in total). That's 6.7 point differential between the two of them from what they actually received, simply because of ridiculous random tech call and the other skater being held up for their Nationality and having more "momentum" (a much better SP score in this case).

 

[ancientpeas]

Sorry. I spelled it several different ways and settled on that spelling because it looked right. I actually changed it. Spelling has never been my strong suit. I should have looked it up before I wrote my response. Normally I stick to her first name because I risk butchering her last. That doesn't mean I'm not a fan of her skating. I think she's has lovely skating skills and expression and I enjoy watching her.


Edited this because... well just because I didn't like my own tone.

Thanks for correcting me. I'll try not make the same mistake again.

 

[Mango]

This instagram account has been breaking down the scoring in an interesting way: https://www.instagram.com/skatingprotocol/

I would recommend the book The Second Mark for a look at how one instance of flawed judging came about.

I see this person is affiliated with The Skating Lesson. Some interesting biases. Even using ordinals for PCS evaluation. Can our technical experts give us an evaluation on the credibility of the analyses posted in this link?

 

[Biellmann]

I guess that makes sense. But even with a judge from a country with their best skater down in the 30s, they will likely not be biased of the top skaters. I feel like the ISU should have a system where if there is clear favoritism (like the Italian judge of ice dance at Skate Canada), they should be replaced or suspended for not complying with the wants or ideals of good PCS of the ISU. I'm just spit balling here.

As if judges from random countries doesn't have their favorites
They don't have a strong skating federation and top skaters and things like this, but they're not living under a rock

 

[Osmond4gold]

Thanks for correcting me. I'll try not make the same mistake again.

Don't sweat it ancientpeas, fans gets Kaetlyn's name wrong as well, on a continued basis. It does not mean they aren't fans and like you, forgiveness for spelling errors is always a given.

 

[Osmond4gold]

It did fulfill the requirements, that call was ridiculous and completely random. Her step sequence was better than Osmond's, who got a Level 4 call with tons of +GOE when I saw that one of her Loops in the sequence was not completed properly, her hops and reverse twizzles within the sequence were pretty weak, and overall it didn't actually have much content.

So really, Osmond deserved Level 3 with +1 GOE at most for her Step Sequence (3.8 points, instead of the 5.5 she got) and Satoko deserved a Level 4 with +1/+2 GOE's (let's say it comes out to 5 points in total). That's 6.7 point differential between the two of them from what they actually received, simply because of ridiculous random tech call and the other skater being held up for their Nationality and having more "momentum" (a much better SP score in this case).

Fans seem to ignore that judges at ISU competitions come from different countries and home cooking and over/under scoring, gets tiresome. Rather than criticize the judging and blaming other skaters, I am sure Satoko is the type of athlete who would rather focus on what the technical panel was telling her here. Most athletes at this level in a pre-Olympic year, appreciate feedback in an effort to improve and to maximize their future skating results.

This is from the ISU webpage:

/////////////

Technical Panel

The Technical Panel is composed of the Technical Controller, the Technical Specialist and the Assistant Technical Specialist, each one from different ISU Members (countries). A Data Operator assists them for recording purposes. An instantaneous slow-motion video replay system operated by a Replay Operator supports the Technical Panel in the identification of the performed elements. The Technical Specialist, assisted by the Assistant Technical Specialist, identifies and calls the performed elements and the specific Levels of Difficulty of certain performed elements (e.g. spins, footwork,). He/She identifies illegal or additional elements and falls. The Technical Controller authorizes and/or corrects all calls, supervises the Data Operator and can propose corrections, if necessary.

In case of disagreement, the majority among the three Officials prevails.