ISU experiment: dividing tasks among judges?? | Golden Skate

ISU experiment: dividing tasks among judges??

ice coverage

avatar credit: @miyan5605
Record Breaker
Joined
Feb 27, 2012
ISU experiment: dividing tasks among judges??

@_David_Molina (who is an ISU judge)
Oberstdorf Sept 25: ISU will explore a new way of judging #figureskate by sharing tasks between judges eg eval 3 PCs out of 5, or GOE only..
5:46 AM - 3 Jul 2014
https://twitter.com/_David_Molina/status/484679575344283648

(Did another thread already have discussion of this experiment? Sorry if I missed it. In any case, I am going to add a post to the existing Nebelhorn thread.)

ETA, I see that the Nebelhorn announcement has more information.

ISU Test policy:

Two of the skating disciplines (Men and Ice Dance) are to be judged by the panel formed in the way described below:

 The two judging panels will be selected by the Technical Committees and will be trained at a one-day seminar before the start of the event.
 The panel will consist of 12 judges
 GOE is evaluated by 5 judges
 Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges
 The 5 components are also evaluated by the remaining 7 judges
- Each of these 7 judges is evaluating only 3 components
- The assignment of these components is done so that each component is finally evaluated with 5 scores.​
 The final result is calculated according to the current ISU system using the trimmed mean of the GOE and components scores.​

Prior to the test a questionnaire will be prepared by the committees to evaluate the opinions of the judges used for the test. The success of the test will also be measured by the opinions of TC Members and Sports Directors and a final report will be sent to the ISU Council.

Page 7 of the Nebelhorn announcement has a chart showing a sample configuration of how individual judges would take on different sets of responsibilities.


ETA:
Thx to Mathman for figuring out how to convert the chart within the Nebelhorn PDF to a version that is compatible with the limitations of GS formatting.​

...Here is the proposal for the tasks of the twelve judges, which will be tested at Nebelhorn.

Judge #1 SS TR PE
Judge#2 TR PE CH
Judge #3 PE CH INT
Judge #4 SS TR CH
Judge #5 SS TR INT
Judge #6 SS PE INT
Judge#7 TR CH INT

Judge #8 GOEs and also SS
Judge #9 GOEs and PE
Judge #10 GOEs and CH
Judge #11 GOEs and INT
Judge #12 Just GOEs

This way each of the five program components is evaluated by five judges, and GOEs are judged by 5 judges also. (Too bad the math worked out so that there was one GOE judge left over with no component to judge -- TR already had five judges. Oh well, nobody's perfect.) ...
 

jace93

On the Ice
Joined
Jan 8, 2014
It does eem kinda complicated, but hopefully this would resolve the problem of corridor judiging...:think:
 

Coltrocks12

On the Ice
Joined
May 18, 2014
I hope they release the results of their testing on this. This is interesting and with anything new will meet with people being resistant to change and others being excited. Who knows, it may help clear up controversy in the long run. Count me in the guardedly optimistically hopeful crowd.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
Interesting.

If this proves successful, the division of labor will have to vary for smaller competitions that can't afford 12 judges per event. Championships should certainly be able to afford it.

Here, one judge has nothing to do except evaluate GOEs. What I heard from the last time they tried a similar test at Nebelhorn, with two separate panels judging either GOEs or PCS, was that the GOE judges were bored because they had so little to do. Nothing to think about between elements. If this division were adopted, I wonder if judges would welcome that assignment or disdain it. Probably depends on the schedule.
 

alebi

Medalist
Joined
Jan 11, 2014
a very complicated way to improve :biggrin:
try to split the judgement for elements/GOEs from components was something requested, I'll give a try to this system, even if this seem a bit expensive :)
 

Alba

Record Breaker
Joined
Feb 26, 2014
ISU Test policy:

Two of the skating disciplines (Men and Ice Dance) are to be judged by the panel formed in the way described below:

 The two judging panels will be selected by the Technical Committees and will be trained at a one-day seminar before the start of the event.
 The panel will consist of 12 judges
 GOE is evaluated by 5 judges
 Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges
 The 5 components are also evaluated by the remaining 7 judges
- Each of these 7 judges is evaluating only 3 components
- The assignment of these components is done so that each component is finally evaluated with 5 scores.
 The final result is calculated according to the current ISU system using the trimmed mean of the GOE and components scores.

Prior to the test a questionnaire will be prepared by the committees to evaluate the opinions of the judges used for the test. The success of the test will also be measured by the opinions of TC Members and Sports Directors and a final report will be sent to the ISU Council.


Can someone explain this to me, in a sort of human language? :rolleye::confused:
 

alebi

Medalist
Joined
Jan 11, 2014
you have 5 judges for GOE and 7 judges for components

GOE's judges also evaluate one different component (except one judge.. LOL )
components' judges evaluate only 3 components, not the whole set

LOL... we have big minds in the ISU council, don't you know? :laugh:
 

Alba

Record Breaker
Joined
Feb 26, 2014
you have 5 judges for GOE and 7 judges for components

GOE's judges also evaluate one different component (except one judge.. LOL )
components' judges evaluate only 3 components, not the whole set

LOL... we have big minds in the ISU council, don't you know? :laugh:


Sorry everyone. I really have to use italian here:
So, 5 giudici su 7 danno il goe, 4 di questi 5 anche un solo aspetto dei PCS. Gli altri 7 giudicano solo tre PCS. Il tutto fatto in modo che comunque l'atleta abbia 5 valutazioni su ogni PCS.
Did I got that right? :rolleye:
 

Blades of Passion

Skating is Art, if you let it be
Record Breaker
Joined
Sep 14, 2008
Country
France
The ISU continues to take good ideas and then implement them incorrectly.

There should just be 6 technical judges and 6 program component judges. It's so easy to figure out. You only need 1 technical specialist, rather than a panel of 3, because all of the technical judges need to be tech specialists in the first place. They would all vote to determine rotation calls for jumps and they would each look for a different criteria in the footwork sequences.
 

Alba

Record Breaker
Joined
Feb 26, 2014
The ISU continues to take good ideas and then implement them incorrectly.

There should just be 6 technical judges and 6 program component judges. It's so easy to figure out.

Agreed.


You only need 1 technical specialist, rather than a panel of 3, because all of the technical judges need to be tech specialists in the first place. They would all vote to determine rotation calls for jumps and they would each look for a different criteria in the footwork sequences.

I guess here they want to apply the majority vote.
 

gravy

¿No ven quién soy yo?
Record Breaker
Joined
Mar 28, 2014
The ISU needs to realize that the current system of judging (or the last for that matter) is not the problem. It's the judges themselves.

Until they refuse to hire judges with conflicts of interests on the field and former cheaters as well as implementing non-anonymous scores, it really won't matter what they come up with. Corrupt judges on the stand will find a way to muck up the results.
 

satine

v Yuki Ishikawa v
Record Breaker
Joined
Feb 13, 2014
Can someone explain this to me, in a sort of human language? :rolleye::confused:

I may be totally, completely wrong here, but I will try :biggrin: (I really might be wrong here :slink:)



There are two groups of judges. One of five & one of seven.

The first group of five will score GOE. *Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges.*

The second group of seven judges will also evalute the 5 components. **Each judge will only score 3 components each.**

***The assignment of these components is done so that each component is finally evaluated with 5 scores.***



***5 components with 5 scores each = 25 component scores altogether.
** accounts for 21 of the 25 scores since 7 judges x 3 components each = 21. This leaves 4 scores to be reached by the first group of GOE judges.
SO, * This means that 1/5 judges will score no components, but 4/5 of them will judge 1 component each.


If I understood this right, that means that even though there will be more judges (12 instead of 9) there will only be 5/12 giving GOE overall & 11/12 giving PCS scores but the total amount of PCS scores will be 25 as opposed to the normal 45 :unsure:
 

Blades of Passion

Skating is Art, if you let it be
Record Breaker
Joined
Sep 14, 2008
Country
France
I guess here they want to apply the majority vote.

It still would be majority vote. 6 tech judges + 1 tech specialist. They would each review the jumps and vote. 4 out of 7 would determine how the call goes.
 

Alba

Record Breaker
Joined
Feb 26, 2014
I may be totally, completely wrong here, but I will try :biggrin: (I really might be wrong here :slink:)



There are two groups of judges. One of five & one of seven.

The first group of five will score GOE. *Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges.*

The second group of seven judges will also evalute the 5 components. **Each judge will only score 3 components each.**

***The assignment of these components is done so that each component is finally evaluated with 5 scores.***



***5 components with 5 scores each = 25 component scores altogether.
** accounts for 21 of the 25 scores since 7 judges x 3 components each = 21. This leaves 4 scores to be reached by the first group of GOE judges.
SO, * This means that 1/5 judges will score no components, but 4/5 of them will judge 1 component each.


If I understood this right, that means that even though there will be more judges (12 instead of 9) there will only be 5/12 giving GOE overall & 11/12 giving PCS scores but the total amount of PCS scores will be 25 as opposed to the normal 45 :unsure:

Thanks satin.

hmm :eek:hwell:
Mathman, how much time it will take you to come up with an idea to beat this system? :biggrin:
 

alebi

Medalist
Joined
Jan 11, 2014
Sorry everyone. I really have to use italian here:
So, 5 giudici su 7 danno il goe, 4 di questi 5 anche un solo aspetto dei PCS. Gli altri 7 giudicano solo tre PCS. Il tutto fatto in modo che comunque l'atleta abbia 5 valutazioni su ogni PCS.
Did I got that right? :rolleye:


we have a total of 12 judges so

5 giudici valutano i GOES
7 giudici valutano i components


4 dei 5 giudici dei GOE valutano anche una sola voce dei components (differente per ogni giudice)
i 7 giudici dei components valutano 3 voci, non tutte quelle presenti

this is how I got it :scratch:
 

alebi

Medalist
Joined
Jan 11, 2014
The ISU continues to take good ideas and then implement them incorrectly.

There should just be 6 technical judges and 6 program component judges. It's so easy to figure out. You only need 1 technical specialist, rather than a panel of 3, because all of the technical judges need to be tech specialists in the first place. They would all vote to determine rotation calls for jumps and they would each look for a different criteria in the footwork sequences.

easy things are always the most difficult to be done :bang:
 

Alba

Record Breaker
Joined
Feb 26, 2014
It still would be majority vote. 6 tech judges + 1 tech specialist. They would each review the jumps and vote. 4 out of 7 would determine how the call goes.

By the way, not completely right though (I'm in total confusion with this topic :confused:).
The tech specialist will determine the level right? While the tech judges will give the GoE's only. So in the end we still have a problem here.
 

FSGMT

Record Breaker
Joined
Sep 10, 2012
The idea of dividing tasks is good (one person can't give GOEs and at the same time evaluate 5 different PCS categories accurately), but why do we need all these complications, with some judges doing more things than the others? :rolleye: Just make one GOE panel and one PCS panel! (But giving only GOE scores would be extremely boring, yes :slink: )
 
Joined
Jun 21, 2003
If they had six "technical judges" they wouldn't need technical specialists at all. Each of the six judges could separately judge the level, under-rotations, wrong edge, etc., and come up with a total score for each element.
 
Top