Page 1 of 4 1 2 3 4 LastLast
Results 1 to 15 of 56

Thread: ISU experiment: dividing tasks among judges??

  1. #1
    Missing Tdizzle and SDiggity golden411's Avatar
    Join Date
    Feb 2012
    Posts
    5,623

    ISU experiment: dividing tasks among judges??

    ISU experiment: dividing tasks among judges??

    @_David_Molina (who is an ISU judge)
    Oberstdorf Sept 25: ISU will explore a new way of judging #figureskate by sharing tasks between judges eg eval 3 PCs out of 5, or GOE only..
    5:46 AM - 3 Jul 2014
    https://twitter.com/_David_Molina/st...79575344283648

    (Did another thread already have discussion of this experiment? Sorry if I missed it. In any case, I am going to add a post to the existing Nebelhorn thread.)

    ETA, I see that the Nebelhorn announcement has more information.

    ISU Test policy:

    Two of the skating disciplines (Men and Ice Dance) are to be judged by the panel formed in the way described below:

     The two judging panels will be selected by the Technical Committees and will be trained at a one-day seminar before the start of the event.
     The panel will consist of 12 judges
     GOE is evaluated by 5 judges
     Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges
     The 5 components are also evaluated by the remaining 7 judges
    - Each of these 7 judges is evaluating only 3 components
    - The assignment of these components is done so that each component is finally evaluated with 5 scores.
     The final result is calculated according to the current ISU system using the trimmed mean of the GOE and components scores.

    Prior to the test a questionnaire will be prepared by the committees to evaluate the opinions of the judges used for the test. The success of the test will also be measured by the opinions of TC Members and Sports Directors and a final report will be sent to the ISU Council.

    Page 7 of the Nebelhorn announcement has a chart showing a sample configuration of how individual judges would take on different sets of responsibilities.


    ETA:
    Thx to Mathman for figuring out how to convert the chart within the Nebelhorn PDF to a version that is compatible with the limitations of GS formatting.

    Quote Originally Posted by Mathman View Post
    ...Here is the proposal for the tasks of the twelve judges, which will be tested at Nebelhorn.

    Judge #1 SS TR PE
    Judge#2 TR PE CH
    Judge #3 PE CH INT
    Judge #4 SS TR CH
    Judge #5 SS TR INT
    Judge #6 SS PE INT
    Judge#7 TR CH INT

    Judge #8 GOEs and also SS
    Judge #9 GOEs and PE
    Judge #10 GOEs and CH
    Judge #11 GOEs and INT
    Judge #12 Just GOEs

    This way each of the five program components is evaluated by five judges, and GOEs are judged by 5 judges also. (Too bad the math worked out so that there was one GOE judge left over with no component to judge -- TR already had five judges. Oh well, nobody's perfect.) ...

  2. #2
    Custom Title
    Join Date
    Jan 2014
    Posts
    283
    It does eem kinda complicated, but hopefully this would resolve the problem of corridor judiging...

  3. #3
    Custom Title Coltrocks12's Avatar
    Join Date
    May 2014
    Location
    Leander, TX
    Posts
    125
    I hope they release the results of their testing on this. This is interesting and with anything new will meet with people being resistant to change and others being excited. Who knows, it may help clear up controversy in the long run. Count me in the guardedly optimistically hopeful crowd.

  4. #4
    Custom Title
    Join Date
    Jul 2003
    Posts
    3,965
    Interesting.

    If this proves successful, the division of labor will have to vary for smaller competitions that can't afford 12 judges per event. Championships should certainly be able to afford it.

    Here, one judge has nothing to do except evaluate GOEs. What I heard from the last time they tried a similar test at Nebelhorn, with two separate panels judging either GOEs or PCS, was that the GOE judges were bored because they had so little to do. Nothing to think about between elements. If this division were adopted, I wonder if judges would welcome that assignment or disdain it. Probably depends on the schedule.

  5. #5
    a cat watching figure skating alebi's Avatar
    Join Date
    Jan 2014
    Location
    Italy
    Posts
    400
    a very complicated way to improve
    try to split the judgement for elements/GOEs from components was something requested, I'll give a try to this system, even if this seem a bit expensive

  6. #6
    Yuzulia & Ruslena Team Alba's Avatar
    Join Date
    Feb 2014
    Location
    Milan
    Posts
    4,952
    ISU Test policy:

    Two of the skating disciplines (Men and Ice Dance) are to be judged by the panel formed in the way described below:

     The two judging panels will be selected by the Technical Committees and will be trained at a one-day seminar before the start of the event.
     The panel will consist of 12 judges
     GOE is evaluated by 5 judges
     Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges
     The 5 components are also evaluated by the remaining 7 judges
    - Each of these 7 judges is evaluating only 3 components
    - The assignment of these components is done so that each component is finally evaluated with 5 scores.
     The final result is calculated according to the current ISU system using the trimmed mean of the GOE and components scores.

    Prior to the test a questionnaire will be prepared by the committees to evaluate the opinions of the judges used for the test. The success of the test will also be measured by the opinions of TC Members and Sports Directors and a final report will be sent to the ISU Council.

    Can someone explain this to me, in a sort of human language?

  7. #7
    a cat watching figure skating alebi's Avatar
    Join Date
    Jan 2014
    Location
    Italy
    Posts
    400
    you have 5 judges for GOE and 7 judges for components

    GOE's judges also evaluate one different component (except one judge.. LOL )
    components' judges evaluate only 3 components, not the whole set

    LOL... we have big minds in the ISU council, don't you know?

  8. #8
    Yuzulia & Ruslena Team Alba's Avatar
    Join Date
    Feb 2014
    Location
    Milan
    Posts
    4,952
    Quote Originally Posted by alebi View Post
    you have 5 judges for GOE and 7 judges for components

    GOE's judges also evaluate one different component (except one judge.. LOL )
    components' judges evaluate only 3 components, not the whole set

    LOL... we have big minds in the ISU council, don't you know?

    Sorry everyone. I really have to use italian here:
    So, 5 giudici su 7 danno il goe, 4 di questi 5 anche un solo aspetto dei PCS. Gli altri 7 giudicano solo tre PCS. Il tutto fatto in modo che comunque l'atleta abbia 5 valutazioni su ogni PCS.
    Did I got that right?

  9. #9
    Skating is art, if you let it be. Blades of Passion's Avatar
    Join Date
    Sep 2008
    Location
    Hollywood, CA
    Posts
    4,152
    The ISU continues to take good ideas and then implement them incorrectly.

    There should just be 6 technical judges and 6 program component judges. It's so easy to figure out. You only need 1 technical specialist, rather than a panel of 3, because all of the technical judges need to be tech specialists in the first place. They would all vote to determine rotation calls for jumps and they would each look for a different criteria in the footwork sequences.

  10. #10
    Yuzulia & Ruslena Team Alba's Avatar
    Join Date
    Feb 2014
    Location
    Milan
    Posts
    4,952
    Quote Originally Posted by Blades of Passion View Post
    The ISU continues to take good ideas and then implement them incorrectly.

    There should just be 6 technical judges and 6 program component judges. It's so easy to figure out.
    Agreed.


    You only need 1 technical specialist, rather than a panel of 3, because all of the technical judges need to be tech specialists in the first place. They would all vote to determine rotation calls for jumps and they would each look for a different criteria in the footwork sequences.
    I guess here they want to apply the majority vote.

  11. #11
    Custom Title
    Join Date
    Mar 2014
    Location
    Oslo, Norway
    Posts
    443
    The ISU needs to realize that the current system of judging (or the last for that matter) is not the problem. It's the judges themselves.

    Until they refuse to hire judges with conflicts of interests on the field and former cheaters as well as implementing non-anonymous scores, it really won't matter what they come up with. Corrupt judges on the stand will find a way to muck up the results.

  12. #12
    missing Mao & Yuna satine94's Avatar
    Join Date
    Feb 2014
    Location
    running the vuvuzela crematorium
    Posts
    1,741
    Quote Originally Posted by Alba View Post
    Can someone explain this to me, in a sort of human language?
    I may be totally, completely wrong here, but I will try (I really might be wrong here )



    There are two groups of judges. One of five & one of seven.

    The first group of five will score GOE. *Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges.*

    The second group of seven judges will also evalute the 5 components. **Each judge will only score 3 components each.**

    ***The assignment of these components is done so that each component is finally evaluated with 5 scores.***



    ***5 components with 5 scores each = 25 component scores altogether.
    ** accounts for 21 of the 25 scores since 7 judges x 3 components each = 21. This leaves 4 scores to be reached by the first group of GOE judges.
    SO, * This means that 1/5 judges will score no components, but 4/5 of them will judge 1 component each.


    If I understood this right, that means that even though there will be more judges (12 instead of 9) there will only be 5/12 giving GOE overall & 11/12 giving PCS scores but the total amount of PCS scores will be 25 as opposed to the normal 45

  13. #13
    Skating is art, if you let it be. Blades of Passion's Avatar
    Join Date
    Sep 2008
    Location
    Hollywood, CA
    Posts
    4,152
    Quote Originally Posted by Alba View Post
    I guess here they want to apply the majority vote.
    It still would be majority vote. 6 tech judges + 1 tech specialist. They would each review the jumps and vote. 4 out of 7 would determine how the call goes.

  14. #14
    Yuzulia & Ruslena Team Alba's Avatar
    Join Date
    Feb 2014
    Location
    Milan
    Posts
    4,952
    Quote Originally Posted by satine94 View Post
    I may be totally, completely wrong here, but I will try (I really might be wrong here )



    There are two groups of judges. One of five & one of seven.

    The first group of five will score GOE. *Four of these 5 Judges also evaluate one component different from the components evaluated by the other of these 4 Judges.*

    The second group of seven judges will also evalute the 5 components. **Each judge will only score 3 components each.**

    ***The assignment of these components is done so that each component is finally evaluated with 5 scores.***



    ***5 components with 5 scores each = 25 component scores altogether.
    ** accounts for 21 of the 25 scores since 7 judges x 3 components each = 21. This leaves 4 scores to be reached by the first group of GOE judges.
    SO, * This means that 1/5 judges will score no components, but 4/5 of them will judge 1 component each.


    If I understood this right, that means that even though there will be more judges (12 instead of 9) there will only be 5/12 giving GOE overall & 11/12 giving PCS scores but the total amount of PCS scores will be 25 as opposed to the normal 45
    Thanks satin.

    hmm
    Mathman, how much time it will take you to come up with an idea to beat this system?

  15. #15
    Yuzulia & Ruslena Team Alba's Avatar
    Join Date
    Feb 2014
    Location
    Milan
    Posts
    4,952
    Quote Originally Posted by Blades of Passion View Post
    It still would be majority vote. 6 tech judges + 1 tech specialist. They would each review the jumps and vote. 4 out of 7 would determine how the call goes.
    Oh yes. You are right. I was thinking how it is now.

Page 1 of 4 1 2 3 4 LastLast

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •