Results 1 to 15 of 17

Thread: How can we measure the degree of agreement between two judges?

Threaded View

  1. #1
    Custom Title Mathman's Avatar
    Join Date
    Jun 2003
    Location
    Detroit, Michigan
    Posts
    28,334

    How can we measure the degree of agreement between two judges?

    Under the CoP the judges’ protocol sheets contain so much data that it is hard to know how to get started in addressing the question of whether two judges see things pretty much the same way or not.

    When judging is not anonymous, there is an easy way it to do it. Convert each judges’ scores to ordinals. Then compute something called the “Spearman rank correlation coefficient.”

    Here is how to do it.

    ..................RUS....…FRA..difference..d squared

    Yagudin.........1..........2..........1..........1
    Plushenko......2..........3..........1..........1
    Goebel..........3..........1..........2..........4
    Honda..........4...........4..........0..........0

    Total............................................. .6

    Now compute r = 6 times (sum of the squared differences) divided by nx(n^2-1)

    = (6x6)/(4x15) = .6

    This is 60% correlation.
    Last edited by Mathman; 09-11-2010 at 10:59 PM.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •