Is ordinal judging easier to understand than the CoP?

[Mathman]

Here is a test for fans who think that the new judging system is harder to understand than the old one was. This information was taken from the following article by Sandra Loosemore.

http://www.frogsonice.com/skateweb/obo/score-obo.shtml

Some background. The results of the 1997 men’s event at Europeans (among several other competitions that year) were so screwy that neither the average fan nor the television commentators (including Dick Button) were able to understand or explain why it came out the way it did.

In response, ISU President Ottavio Cinquanta decided that what the sport needed was a new judging system. With support from the German Federation, he came up with what became known as the OBO system, which supposedly corrected some of the counterintuitive anomalies of the then-current system, the “majority of ordinals” method.

Cinquanta did an end run around the ISU rules and member federations and rushed the new system into place for the 1998 Olympics.

Test: Here are the ordinals of the nine judges, taken from 1997 Europeans (omitting some of the skaters who’s scores are irrelevant to this discussion.)

Skater A 2 1 4 1 2 2 4 5 5
Skater B 4 4 3 3 1 3 1 1 2
Skater C 3 2 2 5 3 5 3 2 1
Skater D 1 3 1 2 5 4 2 3 4

Who won?

[sequinsgalore]

I first began watching figure skating after CoP so I haven't got a clue how the ordinals worked? So I'd say CoP is easier to understand...

Can someone briefly say how ordinals worked? On old Youtube clips I can see the judges sometime use calculators. So what were they calculating?

[cleopatra20042004]

This looks like a very interesting thread. I'll take a gander.
1. Skater B
2. Skater D
3. Skater A
4. Skater C.
Now I'm off to find the actual results.

[Joesitz]

There have been judges, maybe most judges, who were aware of the 'Content of Program' and did not get carried away with the so-called 'Artistry'. These judges really didn't need the CoP. However, those that were carried away with presentation may well have overlooked other skaters who did all the tech. Others, of course, showd country favoritism, and that's what brought about the CoP.

I am basically pleased with the CoP, although it should be on a continuous study for improvement. (Those. plus GoEs and Levels often get to me.). i do not like the breakdwon of PC scores. It would suffice if an additional amount were placed on the program representing these judges who could understand the 5 major parts of the PC scores (i.e., those without the bullets). It would simplify the scoring for both fan and audience.

To answer the topic question: it takes awhile for both systems to be understood. An ardent fan does get to understand both systems. It takes a while longer for the audience.

[gkelly]

Test: Here are the ordinals of the nine judges, taken from 1997 Europeans (omitting some of the skaters who’s scores are irrelevant to this discussion.)

Skater A 2 1 4 1 2 2 4 5 5
Skater B 4 4 3 3 1 3 1 1 2
Skater C 3 2 2 5 3 5 3 2 1
Skater D 1 3 1 2 5 4 2 3 4

Who won?

If those are the actual ordinals, and the majority system was in use, then Skater A won this segment, since he was the only one who had a majority for 2nd place (five ordinals of 2 or better).

If OBO were in place, I wouldn't have a clue.

[mskater93]

In response, ISU President Ottavio Cinquanta decided that what the sport needed was a new judging system. With support from the German Federation, he came up with what became known as the OBO system, which supposedly corrected some of the counterintuitive anomalies of the then-current system, the “majority of ordinals” method.

Cinquanta did an end run around the ISU rules and member federations and rushed the new system into place for the 1998 Olympics.

Test: Here are the ordinals of the nine judges, taken from 1997 Europeans (omitting some of the skaters who’s scores are irrelevant to this discussion.)

Skater A 2 1 4 1 2 2 4 5 5
Skater B 4 4 3 3 1 3 1 1 2
Skater C 3 2 2 5 3 5 3 2 1
Skater D 1 3 1 2 5 4 2 3 4

Who won?

Based on the simple ordinal majority system used in the US (I am not good with OBO because I've not experienced it):

No skater had a majority of first place votes
Skater A had a majority of 5 judges who placed him 2nd or better
Skater B had a majority of 7 judges who placed him 3rd or better
Skater C had a majority of 7 judges who placed him 3rd or better
Skater D had a majority of 6 judges who placed him 3rd or better

By the ordinal majority system used, Skater A would be declared the winner.

Between Skater B and Skater C, who are tied under majority rules, the next tie breaker is the total ordinals of majority (TOM) since they both had 7 judges who placed them third or better. When I added up their TOMs, skater B had a TOM of 14 (3+3+1+3+1+1+2) and skater C had a TOM of 16 (3+2+2+3+3+2+1). The tie breaker has declared Skater B to be the Silver medalist and Skater C to be the Bronze medalist under the ordinal majority system. Skater D is left out, even with all those 3's and better.

[Mathman]

I first began watching figure skating after CoP so I haven't got a clue how the ordinals worked? So I'd say CoP is easier to understand...

Can someone briefly say how ordinals worked? On old Youtube clips I can see the judges sometime use calculators. So what were they calculating?

Ordinal judging goes like this. All the skaters skate. After they have all skated, each judge ranks them from best to worst – first, second, third, etc. down to last.

The end.

But…since there were so many performances to keep track of, the judges used little markers to help them remember. These markers consisted of a “technical score” and a “presentation score.” These scores went from 0 to 6, graduated in tenths.

So for instance, a judge might give a skater a 5.4 in tech and a 5.7 in presentation.

All of this, however, was a red herring and something to show to the cheering crowd. The real judging was simplicity itself – who did you think was best, who was second best, etc., and the 5.4’s and so on were just place-holders for the ordinals. (By the way, one criticism of the CoP is that judges are still using the program component scores in the same way.)

The problems start up when the judges disagree about who was best, second best, etc. In the example of 1997 Europeans, there are a number of different standard techniques for deciding who wins overall, and you get a different outcome depending on which method you use. These methods go by names like the Condorcet method, the Borda count, the Hare method – and the two used in figures skating judging, the majority of ordinals method and the OBO (one-by-one) method.

(Historical note – many of the names associated with the theory of ordinal judging are French. This is because this topic was extensively investigated by French mathematicians immediately after the French revolution when France was struggling to come up with a new republican form of government.)

[gkelly]

The tie breaker has declared Skater B to be the Silver medalist and Skater C to be the Bronze medalist under the ordinal majority system.

No, the tie breaker declares Skater B to be second in this segment and Skater C to be third.

That may or may not turn out to be the final medal results; you also have to take the other competition segment into account, the factored placements for both short and long.

These ordinals are for the long program, right? IIRC, 1-2-3 in the long program at that event did eventually line up with the final medals results, but that was not the case before the final skater (who did not place in the top 3 of either the LP or the final results) received his marks -- there was place switching in both the LP results due to the final skaters' ordinals and then in the factored placements.

[Spun Silver]

I never understood it! The only clear part was the "perfect 10" or whoever got closest to it. That much is understandable and creates emotion.

The principle of COP is easier - "only connect" (add) - but who can remember all the numbers, let alone get excited about, a 197.63?

(Which next time will be replaced by a 198.01... then a 200.78... ad infinitum....)

[Mathman]

This looks like a very interesting thread. I'll take a gander.
1. Skater B
2. Skater D
3. Skater A
4. Skater C.
Now I'm off to find the actual results.

This is correct for OBO.

Unfortunately for skater B, OBO wasn't in place for this competition, so skater B lost to skater A, as computed by gkelly and mskater93.

(You can see the actual result in the Loosemore article referenced. The data in my original post was for skaters 2, 3, 4 and 5, with ordinals adjusted appropriately. In the actual competition, another skater was way ahead, and the controversy was for second and third.)

So, why was there a big uproar when stater A was declared the winner?

There are two reasons. One was the "flip-flop" mentioned by gkelly (a failure of the criterion of "independence of irrelevent alternatives" in election theory.)

But here is something else to notice. Skater B beat skater A head-to-head by a count of five judges to four. Skater B also beat skater C by five to four. Skater B also beat skater D, five to four.

Well, if skater B beat everybody (in particular, A) why didn't skater B win?

This is the basis of OBO. You match each pair of skaters head to head and see who wins and by how much. Then you count up the number of wins and the "JiF's" -- judges in favor.

Here are the hash sheets for this contest (think of them as protocols. )


…...A..…B..….C…..D….wins…JiF
A………0/4…1/5…0/4…..1……13
B…1/5……..,,1/5…1/5…..3……15
C…0/4………0/4…0/4…..0……12
D…1/5…0/4...1/5………..2…….14

B is the clear winner with 3 wins and 15 Jif's, followed by D, A and C.

Is this easy for the average fan or television viewer to compute in real time at a live event?

[antmanb]

I never understood it! The only clear part was the "perfect 10" or whoever got closest to it. That much is understandable and creates emotion.

The perfect 10?? Spun Silver has it really been that long or are you just still on a high from Sasha's annoucement it was called 6.0 for a reason you know!

Ant

[seniorita]

maybe he just confused it with gymnastics?
new system is more comprehensible because it is all mathematics, you add, multiply and substract, the one who has more apples in the coffin wins, it seems more fair so far but i agree the old was more emotional, par example when you could see 6.0s from the crowd, that was cute. This from tv.
Live i have no idea how it would look, probably in old system i would have to wait the official result on the screen if there was a tie win, maybe it was clear for 1st and 2nd position in general but after that ....Now in new system you know around where a top skater will score anyway so you suspect how it goes as competition goes on, it just takes more time, but do all fans have calculators in live competitions to calculate if someone's pcs or Goe or deductions were correct before they would cheer?It just misses some of the magic.
(By the way, it s been three years and i still dont know why takahashi was so low marked in Lp in Olympics, has this been discussed somewhere?)
But thank you for ordinals example, nice way to study it.

[mskater93]

No, the tie breaker declares Skater B to be second in this segment and Skater C to be third.

I assumed (and possibly incorrectly) that they were close enough after the SP for the results of the long to basically carry over to the overall results. I guess I am more used to (under ordinal system for myself, anyway) a one segment result! Oops!

[Mathman]

The main reason the ISU changed from the majority of ordinals system to OBO for international competitions was to prevent “flip-flops.” Two skaters have already skated, and A is ahead of B. Then C skates. After C skates, now B is ahead of A.

Why should the performance of C, whether good, bad, or indifferent, have any effect on whether A beat B or not in the final standings?

Sandra Loosemore gives this example, taken from 1998 world juniors.

…Julia Sebestyen was ahead of Angela Nikodinov with a greater number of "Wins" until Anina Fivian skated. At this point, all three of the skaters were thrown into a tie with an equal number of "Wins", and breaking the tie by "Judges in Favor" ranked the skaters in order Fivian, Nikodinov, and Sebestyen, so that Sebestyen and Nikodinov "flip-flopped" from their previous standings.

I can imagine that this would be very confusing to spectators in the arena. The leader board shows Sebestyen first and Nikodinov second with one skater left to skate. The worst Sebestyen can finish is second, right?

Wrong. Under both majority of ordinals and OBO, somehow or other Sebestyen dropped to third and Nikodinov – by doing nothing at all except watching a third skater – leap-frogged ahead of Sebestyen.

OBO was supposed to prevent this from happening. But it doesn’t. In fact, it can’t. There is a famous mathematical theorem, called Arrow’s Impossibility Theorem, that states that it is impossible to design a ranked voting system in which flip-flops cannot occur (and also satisfies other desirable criteria, like, a unique winner is always determined, the system is not a dictatorship with only one judge, etc.)

(This theorem was proven by Kenneth Arrow in 1951 as part of his doctoral dissertation. He later (1979) won a the Nobel Prize in economics.)

In contrast, with point-total systems like the CoP, once skater A is on the board with more points than skater B, that’s it. Nothing that happens later can change the relative ranking of A and B. This is certainly a desirable feature of a judging system, especially from the point of view of the fans of Julia Sebestyan, who are left scratching their heads at what just happened.

Anyway, the whole point of this thread is to raise this question:

Why do so many people think that the CoP is “too hard for fans to understand,” to the detriment of the sport, when – I think it is safe to say – those same fans really had no clue about how the 6.0 ordinal system worked?

[steyn]

^
I think that's mainly due to layman's psychology. Whether he is an expert or just a 10 year-old kid, one can give an ordinal number to each skater. "This guy is better than the previous guy but is worse than another guy" and so on. Then he has exactly the same set of numbers which can be directly compared with the official result. He may like it or not but in any case he can comment on it and be happy except a few complicated cases.

On the other hand in the current system, one has to decide the level of each element and GOE, multiply factor 0.8 or 1.6, adds the scores, determines PCS and so on to really compare (or compete) with the official protocol. But one cannot really do this in real time. What he has is some vague number at most which is not much useful because it cannot be directly compared with the official score.

In other words, you need a lot of analysis to get a final score in the current system, which is almost impossible for a relaxed viewers having a glass of wine in an armchair. You cannot just give one simple number with which you can declare you have finished your job and are ready to comment on judging.