A new article from Sonia

A new essay by Sonia has been released.

http://www.iceskatingintnl.com/current/content/what else.htm

IMHO this trumps everything:

At the heart of the new judging system there was a fundamental change in the method of evaluating skating performances. The former ordinal method of scoring was based on the recognition that humans can make relative judgments with greater precision than absolute judgments.

Click to expand...

If you agree with Mrs. B. on that point (I do...I think :scratch, then all the nitpicking about rounding, tweaking and diddling is just a red herring to cover the trail of the real culprit -- the whole concept of a "code of points" in the first place.

One thing she failed to mention is:

The mechanism of mathematical rounding of numbers is different from that in software. Rounding in computer is all about bit shifting and has to be done every step of the way, especially if a variable is a floating point and double (which we use for COP). Otherwise, computer will spit out all the garbage bits that were already in the registers before you perform a calculation. Software engineers borrow the idea from the mathematicians, but the implementation of mathematical idea could be a little bit different.

Hence comes numerical analysis of the software, which is basically the hybrid of mathematical/software principles to quantify the degree of numerical accuracy and the margins of errors.

fumie_fumie said:

One thing she failed to mention is:

The mechanism of mathematical rounding of numbers is different from that in software. Rounding in computer is all about bit shifting and has to be done every step of the way, especially if a variable is a floating point and double (which we use for COP). Otherwise, computer will spit out all the garbage bits that were already in the registers before you perform a calculation. Software engineers borrow the idea from the mathematicians, but the implementation of mathematical idea could be a little bit different.

Hence comes numerical analysis of the software, which is basically the hybrid of mathematical/software principles to quantify the degree of numerical accuracy and the margins of errors.

Click to expand...

However, it is child's play to get around any problems of that sort. One way is simply to carry out all internal calculations to, say, 20 decimal places, and round to two at the very end. (Well, OK, someone might get .499999999999999999999 points for something and throw it off, LOL.)

The other is to compute all fractions by keeping track of numerators and denominators separately as integers, do the required fraction arithmetic, and divide (and apply the PCS factor) at the end. This method is absolutely 100% accurate with no rounding considerations ever coming up.

In fact, the only reason that there is a problem at all is because PCS calculations are done in 28ths of a point (increments of a quarter of a point, averaged over seven judges' scores). The computer could just keep track of how many 28ths a skater received (an integer) and there would be no question of floating point calculations or anything else.

For the tech scores, the problem does not arise because the smallest gradation is one one-hundredth of a point exactly (when the 10% bonus is applied to a base value like 4.5).

Edited to add: By the way, according to Dr. Rossano's calculations, the men's event at U.S. Nationals comes out like this:

The Icecalc/Canadian federation proposal way: A tie.

The method mandated by the ISU rules: Johnny wins by .01 point.

The 100% accurate, no rounding, way: Evan wins by .01 point.

Mathman said:

However, it is child's play to get around any problems of that sort. One way is simply to carry out all internal calculations to, say, 20 decimal places, and round to two at the very end. (Well, OK, someone might get .499999999999999999999 points for something and throw it off, LOL.)

The other is to compute all fractions by keeping track of numerators and demoninators separately as integers, do the required fraction arithmetic, and divide (and apply the PCSs factor) at the end. This method is absolutely 100% accurate with no rounding considerations ever coming up.

In fact, the only reason that there is a problem at all is because PCS calculations are done in 28ths of a point (increments of a quarter of a point, averaged over seven judges' scores). The computer could just keep track of how many 28ths a skater received (an integer) and there would be no question of floating point calculations or anything else.

For the tech scores, the problem does not arise because the smallest gradation is one one-hundredth of a point exactly (when the 10% bonus is applied to a base value like 4.5).

Edited to add: By the way, according to Dr. Rossano's calculations, the men's event at U.S. Nationals comes out like this:

The Icecalc/Canadian federation proposal way: A tie.

The method mandated by the ISU rules: Johnny wins by .01 point.

The 100% accurate, no rounding, way: Evan wins by .01 point.

Click to expand...

I guess I oversimplified the gist of the problem here. You are right. A floating point, like integers, is a layout of 1 and 0 over the larger memory space than the integer. Yes, numerators end and denominators are treated as integers, basically. We decide where the numerators end and denominators begin and then message the numbers to make it readable for human eyes for presentation. The one difference and constraint with the computer is we carve out the memory space where a calculation will take place, and whatever bit that flows over MSB or LSB of the constrained bit will be lost, right? But, I am still not clear where a quarter of a point is coming from?

If COP can be done including within a limited space, without having to lose the least significant bit, then you are absolutely right. I am guessing, whatever software ISU is using is probably doing repetitions of subtractions for division though.

Overall, the COP calculator shouldn't be that complex, because it is mostly addition it is doing, which is the numerically safest calculation of all.

Mathman said:

Edited to add: By the way, according to Dr. Rossano's calculations, the men's event at U.S. Nationals comes out like this:

The Icecalc/Canadian federation proposal way: A tie.

The method mandated by the ISU rules: Johnny wins by .01 point.

The 100% accurate, no rounding, way: Evan wins by .01 point.

Click to expand...

Yeah, I remember him stating this in an earlier article. And since he's a supporter of the method #3, what's the fuss? The right guy's won, the next tie won't happen again in the foreseeable future. Let it go, then?

Hsuhs said:

Yeah, I remember him stating this in an earlier article. And since he's a supporter of the method #3, what's the fuss? The right guy's won, the next tie won't happen again in the foreseeable future. Let it go, then?

Click to expand...

Yes, but most people's issue isn't with the calculations - it was with the awarding and not awarding of points (i.e Evan was given more credit than he should have and Johnny was not given enough credit).

dwiggin3 said:

Yes, but most people's issue isn't with the calculations - it was with the awarding and not awarding of points (i.e Evan was given more credit than he should have and Johnny was not given enough credit).

Click to expand...

I believe, Dr. Rossano's primary interest is in math. While Mrs. Bianchetti isn't happy with the IJS in general. I'm not sure what most ppl's issue is.

Did COP make the life of mathematicians more interesting? Before COP there wasn not that much to calculate. You could calculate the probability for something to happen (e.g. fall on a jump etc.). But apart from that...

Oh, one thing I really like about COP (and there's not much) is that you can screw up the short and still win. For example a fall on a jump in the short was pretty devastating back then - but now you can still win if you are that much better than everyone else.

I like Mrs. Bianchetti's thoughts on everything, but going back to 6.0 - it's also a bit tricky. It's not just the "screw-the-short - rock-the-free - and-win" part. The men do much nicer spins nowadays (I just watched my 2002 video) and your probability to win without a Quad increased...

fumie_fumie said:

But, I am still not clear where a quarter of a point is coming from?

Click to expand...

Program component scores are graduated in increments of .25. 6.5, 6.75, 7.00, etc.

If COP can be done including within a limited space, without having to lose the least significant bit, then you are absolutely right. I am guessing, whatever software ISU is using is probably doing repetitions of subtractions for division though.

Overall, the COP calculator shouldn't be that complex, because it is mostly addition it is doing, which is the numerically safest calculation of all.

Click to expand...

I think the hardware and software requirements are miniscule. You only have to deal with a few hundred numbers, none bigger than 20 or so, none with more than 2 decimal place accuracy. Even if you don't try to be clever, the only thing you have to do besides add is multiply a number like 213.75 by 1.6 and divide by 7. (You don't have to do any multiplying or dividing before the last step.)

Hsuh said:

Yeah, I remember him stating this in an earlier article. And since he's a supporter of the method #3, what's the fuss? The right guy's won, the next tie won't happen again in the foreseeable future. Let it go, then?

Click to expand...

I agree. I think Mr. Cinquanta is deliberately setting us to squabbling about whether a triple twist with backward upsidedown dismount should get a 3.2 base value with 0.54 GOE, or whether a flutzy Lutz or a flippy dip needs an "e" or an "f" beside it. Thus deflecting attention from questions like

(a) Does the whole concept of a point-value scroring system fly in the face of common sense?

(b) Is the CoP killing the sport?

(c) Are the judges still cheating merrily away?

Medusa said:

Did COP make the life of mathematicians more interesting?

Click to expand...

It kind of did at first. But no, basically, ordinal judging is much more subtle mathematically than add-up-the points.

There is a famous theorem, proved in 1951 by economist Kenneth Arrow, that says "no ordinal voting system is fair" (in the sense of producing the "right" winner under all circumstances). There is a huge literature in non-parametric statistics about how to minimize the problems associated with ordinal judging. Anyone interested can look up topics like "Borda count" and "Condorcet's paradox" (A beats B, B beats C, and C beats A. Who wins?)

Before COP there was not that much to calculate.

Click to expand...

There were two main systems, both with their flaws. The majority of ordinals system was used at events like the U.S. National championship, as well as many smaller competitions because it is relatively easy to do.

But in international competitions the ISU used the OBO ("One-by-one") method of comparing ordinals. This was kind of hard to understand. You had to compare each pair of skaters and each judge and keep track of running totals. (The problem with OBO is that you might be winning the gold medal, having soundly thrashed your only threat, the number two skater. But then the sixth place skater beats the fifth place skater, everything is recomputed, and you lost!)

There was a big whoop-de-do about this in the 2002 Olympics. In the short program Michelle Kwan was first, Irina Slutskaya was second, Sasha Cohen was third and Sarah Hughes was fourth. In the LP, Sarah got first (5 votes to 4 over Irina). Michelle got 4 second place votes to Irina's 1.

The New York Times reported the next day that if one judge had changed his mind, then Michelle would have got second over Irina and would have won the gold medal over all. (The particular judge that everyone thought should have changed his mind was Joe Inman of the U.S., who placed Irina ahead of Michelle, LOL.)

Under majority of ordinals judging, this was correct (Michelle would have has five 2nd-or-better votes to Irina's four). But Inman was let off the hook when it was pointed out that, no, that's wrong, under the ISU OBO system Irina would have won second anyway.

Mathman said:

Program component scores are graduated in increments of .25. 6.5, 6.75, 7.00, etc.I think the hardware and software requirements are miniscule. You only have to deal with a few hundred numbers, none bigger than 20 or so, none with more than 2 decimal place accuracy. Even if you don't try to be clever, the only thing you have to do besides add is multiply a number like 213.75 by 1.6 and divide by 7.

Click to expand...

I got the impression that the scoring system must be easy enough so that it can be done by hand (with a hand held calculator) and done so in a reasonable amount of time. Is that true? Rounding earlier makes the calculating easier.

If the hand calculation limitation is dropped, I think that there are ways to highly improve the CoP system. For example, rather than grading elements from -3 to +3 and then consulting the table of values, each element could instead be scored from 0 to 10 (with a 0 giving 50% of the base value and a 10 giving 150% of the base value or something like that, it could vary for jumps vs. spins) and always entered in a computer. The latter would be harder to calculate by hand but be more accurate and intuitive IMO.

Anyways, I prefer the CoP because it is more transparent. I agree with Sonia about dropping anonymous judging and dropping the random elimination of scores.

ChrisH said:

I got the impression that the scoring system must be easy enough so that it can be done by hand (with a hand held calculator) and done so in a reasonable amount of time. Is that true? Rounding earlier makes the calculating easier.

Click to expand...

I don't think rounding earlier makes the calulation easier.

{10, 13, 16}

(a) Divide these numbers by 7, then add.

10/7 = 1.43
13/7 = 1.86
16/7 = 2.29

Total = 5.58

(b) Add first, then divide at the end

!0 + 13 + 16 = 39.

39/7 = 5.57

The last method avoids the rounding error of the first, AND it's easier because you only have to divide once instead of three times.

Mathman said:

(b) Add first, then divide at the end

!0 + 13 + 16 = 39.

39/7 = 5.57

The last method avoids the rounding error of the first, AND it's easier because you only have to divide once instead of three times.

Click to expand...

We were taught at school to calculate with fractions (don't know if that's the English expression?) as long as possible - to avoid rounding errors and to make it easier for us. If they even teach that at school - why didn't the ISU decide to calculate everything properly in the first place? They are practically begging people to criticise the system!

I just read the propositions from Skate Canada from Dr. Rossano's site, concerning points for jump combos. That is so confusing, how are people supposed to understand that? And the fun thing is - if they change the rules according to the proposition and this year someone wins with let's say 0.5 points the World Title. The winner had one more difficult jump (let's say Triple Axel instead of Triple Lutz) but just two combos - and the second placed did three jump combos including two Triple-Triple's. So then the guy in second place can say: Well, in 2009 I would have won with my program because I had more difficult combos and next year would haved earned more points with them. Cute, isn't it?

Mathman said:

But in international competitions the ISU used the OBO ("One-by-one") method of comparing ordinals. This was kind of hard to understand. You had to compare each pair of skaters and each judge and keep track of running totals. (The problem with OBO is that you might be winning the gold medal, having soundly thrashed your only threat, the number two skater. But then the sixth place skater beats the fifth place skater, everything is recomputed, and you lost!)

Click to expand...

Actually, that could also happen under the majority system. In fact, the main reason that the OBO system was introduced was to try to prevent that kind of flipflopping in the medal placements due to changes in the majorities of skaters lower in the standings. The inciting incident was the men's event at 1997 Europeans:
http://www.frogsonice.com/skateweb/obo/score-tech.shtml

OBO doesn't prevent that sort of thing entirely though. And of course in cases of mixed ordinals the results could be notably different depending on which system was used, and could change significantly for several skaters if only one judge happened to switch his or her relative placements of two skaters.

And under either majority or OBO calculations, as long as factored placements were being used, there will inevitably be changes in the overall standings when subsequent skaters get "between" previous skaters in the long program standings when the one who placed higher in the long program had been more than two places behind in the short. (More complicated than that with more than two phases to the competition, e.g., with figures or qual rounds, or in dance events with compulsories, OD, and free dance.)

Medusa said:

So then the guy in second place can say: Well, in 2009 I would have won with my program because I had more difficult combos and next year would haved earned more points with them. Cute, isn't it?

Click to expand...

That sort of thing happens all the time; the year after Linda Fratianne lost they changed the rules so that she would've won. That's one scoring system that I would *not* want to return to, if only because people only realized Poetsch has won about a minute into poor Lisa Marie's program, which everyone then neglected to pay attention to! Honestly, if figure skating could survive that kind of anticlimax all the time? I think it can survive CoP!

Medusa said:

We were taught at school to calculate with fractions (don't know if that's the English expression?) as long as possible - to avoid rounding errors and to make it easier for us. If they even teach that at school - why didn't the ISU decide to calculate everything properly in the first place? They are practically begging people to criticise the system!

Click to expand...

Actually, the anomaly that Rossano first caught was not about rounding errors per se. The "round first, then add" method can be criticized from the point of view of mathematical principles, but at least, for better or for worse, that is what the ISU rules say to do. Rather the problem was in applying the 2.0 PCS factor (1.6 for ladies' LPs).

Applying the PCSs factor prematurely (before adding instead of after) could affect the result in either direction, but in any case runs the risk of doubling the rounding errors. Rossano's beef, however, was not that. It was that the method that the IceCalc software was using did not follow the prescription of the ISU rules.

Bianchetti's lament is that the Canadian propsal says, if Icecalc didn't follow the ISU rules, then the ISU must change it's rules. This is the tail wagging the dog. Who is working for whom here?

I just read the propositions from Skate Canada from Dr. Rossano's site, concerning points for jump combos. That is so confusing, how are people supposed to understand that?

Click to expand...

O, Canada! The Canadian Federation, in the person of David Dore, is the priciple architect of the CoP. They love this stuff, thinking up obscure geeky minutae and rolling around in it!

Thank you very much for the link, gsrossano! Another great essay from Sonia!

I'd like to read the opposit opinions from ISU about all the controversy.

I would say that current CoP system has failed. It requires change immidiately before it makes more and deeper damage to the sport. I don't think that going back to 6.0 would be a good solution though. The 6.0 system is too simple. It couldn't explain clear enough for a sport as complex as figure skating. I think we should find a solution in between the two systems, which means that ISU will have to invent a new system to simplify the CoP and adopt all the goodness of 6.0. Is that possible?

In real work scenarios, it may be easier to fix the code than to change the rules, but then you risk breaking something else or exposing other anomalies, and testing can be a greater cost.

People can make all sorts of arguments about the ISU's actions and motives, but this is the type of decision making I see all the time on my job.

jennylovskt said:

Is that possible?

Click to expand...

yes