Rounding errors – those hundredths of a point add up

In the men’s event at U.S. nationals, Lysacek and Weir exactly tied, with Lysacek winning the title on a tie breaker. Analysis of the scores by Dr. George Rossano revealed a peculiarity in the way the last decimal place is computed. The tricky point was this.

In the men’s LP, the program component scores are rounded to two decimal places and multiplied by two. The factor of two is used in order to bring the total PCSs up to a level comparable to the TESs, so that the scoring is balanced between the technical elements and the program components.

But the question is, should we round first and then multiply by two, or should we multiply by two first and then round.

Here are Lysacek’s program component scores, computed by both methods.

Round first, then multiply by 2:

7.79X2 + 7.50X2 + 8.07X2 + 7.86X2 + 8.14X2

= 15.58 + 15.00 + 16.14 + 15.72 + 16.28 = 78.72. This was Evan’s actual score.

Multiply by 2 first, then round

7.786X2 + 7.5000X2 + 8.071X2 + 7.857X2 + 8.143X2

= 15.57 + 15.00 + 16.14 + 15.71 + 16.29 = 78.71. Johnny wins the championship!

The language in the ISU rules was unclear as to which method was intended. It looked like the ISU was not following its own rules. Which might have cost Weir the gold medal.

At the ISU Congress over the summer they addressed this by putting new language into the rules

http://www.isu.org/vsite/vfile/page/fileurl/0,11040,4844-191592-208815-140518-0-file,00.pdf

l) The trimmed mean of each Program Component Score is rounded to two decimal places,...

Click to expand...

The underlined part are new words added this year. This makes it clear that the intention is to round first before adding and before applying the factor of two.

Here is another interesting rule change – one of greater substance. It says that GOEs no longer get an extra bonus after the halfway mark, only base scores.

vi) In the Free Skating of Single Skating the base values (but not the GOE’s) for all jump elements started in the second half of the program will be multiplied by a special factor 1.1 in order to give credit for even distribution of difficulties in the program. In Pair Skating the base value (but not the GOE’s) for all throw jumps, jump elements, lifts and twist lifts, started in the second half of the program will be multiplied by a special factor 1.1. started in the second half of the program will be multiplied by a special factor 1.1.

Click to expand...

This means that, for instance, if you do a triple Lutz after the halfway mark and get a +1 GOE, under last year’s rules your score would be (6.00+1.00)X1.10 = 7.70. Under the new rules your score will be 6.00X1.10 + 1.00 = 7.60.

On the other hand, if you get an edge call and a subsequent negative GOE, the penalty will be slightly less.

Will this change cause skaters to plan their jump layout differently, or is it just a minor bookkeeping thing?

Will it embolden skaters to put more difficult jumps in the second half of the program, by reducing the penalty for sloppy landings, etc. Or is the point difference too small to pay any attention to?

It's funny how bad the World is at math.

And very disturbing that Weir wasn't at least 3-4 points ahead of Lysacek in the first place.

Mathman - I love your discussion on the Tie Breaker for US Nats. I think the two methods show it was just a tie.

Now, the Jr GP series is winding up shortly and I presume there will be ties as to who makes it to the Finals, so stay close.

MM - what an interesting discussion!

(so, I'm part of that world that is bad at math, but...) if I understand what you are saying, I would think we would or will see more Lutz and Flip attempts by flutzers and lippers in the second half of the program. Although, the impact, again if I understand it, will really only be for that group of skaters at or near the top with close scores. Is that what you are thinking too?

very interesting. I guess the ISU never thought it would actually be that close, but still they should have made it clear how to do the math.
Like Blade of Passion, i still think it should never have been that close in the first place. Watching Johnny skate after Evan I was positive Johnny had won. I am an Evan fan, but i think he was handed that title.

Mathman said:

http://www.isu.org/vsite/vfile/page/fileurl/0,11040,4844-191592-208815-140518-0-file,00.pdf

l) The trimmed mean of each Program Component Score is rounded to two decimal places,...

Click to expand...

The underlined part are new words added this year. This makes it clear that the intention is to round first before adding and before applying the factor of two.

Click to expand...

I like that. For those of us checking the protocols, the math will work out. That is, the numbers printed (which are rounded to two decimal places) are actually the numbers used in further calculations.

Here is another interesting rule change – one of greater substance. It says that GOEs no longer get an extra bonus after the halfway mark, only base scores.

vi) In the Free Skating of Single Skating the base values (but not the GOE’s) for all jump elements started in the second half of the program will be multiplied by a special factor 1.1 in order to give credit for even distribution of difficulties in the program. In Pair Skating the base value (but not the GOE’s) for all throw jumps, jump elements, lifts and twist lifts, started in the second half of the program will be multiplied by a special factor 1.1. started in the second half of the program will be multiplied by a special factor 1.1.

Click to expand...

Click to expand...

I don't believe that the GOEs got a bonus after the halfway mark, at least not in the protocols that I looked at. Thus I read that as the ISU specifying what they already do. It makes the calculations easier, especially with combination/sequence jumps.

Based on significance arithmetic

I am not sure if ISU referred to the significance arithmetic to come up with this calculation or not, but the rule follows the arithmetic if it means the first caluation listed in Mathman's post.

As you might know, the significance arithmetic can be applied to measured values that can have uncertainty after series of calculation. It is used to determine the significant figures to represent the result of calculation in a way that more precision than is known or too less precision is avoided.

The arithmetic says (from wikipedia):

When multiplying or dividing numbers, the result is rounded to the number of significant figures in the factor with the least significant figures. Here, the quantity of significant figures in each of the factors is important—not the position of the significant figures.

When adding or subtracting using significant figures rules, results are rounded to the position of the least significant digit in the most uncertain of the numbers being summed (or subtracted).

- Since original PCS from judges have three significant digits, an average value of PCS is not 7.786 but 7.79 based on the arithmetic. The sum of all average PCS multiplied by 2, which are significant to the hundredth place, has significant value to the hundredth. That is 78.72, not 78.7 nor 79. The factor of 2 does not affect significant figures since it is not measured value.

Love_Skate said:

Since original PCS from judges have three significant figures, an average value of PCS is not 7.786 but 7.79 based on the arithmetic. The sum of all average PCS multiplied by 2, which are significant to the hundredth place, has significant value to the hundredth. That is 78.72, not 78.7 nor 79. The factor of 2 does not affect significant figures since it is not measured value.

Click to expand...

Cool.

In fact, the individual judges' scores do not have even three significant decimal digits, because the scores are graduated in quarter-point increments. You have to give a 6.25 or a 6.50, you can't give a 6.26, 6.27, etc.

And I am not even sure they have that much intrinsic accuracy. When a judge sees a performance he must assign one of 41 different scores, 0.00, 0.25,...,9.75, 10.00. Is it reasonable to think that a judge can distinquish objectively between a performance that deserves to be in category 36 instead of category 37 in interpretation or transitions?

There is, however, another way to look at it. Since the judges are judging and not measuring, one could take their judgments as having infinite precission. (As, for instance, with ordinal judging. If you get third place this does not mean you really got 2.99357th place and they rounded it up.) In that case the only question is which rounding convention to use in the averaging process.

You could get the absolutely correct answer, no rounding questions at all, by doing all the arithmetic as fractions. The total would be so many 28ths (the average is typically taken over 7 judges, each reporting scores in quarters of a point).

This method, as well as the other two under discussion, are analyzed by Dr. Rossano here, with some nice graphs showing the various things that could happen.

http://www.iceskatingintnl.com/current/content/rounding ties and tie breakers.htm

ChrisH said:

I like that. For those of us checking the protocols, the math will work out. That is, the numbers printed (which are rounded to two decimal places) are actually the numbers used in further calculations.

Click to expand...

I like that feature of the ISU method, too. It would look awkward if one skater were declared the winner and then someone said, hey look, you made a mistake in adding up the score, the other guy won!

ChrisH said:

I don't believe that the GOEs got a bonus after the halfway mark, at least not in the protocols that I looked at. Thus I read that as the ISU specifying what they already do. It makes the calculations easier, especially with combination/sequence jumps.

Click to expand...

I just checked it out, and you're right about that. So both of these points are just clarifying the wording rather than changing anything.

If I understand what you are saying, I would think we would or will see more Lutz and Flip attempts by flutzers and lippers in the second half of the program. Although, the impact, again if I understand it, will really only be for that group of skaters at or near the top with close scores. Is that what you are thinking too?

Click to expand...

It was, but now that ChrisH has pointed out that this really isn't a change after all, I guess the point is moot.

In general, though, I wonder how much effort skaters, coaches and choreographers put into trying to squeeze the last hundredth of a point out of their programs. Even at the highest level, it's a lot more important to deliver your planned elements cleanly.

At Worlds, Mao Asada lost 7.5 points when she flubbed her triple Axel (she still won!), which is a lot more important than whether or not she gets a tenth of a point bonus on a GOE.

ChrisH said:

I don't believe that the GOEs got a bonus after the halfway mark, at least not in the protocols that I looked at. Thus I read that as the ISU specifying what they already do. It makes the calculations easier, especially with combination/sequence jumps.

Click to expand...

You are correct. The wording only corrects the rule to conform with what the software has been doing since the second season of IJS. Only in the first season did the 1.1 apply to the sum of the base value and GoE.

Another interesting thing about the new calculation rule (ISU 353) is that USFSA forgot to put the current text of the calculation method into the USFSA rulebook. So the calculation method in the USFSA rulebook is the rule from from the 2006 ISU congress not the 2008 congress.

It is also my understanding that USFSA is considering not using the factor of 1.1 for lifts in the second half for Novice pairs and below. It appears some do not understand that the factor of 1.1 is a calculation rule and not a program requirements rule.

Calculations rules apply to all disciplines and all divisions -- there is only one IJS calculation method. Program requirement rules, however, can vary from one division to another.

gsrossano said:

It is also my understanding that USFSA is considering not using the factor of 1.1 for lifts in the second half for Novice pairs and below.

Click to expand...

Is this because the USFSA does not want to encourage children to attempt lifts when they are tired?

Love_Skate said:

- Since original PCS from judges have three significant digits, an average value of PCS is not 7.786 but 7.79 based on the arithmetic. The sum of all average PCS multiplied by 2, which are significant to the hundredth place, has significant value to the hundredth. That is 78.72, not 78.7 nor 79. The factor of 2 does not affect significant figures since it is not measured value.

Click to expand...

The only problem with this is that when performing calculations you always carry one more than the # of sig digs needed for the final answer. So to follow proper mathematical conventions using the averages of the PCS to 3 decimal places would be the correct way to do the calculations, rounding to 2 decimal places for the final answer.

silver.blades said:

The only problem with this is that when performing calculations you always carry one more than the # of sig digs needed for the final answer. So to follow proper mathematical conventions using the averages of the PCS to 3 decimal places would be the correct way to do the calculations, rounding to 2 decimal places for the final answer.

Click to expand...

I was thinking that, too. Rounding earlier in the process is less precise, and significance requires the one more digit carried until the rounding takes place, as silver blades says, right?

rallycairn said:

I was thinking that, too. Rounding earlier in the process is less precise, and significance requires the one more digit carried until the rounding takes place, as silver blades says, right?

Click to expand...

That is indeed the way it is done in science programs. Well, we just keep doing operations in a row on the calculator and then round off the last number :yes:. If you round it off at each step, you risk having important changes.
Ex:
12,46 --> 12,5
+13,24 --> 13,2
+15,5 ---> 16 (if the last digit is a five, round up if the preceding number is odd only)
=41,2 --->=41,7
The first rounds off to 41, the second to 42. (I just took random numbers, and not sure of this proof! QED?)

-kypma

More digits don’t necessarily mean more accurate calculation result

More digits doesn't necessarily mean more accurate calculation result.

It depends on what number we are dealing with.
Significant figures are not necessary to calculate the average of the number of apples in three bags where one, two, and seven apples are contained, respectively. The average is exactly 3.333333…. It will be more accurate as more digits are used.
However, if the number can contain intrinsic error from the start, the significant figures are necessary. For example, when exact height of a student is 6.1112345 measured by a height measuring meter that has the highest accuracy in the world (even this number is not accurate if we can develop a meter that has more precision), since the meter is not available to a school, 6.1 or 6.11 is measured by a normal meter and is used to calculate the average height of a class. In this case, 0.0112345 or 0.0012345 will be lost and this error and errors from heights of the other students will propagate when calculating the average height of the class. Let’s say the calculator reports 5.89440291 as an average. Now, can we still argue that 5.89440291 is more accurate than 5.9 or 5.89?

I think the scores from judges are more like the height than like the number of apples. I think judges give a score of 7.25 to a performance that values scores ranging from 7.13 to 7.37 as a height meter reports 6.1 for a height ranging from 6.05 to 6.14. Since judges must score not with real numbers but with quarter-based points, they must choose one of the scores that is close to what they really think.

Following is the introduction of significance arithmetic in Wikipedia, and short summary is written in my previous post.
http://en.wikipedia.org/wiki/Significance_arithmetic

I think there are two separate questions going on here, but they are basically the same.

Question #1. The program component scores in the men’s LP are doubled (for the ladies LP they are multiplied by 1.6). Should we double them before or after we round? The ISU method says round first, then double – but that method also doubles the rounding error.

Question #2. In general, should we round a bunch of numbers and then add, or should we add first and then round the answer? The principle is the same – As Silver Blades, Rallycairnand Kypma say, if you round first then when you add you run the risk that all the little rounding errors will snowball in the sum.

But Love Skate raised the point that if the original numbers are accurate only up to two decimal places from the get-go, then we can’t squeeze any more accuracy out of the statistics no matter what we do.

I think the right way to think about it is something like this.

The scores of the individual judges are not rounded, they are not estimates or measurements. When a judge gives a score of 6.25 this means 6.250000000000000000000000000000000000 as many 0’s as you like. This is the exact number, to infinite precision. When you add these scores the sum is likewise an exact number.

The only time rounding becomes an issue is when you average by dividing by 7. Then you get a repeating decimal (repeating in groups of 6 digits after the first one or two). For instance, the mean (after the random draw and throwing out the highest and lowest) for Lysacek’s Skating Skill’s component was exactly
6.7 857142 857142 857142 856142… There is no mathematical reason to round this number at all. This is Lysacek’s score for skating skills.

Now it doesn’t matter whether we multiply by two first or add first, we get the same exact answer at the end (by the distributive law ) The only reason to round these numbers is so they will fit on the protocol sheet (but a computer can hold them internally, no problem.)

(How did I know that Lysacek’s skating skills decimal will repeat in six-digit groups? Because 7 is a factor of 111,111.

41 is a factor of 11,111, so the decimal expansion of 1/41 repeats in groups of 5: 1/41 = .02439 02439 02439 … )

Anyway, this method would avoid all questions raised by rounding issues, it would be trivial to modify the software to implement it, and the only objection that I can think of is the one brought up by Chris H. Once in a while the numbers on the printed protocal sheets wouldn't appear to add up quite right.

Love_Skate said:

I think the scores from judges are more like the height than like the number of apples. I think judges give a score of 7.25 to a performance that values scores ranging from 7.13 to 7.37 as a height meter reports 6.1 for a height ranging from 6.05 to 6.14. Since judges must score not with real numbers but with quarter-based points, they must choose one of the scores that is close to what they really think.

Click to expand...

Sorry, i didn't see your post before I submitted mine, above.

I guess I my opinion is that judges scores are more like the number of apples. I really don't think the judges say to themselves, that performance was worth 6.32 points, and that's closest to 6.25 so i'll go with 6.25. I think it's more like, here are the grades I can give: A (8.00), A- (7.75), A-- (7.50), B++ (7.50), etc.

I don't think that the judges, who are, after all, passing judgement on the quality of the performance, are doing the same thing as when we hold up a meter stick to measure someone's height.

In fact, that's my real beef about the whole concept of the CoP. It affects to use quantitative measures to evaluate quality.

Just my opinion.

Mathman said:

I really don't think the judges say to themselves, that performance was worth 6.32 points, and that's closest to 6.25 so i'll go with 6.25.

Click to expand...

I don't think so, either. But, what I think is that judges can think one skater was slightly better than the other, but the difference was small enough to give them the same score of 6.25. It is like 95 and 96 get the same A+. In that sense, I think the scores are given as when the height is measured. One can be slightly above 6.0, the other slightly below 6.0, and both are measured 6.0.

Indeed, it could be a very different story, depending on how we think of the scores. estimate? or exact number?

As Mathman noted, the numbers that go into the scoring calculation are exact numbers and essentially have infinite significant digits. In that sense, all digits in the calculation, both intermediate and final result, are significant.

But in addition, there is what is know as a quantization error, since the marks the Judges give are not continuous, but are limited to certain discrete values. There are also measurement errors in the sense that the judges can enter the wrong numbers in error, or forget to apply a rule that requires a specific reduction. The main source of measurement error, however, is the variation in evaluation among the judges for each GoE and PC.

With all of these potential sources of inaccuracy in the calculation, one would still carry several extra digits in the the intermediate calculation. Then, at the end of the calculation the result should be rounded once to the number of significant digits justified by the least precise number (or error) in the calculation.

By quoting final results to two decimal places the ISU is claiming the two decimal places are significant digits. In that case, the correct process would be to carry at least three decimal places in the intermediate calculations. It doesn't hurt to keep more, and when doing the calculation with a computer, generally more are kept. With pencil and paper, it would be labor saving to just keep the three digits.

On the other hand, if you think the errors make the decimal places NOT significant, then the intermediate calculations can be done to 1-2 decimal places, but the final result must be rounded to 0-1 decimal places.

The spread among the judges is so large, my view is that there are no significant decimal places in the final total, so results should be rounded to the nearest whole point. If you did that, though, many skaters would end up tied in events, so the ISU quotes results with unjustified precision, which makes many of the results statistically meaningless.

This is a fundamental problem with the mathematics of IJS that has been present from the begining and remains to be solved.

Love_Skate said:

But, what I think is that judges can think one skater was slightly better than the other, but the difference was small enough to give them the same score of 6.25.

Click to expand...

But I thought the system wasn't originally designed to compare skaters to each other. I thought there was more objectivity pretence to it. I might have been wrong.

Love_Skate said:

But, what I think is that judges can think one skater was slightly better than the other, but the difference was small enough to give them the same score of 6.25...

Click to expand...

Hsuhs said:

But I thought the system wasn't originally designed to compare skaters to each other. I thought there was more objectivity pretence to it.

Click to expand...

In my humble opinion, that is the arrow through the Achilles heel of the CoP. I think that is exactly how the judges think.

I think the judges say to themselves, well, Miko Ando was really good, a little better than Yukari Nakano, who I just gave a 7.25 to, but not quite as good as Mao Asada, who I just gave a 7.75 to.

Hmm...what to do, what to do?

At best, the judges fall back on their experience at other competitions and compare the current performance to others that they have judged and seen judged.

Frankly, i do not see how it could be otherwise, under any judging system. If you ask a judge why he or she gave a higher score to skater A than to skater B, that judge can say, in all honesty and candor, "I liked skater A's performance better." He could even explain why, in quite satisfactory detail.

But if, out of the blue, that same judge watches a single performance in isolation, I don't think he would be able to give a convincing reason why, no, that performance is not worth a 7.75, but then again, yes, it is better than a 7.25.

The more things change the more they stay the same. I do not see the Program Component Scores as being anything different from the old 5.7s and 5.9s -- they are surrogates and mnemonic aids for ordinals.

JMO