Predicting the PCSs from the TESs

Hsuhs said:

I mean that skaters have different skill levels to begin with, and a different number of skaters of different levels participates in different events.

Click to expand...

Hsuhs: So your questions are ... can Mathman's equations be used to compare the results of one event to the results of another event? And why would we expect similar averages or ranges of scores across different competitions anyway?

In reverse order ... we don't expect such similarity. Some days the ice is slippery. Some days Brian Joubert nails 3 quads in one program.

Mathman -- I think the question is whether one can test homogeniety of variance across competitions. If I understand Hsuhs's question correctly, the answer is: yes, but you would have to do between-groups ANOVA. The correlation coefficient by itself only shows the association of TES with PCS within a single subject. And again, we would not expect homogeneity anyway.

But I'm just an English major converting to public health in my middle age. What say you?

Susan

I wonder what the correlation would be if there were different panels judging elements and components.

Ptichka said:

I wonder what the correlation would be if there were different panels judging elements and components.

Click to expand...

I predict it would be essentially the same, since the TES comes mainly from the base values.

decker said:

In reverse order ... we don't expect such similarity. Some days the ice is slippery. Some days Brian Joubert nails 3 quads in one program.

Mathman -- I think the question is whether one can test homogeniety of variance across competitions. If I understand Hsuhs's question correctly, the answer is: yes, but you would have to do between-groups ANOVA. The correlation coefficient by itself only shows the association of TES with PCS within a single subject. And again, we would not expect homogeneity anyway.

Click to expand...

Regardless of whether the skaters have a good day or a bad day, when you average out over many skaters what you get is a band of PCS as a function of TES. For any given competition some skaters will be having good days and some bad. It all averages out, so even over many competitions the relation for a given discipline and division holds over multiple competitions.

In the U.S. the test rules and the well balanced program rules result in all the skaters in a given division nationwide spanning a fairly well defined range of the TES, PCS scoring space. Also for a given division the judges have guidelines for what the typical skater in a division should be accomplishing in terms of PSC.

As a result, in the recent qualifying competitions, panels nationwide for each division agree very well in terms of the range of TES and PCS marked. The exceptions are cases where the range of TES might agree for several panels, but one panel marks PCS systematically low or high. I interpret this as an anomaly for the outlier panel, where that particular panel of judges did not mark the PCs to the same standard as everyone else -- and therefore in need correction.

Ptichka said:

I wonder what the correlation would be if there were different panels judging elements and components.

Click to expand...

You are on to what I am thinking, Ptichka. According to the total scores in Pairs, Keaunu and Rockne would be 8th if it came to a tie. So different Panels of Judges bring in different GoE scores and when adding them up just may knock out a better team when breaking a tie in the 2nd round of scoring. With all due respect to the judges, each one has the option to plus or minus GoEs from whatever they see in whatever panel.

As to the carry-over of Tech pts to PCS pts, it is similar to the nature of the panels as above, imo

However, Much credit for MM in working out such percentages. I would be interested to see what works in the NHK Pairs, since an unseeded team won over a seeded team.

Just trying to understand.

Joe

decker said:

Hsuhs: So your questions are ... can Mathman's equations be used to compare the results of one event to the results of another event? And why would we expect similar averages or ranges of scores across different competitions anyway?

In reverse order ... we don't expect such similarity. Some days the ice is slippery. Some days Brian Joubert nails 3 quads in one program.

Click to expand...

I was thinking about possible practical implications. There will be different equations (or equations with different numbers) for every single event. So what exactly is there to compare? The correlations between PCSs and TESs in the ladies LP at 2007 Worlds range from .76 to .79. What if these correlations for the same event in 2008 range from .4 to .5? Can we conclude that the judges are now finally starting to separate two sets of marks and use PCS the way they schould be used and not just for placing? No. Can the next year's PCSs be predicted from the last year's ones? No.

I do understand that we compare oranges to oranges, but the oranges are not all the same. Could someone explain what we have here?

Mathman said:

For most skaters, the equations should work pretty well in predicting the PCSs in that particular event.

There is a statistic called the coefficient of correlation that addresses the question of how acurate these predictions are likely to be. For the data that I used here (2007 Worlds LP, men and ladies separately), these coefficients all turned out to be about 75%.

That's not bad. If it were 100%, then each prediction would be absolutely accurate. If it were 0, the predictions would be totally worthless.

How to figure it out? This is a topic in the general subject of "curve fitting." If you have a bunch of data points scattered over a piece of graph paper, try to draw a standard type of curve which matches the data points as closely as possible. The most usual model is a straight line. This is formula #1. Formula #1 is the equation of the straight line (called the least squares regression line) that matches the data the best.

The others are different kinds of curves. For instance, formula #3 represents exponential growth. This would be the right model if the top skaters had PCSs through the roof with just a small increase in tech, while the lower level skaters were basically stuck with uniform low PCS scores which did not rise much even when they increased their tech a lot.

Click to expand...

Interesting! :thumbsup:

How many sampling data did you use for your curves to get those parameters?Are they all from 2007 worlds? If so the example score for woman and man are also from 2007 worlds? Then I guess naturally it would be close, weren't they? since they also part of data that you used in your curve to derived those formulas.

Can these formular be applied to the different competetion?

Hsuhs said:

This is what I don't understand. We have events (like CoC) where the majority of the participants are ninth- and eighth-graders, and then there are events (like NHK) with seventh- and sixth-graders for the most part. All these 'students' get the same math test. How come the group GPAs are comparable?

Click to expand...

I think that is a separate question. If our focus is on the relation between the PCS and the TES, it might go like this:

The seveth graders get TES = 50 and PCS = 55. The ninth graders get TES = 60 and PCS = 66. The relation is the same. In each case the PCS is 10% higher than the TES.

But the new question that you raise is even more interesting. Suppose Verner skates against Takahashi, Joubert, and Lambiel. He scores 60 points in TES and finishes fourth.

The next week Verner skates against Kozuka, Preubert, and Othman. He scores 60 points in TES and finishes first.

Will he get higher PCS in the second event than in the first? (I think, yes, very likely he would, especially if he skates last.)

Mathman said:

The seveth graders get TES = 50 and PCS = 55. The ninth graders get TES = 60 and PCS = 66. The relation is the same. In each case the PCS is 10% higher than the TES.

Click to expand...

OK, I get it so far.

Ptichka said:

I wonder what the correlation would be if there were different panels judging elements and components.

Click to expand...

gsrossano said:

I predict it would be essentially the same, since the TES comes mainly from the base values.

Click to expand...

Maybe not, though. It seems possible that the judges are looking hard at the technical elements, for GOE purposes. At the end, when they have to give out scores for choreography, they might still be in the mindset, wow, he did a quad, and a pretty good one.

A panel with nothing to do but look at edges, ice coverage, transitions, musicality, and the overall coherence of the program might feel itself farther removed from the technical elements and give out PSC more independently.

Regardless of whether the skaters have a good day or a bad day, when you average out over many skaters what you get is a band of PCS as a function of TES.

Click to expand...

I was thinking about that when you mentioned that you found a wider spread in PCS for given TES at the lower end than the higher.

Possibly one factor is this. When the top skaters hit all their stuff, they get top TES and, generally speaking, top PCS, too.

But in the lower TES group you have a less homogeneous group. You have good skaters who had a bad day, but deserve relatively high PCS for the overall quality of their skating. And then you have weaker skaters who had a great day technically, but whose presentation skills are not up to par. So you naturally have a wider range of PCS.

Very interesting Mathman!

Have you ever tried doing the TES / PCS equation for ice dance?

I would like to know whether the correlation coefficient is a lot worse for ice dance...my guess would be that it is.

mzheng said:

How many sampling data did you use for your curves to get those parameters? Are they all from 2007 worlds? If so the example score for woman and man are also from 2007 worlds? Then I guess naturally it would be close, weren't they? since they also part of data that you used in your curve to derived those formulas.

Can these formular be applied to the different competetion?

Click to expand...

Yes, the data came from the 24 sets of scores from the ladies LP at 2007, and separately the 24 sets of scores for the men.

Yes, these results apply only to that competition. To find out if they are of any use for other competitions, we would have to look at those other competitons and see. If we used a large enough data base, like every competiton over a several year period, we might be able to come up with something that has overall predictive value, if everything in the sport stays the same. (We can check out GSRossano's article when it's finished, which does include data from many competitions at many levels.)

About the examples, it's not guaranteed that any individual set of scores will match the overall pattern, but "most will be fairly close." (How many are "most" and how close is "fairly close" is what all of this analysis is really about, LOL.)

However, I have to confess. I deliberately chose examples that worked out really well, just to make myself look smarter.

ETA: BTW, there was one interesting difference between the men and the ladies. For the men, on the average, each extra point in technical scores was expected to be accompanied by an increase of .6 points in program component scores.

For the ladies, it was .75.

Mathman said:

Yes, these results apply only to that competition. To find out if they are of any use for other competitions, we would have to look at those other competitons and see. If we used a large enough data base, like every competiton over a several year period, we might be able to come up with something that has overall predictive balue, if everything in the sport stays the same.

I deliberately chose examples that worked out really well

Click to expand...

OK.

I claim the fundamental reason there is an identifiable relation between TES and PCS is that skaters can either skate or they can't. If they can do the elements well they can generally do the PC stuff too, so those skaters get high marks in both on the average. If they can't do the elements well, then those skaters generally also can't do the PC stuff well and they get low marks in both (again averaged over many skaters).

It is in no way remarkable to me that PCS goes up with TES.

That being said, the tightness of the correlation, the spread of the PCS for a given TES, and the coefficients of the fit are also influenced by other things. Such as:

The way the judges have been calibrated, or not, to reward the skaters in a given division for components. (Probably the most significant one in this list.)
The content the skaters are permitted in the program. (Next most important.)
How many skaters are having a good day or bad.
The problem of the judges giving knee jerk high PC scores to the best skaters or name skaters. (Why the spread in PCS is less for skates with high TES???)
The individual variation in skills from one skater to the next (that more or less follows a bell curve) for the particular group of skaters in the event.
And probably other things.

IMO this relation is useful for telling us about the judging process and the consistency of the calibration of the judges. It also is useful for skaters to gauge where they stand relative to their peers in terms of their scores being competitive. I don't see it having much predictive value.

Say a skater has a program choreographed to usually earn 30 TES points. That does not guarantee they will earn a particular PCS. It only says, if they earn less than the number of PCS points in the relation they are not keeping up with their peers, and if they earn more points than the relation they are doing better than their peers. For a given TES, the spread in PCS points is quite large.

I think a good analogy is that taller people also weight more on the average and you can fit a curve to that relation. But if I tell you how tall I am that does not allow you to predict how much I weigh. It only tells you what the average weight is for someone my height. I might actually be a toothpick, or a big blob.

decker said:

Mathman -- I think the question is whether one can test homogeniety of variance across competitions. If I understand Hsuhs's question correctly, the answer is: yes, but you would have to do between-groups ANOVA. The correlation coefficient by itself only shows the association of TES with PCS within a single subject. And again, we would not expect homogeneity anyway.

Click to expand...

The way the theory goes, we would have to statisfy ourselves as to whether the 24 data taken from a single competition comprises a suitably random sample from the entire population of all possible skaters facing all possible panels of judges over all possible circumstances. GSRossano is saying that the results form one competition to another (that is, the relation between the TES and the PCS) are quite similar.

Now I am wondering if the correlation is stronger or weaker under CoP judging than under 6.0?

Mathman said:

Now I am wondering if the correlation is stronger or weaker under CoP judging than under 6.0?

Click to expand...

Just as a guess, I would expect it to be weaker under CoP, just because of the way that the numbers are arrived at.

Mathman - I'm not sure what the intended point of all this is. Is there an over-arching conclusion you are trying to draw, or are you just throwing out some pattern you saw, and that's the end of it? I tutor in math and I've taken stat/probability, so I understand the equations. But first off, you've only given a couple hand-picked examples, which isn't really compelling to me. Second, if you ARE correct, how is this anything but an indictment of the entire CoP system? Most of the point of moving to CoP was to "get rid of judging improprieties." If the corruption is indeed still there, to the extent you imply, then not only have we lost the rewarding of artistry, and burdened the skaters, judges and audience with an ever increasing list of technicalities to follow, but gained NOTHING, other than the new, strict observation of some technical requirements.

As for the "mystery" of PCS... I have thought about it before, and here's another probable reason I see behind it. If you look at how many points the skaters are earning through TES in this technical-element-worshipping system, you would see that if scored fairly, it would be very likely for MOST of them to have TES far above PCS. Which would then deepen the perception that CoP has killed artistry. "They get almost nothing for PCS -- obviously it doesn't matter!!" So instead they inflate the PCS (perhaps as you claim) to convince us that artistry still matters, when it actually doesn't, or at least not nearly as much as it should. That may even be a bigger crime than favoritism...

There is no intended point to this, nor (as far as I can see) any overarching conclusion to be drawn. Certainly there is no implication of corruption or impropriety in these numbers.

Actually, I expected the correlation between TES and PCS to be higher than it was (about 75%), for the reason that GSRossano mentions above: The best skaters are the best skaters. The only point of the exercise was to investigate, in the case of 2007 worlds, the extent to which the top tech guys also get top PCS, as contrasted with the possibility that there might be a wide variation in PCSs even among skaters with equal technical marks.

As for a suggestion that PCSs are unimportant, we can just as well turn the equations around and use them to predict the technical score, given the program component scores. No one would argue that this proves that technical scores are unimportant!

As a separate issue, however, you are quite right that the CoP deliberately adjusts the program component scores to make them turn out to be in the same ballpark as the technical. For ladies, the PCSs are multiplied by 1.6 and for men by 2.0. This reflects the assumption that that men will achieve about 25% higher tech scores than ladies, so these factors keep each part of the competition in balance with the others.

The ideal would be something like this. Ladies: short program 60, LP tech 60, LP PCS (after the 1.6 factor) 60 = 180 total.

For men, it would be 25% higher: 75+75+75=225.

I don't see this as downplaying the importance of the program component scores, however.

The larger question of whether the CoP is killing artistry -- I don't think that can be answered by looking at numbers.

Mathman said:

Actually, I expected the correlation between TES and PCS to be higher than it was (about 75%)

Click to expand...

In interpreting the correlation, I think it is like porridge. It should neither be too hot (too high) or too cold (too low).

If it were too high, say 1.0 (perfect correlation) that would say the judges were marking the PCS on their impressions of the anticipated TES, and then there would be no purpose in judging the PCS. If it were too low, say 0.0 (no correlation) that would say there is something fundamentally wrong with the requirements of the components and/or the way the judges understand them. Remember, two of the components (SS, TR) are basically technical in nature, and the elements do play into the artistic components as well (PE, CH, IN). So good elements do help the skaters get good component scores.

When I look at the plots of PCS vs. TES for USFSA competition (and I have done a few dozen competitions since the 2005/06 season) they seem reasonable to me in terms of the range of TES the skaters get in each division and the range of PCS you get for a given TES. Yes, there are a few details that look "fishy", but overall they seem to reflect what has happened on the ice.

Mathman said:

you are quite right that the CoP deliberately adjusts the program component scores to make them turn out to be in the same ballpark as the technical... I don't see this as downplaying the importance of the program component scores, however.

Click to expand...

...

DORISPULASKI said:

Very interesting Mathman!

Have you ever tried doing the TES / PCS equation for ice dance?

I would like to know whether the correlation coefficient is a lot worse for ice dance...my guess would be that it is.

Click to expand...

By worse, did you mean higher or lower, LOL? I just did it for the CD, OD, and FD at 2007 worlds. The Pearson coefficient turned out to be much higher than for men (r= .75) and ladies (r=.78).

Compulsory Dance:

PCS = 1.45 x TES - 9.1
r = .98

Original Dance:

PCS = 1.45 x TES - 18.7
r=.89

Free Dance:

PCS = 1.75 x TES - 45
r=.92

The difference betwen dance and singles skating is especially evident when you consider that it is actually r squared that is the primary statistic.

Men LP: r^2 = .56. This means that 56% of the variation in program component scores is correlated with variation in the technical scores, with the remaining 44% reflecting random variation in the sample.

Ladies LP: r^2 = .61.

CD: r^2 = .96. A whopping 96% of the variation in PCS is correlated with variation in TES.

OD: r^2 = .79.

FD: r^2 = .85.

I assume this has something to do with the fact in dance there is almost never an equivalent disaster to a very good skater underrotating and falling on his quad and losing 9 points (which would cause large variation in TES without affecting PCS as much).