- Joined
- Jul 11, 2003
The bloc judging is cultural - not political - not conspiritorily. there are more slavic countries judging than there are celtic judges
Joe
Joe
Joesitz, Don't you think that the sociological foundation (theoretical) of bloc judging is irrelevant to its impact on skating results (empirical)? If judges are biased, showing favoritism -- these issues have to be addressed.The bloc judging is cultural - not political - not conspiritorily.
Joesitz,Joesitz said:But my argument about cultural bias does exis, imo, and the slavic world has the edge on this form of non-intential bias. That was the reason for so many complaints about the old system.
I'm curious how you'd explain what criteria specifically are involved in cultural bias (non-skating-related judgments) that discriminate two skaters such as Michelle Kwan and Irina Slutskaya in the eyes of the judges. For example, skating-related factors that sometimes put Slutskaya rightly ahead of Kwan were speed, spins, and jumps. But what cultural aspects of these skaters' skating would you say made one skater more favorable to Eastern European judges? I felt that Kwan's nuanced skating and choreography to Scheherazade were much more in the classical European style than Slutskaya's Tosca. Aside from the fact that Slutskaya is from Eastern Europe, how would the judges "unwittingly" identify more with Slutskaya's skating than with Kwan's skating?There was no conspiracy here. It was a matter of communal taste in art (and sport).
Mathman, I haven't forgotten. Just only so much at a time that we can deal with, and very little it feels like we can do (as spectators).Speedy has thrown out so many red herrings, statistical and otherwise, that we have forgotten that this was supposed to have something to do with corruption, scandal and pre-judging via deal-making in smoke-filled back rooms.
I agree. And I appreciate your post. But I think you said some interesting things about the CoP that I want to address, because I find them irreconcilable:I think these debates mostly serve to deflect attention from the real issues, namely, doing everything we can to catch cheaters and banning them for life when we do.
If I understand you correctly, we both agree that the CoP is inadequate. We both agree that a "true score" in skating cannot possibly exist. The only thing we can hope to achieve with some certainty is to rank the skaters. In effect, this is the first objective of both the ordinal and CoP systems. A secondary goal of the CoP is to give a meaningful score for each skater. Ironically, the secondary goal is the one being hyped by the ISU, but it is the first goal where the CoP is severely deficient. Deficient because results based on these scores -- of dubious value and meaning -- will themselves be dubious.But I have a little bit of a problem with the sampling theory model anyway. This depends on a tacit assumption that there is somehow a "true score" for every performance, that this "true score" is in principle quantifiable (as being the mean score of the hypothetical population of all possible qualified judges past, present or future, for instance), and that we can then treat the judges' panel as a sample of this population (like a political poll in trying to predict the outcome of national election).
...
But I am not convinced that this mythical "true score" actually exists. As we all admit, skating is subjective. In sober reality, the only thing that counts is, do these 9 judges like your performance of not. Thus the voting panel is the entire population, and everything that we learned in our statistics classes goes out the window.
It is only true that a random draw is insignificant when we are talking about the ordinal system. And first, we have to bar the issue of bloc judging, which is a separate issue that confounds both systems, and in which case we do often say, "Oh but with a different set of judges, so-and-so skater would have won..." For a discussion of the ordinal system (as for the CoP), we have to assume that all judges in the panel are marking each skater the same, regardless of nationality and using the same standards and criteria. With these standards in place, we have to assume that the results of a judging panel with a 5-4 or 6-3 split are indicative of bloc judging, which is why the majority of judges' placements determines the final results under the ordinal system, and which is why the actual scores themselves don't matter. (Under the CoP, the scores matter, to the detriment of the final placements.) For the ordinal system only, the placements of the 9 judges ARE representative of the larger population and how the population would have judged the same event. Your argument is that a sample of a sample will be representative of the population, such that a sample of 5 marbles out of a sample of 9 marbles out of 100 would probably be equal to a sample of 5 marbles out of a population of 100. Only in the ordinal system would this be a correct assumption. Under the CoP system, however, every judge introduces error and variability through each little mark that they generate, therefore each judge can and will individually affect the final outcome regardless of the majority opinion of the panel. Under the ordinal system, 9 judges' placements are compared, but in the CoP system, each judge contributes MULTIPLE scores -- with error -- into a sum total. Because actual scores are being used to compute the results and not ranks, every time you take a random sample of the judges' marks in the CoP, the results of the competition will be different! As we have seen, the GP competitions have been coming down to within hundredths of a point margins. Therefore, random count of judges' marks has a significant impact in the CoP system.One of the points that Dr. Rossano is particularly concerned with is the effect of the random draw. I agree. But this is a public relations issue, not a statistical one. The public says, hey, wait a minute, doesn't this introduce an unwanted element of random chance into the mix? Not really. If you have a sitting panel of, let us say, 9 judges, it does not matter statistically whether or not an additional 5 dummy judges, whose votes are predetermined not to count, are taking up space at the judges' table. It's rather silly, of course, and it does give the public something to howl about. But statistically speaking, choosing 9 judges out of a pool of 14 by computer 15 minutes before the competition starts, has the same effect as if the voting judges had been chosen months in advance by drawing names out of a hat, which is how they did it under the old system. No matter when the judges' draw takes place, or how, skaters face the same probability of obtaining a draw that is favorable to them, or not, for whatever reason.
I think this is the price we pay for living in the real world. No matter how clever we think we are, all of our mathematical and statistical models are very quickly exposed as being just that -- models (in the sense of a model airplane, say) of something that can't really be captured in such a simple fashion. The great Scottish philosopher David Hume staked his reputation on his classic work, A Treatise on Human Nature, in three volumes. The thrust of volume 1, proved in a couple hundred pages of close argument, was that there is no such thing as causality. Volume three, on ethics, begins, no theory of ethics is possible without causality, so forget everything I said in volume 1.But I think you said some interesting things about the CoP that I want to address, because I find them irreconcilable.
No, I don't think so. I think that the same arguments against the reliability of CoP-type scores weigh also against ordinal placements.I took your quote above to mean that it is impossible to sample a population for a true score that doesn't exist. I'm unclear however, if we both agree that we can indeed "treat the judges' panel as a sample of this population" under the ordinal system.
I think that such a sampling distribution would have variations like any other sampling distribution. True, two different panels could give the same rankings, while it would be virtually impossible to achieve a perfect match of CoP scores. But I do not see any reason to think that there would be less variation in one system compared to the other in the final outcomes of the contest if we looked at all 2.7 * 10^21 possible ways of choosing a 9 judge panel out of our 1000 candidates.In the ordinal system, if a different sample of 9 judges out of a population of 1000 was taken multiple times, the system could produce samples with consistent results, i.e. ranking of skaters (but NOT assigning individual scores as in CoP).
This is really the only point that I am fairly confident about. Any statistic whatever, be it mean, median, ordinal ranking or whatever, will have the same sampling distribution no matter how convoluted the process is of selecting the sample, so long as each prospective judge has an equal chance of being included in the final group of nine.It is only true that a random draw is insignificant when we are talking about the ordinal system.
I think that each judge should be judging fairly from one skater to another, but the judges need not agree among themselves. Therefore a 5-4 or 6-3 split may indicate nothing more than a difference of opinion.For a discussion of the ordinal system (as for the CoP), we have to assume that all judges in the panel are marking each skater the same, regardless of nationality and using the same standards and criteria. With these standards in place, we have to assume that the results of a judging panel with a 5-4 or 6-3 split are indicative of bloc judging,
I do not see how this can possibly be asserted? For any statistic whatever which admits of variation within a population, there is always an associated sampling error, by which we understand the difference between the population statistic and the sample statistic. The expected value of the sampling error (the "standard error") is typically easy to quantify for most commonly arising statistics (sigma/sqrt for the mean, sigma*sqrt(pi/2n) for the median, etc., etc.) Non-parametric statistics such as ordinal rankings are certainly not immune to this phenomenon.For the ordinal system only, the placements of the 9 judges ARE representative of the larger population and how the population would have judged the same event.
On the contrary, this is absolutely and incontestably true for any statistic whatever.Your argument is that a sample of a sample will be representative of the population, such that a sample of 5 marbles out of a sample of 9 marbles out of 100 would probably be equal to a sample of 5 marbles out of a population of 100. Only in the ordinal system would this be a correct assumption.
Yes. That's what the judges are there for. To affect the final outcome.Therefore each judge can and will individually affect the final outcome regardless of the majority opinion of the panel.
I hope you mean that "every time you take a random sample (i.e., seat this particular 5-judge panel instead of that one), the total scores under the CoP scoring will be slightly different." If by "results" you mean "who won, who came in second, etc." then of course your underlined and exclamation-pointed sentence cannot possibly be true.Because actual scores are being used to compute the results and not ranks, every time you take a random sample of the judges' marks in the CoP, the results of the competition will be different!
In my previous post I jokingly complimented Cinquanta by calling him a good con man. In fact, however, I don't think he's conning anybody at all. Everybody knows the CoP is crap -- but it's so much fun to roll around in even so!It'd be nice if it were possible, but it's not, and the ISU is misleading people.
Again, you mean that the total scores will be different, not necessarily the final placements, right?...because 20 out of 20 draws of a hat, the competition results will be different.
I guess I'm just more pessimistic by nature. Whenever I see a claim that someone has "eliminated human error" I think first of Murphy's law. I really think that you are way too impressed at the sublime glories of the ordinal system....the majority count should effectively eliminate human error and produce reliable results, consistent results time and again.
I take it that your objection here is to the "touting," not to the method of determining the scores. As you say, the method of determining the "presentation" scores is about the same in both systems....just like the way it was in the ordinal system, except now there are 5 subjective marks instead of 1. These component scores are all as subjective and liable to human error as the single presentation mark in the 6.0 system. The difference is that these 5 subjective marks are added together to form part of a total that is touted to be an objective measure of performance.
That is only partially correct. I think the judges have made an effort to judge Skating Skills (the fist program component) somewhat fairly. Look, for instance, at the Men's SP at Cup of China. There, even though Gao's overall TCS is 9th, his SS is 4th. I know this is an extreme case since the judges probably could not justify saying that a skater with the STRONGEST TES has very poor SS, but I have noticed this in other competitions.This is one of the points about the CoP that statisticians have jumped all over with great glee, because it's easy to run statistical tests about it. So far under the CoP the judges have not made any pretense at figuring out the five program components and the multiple categories within each. If a judge likes a skater, he or she just gives that skater uniformly high marks across the board. Just like the old system, where a skater got just one score which said, OK, that was pretty, or maybe not so pretty.
In a close competition (see Trophee Lalique men's event), random sampling DOES affect the results. Every judge will mark a skater's component scores differently (see http://www.frogsonice.com/skateweb/articles/cop-components.shtml ) with huge amounts of variability between any two judges' scores for each skater. Therefore, the final outcome is completely dependent on the random draw since the final placements are determined by the results of the total scores, which will ALWAYS be different given a different set of judges. I think we both agree on this. But you don't agree that this will affect the final placements.I hope you mean that "every time you take a random sample (i.e., seat this particular 5-judge panel instead of that one), the total scores under the CoP scoring will be slightly different." If by "results" you mean "who won, who came in second, etc." then of course your underlined and exclamation-pointed sentence cannot possibly be true.
moyesii said:final outcome is completely dependent on the random draw since the final placements are determined by the results of the total scores, which will ALWAYS be different given a different set of judges. I think we both agree on this. But you don't agree that this will affect the final placements.
In the ordinal system, it is true that two different samples of 9 judges might produce results like 5-4 and 6-3 in favor of the same skater, but this does NOT affect the placements.
The majority of 1st place ordinals is what counts, and the others are just error.
Also keep in mind that statistically, this is expected since no sample will be an exact representation of the population. The ordinal system is much more robust than the CoP system, because taking random samples from the population of judges has a very likely chance of producing consistent results (outcomes) time and time again, even in a close competition
Under CoP, there is no such a thing as unanimous decisions of the panel, because of the large contributing factor or error and chance into the component marks. The sum of these marks will produce results that no one could have predicted under a valid system.If you look at unanimous or near-unanimous decisions, any valid system should produce the same results.