Well, first I should say that I am not a statistician. I don't think that either Sandra Loosemore or George Rossano is, either. Dr. George Rossano works in aerospace engineering and Dr. Sandra Loosemore, I believe, in the theory of computation. My field is geometry with applications to cosmology. Still, I have taught statistics courses at the undergraduate and graduate levels off and on for quite a number of years.
I am not exactly enamored of Mr. Cinquanta and the ISU, but I do appreciate a good con man when I see one. It's not so much that I am bored with debating the statistical merits of various scoring systems as that I think these debates mostly serve to deflect attention from the real issues, namely, doing everything we can to catch cheaters and banning them for life when we do. Why are people who have been caught red-handed with their fingers in the till still judges in good standing? I think that issues such as regional representation on judging panels and weakening the grip of national federations over the judging process are more important than splitting hairs about how points are tallied up.
One of the points that Dr. Rossano is particularly concerned with is the effect of the random draw. I agree. But this is a public relations issue, not a statistical one. The public says, hey, wait a minute, doesn't this introduce an unwanted element of random chance into the mix? Not really. If you have a sitting panel of, let us say, 9 judges, it does not matter statistically whether or not an additional 5 dummy judges, whose votes are predetermined not to count, are taking up space at the judges' table. It's rather silly, of course, and it does give the public something to howl about. But statistically speaking, choosing 9 judges out of a pool of 14 by computer 15 minutes before the competition starts, has the same effect as if the voting judges had been chosen months in advance by drawing names out of a hat, which is how they did it under the old system. No matter when the judges' draw takes place, or how, skaters face the same probability of obtaining a draw that is favorable to them, or not, for whatever reason.
Rossano and others go on to say that the best thing would be to increase the judges' panel to, say, 25 and count every vote. Of course it would. It would be better yet to increase it to 1000 judges, if we could find that many qualified. But it would be no better or worse first to choose 2000 judges to crowd around the table and then eliminate half of them by a random draw just before the contest started. If we have in mind the paradigm of sampling theory, the only thing that counts is the sample size, along with a guarantee that each judge has an equally likely a priori chance of being selected.
But I have a little bit of a problem with the sampling theory model anyway. This depends on a tacit assumption that there is somehow a "true score" for every performance, that this "true score" is in principle quantifiable (as being the mean score of the hypothetical population of all possible qualified judges past, present or future, for instance), and that we can then treat the judges' panel as a sample of this population (like a political poll in trying to predict the outcome of national election).
Statisticians like this model because for many statistics it has been thoroughly understood for 100 years -- just pop the numbers into the formula and you're done. (Like everyone who likes both numbers and skating I jumped right up after the Nebelhorn competition and ran the numbers through every test I could think of. This was amusing for a while.)
But I am not convinced that this mythical "true score" actually exists. As we all admit, skating is subjective. In sober reality, the only thing that counts is, do these 9 judges like your performance of not. Thus the voting panel is the entire population, and everything that we learned in our statistics classes goes out the window.
Another point at which I disagree with the critics of the CoP is about the whole secrecy issue, and whether we can tell (as easily as before) when individuals and blocs are cheating. Yes, it's true that the public does not know that judge number 4 is Joe Blow from Outer Slobovia. But if the ISU continues to give us all the details of the scoring of each element, it will be easy enough to figure out which judges (by number) were eliminated in the random draw, and then to work up statistical analyses like, "judges 2, 7 and 9 are obviously conspiring to hold down skater A." Much more to the point, IMO, is what the ISU plans to do with this information, if anything.
Anyway, I guess the bottom line is, I don't really have a horse in this race. The mean, trimmed mean, median, ordinals -- none of this can stop people from cheating if they are determined to do so. Like Pollyanna I hold out a foolish hope that somewhere down the line the ISU folks will come to realize that they are killing the sport because the public, which after all pays the bills, think they're a bunch of crooks. Maybe then, when it hits them in the pocketbook, they will actually do something about it rather than just tinkering with the scoring system.
If I had my druthers, I'd rather see the USFSA sign up with the WSF.
PS. Despite this rant, I love this sport! Go AP and Jenny at U.S. Nationals!