You Be the Judge

skatinginbc said:

What I said is an argument in favor of scoring against a set of well-defined criterion. It not only complies with the spirit of sport (Most, if not all sports are scored against criterion) but also makes most sense in terms of measurement practices (Performance-based assessments are usually criterion-based).

Click to expand...

Desirable as this might be, I think the counter-argument is that it is not possible to achieve this in figure skating.

Indeed, we push too hard in this direction only at mortal peril. What would it profit the sport of figure skating to gain the most comprehensive and definitive list of criteria, but lose its soul.

The first words of the Tao Te Ching are (as I am sure you know , "The Tao that can be written is not the true Tao."

In other words, any attempt to confine something within the bounds of syntax and grammar automatically leaves something out -- perhaps the most important part. (In mathematics, this statement can actually be given a formal proof -- it is Goedel's Incompleteness Theorem. [A big private to Seniorita.])

The translation of the Tao te Ching into skating parlance would go something like this: Any attempt to reduce to measurement the true Kwanliness of a skating performance is doomed before we begin. :yes:

Mathman said:

The first words of the Tao Te Ching are (as I am sure you know , "The Tao that can be written is not the true Tao."

Click to expand...

"道可道，非常道，名可名，非常名。無名，天地之始，有名，萬物之母". The literal translation of its first sentence: The Tao (= The way of all things) that can be described is not the universal, eternal Tao. A name that can be assigned is not the universal, eternal name. Nameless is what it was when the sky and the earth first formed. Naming is the mother of all things (= By naming a particular thing, we thus recognize its existence. For instance, English contains so many words for different colors. Without those color labels in Chinese, I didn't grow up with the concept of a fine gradation of color like English speakers do).

Without the 5000 words Laozi used to describe Tao, most of us would probably not even know the existence of such thing called "Tao". Some of us may sense that something is out there. What is it? Nobody can tell. The importance of "有名，萬物之母" (Naming is the mother of all things) shall not be underestimated. The category of "Presentation" in CoP is so broad that different judges may focus on different aspects and overlook others. By naming, the existence of "Transition" becomes obvious so much so that no one can easily ignore.

It is the "Golden Middle" or fine balance between naming (describing) and not naming (not describing) we are searching for. I'm against a strict count of successful features in level rating because it defines skills in such great detail that the skills (Tao) named are no longer the universal skills (Tao). Yet, at the same time, I'm against a category so broad that is basically useless in defining the skills (Tao).

Would it help if COP referenced this book:

http://www.amazon.com/Tao-Pooh-publ...=sr_1_3?s=books&ie=UTF8&qid=1330378814&sr=1-3

dorispulaski said:

Would it help if COP referenced this book:

http://www.amazon.com/Tao-Pooh-publ...=sr_1_3?s=books&ie=UTF8&qid=1330378814&sr=1-3

Click to expand...

"The Tao of Pooh!... in which it is revealed that one of the world's great Taoist masters isn't Chinese..."

A bit off topic: The great Taoist master Laozi was likely not a Han Chinese. Laozi was an honorific title for the founder of Taoism Li Er 李耳, which means "tiger" in several minority languages (揚雄方言: 虎…江, 淮, 南楚之間謂之李耳). Due to possible language infusion, we cannot tell for certain if Li Er's native language was Austro-Asiatic or a branch of Sino-Tibetan such as the Tujia language. Regardless, he was very likely an ethnic minority.

My conclusion from my exploratory "Be the Judge" study:

CoP has been criticized for its disproportional emphasis on technical elements. The statistics based on 2011 Worlds data show that TESs contributed most to a skater's total score and to the variances among them (e.g., men's SP: TES Mean = 37.18, Standard Deviation (SD) = 4.96, while PCS mean = 33.83, SD = 3.91; men's LP: TES mean = 73.06, SD = 11.42, while PCS mean = 69.73, SD = 9.46). This scoring practice contradicts the findings of my exploratory study, in which expert opinions (albeit limited due to insufficient participants) suggested a heavier weight for PCS. It is therefore a legitimate concern about CoP's validity in terms of its relative weight on each category.

Have you looked at other disciplines at 2011 Worlds, or at other events? Maybe next week you could do 2012 Junior Worlds.

One discipline from one competition probably isn't enough of a sample to draw any conclusion one way or another. I wouldn't be surprised if the effects of TES vs. PCS have different trends for men vs. women or juniors vs. seniors.

skatinginbc said:

...the findings of my exploratory study, in which expert opinions (albeit limited due to insufficient participants)...

Click to expand...

That is, due to an insignificant number of participants.

Mathman said:

That is, due to an insignificant number of participants.

Click to expand...

Bad English!!! (English is bad, not me ).

gkelly said:

Have you looked at other disciplines at 2011 Worlds, or at other events? Maybe next week you could do 2012 Junior Worlds. One discipline from one competition probably isn't enough of a sample to draw any conclusion one way or another. I wouldn't be surprised if the effects of TES vs. PCS have different trends for men vs. women or juniors vs. seniors.

Click to expand...

I would not be surprised either. I imagine that in Juniors' or ladies' competitions TES and PCS would be more balanced in weights. Please keep in mind that the performance videos I chose were of elite male skaters. Say, if a panel of experts review the footage and mostly agree that PCS should carry more weight, it is the population of elite male skaters that they are talking about, not the ladies, nor the juniors. Is CoP a valid method in ranking the said population? That's a question needed to be answered by further study (Note: The exploratory study was to identify areas for further research. Due to "insignificant number of participants", I can only identify one question or concern worth further examination.)

skatinginbc said:

The statistics based on 2011 Worlds data show that TESs contributed most to a skater's total score and to the variances among them (e.g., men's SP: TES Mean = 37.18, Standard Deviation (SD) = 4.96, while PCS mean = 33.83, SD = 3.91; men's LP: TES mean = 73.06, SD = 11.42, while PCS mean = 69.73, SD = 9.46).

Click to expand...

This is the way it should be. It means that what the skater's actually put out there on the ice matters more than reputation (which PCS tends to be).

However, I don't agree with your conclusions fully. Just because TES Mean > PCS Mean doesn't contradict a heavier weight for PCS. If I give an exam where you get 60 points for writing your name and then you do 40 questions worth 40 points total, the name writing mean would be 60, and the answer questions mean could be, say, 32. The questions are more outcome determinative but the mean score for writing your name was higher. So mean should not factor in here at all.

Standard deviation matters though But still, I'm not sure exactly how much. First, there is a ceiling on PCS while there isn't on TES. More importantly, the SD for an individual skater's scores between competitions will be greater for TES than PCS because a bombed performances gets you hit harder in TES. So a few great skates and a few bombs are going to throw the TES SD off, where they might not as much in PCS.

gkelly said:

Have you looked at other events?

Click to expand...

2011 GPF (men)
Short TES: Mean = 40.65, SD = 3.35; PCS: Mean: 39.96, SD = 2.90
Long TES: Mean = 82.15, SD = 7.18; PCS: Mean: 81.68, SD = 4.94

drivingmissdaisy said:

the SD for an individual skater's scores between competitions will be greater for TES than PCS because a bombed performances gets you hit harder in TES.

Click to expand...

Is PCS a stabler trait that should result in a smaller variance?
After excluding ""bombed" performances or those with fall(s) from analysis, the 2011 Worlds LP results would be: TES mean 76.38 > PCS mean 71.6 (significant), and TES SD 10.49 > PCS SD 9.07 (insignificant). Or if we count the mandatory deduction as part of PCS, the results would be: TES mean 73.06 > PCS mean 69.15 (significant), and TES SD 11.42 > PCS SD 9.72 (insignificant).

If we look at a small sample (e.g, 2011 GPF), the effect is clear:
After excluding ""bombed" performances, the LP results would be: TES mean 86.38 > PCS mean 82.08, and TES SD 1.30 < PCS SD 3.21. Or if we count the mandatory deduction as part of PCS, the results would be: TES mean 82.15 > PCS mean 81.01, and TES SD 7.18 > PCS SD 4.99.

Indeed, the PCS seems quite stable, but should it be so?

BTW: Javier FERNANDEZ improved his skating skills by hook or by crook less than a year from 6.68 at 2011 Worlds (PCS = 65.78) to 8.21 at 2011 GPF (PCS = 81.40). One can add transitions, hire a better choreographer, and with a fitting program suddenly have a better presentation and interpretation skills. Skating skills, however, are in my opinion a rather stable trait, which requires years of hard work, or am I wrong?

Not been around. but read the thread with interest - though some of the statistical and Taoist philosophy went over my head. Laughed looking at the obo and factored placements. Though I understood it at the time I'm glad to be done with it.

A couple thoughts, people complain about skaters raking up big leads in the SP so it's hard to catch them, which isn't fun to watch. The opposite happens just as often I think - where several skaters are basically tied after the short, only a few points might separate the four skaters ranked 3 - 6, but in an ordinal system the skater ranked 6 is way behind the skater ranked 3. I like that aspect of being almost tied.

I do wish the SP and LP were more distinct in how they measured skills, so the SP was more technical and the LP more artistic (I know that's not a CoP approved word ). Maybe decrease the value of PCS in the SP and increase the value of both GoE and PCS for the LP?

When looking at the relative value of TES vs PCS it is worthwhile looking at how GoE impacts those numbers since that gets at the quality of the skate, not just the quantity and difficulty of the tricks. GoE should also capture thing like creativity and originality, expressing character, and musicality. All of which are PCS type of traits even though they're included in TES

ivy said:

A couple thoughts, people complain about skaters raking up big leads in the SP so it's hard to catch them, which is fun to watch. The opposite happens just as often I think - where several skaters are basically tied after the short, only a few points might separate the four skaters ranked 3 - 6, but in an ordinal system the skater ranked 6 is way behind the skater ranked 3. I like that aspect of being almost tied.

Click to expand...

Me too.
Originally there 14 elements in the senior men's LP (8 jump elements) and 13 (7 jump passes) for senior ladies, and 8 elements (3 jump passes) in the SP. So not quite twice as many total elements in the longer program, but more than twice as many of the highest value, most volatile kinds of elements.

Did the variance tend to be twice as great in the LP? It should be, more or less, if the LP is supposed to contribute twice as much to the results.

Then they took out one of the spin in the LP, reducing ratio of total elements between programs to make the SP even more than half. Did that give the SP more value? Or was the extra spin in the LP insignificant in distinguishing skaters in the LP?

Now they've taken one of the sequences out of the SP (and made the corresponding sequence in the LP "choreo" w/o levels) -- does that put the ratio closer back to LP worth twice as much as SP again?

I do wish the SP and LP were more distinct in how they measured skills, so the SP was more technical and the LP more artistic (I know that's not a CoP approved word ). Maybe decrease the value of PCS in the SP and increase the value of both GoE and PCS for the LP?

Click to expand...

I could live with that. Maybe both men and women get the 0.8 PCS factor in the SP and 2.00 factor in the LP?

And both get 13 or even 14 elements in the LP, but the extra element for women doesn't have to be a jump pass and the other extra one for both must not be?

The way I see it, the purposes of the two separate programs should dicate more difference in the kinds of elements required.

The SP should be more about required skills that everyone must do, while allowing skaters who can do more difficult versions of those skills to be rewarded. Maybe the required elements should be even more prescribed for seniors, as they are for juniors and as they were in the 1970s and 80s for seniors (specific jump takeoff for solo jump, specific flying spin, specific
Maybe certain features -- only things that all senior skaters should be expected to do, not flexibility moves for example -- should be required each year in spins and steps, and others not allowed that year.

The LP should have some minimum requirements of expected skills and "well-balanced" distribution of kinds of elements, but primarily it should be an opportunity for skaters to include as many as possible of their own best skills, including unique ones, which may or may not fit SP requirements, and without repeating the same skill too often. It shouldn't be anything-goes like before the Zayak rule and other restrictions, but there should be a lot more flexibility than there is now. Free the free program!

GoE should also capture thing like creativity and originality, expressing character, and musicality. All of which are PCS type of traits even though they're included in TES

Click to expand...

Yes, those are listed among the possible bullet points for positive GOE. In theory they can also add to the GOE of technically flawed elements before the reductions pulling the score back down. E.g., a step sequence that shows great originality and expression of the music and program theme, and also generally good speed and edges, could start out at +2, and then if there's also a visible stumble the final GOE might end up at 0 or even +1 instead of in the negatives.

How many judges actually apply those bullet points to successful much less flawed elements is probably inconsistent at this point.

skatinginbc said:

No. I meant: If that is the case, I will secretly discard your data. The differences among their SP performances are so obvious that any valid scoring system should produce the same result.

Click to expand...

:thumbsup:

In school I once took a test and one of the questions was, "Do you make all your own clothes and shoes?" I said yes, and for some reason my data was systematically excluded from the study.

ivy said:

I do wish the SP and LP were more distinct in how they measured skills, so the SP was more technical and the LP more artistic (I know that's not a CoP approved word ).

Click to expand...

I think it should be the opposite. Let the long program stay as it is -- jump, jump, jump -- and let the skaters use the short program to express themselves artistically.

Did the variance tend to be twice as great in the LP? It should be, more or less, if the LP is supposed to contribute twice as much to the results.

Click to expand...

Maybe not if the SP data included all skaters at Worlds, while the LP was just the skaters who qualified to skate the LP.

Mathman said:

I think it should be the opposite. Let the long program stay as it is -- jump, jump, jump -- and let the skaters use the short program to express themselves artistically.

Click to expand...

Why?

I can understand if you want to have two totally separate one-program competitions, one for showing off as much technical prowess as possible and the other for expressing artistry, that the artistry programs could easily be shorter.

Or not. With less athletically demanding programs, the skaters would have the physical stamina to show their artistry for a longer period of time. They would also have the option of choreographing in rest periods.

But historically the short program has served some specific purposes:
1) demonstration of specific required technical skills
2) seeding the skate order of the free program
3) in large events, cutting the field to a manageable size for the free program

For all of those purposes, it makes no sense to use a program that gives more weight to artistry than to technique.
For 2) and 3), the program in question has to come earlier in the competition sequence than the free program.
For 3), it makes sense to be shorter and take less time per skater because there are more skaters

So when you say "short program" do you mean "a program that only lasts about 3 minutes instead of 4 or 4 1/2"?
Or do you mean "the first phase of a two-phase competition"?
Clearly you don't mean "the technical program" which is what it was called for a few years in the 1990s

Did the variance tend to be twice as great in the LP? It should be, more or less, if the LP is supposed to contribute twice as much to the results.

Click to expand...

Maybe not if the SP data included all skaters at Worlds, while the LP was just the skaters who qualified to skate the LP.

Click to expand...

Probably true, since the skaters who did not qualify for the LP could have included those who completely missed a majority of the 7-8 elements (e.g., all the weaker senior ladies who failed to rotate any of their jumps the 2.5-3 rotations required by senior SP rules), or negative GOE on more than that, whereas those who qualified likely earned base value or better on the majority.

As a measure of the structural difference in variance between SP and LP, maybe look only at smaller events where no one got cut, or only at SPs of the skaters who made the cut for the LP.

gkelly said:

Why?

...But historically the short program has served some specific purposes:

1) demonstration of specific required technical skills

2) seeding the skate order of the free program

3) in large events, cutting the field to a manageable size for the free program

Click to expand...

Yeah, that's true. These are all desirable goals for what the short program should try to accomplish. I withdraw my suggestion.

Always in these discussions I am thinking about it more from an audience perspective than a competitive one. I just really like short programs. Even though it is supposedly the "technical" program, still the skaters are not overburdened with requirements. I have seen many, many SPs that were little gems of choreography and movement. They have a beginning, a middle, and an end. They include big jumps and thrilling spins, but as punctuation marks serving the purpose of the whole. (Bring back the level one blur spin as a requirement to end every program. )

Plus, they are short.

To me, long programs are moving in the direction of a marathon of stamina. Higher, faster. stronger. But the audience needs to stop and catch its breath just like the skaters do. At about the three-and-a-half minute mark I find myself saying, is that same guy still out there?

***

And another thing! In ice dance is it really necessary for every blessed waltz, rhumba and polka to stop in the middle so the dancers can skate down the middle of the ice twizzling?

Yes. There are 2 necessities:

1. Before COP, skaters routinely missed up the one and two turn twizzles in CD's like the Ravensburger Waltz and the Golden Waltz. People were tired of it. Now they can all do one and two turn twizzles.
2. It assures that both member of a team can rotate clockwise and counterclockwise. Before COP, you had teams like Lang and Tchernyshev that pretty much could only rotate one way.

Consider it a school figure of sorts for ice dancers.

What I wish they would lose is the holding your foot style of twizzles, which are ugly.

Mathman said:

I withdraw my suggestion.

Click to expand...

I like your suggestion!!! Short programs in my view have gradually turned into the artistic programs during the past decade and long programs into drawn-out technical programs. The short is my favorite part: well-balanced, just the right length....
I have a crazy, really crazy idea: Get rid of the long and have TWO short programs instead, one under the current format with required elements, and the other free with restrictions on the maximum number of jumps, spins, etc.

And I have an even crazier idea: To shortlist the competitors into 24, we can have a technical competition first:
(1) Every skater skates the same straight line footwork from one end of the rink to the other. It is scored on quality (e.g., clarity, accuracy, etc.) as well as speed (timed with a watch).
(2) Every skater performs 3 solo jumps (one axel, one edge jump, one toe jump) and 2 spins (one solo, one in combination) in one minute. They can retry a failed element as long as it is within the time limit.

Ain't I crazy? Anyway, my point is: I like SPs. They are artistic.

I wouldn't have a problem with the LP being more technical, the SP more about 'artistry', but then I'd want the LP to be skated first with maybe 2/3 points coming from TES and then SP second with PCS much more heavily weighted. That would give a broader technical test to the skaters in the LP and while freeing them from a lot of requirements for their artistic SP skate.

OTOH I'm attracted to gkelly's idea of freeing the free program. Let make some bumper stickers and T-shirts! I would worry about skaters with lots of tricks, but little finesse just steam rolling over the rest. Maybe a maximum number for base value? Then skaters and coaches could figure out what skills to use to get there. I'd be happy to see spins bv's raised to give skates a broader range of high value elements to chose from.

ivy said:

OTOH I'm attracted to gkelly's idea of freeing the free program.

Click to expand...

I'm afraid it would turn into an exhibition contest. For some reason, the EX programs are the most boring and the least artistic for my taste.

skatinginbc said:

I'm afraid it would turn into an exhibition contest. For some reason, the EX programs are the most boring and the least artistic for my taste.

Click to expand...

It's funny that we worry about the opposite with a free free - I worry about someone doing 2 or 3 quads, 3 lutzes, 3 triple axels and a couple flips and loops, and having poor spins, footwork then walking away with the title.

I also tend not to like exhibition skates and rarely enjoy show skating or pro competitions, with exceptions of course. There is a sameness that CoP creates in programs. Giving guys more way to come up with 80 points of base value might make for some interesting programs. Though maybe the sameness makes it easier to judges apple against apples.