Skaters/Judging experts on GS - Question

Interestingly, Patrick received 87.08 and Daisuke 85.58 for their PCS at the 2011 GPF free skate, but if the medians were used instead, Daisuke would have tied with Patrick. This would have been the results:

Patrick CHAN
Skating Skills = 8.75 (8.75 8.75 8.25 9.50 8.75 8.75 8.75 8.50 9.00)(Mean = 8.75)
Transition = 8.75 (7.50 8.75 8.00 9.00 8.50 8.75 9.00 8.50 8.75)(Mean = 8.61)
Performance = 8.50 (9.25 8.00 8.25 8.50 9.00 8.50 8.75 8.50 8.75)(Mean = 8.61)
Choreography = 8.75 (8.75 8.50 8.25 9.00 9.00 8.75 9.25 8.25 9.00)(Mean = 8.75)
Interpretation = 8.75 (9.25 8.75 8.25 8.50 9.00 8.75 9.00 8.50 9.25)(Mean = 8.82)
PCS = (8.75 + 8.75 + 8.50 + 8.75 + 8.75) x 2 = 43.50 x 2 = 87.00

Daisuke TAKAHASHI
Skating Skills = 8.75 (8.00 9.00 8.50 9.00 8.75 8.75 8.75 7.75 8.25)(Mean = 8.57)
Transition = 8.50 (7.50 9.00 8.50 8.75 8.25 8.75 8.25 5.50 8.50)(Mean = 8.36)
Performance = 8.75 (8.00 9.50 8.50 9.00 8.75 9.00 8.00 8.00 9.00)(Mean = 8.61)
Choreography = 8.75 (7.75 9.00 8.25 9.25 9.00 8.75 8.00 6.75 8.75)(Mean = 8.50)
Interpretation = 8.75 (7.75 9.25 8.50 9.50 9.00 8.75 8.50 8.25 9.00)(Mean = 8.75)
PCS = (8.75 + 8.50 + 8.75 + 8.75 + 8.75) x 2 = 43.50 x 2 = 87.00

Which one makes more sense to you? The medians or the trimmed means?

Very interesting. When the mean is different from the median, that is evidence that the distribution is skewed. In this example the differences are so slight as not to raise any red flags. But still we can spot a couple of tendencies.

In the Transitions category, for both skaters the distribution is skewed o the left (the mean is smaller than the median). This suggests that there are one or two or three judges that are scoring by the Joe Inman memo and marking transitions lower as a general rule.

If we compare the two skaters in the three "performance categories," for Chan two of the three means are higher than the medians, with one tie. For Takahashi, it was exactly the opposite: two means were lower and one the same as the medians.

This says that there were one or more "Chan judges" on the panel, who really, really liked Chan better than Takahashi, whereas the typical judge on the panel liked them both about the same overall.

If we look at P&E in isolation, the means are tied but Takahashi wins the median. Thus the typical judge liked Takahashi better, but Chan got a boost in the mean score from one or two judges who scored him well above the consensus.

From a skating perspective, the results for Median seem correct to me. Patrick has better transitions, and Daisuke has better P&E IMO.
When you use the median, would it tend to lessen the inequity that can occur when say there is a judge from one skater's country, but not from the other skater's country, on the panel??

Mathman said:

This says that there were one or more "Chan judges" on the panel, who really, really liked Chan better than Takahashi, whereas the typical judge on the panel liked them both about the same overall.

Click to expand...

Don't forget the competition occurred in Canada. The crowd might have further reinforced their favoritism if they already liked Chan to begin with.

dorispulaski said:

When you use the median, would it tend to lessen the inequity that can occur when say there is a judge from one skater's country, but not from the other skater's country, on the panel??

Click to expand...

I would say so. They tend to be trimmed away if we see the median as the most extreme case of trimming. However, if the minimum gradation of individual judges marks is large, by simply having one judge with national biases on the panel, the balance could be tipped with great consequence (e.g., -3 -2 -3 -2 -3 -2 -2 -3 -3 vs. -2 -2 -3 -2 -3 -2 -2 -3 -3 ==> the medians for the two are a whole point difference). Remember: A whole point difference in a PCS category is huge. It is a crime if Chan ends up having a 7.50 for interpretation instead of 8.50. But somehow we don't care so much about one point difference in GOE. Strange! Isn't it? I think if the medians are used, we have to ensure that all elements and categories have a similar minimum gradation. Otherwise, some categories (e.g., GOEs) may be more sensitive to random errors or luck than the other categories..

dorispulaski said:

From a skating perspective, the results for Median seem correct to me. Patrick has better transitions, and Daisuke has better P&E IMO.

Click to expand...

Skatinginbc has scared me away from talking about "measured sports" versus "judged sports." So let' use this language instead: "in the spirit of CoP" versus "in the spirit of 6.0."

The mean and the median are both, loosely speaking, kinds of "averages" (better, "measures of central tendency"). Here is the difference.

The mean is the average of the judges' scores. (CoP)

The median is the score of the average judge. (6.0)

In CoP the basic unit is the CoP point. In 6.0 the unit is the judge. How many points do you score versus how many judges do you "score." The question consistent with 6.0 would be, what does the typical (average) judge think.

Actually, in 6.0 what we really care about is, what do the majority of judges think. If the majority are in agreement, the number that they agree on is automatically the same as the median. Lacking majority agreement, the median is the next best thing.

If we conclude (somewhat perversely) that the median is the better statistic in CoP, what we are really saying is that the we don't like the whole concept of the Cop all that much after all.

When you use the median, would it tend to lessen the inequity that can occur when say there is a judge from one skater's country, but not from the other skater's country, on the panel??

Click to expand...

One would expect so, but skatinginbc's example for GOEs shows that national bias can have an effect in either system. Again, it is the "point" versus "judge" question. Would you rather a an extra judge on your side (median) or would you rather have a judge that is way, way, way on your side (mean).

Also, do we want a system that starts by assuming that all judges are honest and as impartial as humanly possible, but honest differences of opinion are inevitable in the qualitative areas of judgment that make up so much of what's being judged -- and then after designing a good system for the ideal state affairs builds in additional safeguards to deal with the fact that dishonesty will sometimes occur?

Or one that starts by taking bias and cheating as the baseline and is designed to thwart those tendencies first of all, and then to somehow arrive at some true evaluation of the actual performances through the noise of falsely arrived at scores?

skatinginbc said:

Remember: A whole point difference in a PCS category is huge. It is a crime if Chan ends up having a 7.50 for interpretation instead of 8.50. But somehow we don't care so much about one point difference in GOE. Strange! Isn't it? I think if the medians are used, we have to ensure that all elements and categories have a similar minimum gradation. Otherwise, some categories (e.g., GOEs) may be more sensitive to random errors or luck than the other categories..

Click to expand...

For PCSs the gradation is 0.25. It would not be such a terrible crime if Chan ended up with 8.25 instead of 8.50. However, in the men's long program the PCSs are doubled so this is an effective gradation of 0.5 points.

In contrast, GOEs range from 1 point for quads, to 0.7 points for triple jumps, 0.5 for a double Axel, 0.2 to 0,3 for other double jumps, etc. I think the idea is to scale the GOEs to match up somewhat with the base values.

Spins are 0.5 regardless of level. So for spins, the ISU took the opposite approach. They wanted to give a slight relative pat on the back to skaters who do lower level spins of exceptional quality.

I have to say, whatever criticisms one might bring, that the CoP is far from just thrown together helter-skelter. It is actually quite subtly crafted by people who know a whole lot about the nuances of figure skating.

gkelly said:

Also, do we want a system that starts by assuming that all judges are honest and as impartial as humanly possible, but honest differences of opinion are inevitable in the qualitative areas of judgment that make up so much of what's being judged -- and then after designing a good system for the ideal state affairs builds in additional safeguards to deal with the fact that dishonesty will sometimes occur?

Or one that starts by taking bias and cheating as the baseline and is designed to thwart those tendencies first of all, and then to somehow arrive at some true evaluation of the actual performances through the noise of falsely arrived at scores?

Click to expand...

I think you have no choice but to do the first. If you start off by assuming that everyone is a crook, you can't have a sport at all.

The trimming of the mean, and also the penalties for judging outside the corridor, I suppose fall into the "additional safeguards" category. Although you can still put a good face on it by saying that this is quality control to insure uniformity of judging standards and qualifications, without saying anything about cheaters, misguided patriots, and scoundrels.

skatinginbc said:

How about "flip-flops" under the "median" method? Say, if we use the computer program to standardize each judge's score like some international piano competitions do so that the scores from all judges are in the same distribution (For instance, an 8 in one judge's mind could mean a 7 in the mind of another, or 0.50 point for one judge could mean 0.75 for another. By standardizing their scores, we may correct those systematic variances). And then the adjusted scores, instead of the raw scores, are used to find the medians. As the competition progresses and more data about each judge's score distribution are available, there is a chance that the medians might change and thus "flip-flops" might occur. Will you make a fuss about that?

Click to expand...

I missed this question!

No, I would not make a fuss about that. I would not make a fuss about any partial results announced prematurely before the competition is over, which then are revised when the full set of data becomes known.

Well, I might make a little bit of a fuss if I had to sit through it. "Oh boy, I'm winning! Oh boy, I'm winning! Oh boy, I'm winning...Or darn, I lost."

Mathman said:

If the majority are in agreement, the number that they agree on is automatically the same as the median. Lacking majority agreement, the median is the next best thing.

Click to expand...

If the majority are in disagreement, the median-method can avoid the paradox (rock/paper/scissors) that plagues preferential voting systems.

Mathman said:

If we conclude (somewhat perversely) that the median is the better statistic in CoP, what we are really saying is that the we don't like the whole concept of the Cop all that much after all.

Click to expand...

To me, the main difference between 6.0 and CoP is that one is norm-referenced and the other is criterion-referenced (which is the core spirit of the CoP--about specified criteria: difficulty levels (base marks), execution levels (GOEs) and so on). It is not so much about measures of central tendency (mean, median, mode).

Mathman said:

Would you rather have an extra judge on your side (median) or would you rather have a judge that is way, way, way on your side (mean).

Click to expand...

The true meaning of "judges' pet" is not that most judges rate a skater high, but that a minority of judges rate that skater way, way high.

Mathman said:

GOEs range from 1 point for quads, to 0.7 points for triple jumps, 0.5 for a double Axel, 0.2 to 0,3 for other double jumps...

Click to expand...

Patrick's 4T+2T GOEs: -3 -2 -2 -2 -2 -3 -3 -3 -2 (Median = -2, Mean = -2.43)
Daisuke's 4T GOEs: -3 -2 -3 -2 -3 -2 -2 -3 -3 (Median = -3, Mean = -2.57)
Remember: As suggested in post #142, there seemed to be one or more "Chan judges" on the panel based on the analysis of the PCSs, and it was a very close competition (Chan 173.67 vs Dai 172.63 in FP at the 2011 GFP). If the medians were used instead, Patrick would have won the free skate largely due to the one-point difference in that GOE alone.

I have tested with the median-method on some close competitions of the past (e.g., 2011 Cup of China), and I don't like what I saw: The GOE medians were so sensitive to "luck", almost like tossing a coin with limited trials to determine "head" or "tail". In cases where the judges' scores look like almost binomial and the mean (an estimate of the expected value) is close to the middle point (e..g, Patrick's -2.43 and Dai's -2.57), a gray area apparently exists and the median would have fallen around the middle point should the minimum gradation of scores allow enough freedom. The rating method for the TOFEL writing test, for example, is in principle a method of finding the median. But when the 1st and the 2nd judges disagree with each other by one level, the 3rd judge has the option of assigning a middle grade if he believes it falls into a gray area. Thus a grade of 4.5 occasionally happens instead of the typical 4 or 5.

My position: If the median is used, a gray area score (the middle point) should be allowed in GOEs/difficulty levels for high-scoring jumps/elements so as to maintain a similar minimum gradation in both TES and PCS. One full point gradation in a high-level jump is too large.

VIRTUE/MOIR received 57.68 and DAVIS/WHITE 57.95 for their PCS at the 2011 GPF free skate. If the medians were used instead, VIRTUE/MOIR would have won the free and this would have been the results:

Tessa VIRTUE / Scott MOIR
Skating Skills = 9.50 (9.75 9.50 9.50 9.50 9.75 9.50 9.75 9.00 9.75)(Mean = 9.61)
Transition = 9.50 (9.50 9.00 8.75 9.75 9.25 9.75 9.75 8.75 9.50)(Mean =9.36 )
Performance = 9.75 (9.75 9.25 9.25 10.00 10.00 9.75 9.75 9.50 9.75)(Mean = 9.68)
Choreography = 9.75 (9.75 9.50 9.25 10.00 9.75 10.00 10.00 9.75 9.75)(Mean = 9.79)
Interpretation = 10.00 (10.00 9.50 9.50 9.75 10.00 10.00 10.00 9.50 10.00)(Mean =9.82 )
PCS = (9.50 x 1.25 + 9.50 x 1.75 + 9.75 + 9.75 + 10.00) = 58.00

Meryl DAVIS / Charlie WHITE
Skating Skills = 9.50 (9.50 10.00 9.75 9.25 9.75 9.50 9.50 9.50 9.50)(Mean = 9.57)
Transition = 9.50 (9.25 9.50 9.75 9.25 9.75 9.75 9.25 9.75 9.25)(Mean = 9.50)
Performance = 9.75 (9.50 10.00 10.00 9.50 10.00 9.50 9.75 9.50 10.00)(Mean = 9.75)
Choreography = 9.75 (9.25 10.00 10.00 9.50 9.75 9.75 10.00 9.75 9.75)(Mean = 9.79)
Interpretation = 9.75 (9.25 9.75 10.00 10.00 9.75 9.75 10.00 9.50 10.00)(Mean = 9.82)
PCS = (9.50 x 1.25 + 9.50 x 1.75 + 9.75 + 9.75 + 9.75) = 57.75

Which one makes more sense to you? The medians or the means?

Remember Moir said that it was a "piss-off" because he felt like they laid down a good skate. "It's tough to get beat artistically when we feel we're the best artistic ice dance team in the last five years," he said (http://www.goldenskate.com/forum/showthread.php?35640-Scott-Moir-says-its-a-piss-off).

It's interesting, because PCS is assigned by individual judges in increments of 0.25, so when you use the median, you'll of course get differences between skaters/teams in increments of 0.25. Using the mean, the two teams are virtually tied in PCS, and likewise using the median. In fact, the teams are nearly tied in all 5 components, by whichever method you use to look at the PCS scores.

BTW, you have the components labelled incorrectly. In dance, the components are:

Skating Skills
Linking Footwork/Movement
Performance
Choreography
Interpretation/Timing

So the median method, which is in increments of 0.25, has the 2 teams tied in 4 components with V&M 0.25 ahead in interpretation/timing, and so ahead by 0.25 overall
The mean method, which is in increments of 0.01, has D&W slightly ahead in Linking Footwork/Movement and Performance and V&M ahead in Skating Skills, and ahead overall by 57.95 to 57.68, a difference of 0.27.

So clearly Scott Moir has not kept pace with the times, especially considering that he uses the word "artistic" as though that were a criterion under COP-which it isn't. As far as the judges are concerned, the two teams are equal in PCS.

skatinginbc said:

If the median is used, a gray area score (the middle point) should be allowed in GOEs/difficulty levels for high-scoring jumps/elements so as to maintain a similar minimum gradation in both TES and PCS.

Click to expand...

A "median with gray area" is very similar in spirit and effect to the trimmed mean.

I find that the more I try to come up with improvements to the IJS the more I come back to the view that the ISU folks have done a pretty good job with the details of the scoring system after all.

DorisPulaski said:

The mean method, which is in increments of 0.01...

Click to expand...

Actually, in increments of 0.0357 (1/28th of a point). Davis and White's 9.57 for instance is 9 and 16/28th.

DorisPulaski said:

Linking Footwork/Movement

Click to expand...

That is very interesting! I guess in singles skating a "transition" is something you do between two highlight elements like jumps and spins. What does "Linking Footwork" link? Different dance segments?

I also see that this component is regarded as the most important of the five, weighted by 1.75.

skatinginbc said:

Remember Moir said that it was a "piss-off" because he felt like they laid down a good skate. "It's tough to get beat artistically when we feel we're the best artistic ice dance team in the last five years," he said.

Click to expand...

Well, going by Interpretation/Timing alone, five of the nine judges agreed.

It's a shame we can't match up the scores of the different judges. Davis and White got one 9.25 in I/T. Did this judge give Virtue and Moir 9.50 (a stingy judge) or 10.00 (a V&M fan).

That line (I/T for Virtue and Moir) is the test case for whether we like the median or mean better. A clear majority (5 out of 9) gave V&M a score of 10.00. Obviously the mean cannot be that high unless all judges concur. So should we go with the ratings of the majority, or should that score be mitigated to take into account the minority opinion?

Interestingly enough, at 4CC's, which V&M won, D&W won the PCS more convincingly by the median method, mostly due to the weighting of the Linking Footwork/Movements component.

D&W
Median 57.25 Mean 57.25
Skating Skills (9.25 9.50 9.50 9.50 9.50 9.50 9.50 9.75 10.00 ) Median 9.50 Mean 9.54
Linking Footwork/ Movements ( 9.00 9.00 9.25 9.25 9.50 9.50 9.50 9.50 9.75) Median 9.50 Mean 9.36
Performance ( 9.50 9.50 9.50 .50 9.50 9.75 10.00 10.00 10.00) Median 9.50 Mean 9.68
Choreography ( 9.25 9.50 9.50 9.50 9.50 9.75 9.75 9.75 10.00 ) Median 9.50 Mean 9.61
Interpretation & Timing ( 9.25 9.50 9.50 9.50 9.75 9.75 9.75 9.75 10.00 ) Median 9.75 Mean 9.64

V&M
Median 56.75 Mean 57.17
Skating Skills (9.00, 9.25, .9.25, 9.50, 9.50, 9.50, 9.50, 9.75, 9.75) Median 9.50 Mean 9.46
Linking Footwork/Movements (9.00, 9.00, 9.25, 9.25, 9.25, 9.25, 9.50, 9.75 ,9.75 ) Median 9.25 Mean 9.32
Performance ( 9.25 , 9.25, 9.50, 9.75, 9.75, 9.75, 10.00, 10.00,10.00 ) Median 9.75 Mean 9.71
Choreography ( 9.50 9.50 9.50 9.50 9.50 9.50 9.75 10.00 10.00 ) Median 9.50 Mean 9.61
Interpretation & Timing ( 9.25 9.50 9.50 9.75 9.75 9.75 9.75 10.00 10.00 ) Median 9.75 Mean 9.71

If you're interested in how the Linking Footwork/Movements component is defined, we discussed it here:

http://www.goldenskate.com/forum/sh...cing-PCS-Rules&p=602071&viewfull=1#post602071

It's funny that in the documentation it is called Transitions & Linking Movements.

And in fact, given the definition of this component, Funny Face should score lower than Der Fledermaus, exactly because the transitional movements of Funny Face are staccato, edgeless steps, done hand in hand or with no hold at all. Such steps do not count tas much oward transitions in dance than steps done in hold, with clear edges. This is what V&M tried to change between GPF & 4CC's, and apparently the judges were still not quite as happy with their transitions/linking movements than D&W's.

dorispulaski said:

And in fact, given the definition of this component, Funny Face should score lower than Der Fledermaus, exactly because the transitional movements of Funny Face are staccato, edgeless steps, done hand in hand or with no hold at all. Such steps do not count tas much oward transitions in dance than steps done in hold, with clear edges. This is what V&M tried to change between GPF & 4CC's, and apparently the judges were still not quite as happy with their transitions/linking movements than D&W's.

Click to expand...

Do you think that this aspect of the scoring system could have an adverse effect on originality and expression? Maybe Virtue and Moir (and their choreographer) felt that a certain kind of linking step is more Fred and Ginger, but they had to change it to get more CoP points?

Mathman said:

A "median with gray area"...

Click to expand...

Let me try again. Say, if the increment of GOE is 6 points, so large that the judges are forced to give either -3 (fail) or +3 (succeed). The median-method is OK if all we care about is whether the majority of judges think that a skater fails or succeeds the required element. But if we add the score to categories of different increments/levels (e.g, 0 very poor, 2.5 poor, 5 average, 7.5 good, 10 very good), the interpretation of the total score becomes problematic. It becomes an arbitrary score that one has a hard time to define clearly. Adding medians is meaningful only when interpretation of the scores is consistent across categories. The skater with the highest total score in the -3 vs +3 scoring is one that "succeeds" in most elements. Similarly, adding Program Component scores together is fine because all categories have an identical increment despite their differences in weights (e.g., Skating Skills = x 1.25). Weight is weight, meaning different emphasis on individual categories. Still, each category is judged on the same increment of 0.25, apparently set on the basis that it is the minimum gradation that a judge is capable of differentiating under a holistic approach. My point is: GOE medians and PCS medians cannot be added unless we make changes to their increments so that they are identical in size and interpretation.

Say, if the ISU adopts the following point system:

Total score = (Element D Score x Element E score) + (Skating Skill x Presentation)
Element D Score = the total base marks of all executed elements in increments of 0.25.
Element E score = the total quality scores of all executed elements in increments of 0.25 (e.g., one fall = -1, two falls = -2, two-foot without disrupting the flow = -0.25, two-foot causing a momentarily jerk-like disruption = -0.50, etc.).
Skating Skill = footwork, linking movements and the overall skating skill level and quality judged analytically in increments of 0.25.
Presentation = the overall performance, choreography and interpretation of the program judged holistically in increments of 0.25.
Note: One can have a perfect performance (say 10 out of 10) but not on the ice (skating skill = 0). The PCS for that wonderful performance would be zero.

Since the increment is 0.25 across all categories. The median-method can be used for all categories.

What do you think?

^ Do you really mean to multiply the D and E scores? Couldn't the E score be negative?

Are you proposing to abandon the fixed scale for elements (Triple Lutz = 6.0, triple filp - 5.3, etc.), and instead have each judge give just a total sore for how difficult he thought the whole program was in terms of elements performed?

Am I understanding this proposal right, that essentially the idea is to scap the CoP, have a 6.0-type first mark and second mark for total elements and then again for total program? And the only thing we would be taking the median or mean of is the final combination of these scores over all judges? (No tech panel, right?)

This discussion has been going on so long that now I have forgotten exactly what perceived flaw in the CoP it is that we are trying to fix.

dorispulaski said:

It's interesting, because PCS is assigned by individual judges in increments of 0.25, so when you use the median, you'll of course get differences between skaters/teams in increments of 0.25. Using the mean, the two teams are virtually tied in PCS, and likewise using the median. In fact, the teams are nearly tied in all 5 components, by whichever method you use to look at the PCS scores.

Click to expand...

When the distance between two skaters' performances is clear, the two methods produce an identical result. Our question is: When the judges have to split hairs between two performances, which method is better?
T&M GPF Interpretation = (10.00, 9.50, 9.50, 9.75, 10.00, 10.00, 10.00, 9.50, 10.00)(Median = 10.00, Mean = 9.81, SD = 0.24, Trimmed Mean = 9.82)
D&W GPF Interpretation = (9.25, 9.75, 10.00, 10.00, 9.75, 9.75, 10.00, 9.50, 10.00)(Median = 9.75, Mean = 9.78, SD = 0.26, Trimmed Mean = 9.82)

T&M 4CC Linking Footwork/Movements (9.00, 9.00, 9.25, 9.25, 9.25, 9.25, 9.50, 9.75, 9.75)(Median = 9.25, Mean = 9.33, SD = 0.28, Trimmed Mean = 9.32)
D&W 4CC Linking Footwork/ Movements = (9.00, 9.00, 9.25, 9.25, 9.50, 9.50, 9.50, 9.50, 9.75)(Median = 9.50, Mean = 9.36, SD = 0.25, Trimmed Mean = 9.36)

If T&M's 0.25 median deficit in 4CC Linking Footwork can be justified with this:

And in fact, given the definition of this component, Funny Face should score lower than Der Fledermaus, exactly because the transitional movements of Funny Face are staccato, edgeless steps, done hand in hand or with no hold at all. Such steps do not count tas much oward transitions in dance than steps done in hold, with clear edges. This is what V&M tried to change between GPF & 4CC's, and apparently the judges were still not quite as happy with their transitions/linking movements than D&W's.

Click to expand...

Can D&W's 0.25 median deficit in GPF Interpretation be justified from the skating perspective?

OK this post talks about Interpretation & Timing:

http://www.goldenskate.com/forum/sh...ules-2011-2012&p=626496&viewfull=1#post626496

The Timing part can be ruled out, at least as far as the rules are concerned, because any PCS component grade between 9.0 and 10.0 requires 100% perfect timing as prerequisite for the range.

One of the words used is "effortless". D&W's GPF performance was indeed effortless, which the last lift and final posing sections of their 4CC's performance showed the effects of Charlie being "gassed".

This also is the section where the relationship between the partners is a criterion:

•Relationship between the partners reflecting the character and content of the music

Click to expand...

It's important to take account of the first part of the post, of how grading is done, however:

From Communication 1677:

The mark for each Program Component is established at a certain degree according to the majority of Characteristics of Program Component which are met. This mark is further adjusted, if necessary, according to the Adjustments to Program Component.

Click to expand...

So perhaps a judge could perhaps fairly take a small deduction here? If D&W didn't meet the relationship criterion at GPF, they strongly met all the other criteria. However, here, you could consider the not so effortless stuff and drop the mark a bit.