Time for New US World Selection System? | Golden Skate

Time for New US World Selection System?

willdu

On the Ice
Joined
Nov 13, 2012
Over the past few years and majorly debated over at the 4cc thread between Gold and Gao, the US selection system at worlds has seemed to be take the top two at nationals and be done with it. Whereas the 4cc selection for the US seems to be more flexible. Over the past three years, we've had more medalists at 4CC than at worlds obviously from a competitive standpoint (not all competitors are there), but many times the competitor that medals is not present at worlds (ie Zhang 2012, Nagasu 2011). USFSA needs to re-evaluate its system of selection in order to maximize or world rankings and not base selections for the World team over 1 performance which could be a fluke (both good or bad).

If I had to create a selection system it would be as follows (for all disciplines):

1 berth for the National Champion, they have earned it.

2 and 3rd berth decided as follows:

Scoring:
  • Senior A events such as the Grand Prix are counted
  • Nationals counted
  • 3 point bonus for 1st place finish
  • 2 point bonus for 2nd place finish
  • 1 point bonus for 3rd place finish
  • 3 point bonus for GPF qualification

  • Add total scores and divide by number of competitions thus calculating mean. Calculate standard deviation. Then add bonuses to the mean.
  • Subtract the standard deviation from the mean with bonus.
  • Rank.

Competitors with the highest totals and relatively low standard deviations are selected.

Let's calculate ladies for the past season as our field is rather volatile:

Mean of Ladies in order:
  • Gao: 167.445
  • Zawadzki: 168.87
  • Gold: 171.0566
  • Nagasu: 171.296
  • Wagner:187.4425

Bonuses:
  • Gao: 5
  • Zawadzki: 2
  • Gold: 4
  • Nagasu: 1
  • Wagner:14

Mean + Bonuses in order:
  • Zawadzki: 170.87
  • Nagasu: 172.2966
  • Gao: 172.445
  • Gold: 175.0566
  • Wagner: 201.4425

Standard Deviations/Measure of Range:
  • Zawadzki: 9.8268
  • Nagasu: 6.94
  • Gao: 9.97
  • Gold: 17.835
  • Wagner: 3.801

Mean-Standard Deviation (final Ranking):
  • Gold: 157.2
  • Zawadski:161.04
  • Gao: 162.47
  • Nagasu: 165.35
  • Wagner: 197.64

Based on these set of skaters my world team would be Wagner and Nagasu or Gao.

Interesting facts or observations:
After Wagner, Gold has the highest mean scores of the season followed by Nagasu then Gao. (Not including 4CC). However, she is really harmed by her inconsistency.

Our most consistently scoring skater is by far Wagner despite her falls and then what suprised me was Nagasu came in second in the category. This is what propelled her in front of Gao and Gold.

Based on raw placement and qualification bonuses Gao, Gold and Wagner have the upper hand.

What I like about this system:
  • It takes into account entire seasons worth of work
  • Encourages athletes to be ready for Grand Prix events and score well
  • Takes into account consistency

Criticisms:
  • It does not take into account maximum scoring potential or else Gold would be higher
  • Standardization of scores accross events is hard
  • Some events are stacked compared to others thus making bonuses unfair in some situations

Any comments? Other ideas? I tried to model this after the US Gymnastics trials process where the top competitor is automatically qualified and the rest of the team is up to discretion.
 
Joined
Jun 21, 2003
Cool!

The biggest question I have is the use of the standard deviation. Consistency by itself is not especially to be valued unless you are consistently good.

Interesting, though, how consistent Wagner's scores have been, considering that she has mixed some really good skates in with some bad ones this season.
 

mskater93

Record Breaker
Joined
Oct 22, 2005
Didn't JSF use a similar process which resulted in Nakano NOT going to the Olympics and Ando completely tanking in 2006?
 

FSGMT

Record Breaker
Joined
Sep 10, 2012
Didn't JSF use a similar process which resulted in Nakano NOT going to the Olympics and Ando completely tanking in 2006?
It can happen, but in 2010 Ando was sent to Vancouver instead of Nakano (again) but skated really well and placed 5th (4th in SP), probably higher than what Yukari could have done...
 

b-man

Final Flight
Joined
Jun 25, 2010
Interesting, though, how consistent Wagner's scores have been, considering that she has mixed some really good skates in with some bad ones this season.
That is what happens with skaters with high PCS. You can have a couple of bad skates, and your scores won't go down much.
 

clairecloutier

Final Flight
Joined
Aug 27, 2003
I do think the USFSA should consider a different method of selecting the World teams. We need more flexibility. Placements at Nationals are just not producing the best World teams for us on the ladies' side.

On the other hand, if they could just fix the bizarre judging at Nationals, we wouldn't be having this conversation. You could argue that's the real problem.
 

CanadianSkaterGuy

Record Breaker
Joined
Jan 25, 2013
So this doesn't account for events such as Nationals and 4CC? Another thing is that certain GP events are way harder than others. For example, Gao would have easily placed top 3 at Skate America, but it would have been difficult for her to place top 3 at TEB.

At SA: Wagner 188, Gao 174, Sotnikova 169, Marchei 159
At Rostelcom: Korpi 177, Gold 175, Zawadski 166, Murakami 166
At NHK: Asada 185, Suzuki 185, Nagasu 176, Li 174
At TEB: Wagner 190, Tuktamysheva 179, Lipnitskaia 179, Gao 165

Also, does this bear in mind that Gao was actually a replacement for Lipnitskaia who withdrew at the GPF? I technically wouldn't count her 3-point bonus for having made the GPF (where she performed poorly anyways). I also wonder how the results would look if you dropped each woman's poorest showing (so Gracie's SC skate, Gao's TEB skate, and Mirai at CoC). It would certainly benefit Gold, whose SC performance skews her off the World team. Then again, it wouldn't take into consideration Gao beating her at 4CC either.

They really should do the whole - winner goes to Worlds and the 2nd berth is decided at 4CC. Gao deserves to go, but I think if they skated their bests, Gold has more potential to place higher.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
Criticisms:
  • It does not take into account maximum scoring potential or else Gold would be higher
  • Standardization of scores accross events is hard
  • Some events are stacked compared to others thus making bonuses unfair in some situations

Any comments? Other ideas?

Another criticism or weakness in this system is that it penalizes skaters for not doing well enough last year to get Grand Prix assignments this year, or for being injured or otherwise out of competition in the fall, even if they finish a convincingly strong second at Nationals.

For example, Max Aaron. No Grand Prix events, although he did win the senior B this summer.
As national champion he would still get the spot under your rules. But suppose that he had skated exactly the same as he did, but either Abbott or Miner had done one more successful triple in the long program to hold onto a tiny lead, and the other one completely bombed. Would Aaron have no chance of being selected for Worlds over a distant third- or fourth-place finisher who had had the best GP season this year?

Because there are so many variables, I don't think a strict formula that tries to weight all past results absolutely would be the way to go.

If you're going to consider past results, then it's probably better to build in some flexibility by allowing a committee to take into account previous competitive results without a precise formula so that they can also take into account factors like injuries, strength of field, etc. Of course, flexibility also leaves room for disagreements.

If you're going to go with a strict formula, no exceptions, then why not just go with straight Nationals results?
 

mskater93

Record Breaker
Joined
Oct 22, 2005
Nationals is the one time you face everybody else who's a candidate to go to Worlds with the same judges for all. They get selected to different GPs which have different scoring (at least borderline URs, PCs, and GOEs) and as they say, you can't really compare scores across events (although I suppose you COULD compare BVs and automatic GOE reducers like falls in an equation of some sort). Also, the pressure was completely OFF Gao at 4C - she KNEW this was her last event of the year and it was her last chance to leave an impression on the international judging community for this year, she doesn't have to continue training for "in season" now versus Gold who knew that 4 weeks later she has to be at her absolute best in order to help get 3 spots back. The question becomes "how would she have skated at 4C if she knew that this was part of the selection process for Worlds?".
 

noidont

Final Flight
Joined
Mar 27, 2010
This is way too complicated and could result in major politicking on every stage. I remember some European gymnasts were suing each other for their country's only Olympic spot last year because of some odd point based system. I think sending 2nd, 3rd and 4th placed skaters at Nationals to 4CC and pick whoever finishes first/top 2 is probably a better idea. Or do some sort of score averaging between the two. At least that's what the Russians did with Kovtun and hopefully what the Chinese would do with Han Yan.
The only problem is when National champion wants/needs to go to 4CC. Although technically that would only be a problem if there are three spots...
But all these won't make a real difference if the US judges keep up with their blatant bias toward Asian skaters.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
USFS could use Four Continents as a skate-off for a second or third world team spot IF they rewrite their rules to allow that possibility and then IF they announce right after Nationals that they are going to hold off naming the world team until after 4Cs. As long as everyone knows what to expect.

I don't think it would be a bad idea to give themselves that flexibility.

Some years, especially Olympic years, they wouldn't use it at all. And other years they would only use it for one or more disciplines but not all.
 

Cherryy

On the Ice
Joined
Aug 27, 2012
Would it really have to be so strict? One thing for sure - they could wait with announcing the worlds team after 4cc. I would send the national champion to worlds and the other 1/2 spots could be given after a meeting of USFS's members. They want their best skaters to go so I doubt it would be unfair. They would see who placed best at 4cc and that should remain as the most important factor but consistency and all other competitions would be also considered.
 
Joined
Jun 21, 2003
This sudden emphasis on Four Continents is quite a departure from tradition. There is no guarantee that a skater who does well at Four Continents will do well at Worlds. Any more than there is a guarantee that a skater who does well at Nationals will do well at Worlds. It's a crap shoot any way you go.

I confess that I like the idea of one big shoot-out, winner take all (U.S. Nationals). Taking it out of the skaters' hands and putting it in the hands of "The Committee" just begs for politics and favoritism to trump performance on the ice.
 

clairecloutier

Final Flight
Joined
Aug 27, 2003
This sudden emphasis on Four Continents is quite a departure from tradition. There is no guarantee that a skater who does well at Four Continents will do well at Worlds. Any more than there is a guarantee that a skater who does well at Nationals will do well at Worlds. It's a crap shoot any way you go.

I confess that I like the idea of one big shoot-out, winner take all (U.S. Nationals). Taking it out of the skaters' hands and putting it in the hands of "The Committee" just begs for politics and favoritism to trump performance on the ice.


Until recently, I would have agreed. However, lately, the judging of the ladies' event at Nationals seems nearly as subject to politics and favoritism as any committee possibly could be.
 

mskater93

Record Breaker
Joined
Oct 22, 2005
I want to know everyone's proof of this supposed "politics and favoritism" at Nationals. You'd have to have 9 mathmeticians on the panel...
 
Joined
Jun 21, 2003
Until recently, I would have agreed. However, lately, the judging of the ladies' event at Nationals seems nearly as subject to politics and favoritism as any committee possibly could be.

For myself, I always have the same exact same feeling immediately after Nationals. This year, Caroline Zhang should have won, with Jasmin Saraj second. Booooo!

But then...are the results really all that outlandish? Wagner screwed up big time. Sure enough, she got nailed on TES, scoring 14 points below Gold and also behind Hicks, Gao, Siraj, Cesario, Wang, and Zhang. Despite technical mistakes, Wagner got solid scores in skating skills, transitions, presentation, etc. -- but then again, she was better than the other girls in these areas.

Zawadski was terrible in the free skate -- fairly, she finished 7th behind Gracie, Ashley, Courtney, Christina, Jasmin, and Samantha. She did place above Caroline :cry:, but objectively speaking Caroline has many weaknesses -- she can't get first just because I like her the best! ;)

Christina Gao skated well, but on replay not quite as well as she had in her two Grand Prix events earlier in the season. She deserved -- and got -- higher PCSs than Hicks, but Hicks out-pointed her in TES to take the bronze.

Gold gave an astonishing display of pyrotechnics and deserved a big score. This overcame a disastrous short, but that's how the points added up.

So...I don't know. I am not nearly so outraged as I was (and usually am) on the day of the competition. ;)
 

avalyn

Rinkside
Joined
Nov 12, 2012
It doesn't have to be a big 'conspiracy' like judges meeting beforehand, or somehow rigging the competiton. But in general there might be some favoritism towards certain skaters due to their reputation, potential, or performance during the season. If such skaters mess up, it seems like they are treated more generously or given the benefit of the doubt compared to less favored skaters who mess up. I don't think this is completely outrageous to suggest; I have seen many posters here say, as if it were fact, that Rachael Flatt was favored at one point and scored especially well nationally because of it, and Ashley Wagner said something like "I think my good season might have helped me here" in an interview after she won Nationals.

But like Mathman said, perceptions of unfairness might just be that, perceptions, and sometimes the scores and placements become more clear with time. It doesn't help that I don't think of the official COP program components and things while I'm watching a competition, and I can't recognize underrotated, but seemingly clean, jumps. I tend to weigh skaters' performances by the balance of technical content and presentation in their programs along with cleanliness and consistency in the short and long programs. So it's unsatisfying for me when the podium skates don't match those ideals, like Agnes' multiple errors across both programs, Ashley's major errors in the long program, and Gracie's poor short program (though I thought her long program deserved to make up for it!). Some of the skaters who skated lights out before them or met my ideals better probably just didn't have various things like the best skating skills, the best packaging, the best reputation in their favor, etc.

As for the actual thread, I think it would be a good idea to base the world team on more factors than just Nationals, though I agree flexibility in weighing the factors is better than a strict formula. Waiting for 4CC's results could be just another factor to consider, it doesn't have to become more important than Nationals placements or other considerations.
 

chuckm

Record Breaker
Joined
Aug 31, 2003
Country
United-States
No matter how the US World team is selected, there will always be disgruntled fans who are mad that their favorite skater didn't get the nod.

Most countries use their Nationals as the selection criteria (or did, until the ISU minimums entered the picture). Canada and Russia are some of the only major federations who waffle and add other post-Nationals criteria. I don't see why the US needs to follow their example.
 

gkelly

Record Breaker
Joined
Jul 26, 2003
It doesn't have to be a big 'conspiracy' like judges meeting beforehand, or somehow rigging the competiton. But in general there might be some favoritism towards certain skaters due to their reputation, potential, or performance during the season. If such skaters mess up, it seems like they are treated more generously or given the benefit of the doubt compared to less favored skaters who mess up.

I think there probably is some of this reputation effect -- judges expect certain skaters to do well and they see what they expect to see and score accordingly.

BUT I also think that oftentimes skaters who earn high scores even with mistakes earned those scores on the basis of better skating skills, more difficult and complex programs, better quality on the successfully completed elements, etc., and that judges are rewarding what the skaters actually did on the ice, penalizing the mistakes and rewarding all the good qualities.

But observers who judge the success of a performance primarily by counting the mistakes and counting the jumps landed might disagree that that skater was better in areas that they don't find obvious or of interest. So they assume that the judges are scoring the skater's reputation rather than their actual skating that day.

Since we can't really read the judges' minds, it's hard to tell when the judges were being more accurate than we were at evaluating the overall package of skills demonstrated in this competition and when they were overly influenced by expectations really got this one wrong.

When it comes to comparing a stronger skater with a few obvious mistakes vs. a less strong skater having a good day, there's always room for disagreement as to which should come out ahead. Under 6.0, each judge had to decide how to rank them, and they didn't always agree with each other let alone with the fans. Under IJS, they just have to mark each element and each component and can only guestimate what that will do to the final results.
 

Bentley

Rinkside
Joined
Mar 5, 2012
Until recently, I would have agreed. However, lately, the judging of the ladies' event at Nationals seems nearly as subject to politics and favoritism as any committee possibly could be.

Lately??? It has always been this way. Perhaps you are too young to remember.
 
Top