What do judges see on their screens? | Golden Skate

What do judges see on their screens?

yupeima

Spectator
Joined
Jan 21, 2019
I am curious about what judges see on their screens. Besides the information on technical elements, do they see any additional details about the skater? For example, in the free program, do judges have access to the short program rankings and scores on their computers? This should not happen, but if judges want to place Skater A above Skater B, how do they manage that without information about the score difference in the short program? Do they simply rely on their memory of the short program scores to make calculations?

Thank you in advance for your input!
 
At Canadian nationals, at some point, I ventured into the restricted area...oups .. I could see the judges screens.
From afar it looked like the element was called in and they had to give GOE on it. There was nothing else I could see.
 
Last edited:
This should not happen, but if judges want to place Skater A above Skater B, how do they manage that without information about the score difference in the short program? Do they simply rely on their memory of the short program scores to make calculations?
I can remember what scores skaters are given at past events, so it wouldn't be hard for a judge who pays closer attention to the sport.

Plus, don't the officials review programs together, if it's felt that a skater is doing something well/badly and not reflected in the scores then it's reviewed in these meetings. This leaves a lot of discretion. Hypothetically, i they wanted to target a skater they would highlight in the meeting what they are doing badly, and just highlight on what someone is doing well if they want to boost their scores while ignoring the negative.

If you're a judge and you want to get invited back, you don't want to be the outlier right?
 
Last edited:
They don't need to see previous results live. They have the same access to public SP results as everybody else has already before the FS. It would be impossible to isolate them so that they could not see any previous results. I also sat behind judges and could see some screens at Finland's GP last time. I did not pay much attention but a couple of times I saw close video about skater's foot. May well have been the same which was shown on TV.
 
I can remember what scores skaters are given at past events, so it wouldn't be hard for a judge who pays closer attention to the sport.

Plus, don't the officials review programs together, if it's felt that a skater is doing something well/badly and not reflected in the scores then it's reviewed in these meetings. This leaves a lot of discretion. Hypothetically, i they wanted to target a skater they would highlight in the meeting what they are doing badly, and just highlight on what someone is doing well if they want to boost their scores while ignoring the negative.

If you're a judge and you want to get invited back, you don't want to be the outlier right?
Thank you so much! This is super helpful information! One quick follow-up: before the removal of anonymous judging in 2016, could ISU identify which judge gave a biased score?
 
As i understand judges watch the skaters live while they're skating and just look at the screen enough to see what goe they're clicking for each element. After the program is over they can replay elements on video. Also after the program they need to check the tech panel calls especially edge and rotation calls for jumps and make sure their goes match what was called. They don't see the level calls for leveled elements.
 
I had a bit of extra time sitting behind the judging panel at JGP Riga in September. They had gotten a slightly revamped interface from last spring and this got me interested in trying to see the possible differences to before (I try to get a seat from the behind the judges if possible, so I have watched them work before many times). I was at a slight distance though, some details I could not see. I did try to take a few photos, but my phone camera was no good and I did not want to go right behind them to photograph...

Each judge has a touch screen plus judging criteria cheat sheets and planned elements lists for each skater as paper printouts. The latter probably also includes spaces for notes for different scores. Many write notes on the during the performances.

The touch screen has the list of elements on the left and feed from the ISU camera on the right. Below there are GOE buttons (red for negative, white for 0, and green for positive) and at the bottom PCS buttons (red, blue yellow for different categories). In addition, buttons for operating the video and finishing judging.

The element list has a column for review flags on the right (red), a column for falls (red) and a column for the TP final calls (dark blue). [This is part I could not see in detail.] The judge can review each element on video, also slow it down, stop or speed it up etc.

They probably watch what happens on the ice, but the screen can be of assistance if an element happens in such a place that they have problems seeing it clearly from where they are located. If you are sitting in one or the other edge, it can be difficult seeing clearly if something happens in or near the other corner on the judges' side.

The judges probably usually mark at least some of the GOEs during the program, but can and do review their scores afterwards. This particularly if the Tech Panel has flagged elements - their decisions can change the GOE.

The PCS buttons open separate screens with the bullet points listed for each category.

The Tech Panel starts the reviews immediately after the skater stops and depending on how manyreviews they have to do can be ready very quickly or then take a bit more - sometimes almost every element can be flagged! The time they have is usually a bit less than 3 min, the first and last get a bit more. The judges start checking scores etc. immediately, but then have to wait for the results of the reviews. Some judges use the entire time they have, some are quicker but you have to be able to finish the job in the allocated time.

If they are ready quickly, they can press the "ready" button to enter their final scores to the system. There is probably an automatic cutoff when the scoring time ends [this could be the reason for some mistakes = the judge just has not had the time to return to some detail], because I saw at times that the screen switched to such a state that the judge could not continue scoring. The system could also notice if scores are missing, but this is me guessing.

The referee also gave scores at Riga which I thought was interesting - not included in the results in the end obviously. It is also possible they used these somehow in the review process afterwards, but this is me guessing.

There is a Swiss Timing tech (or three) to make sure things happen smoothly. Though bcs it is a computer system there are likely to be a glitch or two or three over any given event.

If the process seems to take a long while there can be many reasons. Tech problems are the most common I would imagine - someone's screen freezes mid-process and they have to deal with it, the whole system has a glitch, the announcing system has a glitch, etc. Lots of things can go wrong... At Riga, the judging system meltdown in the men's SP caused such a long delay that the referee declared a new warm-up for the remaining skaters in the last group.

Then I could also think that the Tech Panel review process can sometimes cause delays when there's a lot of things to check or they might have to discuss some problematic situation at length (the latter is probably not very common but happens, I think Papadakis & Cizeron at 2020 Euros could be an example). In these cases, some judges might continue their scoring all the way to the end, but I suspect that most would try to finish their job in the normal time frame since they would not necessarily be aware of a delay.

Then there was also the Pooh collecting time when Hanyu was on ice. They had to clear the ice and give the next skater the same amount of warm-up time as everyone else. If there was no next skater, he was then the last and the first and last skater get a slightly longer time. A lot of his fans thought the judges were giving his skates extra scrutiny, but I think the explanations are far more mundane.
 
The judges can write anything on a paper, including the SP order. ;)

It's not the judges' screen, but this might also be interesting to some: this is how the screen of the technical panel looks like (images are from 2021):

Uzor5DU.png


lNFfGb6.png


OEbjeHR.png
 
The referee also gave scores at Riga which I thought was interesting - not included in the results in the end obviously. It is also possible they used these somehow in the review process afterwards, but this is me guessing.
Referees always have to score the whole event.
 
The judges can write anything on a paper, including the SP order. ;)

The papers they can bring to the judging tables are outlined in Communication No. 2576:

The Judges will receive shortly before the commencement of the segment the “Judges Marking Sheets”, which is also the basis for the manual calculation of the results should this be necessary. These “Judges Marking Sheets” include the planned program content, i.e. the elements of their programs which skaters expect to perform.
In addition to the “Judges Marking Sheets”, the Judges may also take to their places the Rule books, the latest pertinent ISU Communication(s) including the guidelines for the Grade of Execution (GOE) and Program Components and summary of deductions/reductions. For Compulsory Ice Dance Pattern Dance segments, a copy of the relevant dance pattern is also allowed on the Judges stand.
The “Judges Marking Sheets”, one per skater/couple/team, allow Judges to note remarks during the Skaters performances and also serve as the basis for the manual back-up system in the case of an interruption in the electronic/computerized scoring system. However, taking notes by the Judges can be done only during the Skaters performance, but not after the scores have been sent.
 
Not a judge, but from sitting right behind the judges a few times at JGPs and snooping at their screens: @eppen ‘s description is very accurate and the photos of the judging system’s screens from previous years have not changed much into this season. The software looks pretty old-school but I kind like that they’re keeping it simple!

The GOE estimates you see during streams are from the scores judges input while the skater is still on the ice - that’s why they can change a lot between the estimate and final score (and sometimes the estimates are from a third of the panel or less - usually only things that are very clearly -5 or very clearly clean get the quick hits). Also, while the stream/replays for the audience might have shitty shots (e.g. framing a spin without feet/blades visible), the shots on the judging system are way better.

Also, PCSs scores are a separate screen, but some judges enter those right after the performance and some wait until they finished scoring the GOEs.

Almost all judges make pen-on-paper notes during the performance. Whether they refer to the printouts (planned content/rules) seems to be a personal preference, but I’ve seen judges who were cross-referencing almost every time. Which I think makes sense! Especially during JPGs at the start of the season, where any rule changes are pretty new.

Generally, the judges don’t talk to each other during scoring, but tech referees sometimes do.

Judges also have a second tablet/computer on their desks in case there’s sth wrong with the first one, so a tech issue doesn’t hold up the scoring.
 
Generally, the judges don’t talk to each other during scoring, but tech referees sometimes do.

I think the judges are forbidden to discuss the scores during the competition? The tech panel has to discuss the decisions and for that they have headphones with mics. Some panels talk a lot and then some not, depends on the persons and their preferences, I guess?

I think it's interesting how some people have alluded to results being decided beforehand or scores being given so that the ISU/organizers/whoever gets a result they want and not what should be in the writer's opinion. I personally am often in agreement with the final results especially if the score sheets reveal a seemingly fair TES evaluation but sometimes wonder about how skaters get particularly high/low PCS scores (but if the same keeps happening over the years, I just have to accept that what I like/dislike in some skater is not an opinion shared by judges).

I don't think, however, that the end result of any competition (expect maybe in ice dance
;)) can be decided before the competition happens.

In singles and pairs there are quite a few things that can go wrong even with the best and most consistent skaters so that it is difficult to predict the outcome of a performance and score it in such a way that you end up with a predetermined list. The end result can be decided by tenths and even hundreths, so how can you manipulate decisions of the TP and the 9 person panel in just 2-3 minutes times to get to a predetermined result? It would probably require a supercomputer to predict and determine the scores whilst the skater is still on ice and then those scores would have to be relayed to the judges somehow - I don't think anyone has ever seen the panel wearing ear buttons? Not to mention that the majority of the judges should then meekly comply with the suggestions...

The judges give a huge number of scores in each competition that keeping track of what each skater has gotten before and what is still to come would require meticulous bookkeeping by each judge which they really have no time to do. The screens do not allow them to access the leaderboard AFAIK so they cannot check the standings even during warm-ups (some info could be gathered during resurfacing breaks but even then you still have at least 8/10/12 skaters/teams to score afterwards). I cannot remember seeing phones on the tables either. After all, that kind of activity would be visible to the audience and people would start commenting on it pretty quickly, I think.

The time of anonymous judging was bad, of course - I don't know if even the ISU had access to who gave the scores? But you can see from eg the Skating Score's national bias charts that that practice has not been rooted out by the knowing which judge gave which scores. The panels also most times include judges from the most important nations, often reps for rival skaters/teams, so that I really would like to see how the predetermined lists were decided so that everyone in the panel(s) would agree to do it. Or even that the majority would agree to do it.

Then there has been talk of the tech panel which actually has quite a lot of power in their hands since they decide the levels and call mistakes etc. And they have in their disposal not always great material to make those decisions - the one camera angle can be unclear for edge and/or rotation calls. Plus there is a slight tendency at least to look harder at "usual suspects" ie skaters who are known to have problems with certain key aspects. The mistakes made by those who are perceived to be technically correct are not perhaps noticed unless they're very clear. The TP is supposed to composed of independent ISU reps, they do not represent countries, but eg national bias is hard to erase, I think.

I was partly insprired to write this based on Shin Amano's work in the TP of the European's men's competition where the TP was calling seemingly every edge and rotation call - only 4/24 men had no calls in the free. It is annoying that each panel - judges and techies - works a little differently and the results can be very lenient or very strict. How much this makes a difference in the final outcome is more difficult to determine, but eg lack of rotation calls can mean a lot of points gained in the final TES.

E
 
Then there has been talk of the tech panel which actually has quite a lot of power in their hands since they decide the levels and call mistakes etc. And they have in their disposal not always great material to make those decisions - the one camera angle can be unclear for edge and/or rotation calls. Plus there is a slight tendency at least to look harder at "usual suspects" ie skaters who are known to have problems with certain key aspects. The mistakes made by those who are perceived to be technically correct are not perhaps noticed unless they're very clear. The TP is supposed to composed of independent ISU reps, they do not represent countries, but eg national bias is hard to erase, I think.

I was partly insprired to write this based on Shin Amano's work in the TP of the European's men's competition where the TP was calling seemingly every edge and rotation call - only 4/24 men had no calls in the free. It is annoying that each panel - judges and techies - works a little differently and the results can be very lenient or very strict. How much this makes a difference in the final outcome is more difficult to determine, but eg lack of rotation calls can mean a lot of points gained in the final TES.

E
I wish all tech panels were so careful about their calls! But aside from the tech panel, there is a lot of technique assessment that should be done by the judges as well, and I feel like very often that area is lacking. For example, an underrotation of less than a quarter does not get marked by the tech panel, but should be noticed by the judges are get lower GOE. Same with unclear flutz edges, they can get GOE slightly lowered even if tech panel doesn’t mark it. (Although I can for the life of me figure out why pre-rotation is only penalised at half mark, instead of q, and why it’s not grounds for lowering GOE).

But looking at recent comps, I am seeing some changes (unless it’s a freak coincidence 🤣). Like step sequences: used to be that if you had the 2 clusters, you were getting lv3/4 anyway, now I’m seeing more and more that tech panels are paying more attention to the complexity/body movement criteria as well.
 
There is a division of labor between the panels and for the most part, I would imagine it is respected... Maybe already bcs of time constraints. You have only a few minutes to score the skater, so looking for flaws that the TP might have not noticed is usually probably not possible.

The judging panel does sometimes go against the TP when it comes to scoring quads with rotation calls - a material I have at my disposal very easily...

There have been some 1240 quads with qs since 2020 when it was introduced and 100 of them have gotten positive GOEs, about 14 to 25 cases/season, mostly domestic competitions - 30 in international ones over 5 seasons so far.

Usually once or twice per skater, but there are some who have gotten quite a few: Shoma Uno 7 (6 internationally) times, Vincent Zhou 8 (7 internationally) and Petr Gumennik 14 (all domestic).

There are also 19 cases where a < or even a << quad has received a 0 or positive GOE, usually 0,00.

Also these mostly domestic but some international as well. Interestingly, Vincent Zhou turns up again with 3 such quads (all domestic). And then there is also Hanyu who got 2 such scores at the 2019 Autum Classic International!

So, if some skaters get targeted for reviews as "usual suspects" then some can also obviously get away with mistakes...

E
 
The judging panel does sometimes go against the TP when it comes to scoring quads with rotation calls - a material I have at my disposal very easily...

There have been some 1240 quads with qs since 2020 when it was introduced and 100 of them have gotten positive GOEs, about 14 to 25 cases/season, mostly domestic competitions - 30 in international ones over 5 seasons so far.
This is not exactly judging panel going against tech panel - at least in case of q and <; for << it's an obvious judging error.

The reason is, GOE grading first counts the positive things about the jump to determine the "starting GOE", then retracts points for deductions.

Downgraded jumps (<<): starting GOE can be no higher than +2 and the deduction is -3 to -4 - so if final GOE is higher than -1, this was the judge ignoring the rules.

Missing rotation of less than a half does not have the starting GOE limit, so they can start the same as a clean jump, with a max +5 starting GOE (side note: when adding the points for the positives, they're supposed to look only at those! So you can have a jump with issues that also has positive things like good height/length/take-off/position/matches music etc. that would start high and then get dinked on those issues.

For a q jump that's perfect aside from that q, the max GOE allowed by the rules is +3, for underrotated (<) it's +2/3.

Technically even a fall could get a -3 instead of a -5, but I'd be hard pressed to come up with an example of a fall that should get no other deductions. :ROFLMAO:
 
Downgraded jumps (<<): starting GOE can be no higher than +2 and the deduction is -3 to -4 - so if final GOE is higher than -1, this was the judge ignoring the rules.
It also might be that the judge was busy working on scoring a different non-straightforward element, or more likely working on scoring the program components, which are on a separate page/screen, and neglected to check all the final edge/rotation calls from the tech panel before sending their scores.

That's a mistake on the part of the judge for sure -- they're supposed to check and adjust their GOEs to reflect the calls -- but it's not intentionally "ignoring."

Technically even a fall could get a -3 instead of a -5, but I'd be hard pressed to come up with an example of a fall that should get no other deductions. :ROFLMAO:
If there are no other deductions and also no positive qualities, then the final GOE should still be -5.

For jumps with falls and also other errors (underrotation to some degree being the most common), then the total reductions would add up to more than -5 but they're capped (or "floored" if you prefer) at -5. Some really bad jumps might have more errors than that (e.g., underrotation, edge change, telegraphed entry, touchdown of hand or free foot on the way down, poor air position, also very small jump which is more likely at non-elite levels).

But if there are one or more positive qualities going into the element and through most of the element but then a fall at the very end, it's entirely possible the judge is adding up multiple positive bullet points before subtracting -5 for the fall. There may not be any other errors at all.

If there's a fall on the landing, that pretty much automatically negates the "good takeoff and landing" bullet point. And probably the "effortless" bullet point. Still, there could very easily be preceding moves or unexpected entry, very good height and distance, enhancing the music, even good body position throughout including the landing before the fall.

This is probably most common in jump combinations/sequences where there's a lot more opportunity for the element to have multiple positive on the first jump and for much of the second (or third) one before the fall. But even a jump on its own can have at at least

Here's an example of a throw jump that certainly deserves the "very good height and distance" bullet and I don't see any other errors besides the fall:
 
Back
Top