Wow, I didn't expect my random idea would get a new poster
Welcome! I don't have the data myself but I've seen some users on GS have data collected from scoring per judge so this might be interesting to run it through some regression analyses? I saw skatedb doesn't break down by judge AFAIK or that could be a resource.
Unfortunately if I ever get around to this might take a looong time, sigh.
Hello! If it's regression analysis, what's the dependent and independent variable? I was thinking that one of the x is the country where the judge comes from, and the y is the PCS (because TES can't be used, since every skater has different technical aspects to score).
Well, if you're interested in Yuzu's competitions only, it wouldn't take much. Protocols from previous seasons won't have the judges' names revealed, they will stay anonymous, so it would only be a matter of SC, NHK and GPF. You can find them in the
results' sites (the one I linked is only for NHK, for example), and you will find the judges' names and their corresponding number in the protocols in the "panel of judges" column
I don't have a lot of time on my hands right now (or good internet connection) so I was only able to run a test on SP Men NHK result, based on how the judges scored the PCS of all of the men skaters alone. (When I have the time I'll do further analysis with TES but the data mining is... time consuming
)
And this is what I found:
1. On average,
all the judges are consistent with their scoring. This means if a judge is stingy with the score, they will be stingy to ALL skaters. If a judge is generous, they will be generous to all skaters.
2. A single judge, however, will not be consistent. For example, if the panel only comprises of a Japanese judge, their consistency will be highly questionable.
3. It's very interesting how removing the Canadian judge from the data increases the average consistency and the single judge consistency (I use ICC, if anybody is interested). This
might indicate that the Canadian judge is inconsistent with some of their judgement. I would not say that their inconsistency is significant, however. (Removing the US judge also increases the average consistency, but with less impact than the Canadian judge).
4. The absolute agreement among the judges is... medium. For example, Nathan's true score for his transition--if he's judged by god or whatever--is 9. It's not easy to get this score in real life, or in other words, it's not easy to get the judges to *agree* to give nathan 9. The US will give him a 10, or the Russian will give him an 8. This variability is caused by systematical error--in this case, usually it's bias.
So yeah,
I would say the judges are kinda biased. Not all the time, but enough in NHK.
5. ...Removing the Canadian judge... will improve the absolute agreement score.... Hahaha. Disclaimer: I have nothing against the Canadian judge. It's not like the increase in score is significant, but it's there anyways. (Removing the Japanese judge, in fact, will decrease the consistency in point 1 above. It's not significant either.)
So there you go. As long as you have the xls ready I can do any sort of test, really. Pdf is hard because it takes time to convert it to editable xls ;___;
Edit: if anybody is interested with the xls file, here it is
https://drive.google.com/open?id=0B1CrwcK9CJcgcVhKb0YxbUJ5Z1k
nice to meet you, I really like Yuzu and hope can help sub his video to english
...but sadly 
