Use of technology in figure skating tech calls
- Thread starter jinhamasaki
- Start date
Um, you're the one who said an automatic measurement has to be absolutely right. That means that if there's even one failure in a million then it should be thrown out and scrapped and not used. Compare that with the use of human judging systems, where there are quite a few mistakes even at the highest levels of competition like the Olympics and you say it's "good enough".
Whether human eyeballs looking at video or software-based algorithms looking at video is a better form of measurement is not really about manual vs automatic, it's about which alternative gives better results such as higher accuracy, faster, more efficient process, etc. Ultimately those are the figures of merit that we care about for measurement, not whether it's automatic or not. If I can show that for some reason a coin flip can perfectly predict whether or not a skater is under-rotated, then the coin flip is the best form of measurement simply because it is more accurate, faster, and more efficient that the others, regardless of how silly it sounds. (Obviously, the problem is all contained within that "if".) Adding an artificial standard to one alternative -- that it must be "absolutely right" -- without applying the same standard to other alternatives is simply incorrect and biased decision making. Rather, each alternative should be judged based on the things that we actually care about (accuracy, speed, etc.), rather than how those alternatives fulfill their purpose (manual or automatic).
Yup my local rink has the same issue -- the space near the lobby at one of the ends is around maybe 15 feet from rink barrier to wall, while it's more like 10 feet down the side of the rink with the bleachers, and around 5 feet where the zamboni is (at the other end) and where the announcer/rink control box is. Not only are there the hockey glass barriers, but there's also nets above them extending all the way to the ceiling (to catch errant pucks) at each end.
Fortunately, there are a variety of different ways to determine a skater's position from video; a wide-view camera that captures the entire rink is the easiest way in my opinion, but not the only way (although another alternative is that it could be mounted high above the rink). You can also localize a skater's position by triangulation if you know two camera's positions (X-Y coordinates) and where they're pointing at within the rink. For example, you could have the two cameras at two of the corners of the rink, and as they keep the skater in their sights, they'll also see the rink barriers in the background. From the rink barriers you can determine where the cameras were pointing -- put colored dots at evenly spaced distances on the top of the rink barriers if you want. Not only is this possible, but I use precisely this method for some of the skater jump locations for the Olympics, using the light blue pattern they had on the rink barriers. From the wide-angle camera I can figure out the X-Y coordinates of the different network cameras. Assuming they didn't move between skaters, for the skaters where I didn't find a wide-angle fancam video (that is to say, most of them), if I can find videos from two different camera angles then I can determine the skater's position at any point in their routine. An advantage of this setup is that the two cameras can stay zoomed in on the skater, so you can look at jumps, spins, etc. from two different angles if you want (as opposed to the one wide and one close-up setup where you only get the detailed video from one camera). The disadvantage is in the setup -- you have to calibrate the cameras' positions beforehand, so you have to acquire some information about the setup externally (whereas for a wide-view camera, you don't need to know the location of the camera, you just need to be able to see the rink).
Certainly it's a long term prospect to incorporate video analysis into the scoring system (i.e. basing the score linearly on computer-calculated jump rotation angles, like I mentioned above). I mean, looking at automatic race timing, from the wikipedia page, it was first used in 1928 but didn't become a requirement until 1977. So it's not something where you just try it out for a few years and then redo the scoring system around it -- it will take time to work out the kinks (not just software-wise but the judging process flow), for people to become familiar and more confident in it, to have a broader discussion about the technical details (such as how do you define the path of the skater), for the technology to become widespread, and so forth. Introducing technology tends to be done relatively gradually.
What I think is the easiest to implement and probably the most important is determining the skater's X-Y location within the rink at any given time, specifically at the beginning and at the end of a jump, and being able to relate this to the location of the camera that is providing the video replay. Because of perspective reasons, when looking at video it is very difficult to tell how much an object is moving toward or away from the camera; we usually see something as moving to the left or to the right, rather than that it's also gradually approaching or gradually receding. A skater that's mostly moving to the left, but say 20 degrees away from the camera looks almost the same as a skater that's moving say 20 degrees toward the camera, even though this difference -- 40 degrees -- makes a big difference for under-rotations. This is the single biggest reason why a lot of the post-Sochi video analysis was bogus, if they don't correct for the skater's relative motion to the camera, since it is very rare for the skater to be moving directly perpendicular to the camera, and it's a significant difference in the reported jump rotation angles. (The next biggest reason is people just made up angles relative to the camera, rather than justifying the numbers they put up.)
There's a variety of techniques for determining the skater's location within the rink, such as a wide-angle camera, or triangulation with two cameras. The data could be provided to the judges (or anybody, really), to note that when you're viewing this video, the skater is actually moving this much toward/away from the camera, so that the person could take it into account when looking at the skater's jump rotation. For example, it could be that you determine the skater's skate was at 75 degrees relative to the camera at the moment of touchdown (i.e. via eyeball), and you add in the software position-determined path correction of 35 degrees, for a total of 110 degrees. In fact doing this manually is what I'd be writing about once I talk about measuring jump rotation from video. Note though that I'm doing this manually, i.e. determining the skater's position from video with my eyeball (although I get it down to within a pixel, and the rest is software calculations).
Something like that is more like an aid for < and << calls, so that people (including judges) can properly account for perspective when looking at video replays, which leads to more accurate decisions. If this isn't available, well, then, judges can go with their own memory (of about where the skater was when they did a jump), albeit less accurate since it's estimation and not really measurement. Basically, the current system.
The more difficult thing is to actually measure the skater's rotation angle from the video image. It's harder for software to do because it's image recognition, but again, I have some ideas on how to do it. I do think the easiest method though is just to look at when the skate blade lines up with the camera, and then interpolate/extrapolate the skate's rotation from there. If measuring the skater's rotation angle is doable, then the system can incorporate the path correction with this measurement, to report the calculated jump rotation angle. Then the judge can take this into account in deciding whether or not to give a UR.
I think it's only once this system has been pretty established would there be any discussion of having it score jumps directly (i.e. lose points linearly based on how much a jump was under-rotated). Having it available at most venues is also an issue; skaters having enough to keep track of as it is and different scoring systems would just add to the confusion, so at that point it should be pretty much a given that the system is available for any given level of competition where that type of scoring is used.
No, having the data operator doing a sanity double-check doesn't mean that the judges can't do one on their own. It's just so the judges are aware of it when they look over the numbers.
See, the problem is that you keep assuming things about an automatic system that are unnecessary, and then use those assumptions to say why the system wouldn't work, rather than discussing the actual ideas directly. You assume an automatic system must be absolutely right (despite not applying the same standard to human judging) and then use it to argue why the system wouldn't work, rather than talking about the merits (i.e. comparing the accuracy of software vs human, comparing the speed, etc.). Here, I suggested that someone like the data operator could do a sanity double-check of the measurements, and you assume it means that no one else can question the measurements, and use that to argue against automatic measurement, as if judges are suddenly not allowed to view the videos or recall from their memory themselves.
The reason why I suggested the data operator was because that was how it worked in the factory I worked at. We had inspection machines that would visually inspect the product (take pictures, then use image recognition and edge detection software on the pictures to measure the dimensions of the product). Even though they were over 99% accurate, they measured over a thousand points a day, so throughout the course of a day they do produce some errors. So the inspection machine operators also looked at the pictures to do a sanity double-check; the machine inserted graphical arrows into the pictures indicating where on the product it measured, and the operators checked that those arrows lined up with the actual edges of the product.
If the inspection machine indicated that the product was out of spec, and thus a production machine needed to be adjusted, the above process didn't mean that the production engineer took the measurements blindly and adjusted the production machine from that. He was supposed to look at the images himself as well, to doubly confirm that the machine measured correctly. This is because adjusting a production machine could lead to thousands of dollars of product loss if tuned incorrectly, so the factory wanted to make sure that the measurements were correct. But the reason why the inspection machine operators looked at the images first is that in the event that it was a mis-measurement, then it's the inspection machine that needs to be adjusted (in software, which doesn't affect the product directly) and the production engineers don't need to be called to resolve the problem, so that they can, well, do engineering. This frees up time for the production engineer by putting the primary responsibility for correct measurements on the person in charge of the measurement machines.
In skating it's somewhat similar -- just because the data operator flags a video clip for a bad measurement doesn't mean that the judges can't if they think the measurement is wrong, just that they have a heads-up in case they themselves miss it, and so the data operator is aware of how well the system is working. And in a "worst case scenario" where all measurements are flagged as wrong, well, then we basically just have the current system where the judges use their eyeballs.
And the measurements were done quickly as well, roughly within 2 seconds or so; the bottleneck time-wise on the inspection machines was how long it took for the camera to move from location to location, not how long it took the software to analyze each image. The software company admitted to us afterward that it was their first time writing measurement software, and this was nearly 10 years ago; if a first-generation measurement software using the computational power of a computer nearly 10 years ago could analyze an image within 2 seconds, I don't see any reason why a company that's more knowledgeable about measurement algorithms using better computers today couldn't do it in near-real time. (Keep in mind the software wouldn't be analyzing a whole video but just the snippets of jumps during a performance, and that locating the object of interest, which was what took the longest time for the software, is much faster in a video where the object isn't moving much within the field of view frame-by-frame.) So the measurements would already be complete by the end of the performance, or perhaps very shortly after the data operator prepares the video snippets for the judges to review.
There's certainly many ways that it could be used. I don't think we should think of it as "the software says UR or not UR". Rather, it's more likely that the software would report something like "76 +- 9" or something, along with the ancillary information that it used (the path of the skater relative to the rink, the frame where the skate touched down on the ice, etc.). As such, there's no reason why it couldn't be made available to skaters and coaches, in fact it would likely improve their skating because then they could know how well they're rotating jumps day-in day-out in practices.
If the video that the software used is publicly available, then skating fans could certainly have their own debate about if a jump was okay or not, but I'd say that it's better to then actually analyze the videos themselves if they want to back up their case. Realistically, though, I doubt this will happen, because post-Sochi very few fans actually bothered to do any sort of analysis but instead just put screenshots and said "See!!!! it was under-rotated!!!" or "See!!! it was fine!!!" without backing any of it up. Having software provide a measurement at least gives a more neutral basis for the results, unless they want to complain that the algorithms are biased. As we saw, regardless of the evidence anyway some fans are still going to hold whatever opinion they want, and that's going to be true even if accurate measurements are made.
The question is, would it be better than the current system, where the technical panel makes their judgments without appeal (or at least, very unlikely to be appealed or changed) and where the judging process is completely opaque (we know the rules they're supposed to follow, but we have no idea why each judge did or didn't see an under-rotation when they reviewed the video)? With software measurement, at least this type of information is available and can be discussed.
Also, this type of software can be applied to any video, it's not restricted to the video the judges use. In fact, my analysis will rely on Youtube videos from the Olympics taken by the spectators (i.e. fancams) as well as the network feeds that I could find. So there's nothing to prevent people from doing this on their own. The difference is that people doing it on their own won't be official, but if there's a significant difference between what people can figure out on their own and what the official results are, well, then, yes there would be controversy, but that's nothing new. (Not to mention, if people are applying the correct lines of reasoning in measuring, then there shouldn't be any significant differences when done correctly.)
Nope, I didn't say that. I said that any technical assistance is good, but that the final decision should be made by the tech panel if the system is unreliable. You said that one person like the replay operator should make the final decision if the system's output doesn't make sense, that's where we disagree.
Um so there's two different points being discussed together.
#1: The standard that we should hold automatic systems to:
#2: The process flow for incorporating automatic measurement in judging:
Basically, I said that there will almost certainly still be humans checking the software measurements, giving the data operator as an example, from my personal job experience. It does not mean "only the data operator can check the measurements and the judges aren't allowed to disagree" nor does it mean "the data operator will be scoring the jump and the technical panel is powerless against it".
@ Vanshilar
What is your point now exactly? Do you want the TES to be scored automatically with the system you suggested or not?
My opinion still is that this wouldn't be a good idea, because the system you suggested is unreliable in my opinion. Therefore you still need the tech panel in, because otherwise judging would be far worse than it is now. But if it's possible to assist the tech panel in any way to make their work easier I'm for any technical assistance available.
This will be my last post in this thread, because I won't change my mind and I have the feeling that you won't change yours.
What is your point now exactly? Do you want the TES to be scored automatically with the system you suggested or not?
My opinion still is that this wouldn't be a good idea, because the system you suggested is unreliable in my opinion. Therefore you still need the tech panel in, because otherwise judging would be far worse than it is now. But if it's possible to assist the tech panel in any way to make their work easier I'm for any technical assistance available.
This will be my last post in this thread, because I won't change my mind and I have the feeling that you won't change yours.
