This only addresses the viewer. What about the SKATERS? I don't see anyone mentioning the most important group of people in all of this. Should skaters just dedicate all those hours and years to practice, and accept that they'll probably get rewarded with unfair scores? We may never achieve a perfect system, but we should aim for it.
I don't think anyone is suggesting that. It's just that no matter how perfect the system and how honest the judges, there probably will always be some disagreement about results. Unfortunately, that's the nature of a judged sport. The goal is (or should be) to construct a system that everyone trusts and that minimizes the chance of cheating/manipulation as much as possible.