I may be playing the devil's advocate, but money side of things are important and should not be disregarded. It may sounds selfish but at the moment the JSF benefits significantly from the popularity of figure skating in Japan and they need it. Money generated from the Worlds will be part distributed to a number of skaters, not only a handful of those who will be skating at the Worlds and some others you ever hear about, but novices, juniors, future hopefuls, AND BOTH SHORT AND LONG TRACK SPEED SKATERS TOO. It will be used not only on athletes, but supporting coaches, lobbying the ISU and the national government, organising training camps, etc, etc. Money is needed to support the future of skating in Japan. If money JSF has already spent cannot be recouped at least and they ended up loosing a significant amount of money, it'd be the athletes, especially those who do not have private sponsors, i.e., most of Japanese skaters will be the ones who would loose out. And the effect could last some time. It is understandable if JSF is trying to protect them.
For example, Yuzuru Hanyu lives in Sendai, and whose home rink is, I heard, badly damaged by the earthquake - so JSF may need money to support him continue skating elsewhere. Historically, JSF helped financially our world champions, Daisuke Takahashi and Shizuka Arakawa, who were not from very affluent family, to be trained abroad.
When figure skating was not bringing in any money to the fed and speed skaters were national heroes in Japan back in the days, the money made by speed skating was used to find and train the next generation of figure skaters systematically. Without speed skaters contribution, we may not have had the current crop of Japanese top figure skaters. Now the situation's turned round, figure skating is supporting speed skating, I believe.
Please note I am not outright supporting the Worlds in Tokyo in October, actually I too think the idea is bonkers, but am trying to understand why JSF appears so desperate to hang onto the possibility of holding the Worlds in Japan. It takes vast amount and wide range of considerations to run an organisation, which is responsible for the continuing success of the sport in the country. We do not have enough knowledge and information to understand fully how they do it. But it is easy to guess the finance (or financial health of the organisation) must be near the top of the priority list.
This is a very Japan-centred point of view, and of course, it is the ISU who have to take much more broad view on the issue and hopefully they will come up with the most sensible decision which benefits all the athletes (not only figure skaters but all athletes whose interests they are meant to look after) and federations fairly.