I'm pretty sure it was not the IOC decision. It's the home country/organization who decide about the schedule in the end. And they lost a lot of money for not having audience in their venues in first place. Also, to organize the whole event in these conditions request a lot of streinght, so I'm not blaming Japan... But my point was that many of those requests and decisions are against athletes to begin with...Your complaint, of course, is with the IOC who sold the broadcast rights to NBC with these provisos.
If the IOC and Mr. Thomas Bach did not want to have the events at unusual local times, they had the capacity to set them whenever they wanted. They chose to accept the money.
If I offer to buy something from you, and you agree to my conditions for purchase, that's not my fault that you agreed. ETA: You could always choose not to sell, or to sell to someone with different conditions.